File size: 1,905 Bytes
a2a30ce db490d7 f10f3d2 db490d7 8bb0749 db490d7 707634e 6c6b419 806cfb4 707634e f9a3a38 806cfb4 f9a3a38 3d573c1 806cfb4 3d573c1 806cfb4 13adbac 277ce71 13adbac 66bf69c 13adbac 66bf69c 13adbac 806cfb4 c817644 3dd8202 c817644 fbce735 3dd8202 292bdf5 fbce735 292bdf5 fbce735 c817644 13adbac 3dd8202 806cfb4 db490d7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- embedder
- embedding
- moedels
- GGUF
- Bert
- Nomic
- Gist
- text-embeddings-inference
misc:
- text-embeddings-inference
language:
- en
- de
architecture:
- GIST
---
# All models tested with ALLM(AnythingLLM) with LM as server
they work more or less (all BGE types iv tested dont work)
my short impression:
- nomic-embed-text
- mxbai-embed-large
- mug-b-1.6
- Ger-RAG-BGE-M3 (german)
working well, all other its up to you!
short hints for using:
set your (Max Tokens)context-lenght 16000t main-model, set your embedder-model (Max Embedding Chunk Length) 1024t,set (Max Context Snippets) 14
-> ok what that mean!
you can receive 14-snippets a 1024t (14336t) from your document ~10000words and 1600t left for the answer ~1000words
you can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512t ...
16000t ~1GB VRAM usage
...
how embedding and search works for now
you have a txt/pdf file maybe 90000words(~300pages). you ask the model lets say "what is described in chapter XYZ in relation to ZYX". now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” , now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point. this text snippet is then used for your answer. if, for example, the word “XYZ” occurs 100 times in one file, not all 100 are found (usually only 4,8, or 16 snippet)
so , a question for "summary of the document" is most time not useful, if the document has an introduction or summaries its searching there if you have luck.
if the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.
...
...
(ALL Licenses and terms of use go to original author) |