Update README.md
Browse files
README.md
CHANGED
@@ -50,8 +50,10 @@ you can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512
|
|
50 |
|
51 |
how embedding and search works for now
|
52 |
|
53 |
-
you have a txt/pdf file maybe 90000words(~300pages). you ask the model lets say "what is described in chapter XYZ in relation to ZYX". now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX
|
|
|
54 |
so , a question for "summary of the document" is most time not useful, if the document has an introduction or summaries its searching there if you have luck.
|
|
|
55 |
if the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.
|
56 |
|
57 |
...
|
|
|
50 |
|
51 |
how embedding and search works for now
|
52 |
|
53 |
+
you have a txt/pdf file maybe 90000words(~300pages). you ask the model lets say "what is described in chapter XYZ in relation to ZYX". now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” , now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point. this text snippet is then used for your answer. if, for example, the word “XYZ” occurs 100 times in one file, not all 100 are found (usually only 4,8, or 16 snippet)
|
54 |
+
|
55 |
so , a question for "summary of the document" is most time not useful, if the document has an introduction or summaries its searching there if you have luck.
|
56 |
+
|
57 |
if the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.
|
58 |
|
59 |
...
|