How can I embed textual input to calculate the cosine similarity?
#95
by
skaes
- opened
Dear community,
all tutorials which I found just talked about how to use a tokenizer to create input based on a prompt. I actually wish to create embeddings for 2 sentences and then compare them using cosine similarity. How can I do that? I did not find a function which simply converts text into tokens.
Thank you in advance!
All the best,
S.