What am i doing wrong?
#38
by
ILTRENTA
- opened
I've been trying to use inceptionai/jais-13b-chat model for translation, altough it seems to work with very small chunk of text, like 4-6 words in arabic. When given longer context it:
- either doesn't answer to translation queries
- says it can't answer beacause it is harfmul content, but it really isn't because i am just giving it the context from wikipedia in arabic.
I am currently loading the model in float16 ( i am now trying bfloat16, but still getting the error).
I am using the code https://huggingface.co/inceptionai/jais-13b-chat you provided in the model card modifying the system prompt to direct it to translation.
is the problem related to the float32 to float16 conversion?
are there parameters i should use?
thank you very much in advance