What am i doing wrong?

#38
by ILTRENTA - opened

I've been trying to use inceptionai/jais-13b-chat model for translation, altough it seems to work with very small chunk of text, like 4-6 words in arabic. When given longer context it:

  • either doesn't answer to translation queries
  • says it can't answer beacause it is harfmul content, but it really isn't because i am just giving it the context from wikipedia in arabic.

I am currently loading the model in float16 ( i am now trying bfloat16, but still getting the error).

I am using the code https://huggingface.co/inceptionai/jais-13b-chat you provided in the model card modifying the system prompt to direct it to translation.

is the problem related to the float32 to float16 conversion?
are there parameters i should use?
thank you very much in advance

Sign up or log in to comment