jpacifico

/

Chocolatine-2-14B-Instruct-v2.0.3

like 14

hi @Farouk09 thanks for your message ! That’s indeed strange — I’ve never observed such behavior from the model before. Did you by use a quantized version ?

Hi @jpacifico , thanks for your reply! I served the model using vLLM with dtype="auto" (which uses FP16 precision for FP32/FP16 models and BF16 for BF16 models), and a temperature setting of 0.3.

The system prompt instructed the LLM to act as "un tuteur pour orienter les étudiants à choisir une formation adéquate."