Model no longer respects structured response requests
#2
by
prolix-oc
- opened
Using Aphrodite Engine and the lm-format-enforcer guided response backend, I was able to generate structured responses with Mistral-Small-24B-Instruct-2501. Using this model, it appears to completely ignore any response_format requests and outputs a single string. I am not using a reasoning prompt for this call, and have specified the example format output in the system prompt.
Any tips or potential explanations towards this? I am fully aware this model is in preview, but I would like to give feedback.