Multilingual?
Hi everyone
Understand, that Glider was trained on english, however base model is multilingual.
Has anyone:
- evaluated how good Glider performs on FR/DE/IT?
- an intuition, how much further Finetuning / Synthetic Data would be necessary to adopt Glider to multilingual use-case?
Hey! You can see the multilingual reward bench results (Table 3) in our paper (https://arxiv.org/pdf/2412.14140)
As you can see, the model already outperforms other open-source, judge models for FR, DE and IT.
great! how should a prompt look like then?
a) whole prompt in german or
b) default prompt with additional instruction that the given trxt and output should be in german/french/italian?
ah, and tag the model be multilingual on huggingface, otherwise you are missing out whole europe
We recommend using the default prompt in English and passing the evaluation fields in german/french or italian. Will tag the model as multilingual, thanks for the suggestion :)