Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
mistralai/Mistral-7B-v0.1 suspiciously high MLLU score
#299
by
ThomasBaruzier
- opened
This new 7b model outperforms the second model by 14% in MLLU.
Conducting a small investigation should be done before flagging the model, of course.
I believe it’s pretrained on completely new data and is not a fine tuned version of llama 2 or llama 1. It’s similar but with gqa. Of course, there could be some contamination.
Oops posted in wrong thread.
Unfortunate it is for now impossible to know on what dataset mistral-7B has been trained, so closing this discussion for now, feel free to reopen if you find something interesting !
SaylorTwift
changed discussion status to
closed