Benchmarks on model card and report don't add up
#9
by
patrickvonplaten
- opened
What benchmark is correct here?
Pretty much nothing adding up here
One appears to be the IT model and the other appears to be the PT model? I might be wrong, however!
Ah yeah that makes sense - on this instruction-tuned model card we only see PT numbers? Would be good to transfer the IT numbers to this model card I think
The first image shows the benchmark evaluated for 5-shot while the 2nd one is for 0-shot.