🇪🇪 Estonian LLM Evaluation
A collection of resources for evaluation of LLM capabilities in the Estonian language.
Viewer • Updated • 541 • 200Note Manually translated and culturally adapted IFEval.
tartuNLP/winogrande_et
Viewer • Updated • 4.83k • 178Note Manually translated and culturally adapted WinoGrande test set.
TalTechNLP/EstQA
Viewer • Updated • 1.38k • 77 • 2Note Manually annotated extractive question answering.
tartuNLP/EstNER
Viewer • Updated • 46.1k • 26Note Manually annotated named entity recognition.
tartuNLP/EstCOPA
Viewer • Updated • 2k • 11 • 1Note Machine-translated and post-edited choice of plausible alternatives.
TalTechNLP/grammar_et
Viewer • Updated • 8.94k • 17 • 1Note Grammatical error correction. Text with errors → corrected text.
TalTechNLP/ERRnews
Updated • 37 • 1Note ERR news story transcripts generated by an ASR pipeline paired with the human written summary.
kardosdrur/estonian-valence
Viewer • Updated • 4.09k • 432 • 1Note Newspaper article sentiment.
TalTechNLP/exam_et
Viewer • Updated • 1.61k • 9Note Exam questions on several subjects in multiple choice format.
TalTechNLP/MMLU_et
Viewer • Updated • 14k • 119 • 1Note Machine-translated MMLU.
TalTechNLP/word_meanings_et
Viewer • Updated • 54.6k • 5Note Matching words with their dictionary definitions.
TalTechNLP/inflection_et
Viewer • Updated • 1.4k • 6 • 1Note Noun phrase inflection.
TalTechNLP/trivia_et
Viewer • Updated • 800 • 22 • 1Note Trivia dataset extracted from the "Eesti Mäng" board game.
TalTechNLP/MMLU_Pro_et
Viewer • Updated • 12.1k • 3Note Machine-translated MMLU-Pro.
TalTechNLP/M_MMLU_et
Viewer • Updated • 15k • 3
tartuNLP/smugri-mt-bench
Viewer • Updated • 500 • 94 • 2Note Extension of the MT-Bench dataset including Estonian, Livonian, Komi, and Võro.
tartuNLP/flores-smugri-pairs
Viewer • Updated • 16.5k • 52Note Extension of the original Flores dataset adding additional translation directions, such as Estonian to Võro, Estonian to Komi-Zyrian, and some others.
facebook/belebele
Viewer • Updated • 110k • 14.4k • 114Note Multiple choice reading comprehension. Includes Estonian as one of the languages.
cambridgeltl/xcopa
Viewer • Updated • 12.6k • 6.41k • 18Note Machine-translated multilingual COPA. Includes validation and test for Estonian.
jumelet/multiblimp
Viewer • Updated • 121k • 735 • 12Note Linguistic acceptability dataset derived from Universal Dependencies with heuristics. Includes Estonian as one of the languages.
openlanguagedata/flores_plus
Viewer • Updated • 872k • 4.35k • 61Note Includes Estonian as one of the languages. English sources are translated to 200 languages. Any translation direction is possible potentially.
google/wmt24pp
Viewer • Updated • 54.9k • 3.52k • 58Note Includes English to Estonian translation direction.
mteb/sib200
Viewer • Updated • 387k • 2.21k • 3Note Includes Estonian as one of the languages, classification labels remain in English.