open-llm-leaderboard/open_llm_leaderboard · Evaluation of my models failed 2 days ago :(

Apr 26, 2024

•

edited Apr 26, 2024

My model failed during eval. It's been 3 days but I have now checked it. Following is the link:

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/ChuGyouk/Llama-3-11.5B-DUS-no-cpt_eval_request_False_bfloat16_Original.json

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/ChuGyouk/Llama-3-11.5B-modified-DUS-nocpt_eval_request_False_bfloat16_Original.json

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/ChuGyouk/Llama-3-6.3B-no-healing_eval_request_False_bfloat16_Original.json

Could you check and tell me why it failed? I cannot resubmit my models.
Thanks :)

ChuGyouk changed discussion title from Evaluation of my models failed 2 days ago... to Evaluation of my models failed 2 days ago :( Apr 26, 2024

alozowski

Open LLM Leaderboard org Apr 29, 2024

Hi @ChuGyouk ,

Thanks for providing the request files! I've checked their logs and resubmitted your models – seems like the recent network failures affected their evaluation

I'll close this issue, please reopen it / write here in case of any other problems with these models

alozowski changed discussion status to closed Apr 29, 2024