Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.7k

Running on CPU Upgrade

Thanks for the report!
cc @SaylorTwift

Hi ! It would really help if you provided the actual request file and not the commit, thanks :)

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/070083da3e2fe0d27419790fdabb9ebdc8d0e581/fblgit/UNAversal-2x7B-v1_eval_request_False_bfloat16_Original.json

{
  "model": "fblgit/UNAversal-2x7B-v1",
  "base_model": "",
  "revision": "main",
  "private": false,
  "precision": "bfloat16",
  "weight_type": "Original",
  "status": "FINISHED",
  "submitted_time": "2024-01-09T08:46:56Z",
  "model_type": "\ud83d\udd36 : fine-tuned",
  "likes": 0,
  "params": 12.879,
  "license": "apache-2.0",
  "architectures": "MixtralForCausalLM",
  "job_start_time": "2024-01-10T15:15:39.219012",
  "job_id": "657923"
}

will this work @SaylorTwift

Yes, thats perfect thanks !
Looking at the logs I can see that there was an server issue with the hub while uploading the details and result files, I will add them back asap

cool

Closing as resolved.

i still dont see the results @clefourrier

Ha, you are correct, I'll reopen and let @SaylorTwift close it when it's good :)

@clefourrier @SaylorTwift
Another fblgit model eval is missing as well.

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/fblgit/UNA-34Beagles-32K-v1_eval_request_False_bfloat16_Original.json

{
  "model": "fblgit/UNA-34Beagles-32K-v1",
  "base_model": "",
  "revision": "main",
  "private": false,
  "precision": "bfloat16",
  "params": 34.389,
  "architectures": "LlamaForCausalLM",
  "weight_type": "Original",
  "status": "FINISHED",
  "submitted_time": "2024-01-14T07:51:58Z",
  "model_type": "\ud83d\udd36 : fine-tuned",
  "job_id": "914733",
  "job_start_time": "2024-01-14T07:57:46.265410"
}

the other still also missing and not uploaded as well, this happens only with my models?

Hi, it seems like the issue affected multiple (which is normal since it's a dataset hub issue), I will add the missing files on Monday. Thanks for your patience !

Well.. ran it with another user, and the model had no problem at all. FBLGIT models twice in a row with this problem which seems a bit too much of a coincidence.

@clefourrier @SaylorTwift any news on the missing evals ? Either u can find it or we can re-trigger it. but nearly a week+ since the event.

^

Hi!
These results have been synced 5 days ago here and here.
Closing this issue as it's been resolved.

then why the models doesnt show on the leaderboard ?

Hi @fblgit ,
Your model appears on the leaderboard, but in the "flagged" category, most likely because your metadata is incorrect. Please update your metadata and open another issue for unflagging.