LeroyDyer commited on
Commit
4f4f2b5
·
verified ·
1 Parent(s): 50c69e5

Adding Evaluation Results (#2)

Browse files

- Adding Evaluation Results (d031fd883ed62246a960f6100974aa16d058de52)

Files changed (1) hide show
  1. README.md +120 -12
README.md CHANGED
@@ -2,6 +2,7 @@
2
  language:
3
  - en
4
  license: apache-2.0
 
5
  tags:
6
  - text-generation-inference
7
  - transformers
@@ -21,17 +22,6 @@ tags:
21
  - mega-series
22
  - SpydazWebAI
23
  base_model: LeroyDyer/Mixtral_AI_CyberTron_Ultra
24
- metrics:
25
- - accuracy
26
- - bertscore
27
- - bleu
28
- - brier_score
29
- - cer
30
- - character
31
- - charcut_mt
32
- - chrf
33
- - code_eval
34
- library_name: transformers
35
  datasets:
36
  - gretelai/synthetic_text_to_sql
37
  - HuggingFaceTB/cosmopedia
@@ -47,6 +37,111 @@ datasets:
47
  - Rogendo/English-Swahili-Sentence-Pairs
48
  - ise-uiuc/Magicoder-Evol-Instruct-110K
49
  - meta-math/MetaMathQA
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  ---
51
 
52
  # Uploaded model
@@ -116,4 +211,17 @@ Im not sure if Lora actually works when you save them but i do save some and use
116
 
117
  This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
118
 
119
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  language:
3
  - en
4
  license: apache-2.0
5
+ library_name: transformers
6
  tags:
7
  - text-generation-inference
8
  - transformers
 
22
  - mega-series
23
  - SpydazWebAI
24
  base_model: LeroyDyer/Mixtral_AI_CyberTron_Ultra
 
 
 
 
 
 
 
 
 
 
 
25
  datasets:
26
  - gretelai/synthetic_text_to_sql
27
  - HuggingFaceTB/cosmopedia
 
37
  - Rogendo/English-Swahili-Sentence-Pairs
38
  - ise-uiuc/Magicoder-Evol-Instruct-110K
39
  - meta-math/MetaMathQA
40
+ metrics:
41
+ - accuracy
42
+ - bertscore
43
+ - bleu
44
+ - brier_score
45
+ - cer
46
+ - character
47
+ - charcut_mt
48
+ - chrf
49
+ - code_eval
50
+ model-index:
51
+ - name: SpydazWeb_AI_CyberTron_Ultra_7b
52
+ results:
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: IFEval (0-Shot)
58
+ type: HuggingFaceH4/ifeval
59
+ args:
60
+ num_few_shot: 0
61
+ metrics:
62
+ - type: inst_level_strict_acc and prompt_level_strict_acc
63
+ value: 15.56
64
+ name: strict accuracy
65
+ source:
66
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
67
+ name: Open LLM Leaderboard
68
+ - task:
69
+ type: text-generation
70
+ name: Text Generation
71
+ dataset:
72
+ name: BBH (3-Shot)
73
+ type: BBH
74
+ args:
75
+ num_few_shot: 3
76
+ metrics:
77
+ - type: acc_norm
78
+ value: 27.75
79
+ name: normalized accuracy
80
+ source:
81
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
82
+ name: Open LLM Leaderboard
83
+ - task:
84
+ type: text-generation
85
+ name: Text Generation
86
+ dataset:
87
+ name: MATH Lvl 5 (4-Shot)
88
+ type: hendrycks/competition_math
89
+ args:
90
+ num_few_shot: 4
91
+ metrics:
92
+ - type: exact_match
93
+ value: 1.36
94
+ name: exact match
95
+ source:
96
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
97
+ name: Open LLM Leaderboard
98
+ - task:
99
+ type: text-generation
100
+ name: Text Generation
101
+ dataset:
102
+ name: GPQA (0-shot)
103
+ type: Idavidrein/gpqa
104
+ args:
105
+ num_few_shot: 0
106
+ metrics:
107
+ - type: acc_norm
108
+ value: 5.7
109
+ name: acc_norm
110
+ source:
111
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
112
+ name: Open LLM Leaderboard
113
+ - task:
114
+ type: text-generation
115
+ name: Text Generation
116
+ dataset:
117
+ name: MuSR (0-shot)
118
+ type: TAUR-Lab/MuSR
119
+ args:
120
+ num_few_shot: 0
121
+ metrics:
122
+ - type: acc_norm
123
+ value: 10.3
124
+ name: acc_norm
125
+ source:
126
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
127
+ name: Open LLM Leaderboard
128
+ - task:
129
+ type: text-generation
130
+ name: Text Generation
131
+ dataset:
132
+ name: MMLU-PRO (5-shot)
133
+ type: TIGER-Lab/MMLU-Pro
134
+ config: main
135
+ split: test
136
+ args:
137
+ num_few_shot: 5
138
+ metrics:
139
+ - type: acc
140
+ value: 20.73
141
+ name: accuracy
142
+ source:
143
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=LeroyDyer/SpydazWeb_AI_CyberTron_Ultra_7b
144
+ name: Open LLM Leaderboard
145
  ---
146
 
147
  # Uploaded model
 
211
 
212
  This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
213
 
214
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
215
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
216
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/LeroyDyer__SpydazWeb_AI_CyberTron_Ultra_7b-details)
217
+
218
+ | Metric |Value|
219
+ |-------------------|----:|
220
+ |Avg. |13.57|
221
+ |IFEval (0-Shot) |15.56|
222
+ |BBH (3-Shot) |27.75|
223
+ |MATH Lvl 5 (4-Shot)| 1.36|
224
+ |GPQA (0-shot) | 5.70|
225
+ |MuSR (0-shot) |10.30|
226
+ |MMLU-PRO (5-shot) |20.73|
227
+