Files changed (1) hide show
  1. README.md +146 -132
README.md CHANGED
@@ -1,132 +1,146 @@
1
- ---
2
- license: other
3
- library_name: transformers
4
- base_model:
5
- - Qwen/Qwen2.5-72B-Instruct
6
- license_name: qwen
7
- license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
8
- model-index:
9
- - name: Replete-LLM-V2.5-Qwen-72b_Duplicated
10
- results:
11
- - task:
12
- type: text-generation
13
- name: Text Generation
14
- dataset:
15
- name: IFEval (0-Shot)
16
- type: HuggingFaceH4/ifeval
17
- args:
18
- num_few_shot: 0
19
- metrics:
20
- - type: inst_level_strict_acc and prompt_level_strict_acc
21
- value: 71.55
22
- name: strict accuracy
23
- source:
24
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
25
- name: Open LLM Leaderboard
26
- - task:
27
- type: text-generation
28
- name: Text Generation
29
- dataset:
30
- name: BBH (3-Shot)
31
- type: BBH
32
- args:
33
- num_few_shot: 3
34
- metrics:
35
- - type: acc_norm
36
- value: 61.27
37
- name: normalized accuracy
38
- source:
39
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
40
- name: Open LLM Leaderboard
41
- - task:
42
- type: text-generation
43
- name: Text Generation
44
- dataset:
45
- name: MATH Lvl 5 (4-Shot)
46
- type: hendrycks/competition_math
47
- args:
48
- num_few_shot: 4
49
- metrics:
50
- - type: exact_match
51
- value: 47.58
52
- name: exact match
53
- source:
54
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
55
- name: Open LLM Leaderboard
56
- - task:
57
- type: text-generation
58
- name: Text Generation
59
- dataset:
60
- name: GPQA (0-shot)
61
- type: Idavidrein/gpqa
62
- args:
63
- num_few_shot: 0
64
- metrics:
65
- - type: acc_norm
66
- value: 19.8
67
- name: acc_norm
68
- source:
69
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
70
- name: Open LLM Leaderboard
71
- - task:
72
- type: text-generation
73
- name: Text Generation
74
- dataset:
75
- name: MuSR (0-shot)
76
- type: TAUR-Lab/MuSR
77
- args:
78
- num_few_shot: 0
79
- metrics:
80
- - type: acc_norm
81
- value: 17.32
82
- name: acc_norm
83
- source:
84
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
85
- name: Open LLM Leaderboard
86
- - task:
87
- type: text-generation
88
- name: Text Generation
89
- dataset:
90
- name: MMLU-PRO (5-shot)
91
- type: TIGER-Lab/MMLU-Pro
92
- config: main
93
- split: test
94
- args:
95
- num_few_shot: 5
96
- metrics:
97
- - type: acc
98
- value: 54.83
99
- name: accuracy
100
- source:
101
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
102
- name: Open LLM Leaderboard
103
- ---
104
- # Rombos-LLM-V2.5-Qwen-72b
105
-
106
- ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/wp9qOi2K2WGzkey0I3SgH.jpeg)
107
-
108
- Rombos-LLM-V2.5-Qwen-72b is a continues finetuned version of Qwen2.5-72B. I noticed recently that the Qwen team did not learn from my methods of continuous finetuning, the great benefits, and no downsides of it. So I took it upon myself to merge the instruct model with the base model myself using the *Ties* merge method
109
-
110
- This version of the model shows higher performance than the original instruct and base models.
111
-
112
- Quants: (Coming soon)
113
-
114
- GGUF: https://huggingface.co/bartowski/Replete-LLM-V2.5-Qwen-72b-GGUF
115
-
116
- EXL2:
117
-
118
- Benchmarks: (Coming soon)
119
-
120
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
121
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_rombodawg__Replete-LLM-V2.5-Qwen-72b_Duplicated)
122
-
123
- | Metric |Value|
124
- |-------------------|----:|
125
- |Avg. |45.39|
126
- |IFEval (0-Shot) |71.55|
127
- |BBH (3-Shot) |61.27|
128
- |MATH Lvl 5 (4-Shot)|47.58|
129
- |GPQA (0-shot) |19.80|
130
- |MuSR (0-shot) |17.32|
131
- |MMLU-PRO (5-shot) |54.83|
132
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: transformers
4
+ base_model:
5
+ - Qwen/Qwen2.5-72B-Instruct
6
+ license_name: qwen
7
+ license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
8
+ language:
9
+ - zho
10
+ - eng
11
+ - fra
12
+ - spa
13
+ - por
14
+ - deu
15
+ - ita
16
+ - rus
17
+ - jpn
18
+ - kor
19
+ - vie
20
+ - tha
21
+ - ara
22
+ model-index:
23
+ - name: Replete-LLM-V2.5-Qwen-72b_Duplicated
24
+ results:
25
+ - task:
26
+ type: text-generation
27
+ name: Text Generation
28
+ dataset:
29
+ name: IFEval (0-Shot)
30
+ type: HuggingFaceH4/ifeval
31
+ args:
32
+ num_few_shot: 0
33
+ metrics:
34
+ - type: inst_level_strict_acc and prompt_level_strict_acc
35
+ value: 71.55
36
+ name: strict accuracy
37
+ source:
38
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
39
+ name: Open LLM Leaderboard
40
+ - task:
41
+ type: text-generation
42
+ name: Text Generation
43
+ dataset:
44
+ name: BBH (3-Shot)
45
+ type: BBH
46
+ args:
47
+ num_few_shot: 3
48
+ metrics:
49
+ - type: acc_norm
50
+ value: 61.27
51
+ name: normalized accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: MATH Lvl 5 (4-Shot)
60
+ type: hendrycks/competition_math
61
+ args:
62
+ num_few_shot: 4
63
+ metrics:
64
+ - type: exact_match
65
+ value: 47.58
66
+ name: exact match
67
+ source:
68
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
69
+ name: Open LLM Leaderboard
70
+ - task:
71
+ type: text-generation
72
+ name: Text Generation
73
+ dataset:
74
+ name: GPQA (0-shot)
75
+ type: Idavidrein/gpqa
76
+ args:
77
+ num_few_shot: 0
78
+ metrics:
79
+ - type: acc_norm
80
+ value: 19.8
81
+ name: acc_norm
82
+ source:
83
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
84
+ name: Open LLM Leaderboard
85
+ - task:
86
+ type: text-generation
87
+ name: Text Generation
88
+ dataset:
89
+ name: MuSR (0-shot)
90
+ type: TAUR-Lab/MuSR
91
+ args:
92
+ num_few_shot: 0
93
+ metrics:
94
+ - type: acc_norm
95
+ value: 17.32
96
+ name: acc_norm
97
+ source:
98
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
99
+ name: Open LLM Leaderboard
100
+ - task:
101
+ type: text-generation
102
+ name: Text Generation
103
+ dataset:
104
+ name: MMLU-PRO (5-shot)
105
+ type: TIGER-Lab/MMLU-Pro
106
+ config: main
107
+ split: test
108
+ args:
109
+ num_few_shot: 5
110
+ metrics:
111
+ - type: acc
112
+ value: 54.83
113
+ name: accuracy
114
+ source:
115
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=rombodawg/Replete-LLM-V2.5-Qwen-72b_Duplicated
116
+ name: Open LLM Leaderboard
117
+ ---
118
+ # Rombos-LLM-V2.5-Qwen-72b
119
+
120
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/wp9qOi2K2WGzkey0I3SgH.jpeg)
121
+
122
+ Rombos-LLM-V2.5-Qwen-72b is a continues finetuned version of Qwen2.5-72B. I noticed recently that the Qwen team did not learn from my methods of continuous finetuning, the great benefits, and no downsides of it. So I took it upon myself to merge the instruct model with the base model myself using the *Ties* merge method
123
+
124
+ This version of the model shows higher performance than the original instruct and base models.
125
+
126
+ Quants: (Coming soon)
127
+
128
+ GGUF: https://huggingface.co/bartowski/Replete-LLM-V2.5-Qwen-72b-GGUF
129
+
130
+ EXL2:
131
+
132
+ Benchmarks: (Coming soon)
133
+
134
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
135
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_rombodawg__Replete-LLM-V2.5-Qwen-72b_Duplicated)
136
+
137
+ | Metric |Value|
138
+ |-------------------|----:|
139
+ |Avg. |45.39|
140
+ |IFEval (0-Shot) |71.55|
141
+ |BBH (3-Shot) |61.27|
142
+ |MATH Lvl 5 (4-Shot)|47.58|
143
+ |GPQA (0-shot) |19.80|
144
+ |MuSR (0-shot) |17.32|
145
+ |MMLU-PRO (5-shot) |54.83|
146
+