lbourdois commited on
Commit
ade8c88
·
verified ·
1 Parent(s): f90fa95

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +82 -69
README.md CHANGED
@@ -1,69 +1,82 @@
1
- ---
2
- base_model:
3
- - Qwen/Qwen2.5-7B
4
- - Qwen/Qwen2.5-Coder-7B
5
- - Qwen/Qwen2.5-7B-Instruct
6
- - Qwen/Qwen2.5-Math-7B
7
- library_name: transformers
8
- tags:
9
- - mergekit
10
- - merge
11
-
12
- ---
13
- # nthehai01/Qwen2.5-7B-Instruct-Math-Code-breadcrumbs
14
-
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
-
17
- ## Performance
18
- | Metric |Value|
19
- |---------------------------------|----:|
20
- |GSM8k (zero-shot) |90.06|
21
- |HellaSwag (zero-Shot) |82.77|
22
- |MBPP (zero-shot) |62.21|
23
-
24
- ## Merge Details
25
- ### Merge Method
26
-
27
- This model was merged using the [Model Breadcrumbs](https://arxiv.org/abs/2312.06795) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
28
-
29
- ### Models Merged
30
-
31
- The following models were included in the merge:
32
- * [Qwen/Qwen2.5-Coder-7B](https://huggingface.co/Qwen/Qwen2.5-Coder-7B)
33
- * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
34
- * [Qwen/Qwen2.5-Math-7B](https://huggingface.co/Qwen/Qwen2.5-Math-7B)
35
-
36
- ### Configuration
37
-
38
- The following YAML configuration was used to produce this model:
39
-
40
- ```yaml
41
- base_model: Qwen/Qwen2.5-7B
42
- dtype: bfloat16
43
- merge_method: breadcrumbs
44
- parameters:
45
- lambda: 0.9075603207928135
46
- normalize: 1.0
47
- slices:
48
- - sources:
49
- - layer_range: [0, 28]
50
- model: Qwen/Qwen2.5-7B
51
- - layer_range: [0, 28]
52
- model: Qwen/Qwen2.5-Math-7B
53
- parameters:
54
- density: 0.11722197443445775
55
- gamma: 0.07547691839721048
56
- weight: 0.17267293536872041
57
- - layer_range: [0, 28]
58
- model: Qwen/Qwen2.5-Coder-7B
59
- parameters:
60
- density: 0.48352747334554935
61
- gamma: 0.0753405327865558
62
- weight: 0.11164770709858211
63
- - layer_range: [0, 28]
64
- model: Qwen/Qwen2.5-7B-Instruct
65
- parameters:
66
- density: 0.8190520808683315
67
- gamma: 0.022307694128235696
68
- weight: 0.7626295102691242
69
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Qwen/Qwen2.5-7B
4
+ - Qwen/Qwen2.5-Coder-7B
5
+ - Qwen/Qwen2.5-7B-Instruct
6
+ - Qwen/Qwen2.5-Math-7B
7
+ library_name: transformers
8
+ tags:
9
+ - mergekit
10
+ - merge
11
+ language:
12
+ - zho
13
+ - eng
14
+ - fra
15
+ - spa
16
+ - por
17
+ - deu
18
+ - ita
19
+ - rus
20
+ - jpn
21
+ - kor
22
+ - vie
23
+ - tha
24
+ - ara
25
+ ---
26
+ # nthehai01/Qwen2.5-7B-Instruct-Math-Code-breadcrumbs
27
+
28
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
29
+
30
+ ## Performance
31
+ | Metric |Value|
32
+ |---------------------------------|----:|
33
+ |GSM8k (zero-shot) |90.06|
34
+ |HellaSwag (zero-Shot) |82.77|
35
+ |MBPP (zero-shot) |62.21|
36
+
37
+ ## Merge Details
38
+ ### Merge Method
39
+
40
+ This model was merged using the [Model Breadcrumbs](https://arxiv.org/abs/2312.06795) merge method using [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B) as a base.
41
+
42
+ ### Models Merged
43
+
44
+ The following models were included in the merge:
45
+ * [Qwen/Qwen2.5-Coder-7B](https://huggingface.co/Qwen/Qwen2.5-Coder-7B)
46
+ * [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
47
+ * [Qwen/Qwen2.5-Math-7B](https://huggingface.co/Qwen/Qwen2.5-Math-7B)
48
+
49
+ ### Configuration
50
+
51
+ The following YAML configuration was used to produce this model:
52
+
53
+ ```yaml
54
+ base_model: Qwen/Qwen2.5-7B
55
+ dtype: bfloat16
56
+ merge_method: breadcrumbs
57
+ parameters:
58
+ lambda: 0.9075603207928135
59
+ normalize: 1.0
60
+ slices:
61
+ - sources:
62
+ - layer_range: [0, 28]
63
+ model: Qwen/Qwen2.5-7B
64
+ - layer_range: [0, 28]
65
+ model: Qwen/Qwen2.5-Math-7B
66
+ parameters:
67
+ density: 0.11722197443445775
68
+ gamma: 0.07547691839721048
69
+ weight: 0.17267293536872041
70
+ - layer_range: [0, 28]
71
+ model: Qwen/Qwen2.5-Coder-7B
72
+ parameters:
73
+ density: 0.48352747334554935
74
+ gamma: 0.0753405327865558
75
+ weight: 0.11164770709858211
76
+ - layer_range: [0, 28]
77
+ model: Qwen/Qwen2.5-7B-Instruct
78
+ parameters:
79
+ density: 0.8190520808683315
80
+ gamma: 0.022307694128235696
81
+ weight: 0.7626295102691242
82
+ ```