gz987 commited on
Commit
9e50bca
·
verified ·
1 Parent(s): ad63138

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -34
README.md CHANGED
@@ -1,40 +1,15 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
 
8
- ---
9
- # Untitled Model (1)
10
 
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
 
13
- ## Merge Details
14
- ### Merge Method
 
15
 
16
- This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using /home/LAB/yangzz/rank/Qwen2.5-7B-Instrust/ as a base.
17
 
18
- ### Models Merged
19
 
20
- The following models were included in the merge:
21
- * /home/LAB/yangzz/rank/Tsunami-0.5-7B-Instruct-model-nm-32-128-fq-first/
22
- * /home/LAB/yangzz/rank/fq2.5-7b-it-normalize_true-model-nm-32-128-fq-first/
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- models:
30
- - model: /home/LAB/yangzz/rank/Tsunami-0.5-7B-Instruct-model-nm-32-128-fq-first/
31
- parameters:
32
- weight: 0.55
33
- - model: /home/LAB/yangzz/rank/fq2.5-7b-it-normalize_true-model-nm-32-128-fq-first/
34
- parameters:
35
- weight: 0.5
36
-
37
- merge_method: task_arithmetic
38
- dtype: bfloat16
39
- base_model: /home/LAB/yangzz/rank/Qwen2.5-7B-Instrust/
40
- ```
 
1
+ This model is a merged model based on [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) using a novel model merging technique.
 
 
 
 
 
2
 
3
+ ## **Performance (Self-Tested on A100)**
 
4
 
5
+ The following results are obtained using **batch_size=6** on an A100 GPU. Official results are pending submission to `open_llm_leaderboard`.
6
 
7
+ | IFEVAL | BBH | MATH | GPQA | MUSR | MMLU-PRO | AVG |
8
+ |--------|------|------|------|------|----------|-------|
9
+ | 75.21 | 36.03 | 48.87 | 7.61 | 15.37 | 37.8 | 36.815 |
10
 
11
+ **Note**: These results will be updated once officially verified.
12
 
13
+ ## **Recipe Coming Soon**
14
 
15
+ We will release details on the merging technique and methodology soon. Stay tuned! 🚀