Update README.md
Browse files
README.md
CHANGED
@@ -15,6 +15,9 @@ abdullah-Qwen-Qwen2.5-0.5B-Instruct-merged-two-same-model-slerp is a merge of th
|
|
15 |
|
16 |
## 🧩 Configuration
|
17 |
|
|
|
|
|
|
|
18 |
```yaml
|
19 |
slices:
|
20 |
- sources:
|
@@ -77,6 +80,12 @@ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|
77 |
```
|
78 |
|
79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
80 |
|
81 |
## Citation
|
82 |
|
|
|
15 |
|
16 |
## 🧩 Configuration
|
17 |
|
18 |
+
Note: You nedd to change the layer range based on `number of layers` in the model. For `Qwen/Qwen2.5-0.5B-Instruct` number of layer is 24. The notebook used (see below in reference section
|
19 |
+
had `layer_range` `[0, 32]`, this is not true for the qwen model I am using here)
|
20 |
+
|
21 |
```yaml
|
22 |
slices:
|
23 |
- sources:
|
|
|
80 |
```
|
81 |
|
82 |
|
83 |
+
## Reference:
|
84 |
+
|
85 |
+
Used this https://huggingface.co/blog/mlabonne/merge-models for creating the merged model
|
86 |
+
|
87 |
+
|
88 |
+
|
89 |
|
90 |
## Citation
|
91 |
|