Update README.md
Browse files
README.md
CHANGED
|
@@ -32,6 +32,8 @@ Stock for the "True Merge" -- This was a TIES Merge, the reasoning is explained
|
|
| 32 |
- Sao10K/L3.3-70B-Euryale-v2.3
|
| 33 |
- (Custom Base Model-Stock Soup -- Recipe Below)
|
| 34 |
|
|
|
|
|
|
|
| 35 |
|
| 36 |
# Why a different approach?
|
| 37 |
As some users had noted, particularly thanks to |GodZio| and The-Istar, the previous Mirai's instruct format was very unclear. Infact, when testing Llama-3 instruct format it seemed just broken, and, it was. Why? Well, the issue was with merging multiple models with different stopping tokens. I'll leave a tecnical explanation below for my assumption about why this happened. The long story short, I changed strategies for this model. It's very different, and expects the Llama-3 format to be used.
|
|
|
|
| 32 |
- Sao10K/L3.3-70B-Euryale-v2.3
|
| 33 |
- (Custom Base Model-Stock Soup -- Recipe Below)
|
| 34 |
|
| 35 |
+
One note here, I wasn't really sure how to state this in the huggingface tags. This model is actually THREE different merges. There's a base history merge, which was rolled into a base model merge, and you can see we merged the bases with our instruct models. Whew. I tried to give a thorough overview of model contributions, but not all of them contribute to the most direct merge.
|
| 36 |
+
|
| 37 |
|
| 38 |
# Why a different approach?
|
| 39 |
As some users had noted, particularly thanks to |GodZio| and The-Istar, the previous Mirai's instruct format was very unclear. Infact, when testing Llama-3 instruct format it seemed just broken, and, it was. Why? Well, the issue was with merging multiple models with different stopping tokens. I'll leave a tecnical explanation below for my assumption about why this happened. The long story short, I changed strategies for this model. It's very different, and expects the Llama-3 format to be used.
|