Update README.md
Browse files
README.md
CHANGED
|
@@ -21,10 +21,13 @@ language:
|
|
| 21 |
- en
|
| 22 |
- ja
|
| 23 |
---
|
| 24 |
-
#
|
| 25 |
|
| 26 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 27 |
|
|
|
|
|
|
|
|
|
|
| 28 |
## Configuration
|
| 29 |
|
| 30 |
The following YAML configuration was used to produce this model:
|
|
|
|
| 21 |
- en
|
| 22 |
- ja
|
| 23 |
---
|
| 24 |
+
# Llama-3-VNTL-Yollisa-8B
|
| 25 |
|
| 26 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 27 |
|
| 28 |
+
This merge is an expansion on the idea of [merging at extremely low weight as an alternitive to finetuning](https://huggingface.co/grimjim/kukulemon-v3-soul_mix-32k-7B) with the added step of subtracting the base model from finetunes before merging.
|
| 29 |
+
Instruct format is the custom version of llama3 that VNTL uses, but you should be able to mix in some regular llama3 formats as well, and it might even help with improving translation quality with the right prompt.
|
| 30 |
+
|
| 31 |
## Configuration
|
| 32 |
|
| 33 |
The following YAML configuration was used to produce this model:
|