Update README.md
Browse files
README.md
CHANGED
@@ -58,13 +58,13 @@ Our approach ensures that the model retains its original strengths while acquiri
|
|
58 |
- **License:** APACHE 2.0
|
59 |
- **Contact:** [Website](https://vago-solutions.de/#Kontakt) [David Golchinfar](mailto:[email protected])
|
60 |
|
61 |
-
###Training Dataset
|
62 |
|
63 |
SauerkrautLM-7b-HerO was trained with mix of German data augmentation and translated data.
|
64 |
We found, that only a simple translation of training data can lead to unnatural German phrasings.
|
65 |
Data augmentation techniques were used to grant grammatical, syntactical correctness and a more natural German wording in our training data.
|
66 |
|
67 |
-
###Merge Procedure
|
68 |
|
69 |
SauerkrautLM-7b-HerO was merged on 1 A100 with [mergekit](https://github.com/cg123/mergekit).
|
70 |
The merged model contains [OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) and [Open-Orca/Mistral-7B-OpenOrca](https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca).
|
@@ -72,7 +72,7 @@ We applied the gradient SLURP method.
|
|
72 |
|
73 |
|
74 |
|
75 |
-
###Prompt Template
|
76 |
```
|
77 |
<|im_start|>system
|
78 |
Du bist Sauerkraut-HerO, ein großes Sprachmodell, das höflich und kompetent antwortet. Schreibe deine Gedanken Schritt für Schritt auf, um Probleme sinnvoll zu lösen.
|
@@ -86,7 +86,7 @@ Bitte erkläre mir, wie die Zusammenführung von Modellen durch bestehende Spitz
|
|
86 |
<|im_start|>assistant
|
87 |
```
|
88 |
## Evaluation
|
89 |
-
###MT-Bench (German)
|
90 |
```
|
91 |
########## First turn ##########
|
92 |
score
|
@@ -145,7 +145,7 @@ SauerkrautLM-3b-v1 2.581250
|
|
145 |
open_llama_3b_v2 1.456250
|
146 |
Llama-2-7b 1.181250
|
147 |
```
|
148 |
-
###MT-Bench (English)
|
149 |
```
|
150 |
########## First turn ##########
|
151 |
score
|
@@ -173,20 +173,20 @@ neural-chat-7b-v3-1 6.812500
|
|
173 |
```
|
174 |
|
175 |
|
176 |
-
###Language Model evaluation Harness
|
177 |
Compared to Aleph Alpha Luminous Models:
|
178 |

|
179 |
|
180 |
*performed with newest Language Model Evaluation Harness
|
181 |
-
###BBH
|
182 |

|
183 |
*performed with newest Language Model Evaluation Harness
|
184 |
-
###GPT4ALL
|
185 |
Compared to Aleph Alpha Luminous Models, LeoLM and EM_German:
|
186 |

|
187 |
|
188 |

|
189 |
-
###Additional German Benchmark results
|
190 |

|
191 |
*performed with newest Language Model Evaluation Harness
|
192 |
## Disclaimer
|
|
|
58 |
- **License:** APACHE 2.0
|
59 |
- **Contact:** [Website](https://vago-solutions.de/#Kontakt) [David Golchinfar](mailto:[email protected])
|
60 |
|
61 |
+
### Training Dataset:
|
62 |
|
63 |
SauerkrautLM-7b-HerO was trained with mix of German data augmentation and translated data.
|
64 |
We found, that only a simple translation of training data can lead to unnatural German phrasings.
|
65 |
Data augmentation techniques were used to grant grammatical, syntactical correctness and a more natural German wording in our training data.
|
66 |
|
67 |
+
### Merge Procedure:
|
68 |
|
69 |
SauerkrautLM-7b-HerO was merged on 1 A100 with [mergekit](https://github.com/cg123/mergekit).
|
70 |
The merged model contains [OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) and [Open-Orca/Mistral-7B-OpenOrca](https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca).
|
|
|
72 |
|
73 |
|
74 |
|
75 |
+
### Prompt Template:
|
76 |
```
|
77 |
<|im_start|>system
|
78 |
Du bist Sauerkraut-HerO, ein großes Sprachmodell, das höflich und kompetent antwortet. Schreibe deine Gedanken Schritt für Schritt auf, um Probleme sinnvoll zu lösen.
|
|
|
86 |
<|im_start|>assistant
|
87 |
```
|
88 |
## Evaluation
|
89 |
+
### MT-Bench (German):
|
90 |
```
|
91 |
########## First turn ##########
|
92 |
score
|
|
|
145 |
open_llama_3b_v2 1.456250
|
146 |
Llama-2-7b 1.181250
|
147 |
```
|
148 |
+
### MT-Bench (English):
|
149 |
```
|
150 |
########## First turn ##########
|
151 |
score
|
|
|
173 |
```
|
174 |
|
175 |
|
176 |
+
### Language Model evaluation Harness:
|
177 |
Compared to Aleph Alpha Luminous Models:
|
178 |

|
179 |
|
180 |
*performed with newest Language Model Evaluation Harness
|
181 |
+
### BBH:
|
182 |

|
183 |
*performed with newest Language Model Evaluation Harness
|
184 |
+
### GPT4ALL:
|
185 |
Compared to Aleph Alpha Luminous Models, LeoLM and EM_German:
|
186 |

|
187 |
|
188 |

|
189 |
+
### Additional German Benchmark results:
|
190 |

|
191 |
*performed with newest Language Model Evaluation Harness
|
192 |
## Disclaimer
|