AlekseiPravdin commited on
Commit
8a49fc9
1 Parent(s): 6701cf9

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -13
README.md CHANGED
@@ -35,23 +35,19 @@ dtype: float16
35
 
36
  ## Model Features
37
 
38
- This merged model combines the advanced capabilities of the HQQ quantized version of Llama-3.1-8B-Instruct with the fine-tuned prowess of the SauerkrautLM variant. The result is a versatile model that excels in both generative tasks and nuanced understanding of multilingual contexts, particularly in German and English. The integration of Spectrum Fine-Tuning from the SauerkrautLM model enhances the efficiency of the model while preserving its extensive knowledge base.
39
 
40
  ## Evaluation Results
41
 
42
- The performance of the parent models provides a solid foundation for the merged model. Here are some evaluation results from the original models:
43
 
44
- ### mobiuslabsgmbh/Llama-3.1-8b-instruct_4bitgs64_hqq_calib
45
- - **ARC (25-shot)**: 60.49
46
- - **HellaSwag (10-shot)**: 80.16
47
- - **MMLU (5-shot)**: 68.98
48
- - **Average Performance**: 69.51
49
-
50
- ### VAGOsolutions/Llama-3.1-SauerkrautLM-8b-Instruct
51
- - Fine-tuned on German-English data, showcasing significant improvements in multilingual capabilities.
52
-
53
- These results indicate that the merged model is likely to inherit and enhance the strengths of both parent models, particularly in text generation and comprehension tasks.
54
 
55
  ## Limitations
56
 
57
- While the merged model benefits from the strengths of both parent models, it may also carry over some limitations. For instance, the potential for uncensored content remains a concern, as noted in the SauerkrautLM documentation. Additionally, the model's performance may vary depending on the specific task and the languages involved, particularly in less common languages or dialects. Users should be aware of these factors when deploying the model in real-world applications.
 
35
 
36
  ## Model Features
37
 
38
+ This merged model combines the advanced capabilities of the HQQ quantized version of Llama-3.1-8B-Instruct with the fine-tuned prowess of Llama-3.1-SauerkrautLM-8b-Instruct. The result is a versatile model that excels in both generative tasks and nuanced understanding of multilingual contexts, particularly in German and English. The integration of Spectrum Fine-Tuning from the Sauerkraut model enhances the model's efficiency and performance, making it suitable for a wide range of applications.
39
 
40
  ## Evaluation Results
41
 
42
+ The evaluation results from the parent models indicate strong performance across various benchmarks. For instance, the HQQ 4-bit version of Llama-3.1-8B-Instruct achieved notable scores on tasks like ARC, HellaSwag, and MMLU, while the SauerkrautLM model demonstrated significant improvements in multilingual capabilities. The combined strengths of these models in the merged version are expected to yield even better results in practical applications.
43
 
44
+ | Benchmark | Llama-3.1-8B-Instruct (HQQ 4-bit) | Llama-3.1-SauerkrautLM |
45
+ |-------------------|-----------------------------------|-------------------------|
46
+ | ARC (25-shot) | 60.32 | - |
47
+ | HellaSwag (10-shot)| 79.21 | - |
48
+ | MMLU (5-shot) | 67.07 | - |
49
+ | Multilingual Tasks | - | Improved |
 
 
 
 
50
 
51
  ## Limitations
52
 
53
+ While the merged model benefits from the strengths of both parent models, it may also inherit some limitations. For instance, the potential for uncensored content remains a concern, as noted in the SauerkrautLM documentation. Additionally, the model's performance may vary depending on the specific task and the languages involved, particularly in less represented languages or dialects. Users should be aware of these factors when deploying the model in real-world applications.