MarinaraSpaghetti commited on
Commit
9ee5e21
·
verified ·
1 Parent(s): 350f719

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +96 -55
README.md CHANGED
@@ -1,55 +1,96 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Nemomix-v0.4-12B
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the della_linear merge method using F:\mergekit\mistralaiMistral-Nemo-Base-2407 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * F:\mergekit\intervitens_mini-magnum-12b-v1.1
22
- * F:\mergekit\mistralaiMistral-Nemo-Instruct-2407
23
- * F:\mergekit\invisietch_Atlantis-v0.1-12B
24
- * F:\mergekit\NeverSleepHistorical_lumi-nemo-e2.0
25
-
26
- ### Configuration
27
-
28
- The following YAML configuration was used to produce this model:
29
-
30
- ```yaml
31
- models:
32
- - model: F:\mergekit\invisietch_Atlantis-v0.1-12B
33
- parameters:
34
- weight: 0.16
35
- density: 0.4
36
- - model: F:\mergekit\mistralaiMistral-Nemo-Instruct-2407
37
- parameters:
38
- weight: 0.23
39
- density: 0.5
40
- - model: F:\mergekit\NeverSleepHistorical_lumi-nemo-e2.0
41
- parameters:
42
- weight: 0.27
43
- density: 0.6
44
- - model: F:\mergekit\intervitens_mini-magnum-12b-v1.1
45
- parameters:
46
- weight: 0.34
47
- density: 0.8
48
- merge_method: della_linear
49
- base_model: F:\mergekit\mistralaiMistral-Nemo-Base-2407
50
- parameters:
51
- epsilon: 0.05
52
- lambda: 1
53
- int8_mask: true
54
- dtype: bfloat16
55
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+ ---
8
+
9
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6550b16f7490049d6237f200/Hj686vH4WgD7ILybOQObi.jpeg)
10
+
11
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6550b16f7490049d6237f200/qC-j_gzwjykZGbkLUq2PH.png)
12
+
13
+ # The best one so far out of all the Nemomixes. Use this one.
14
+
15
+ ## Information
16
+ ### Description
17
+
18
+ My main goal is to merge the smartness of the base Instruct Nemo with the better prose from the different roleplaying fine-tunes. This one seems to be the best out of all, so far. All credits and thanks go to Intervitens, Mistralai, Invisietch, and NeverSleep for providing amazing models used in the merge.
19
+
20
+ ### Instruct
21
+
22
+ Mistral Instruct.
23
+
24
+ ```
25
+ <s>[INST] {system} [/INST]{assistant}</s>[INST] {user} [/INST]
26
+ ```
27
+
28
+ ### Settings
29
+
30
+ Lower Temperature of 0.35 recommended, although I had luck with Temperatures above one (1.0-1.2) if you crank up the Min P (0.01-0.1). Run with base DRY of 0.8/1.75/2/0 and you're good to go.
31
+
32
+ ### GGUF
33
+
34
+ https://huggingface.co/MarinaraSpaghetti/Nemomix-v4.0-12B-GGUF
35
+
36
+ ### Other Versions
37
+
38
+ V1: https://huggingface.co/MarinaraSpaghetti/Nemomix-v1.0-12B
39
+
40
+ V2: https://huggingface.co/MarinaraSpaghetti/Nemomix-v2.0-12B
41
+
42
+ V3: https://huggingface.co/MarinaraSpaghetti/Nemomix-v3.0-12B
43
+
44
+ V4: https://huggingface.co/MarinaraSpaghetti/Nemomix-v4.0-12B
45
+
46
+ # Nemomix-v0.4-12B
47
+
48
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
49
+
50
+ ## Merge Details
51
+ ### Merge Method
52
+
53
+ This model was merged using the della_linear merge method using F:\mergekit\mistralaiMistral-Nemo-Base-2407 as a base.
54
+
55
+ ### Models Merged
56
+
57
+ The following models were included in the merge:
58
+ * F:\mergekit\intervitens_mini-magnum-12b-v1.1
59
+ * F:\mergekit\mistralaiMistral-Nemo-Instruct-2407
60
+ * F:\mergekit\invisietch_Atlantis-v0.1-12B
61
+ * F:\mergekit\NeverSleepHistorical_lumi-nemo-e2.0
62
+
63
+ ### Configuration
64
+
65
+ The following YAML configuration was used to produce this model:
66
+
67
+ ```yaml
68
+ models:
69
+ - model: F:\mergekit\invisietch_Atlantis-v0.1-12B
70
+ parameters:
71
+ weight: 0.16
72
+ density: 0.4
73
+ - model: F:\mergekit\mistralaiMistral-Nemo-Instruct-2407
74
+ parameters:
75
+ weight: 0.23
76
+ density: 0.5
77
+ - model: F:\mergekit\NeverSleepHistorical_lumi-nemo-e2.0
78
+ parameters:
79
+ weight: 0.27
80
+ density: 0.6
81
+ - model: F:\mergekit\intervitens_mini-magnum-12b-v1.1
82
+ parameters:
83
+ weight: 0.34
84
+ density: 0.8
85
+ merge_method: della_linear
86
+ base_model: F:\mergekit\mistralaiMistral-Nemo-Base-2407
87
+ parameters:
88
+ epsilon: 0.05
89
+ lambda: 1
90
+ int8_mask: true
91
+ dtype: bfloat16
92
+ ```
93
+
94
+ ## Ko-fi
95
+ ### Enjoying what I do? Consider donating here, thank you!
96
+ https://ko-fi.com/spicy_marinara