Azazelle commited on
Commit
e723811
1 Parent(s): a6d61c2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -1,3 +1,30 @@
1
  ---
 
 
 
 
2
  license: cc-by-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ pipeline_tag: text-generation
3
+ tags:
4
+ - mistral
5
+ - merge
6
  license: cc-by-4.0
7
  ---
8
+ # Model Card for Argetsu
9
+ <!-- Provide a quick summary of what the model is/does. -->
10
+ Slerp merge of lots of models.
11
+
12
+ .yaml file for mergekit
13
+ ```.yaml:
14
+ slices:
15
+ - sources:
16
+ - model: Azazelle/SlimMelodicMaid
17
+ layer_range: [0, 32]
18
+ - model: Azazelle/Dumb-Maidlet
19
+ layer_range: [0, 32]
20
+ merge_method: slerp
21
+ base_model: mistralai/Mistral-7B-v0.1
22
+ parameters:
23
+ t:
24
+ - filter: self_attn
25
+ value: [0.2, 0.6, 0.44, 0.76, 1]
26
+ - filter: mlp
27
+ value: [0.8, 0.4, 0.56, 0.24, 0]
28
+ - value: 0.5 # fallback for rest of tensors
29
+ dtype: float16
30
+ ```