ryzen88 commited on
Commit
a015a60
·
verified ·
1 Parent(s): 8fa334b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -8
README.md CHANGED
@@ -7,6 +7,13 @@ tags:
7
 
8
  ---
9
  # model
 
 
 
 
 
 
 
10
 
11
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
 
@@ -18,9 +25,9 @@ This model was merged using the breadcrumbs_ties merge method using Z:\Llama-3-G
18
  ### Models Merged
19
 
20
  The following models were included in the merge:
21
- * Z:\Smaug-Llama-3-70B-Instruct
22
- * I:\Llama-3-Lumimaid-70B-v0.1-alt
23
- * I:\Tess-2.0-Llama-3-70B-v0.2
24
 
25
  ### Configuration
26
 
@@ -28,27 +35,27 @@ The following YAML configuration was used to produce this model:
28
 
29
  ```yaml
30
  models:
31
- - model: Z:\LLM's\Llama-3-Giraffe-70B-Instruct
32
  parameters:
33
  weight: 0.25
34
  density: 0.90
35
  gamma: 0.01
36
- - model: Z:\LLM's\Smaug-Llama-3-70B-Instruct
37
  parameters:
38
  weight: 0.30
39
  density: 0.90
40
  gamma: 0.01
41
- - model: I:\Tess-2.0-Llama-3-70B-v0.2
42
  parameters:
43
  weight: 0.15
44
  density: 0.90
45
  gamma: 0.01
46
- - model: I:\Llama-3-Lumimaid-70B-v0.1-alt
47
  parameters:
48
  weight: 0.30
49
  density: 0.90
50
  gamma: 0.01
51
  merge_method: breadcrumbs_ties
52
- base_model: Z:\LLM's\Llama-3-Giraffe-70B-Instruct
53
  dtype: bfloat16
54
  ```
 
7
 
8
  ---
9
  # model
10
+ A large context uncencored Llama 3 model focussed on story & RP
11
+ I found the Smaug version of lama very impressive, exept for a couple of quirks and the default context window.
12
+ This merge is with the Giraffe instruct for long context window, and basically a smaug - lumi tess merger.
13
+ I am planning to do the same with a gradient model and compaire it to this giraffe version.
14
+
15
+ Breadcrumbs_ties really is awesome.
16
+
17
 
18
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
19
 
 
25
  ### Models Merged
26
 
27
  The following models were included in the merge:
28
+ * \Smaug-Llama-3-70B-Instruct
29
+ * \Llama-3-Lumimaid-70B-v0.1-alt
30
+ * \Tess-2.0-Llama-3-70B-v0.2
31
 
32
  ### Configuration
33
 
 
35
 
36
  ```yaml
37
  models:
38
+ - model: \Llama-3-Giraffe-70B-Instruct
39
  parameters:
40
  weight: 0.25
41
  density: 0.90
42
  gamma: 0.01
43
+ - model: \Smaug-Llama-3-70B-Instruct
44
  parameters:
45
  weight: 0.30
46
  density: 0.90
47
  gamma: 0.01
48
+ - model: \Tess-2.0-Llama-3-70B-v0.2
49
  parameters:
50
  weight: 0.15
51
  density: 0.90
52
  gamma: 0.01
53
+ - model: \Llama-3-Lumimaid-70B-v0.1-alt
54
  parameters:
55
  weight: 0.30
56
  density: 0.90
57
  gamma: 0.01
58
  merge_method: breadcrumbs_ties
59
+ base_model: \Llama-3-Giraffe-70B-Instruct
60
  dtype: bfloat16
61
  ```