sophosympatheia
commited on
Commit
•
9804d59
1
Parent(s):
6179695
Update README.md
Browse filesUpdates based on more testing at 16K context
README.md
CHANGED
@@ -22,9 +22,9 @@ This model was designed for roleplaying and storytelling and I think it does wel
|
|
22 |
|
23 |
### Long Context Tips
|
24 |
|
25 |
-
You can run this model past 4096 context with alpha_rope set to 1
|
26 |
-
|
27 |
-
|
28 |
|
29 |
### Sampler Tips
|
30 |
|
@@ -47,7 +47,7 @@ If you save the below settings as a .json file, you can import them directly int
|
|
47 |
"epsilon_cutoff": 0,
|
48 |
"eta_cutoff": 0,
|
49 |
"typical_p": 1,
|
50 |
-
"min_p": 0.
|
51 |
"rep_pen": 1.05,
|
52 |
"rep_pen_range": 2800,
|
53 |
"no_repeat_ngram_size": 0,
|
@@ -64,7 +64,7 @@ If you save the below settings as a .json file, you can import them directly int
|
|
64 |
"min_temp": 0.8,
|
65 |
"max_temp": 1.35,
|
66 |
"dynatemp_exponent": 1,
|
67 |
-
"smoothing_factor": 0.
|
68 |
"add_bos_token": true,
|
69 |
"truncation_length": 2048,
|
70 |
"ban_eos_token": false,
|
@@ -92,7 +92,7 @@ If you save the below settings as a .json file, you can import them directly int
|
|
92 |
"n": 1,
|
93 |
"rep_pen_size": 0,
|
94 |
"genamt": 500,
|
95 |
-
"max_length":
|
96 |
}
|
97 |
```
|
98 |
|
|
|
22 |
|
23 |
### Long Context Tips
|
24 |
|
25 |
+
You can run this model past 4096 context with alpha_rope set to 1.
|
26 |
+
I have tested my 5.0bpw exl2 quant of this model out to 16K context using 8-bit cache with alpha_rope 1 and it performs great without any noticable drop in quality as the context size filled from < 4K to the full 16K context.
|
27 |
+
Miqu can go up to 32K context, so in theory this merge can too. I will test that theory soon.
|
28 |
|
29 |
### Sampler Tips
|
30 |
|
|
|
47 |
"epsilon_cutoff": 0,
|
48 |
"eta_cutoff": 0,
|
49 |
"typical_p": 1,
|
50 |
+
"min_p": 0.2,
|
51 |
"rep_pen": 1.05,
|
52 |
"rep_pen_range": 2800,
|
53 |
"no_repeat_ngram_size": 0,
|
|
|
64 |
"min_temp": 0.8,
|
65 |
"max_temp": 1.35,
|
66 |
"dynatemp_exponent": 1,
|
67 |
+
"smoothing_factor": 0.35,
|
68 |
"add_bos_token": true,
|
69 |
"truncation_length": 2048,
|
70 |
"ban_eos_token": false,
|
|
|
92 |
"n": 1,
|
93 |
"rep_pen_size": 0,
|
94 |
"genamt": 500,
|
95 |
+
"max_length": 16128
|
96 |
}
|
97 |
```
|
98 |
|