Update README.md
Browse files
README.md
CHANGED
|
@@ -36,6 +36,23 @@ instruction following, output quality and stronger connections to ideas, concept
|
|
| 36 |
|
| 37 |
This combines with "MAXing" the quant to improve preformance.
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
<b>Optional : System Prompt</b>
|
| 40 |
|
| 41 |
This is an optional system prompt you can use to enhance operation.
|
|
|
|
| 36 |
|
| 37 |
This combines with "MAXing" the quant to improve preformance.
|
| 38 |
|
| 39 |
+
This chart shows the order in terms of "BPW" for each quant (mapped below with relative "strength" to one another) with "IQ1_S" with the least, and "Q8_0" (F16 is full precision) with the most:
|
| 40 |
+
|
| 41 |
+
<small>
|
| 42 |
+
<PRE>
|
| 43 |
+
IQ1_S | IQ1_M
|
| 44 |
+
IQ2_XXS | IQ2_XS | Q2_K_S | IQ2_S | Q2_K | IQ2_M
|
| 45 |
+
IQ3_XXS | Q3_K_S | IQ3_XS | IQ3_S | IQ3_M | Q3_K_M | Q3_K_L
|
| 46 |
+
Q4_K_S | IQ4_XS | IQ4_NL | Q4_K_M
|
| 47 |
+
Q5_K_S | Q5_K_M
|
| 48 |
+
Q6_K
|
| 49 |
+
Q8_0
|
| 50 |
+
F16
|
| 51 |
+
</pre>
|
| 52 |
+
</small>
|
| 53 |
+
|
| 54 |
+
More information on quants is in the document below "Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers".
|
| 55 |
+
|
| 56 |
<b>Optional : System Prompt</b>
|
| 57 |
|
| 58 |
This is an optional system prompt you can use to enhance operation.
|