Update README.md
Browse files
README.md
CHANGED
@@ -59,33 +59,19 @@ Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or a
|
|
59 |
| -------- | ---------- | --------- | ----- | ----------- |
|
60 |
| [Mistral-Small-3.1-24B-Instruct-2503-bf16.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-bf16.gguf) | bf16 | 47.15GB | false | Full BF16 weights. |
|
61 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf) | Q8_0 | 25.05GB | false | Extremely high quality, generally unneeded but max available quant. |
|
62 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf) | Q8_0 | 25.05GB | false | Extremely high quality, generally unneeded but max available quant. |
|
63 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf) | Q6_K_L | 19.67GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
|
64 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf) | Q6_K_L | 19.67GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
|
65 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf) | Q6_K | 19.35GB | false | Very high quality, near perfect, *recommended*. |
|
66 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf) | Q6_K | 19.35GB | false | Very high quality, near perfect, *recommended*. |
|
67 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf) | Q5_K_L | 17.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
|
68 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf) | Q5_K_L | 17.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
|
69 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf) | Q5_K_M | 16.76GB | false | High quality, *recommended*. |
|
70 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf) | Q5_K_M | 16.76GB | false | High quality, *recommended*. |
|
71 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf) | Q5_K_S | 16.30GB | false | High quality, *recommended*. |
|
72 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf) | Q5_K_S | 16.30GB | false | High quality, *recommended*. |
|
73 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf) | Q4_1 | 14.87GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
|
74 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf) | Q4_1 | 14.87GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
|
75 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf) | Q4_K_L | 14.83GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
76 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf) | Q4_K_L | 14.83GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
77 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf) | Q4_K_M | 14.33GB | false | Good quality, default size for most use cases, *recommended*. |
|
78 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf) | Q4_K_M | 14.33GB | false | Good quality, default size for most use cases, *recommended*. |
|
79 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf) | Q4_K_S | 13.55GB | false | Slightly lower quality with more space savings, *recommended*. |
|
80 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf) | Q4_K_S | 13.55GB | false | Slightly lower quality with more space savings, *recommended*. |
|
81 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf) | Q4_0 | 13.49GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
|
82 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf) | Q4_0 | 13.49GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
|
83 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf) | IQ4_NL | 13.47GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
|
84 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf) | IQ4_NL | 13.47GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
|
85 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf) | Q3_K_XL | 12.99GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
86 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf) | Q3_K_XL | 12.99GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
87 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf) | IQ4_XS | 12.76GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
|
88 |
-
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf) | IQ4_XS | 12.76GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
|
89 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_L.gguf) | Q3_K_L | 12.40GB | false | Lower quality but usable, good for low RAM availability. |
|
90 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_M.gguf) | Q3_K_M | 11.47GB | false | Low quality. |
|
91 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ3_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ3_M.gguf) | IQ3_M | 10.65GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
|
|
|
59 |
| -------- | ---------- | --------- | ----- | ----------- |
|
60 |
| [Mistral-Small-3.1-24B-Instruct-2503-bf16.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-bf16.gguf) | bf16 | 47.15GB | false | Full BF16 weights. |
|
61 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q8_0.gguf) | Q8_0 | 25.05GB | false | Extremely high quality, generally unneeded but max available quant. |
|
|
|
|
|
62 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K_L.gguf) | Q6_K_L | 19.67GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
|
63 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q6_K.gguf) | Q6_K | 19.35GB | false | Very high quality, near perfect, *recommended*. |
|
|
|
|
|
64 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_L.gguf) | Q5_K_L | 17.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
|
65 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_M.gguf) | Q5_K_M | 16.76GB | false | High quality, *recommended*. |
|
|
|
|
|
66 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q5_K_S.gguf) | Q5_K_S | 16.30GB | false | High quality, *recommended*. |
|
67 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_1.gguf) | Q4_1 | 14.87GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
|
|
|
68 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_L.gguf) | Q4_K_L | 14.83GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
|
|
|
|
69 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf) | Q4_K_M | 14.33GB | false | Good quality, default size for most use cases, *recommended*. |
|
70 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf) | Q4_K_S | 13.55GB | false | Slightly lower quality with more space savings, *recommended*. |
|
|
|
|
|
71 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q4_0.gguf) | Q4_0 | 13.49GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
|
72 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_NL.gguf) | IQ4_NL | 13.47GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
|
|
|
|
|
73 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_XL.gguf) | Q3_K_XL | 12.99GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
74 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ4_XS.gguf) | IQ4_XS | 12.76GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
|
|
|
75 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_L.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_L.gguf) | Q3_K_L | 12.40GB | false | Lower quality but usable, good for low RAM availability. |
|
76 |
| [Mistral-Small-3.1-24B-Instruct-2503-Q3_K_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-Q3_K_M.gguf) | Q3_K_M | 11.47GB | false | Low quality. |
|
77 |
| [Mistral-Small-3.1-24B-Instruct-2503-IQ3_M.gguf](https://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/main/mistralai_Mistral-Small-3.1-24B-Instruct-2503-IQ3_M.gguf) | IQ3_M | 10.65GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
|