Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,17 +1,6 @@
|
|
| 1 |
---
|
| 2 |
quantized_by: bartowski
|
| 3 |
pipeline_tag: text-generation
|
| 4 |
-
language:
|
| 5 |
-
- en
|
| 6 |
-
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/blob/main/LICENSE
|
| 7 |
-
tags:
|
| 8 |
-
- code
|
| 9 |
-
- codeqwen
|
| 10 |
-
- chat
|
| 11 |
-
- qwen
|
| 12 |
-
- qwen-coder
|
| 13 |
-
base_model: Qwen/Qwen2.5-Coder-32B-Instruct
|
| 14 |
-
license: apache-2.0
|
| 15 |
---
|
| 16 |
|
| 17 |
## Llamacpp imatrix Quantizations of Qwen2.5-Coder-32B-Instruct
|
|
@@ -60,6 +49,7 @@ Run them in [LM Studio](https://lmstudio.ai/)
|
|
| 60 |
| [Qwen2.5-Coder-32B-Instruct-Q3_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q3_K_S.gguf) | Q3_K_S | 14.39GB | false | Low quality, not recommended. |
|
| 61 |
| [Qwen2.5-Coder-32B-Instruct-IQ3_XS.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ3_XS.gguf) | IQ3_XS | 13.71GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
|
| 62 |
| [Qwen2.5-Coder-32B-Instruct-Q2_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q2_K_L.gguf) | Q2_K_L | 13.07GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
|
|
|
|
| 63 |
| [Qwen2.5-Coder-32B-Instruct-Q2_K.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q2_K.gguf) | Q2_K | 12.31GB | false | Very low quality but surprisingly usable. |
|
| 64 |
| [Qwen2.5-Coder-32B-Instruct-IQ2_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ2_M.gguf) | IQ2_M | 11.26GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
|
| 65 |
| [Qwen2.5-Coder-32B-Instruct-IQ2_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ2_S.gguf) | IQ2_S | 10.39GB | false | Low quality, uses SOTA techniques to be usable. |
|
|
|
|
| 1 |
---
|
| 2 |
quantized_by: bartowski
|
| 3 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
## Llamacpp imatrix Quantizations of Qwen2.5-Coder-32B-Instruct
|
|
|
|
| 49 |
| [Qwen2.5-Coder-32B-Instruct-Q3_K_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q3_K_S.gguf) | Q3_K_S | 14.39GB | false | Low quality, not recommended. |
|
| 50 |
| [Qwen2.5-Coder-32B-Instruct-IQ3_XS.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ3_XS.gguf) | IQ3_XS | 13.71GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
|
| 51 |
| [Qwen2.5-Coder-32B-Instruct-Q2_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q2_K_L.gguf) | Q2_K_L | 13.07GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
|
| 52 |
+
| [Qwen2.5-Coder-32B-Instruct-IQ3_XXS.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ3_XXS.gguf) | IQ3_XXS | 12.84GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
|
| 53 |
| [Qwen2.5-Coder-32B-Instruct-Q2_K.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q2_K.gguf) | Q2_K | 12.31GB | false | Very low quality but surprisingly usable. |
|
| 54 |
| [Qwen2.5-Coder-32B-Instruct-IQ2_M.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ2_M.gguf) | IQ2_M | 11.26GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
|
| 55 |
| [Qwen2.5-Coder-32B-Instruct-IQ2_S.gguf](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-IQ2_S.gguf) | IQ2_S | 10.39GB | false | Low quality, uses SOTA techniques to be usable. |
|