Update README.md
Browse files
README.md
CHANGED
@@ -33,13 +33,14 @@ tags:
|
|
33 |
|
34 |
Our DeepSeek-V3-0324 GGUFs allow you to run the model in llama.cpp, LMStudio, Open WebUI and other inference frameworks.
|
35 |
|
36 |
-
Includes 1.
|
37 |
|
38 |
| MoE Bits | Type | Disk Size | Accuracy | Link | Details |
|
39 |
|----------|----------|-------------|----------|------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
|
40 |
-
| 1.
|
41 |
-
|
|
42 |
-
| 2.
|
|
|
43 |
| 3.5bit | Q3_K_XL | **320GB** | Great | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q3_K_XL) | `down_proj` in MoE mixture of 4.5/3.5bit |
|
44 |
| 4.5bit | Q4_K_XL | **406GB** | Best | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q4_K_XL) | `down_proj` in MoE mixture of 5.5/4.5bit |
|
45 |
|
|
|
33 |
|
34 |
Our DeepSeek-V3-0324 GGUFs allow you to run the model in llama.cpp, LMStudio, Open WebUI and other inference frameworks.
|
35 |
|
36 |
+
Includes 1.78-bit, 2-bit, 3-bit and 4-bit Dynamic versions, which yields better accuracy and results than standard quantization.
|
37 |
|
38 |
| MoE Bits | Type | Disk Size | Accuracy | Link | Details |
|
39 |
|----------|----------|-------------|----------|------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
|
40 |
+
| 1.78bit (prelim) | IQ1_S | **151GB** | Ok | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ1_S) | `down_proj` in MoE mixture of 2.06/1.78bit |
|
41 |
+
| 1.93bit (prelim) | IQ1_M | **178GB** | Fair | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ1_M) | `down_proj` in MoE mixture of 2.06/1.93bit |
|
42 |
+
| 2.42bit | IQ2_XXS | **203GB** | Better | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ2_XXS) | `down_proj` in MoE all 2.42bit |
|
43 |
+
| 2.71bit | Q2_K_XL | **232GB** | Good | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q2_K_XL) | `down_proj` in MoE mixture of 3.5/2.71bit |
|
44 |
| 3.5bit | Q3_K_XL | **320GB** | Great | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q3_K_XL) | `down_proj` in MoE mixture of 4.5/3.5bit |
|
45 |
| 4.5bit | Q4_K_XL | **406GB** | Best | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q4_K_XL) | `down_proj` in MoE mixture of 5.5/4.5bit |
|
46 |
|