shimmyshimmer commited on
Commit
67c4c11
·
verified ·
1 Parent(s): 28f1191

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -33,13 +33,14 @@ tags:
33
 
34
  Our DeepSeek-V3-0324 GGUFs allow you to run the model in llama.cpp, LMStudio, Open WebUI and other inference frameworks.
35
 
36
- Includes 1.58-bit, 2-bit, 3-bit and 4-bit Dynamic versions, which yields better accuracy and results than standard quantization.
37
 
38
  | MoE Bits | Type | Disk Size | Accuracy | Link | Details |
39
  |----------|----------|-------------|----------|------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
40
- | 1.58bit (still uploading) | IQ1_S | **131GB** | Ok | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ1_S) | `down_proj` in MoE mixture of 2.06/1.56bit |
41
- | 2.22bit (still uploading) | IQ2_XXS | **183GB** | Fair | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ2_XXS) | `down_proj` in MoE all 2.06bit |
42
- | 2.51bit | Q2_K_XL | **212GB** | Good | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q2_K_XL) | `down_proj` in MoE mixture of 3.5/2.5bit |
 
43
  | 3.5bit | Q3_K_XL | **320GB** | Great | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q3_K_XL) | `down_proj` in MoE mixture of 4.5/3.5bit |
44
  | 4.5bit | Q4_K_XL | **406GB** | Best | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q4_K_XL) | `down_proj` in MoE mixture of 5.5/4.5bit |
45
 
 
33
 
34
  Our DeepSeek-V3-0324 GGUFs allow you to run the model in llama.cpp, LMStudio, Open WebUI and other inference frameworks.
35
 
36
+ Includes 1.78-bit, 2-bit, 3-bit and 4-bit Dynamic versions, which yields better accuracy and results than standard quantization.
37
 
38
  | MoE Bits | Type | Disk Size | Accuracy | Link | Details |
39
  |----------|----------|-------------|----------|------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
40
+ | 1.78bit (prelim) | IQ1_S | **151GB** | Ok | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ1_S) | `down_proj` in MoE mixture of 2.06/1.78bit |
41
+ | 1.93bit (prelim) | IQ1_M | **178GB** | Fair | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ1_M) | `down_proj` in MoE mixture of 2.06/1.93bit |
42
+ | 2.42bit | IQ2_XXS | **203GB** | Better | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-IQ2_XXS) | `down_proj` in MoE all 2.42bit |
43
+ | 2.71bit | Q2_K_XL | **232GB** | Good | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q2_K_XL) | `down_proj` in MoE mixture of 3.5/2.71bit |
44
  | 3.5bit | Q3_K_XL | **320GB** | Great | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q3_K_XL) | `down_proj` in MoE mixture of 4.5/3.5bit |
45
  | 4.5bit | Q4_K_XL | **406GB** | Best | [Link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF/tree/main/UD-Q4_K_XL) | `down_proj` in MoE mixture of 5.5/4.5bit |
46