Hugging Face's logo

ExceedZhang
/
DeepSeek-R1-Distill-Qwen-32B-W4A16-G128

Safetensors
English
Chinese
qwen2
compressed-tensors
Model card Files Community
1
ExceedZhang commited on
Commit
2f9d725
·
verified ·
1 Parent(s): 4eb1dde

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -8,4 +8,5 @@ base_model:
8
  - deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
9
  ---
10
 
11
- DeepSeekR1蒸馏Qwen2.5 32B版本经过Int4 GPTQ Marlin算法量化的版本，推荐RTX4090 24GB 2块GPU推理，性能达到1700tokens/秒，最优并发128同时使用。比PF16版本性能相当，ceval评测82.3
 
 
8
  - deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
9
  ---
10
 
11
+ DeepSeekR1蒸馏Qwen2.5 32B版本经过Int4 GPTQ Marlin算法量化的版本，推荐RTX4090 24GB 2块GPU推理，性能达到1700tokens/秒，最优并发128同时使用。
12
+ 比PF16版本性能相当，ceval评测82.3，显存降低50%