README.md

metadata

license: apache-2.0
base_model:
  - Qwen/QwQ-32B
base_model_relation: quantized
pipeline_tag: text-generation

Disclaimer: I don't know what I'm doing.

Original Model: https://huggingface.co/Qwen/QwQ-32B

QwQ 32B EXL2	Size
8.0bpw	33.5 GB
7.0bpw	29.6 GB
6.5bpw	27.5 GB
6.0bpw	25.6 GB
5.5bpw	23.6 GB
5.0bpw	21.7 GB
4.5bpw	19.7 GB
4.0bpw	17.8 GB
3.75bpw	16.8 GB
3.5bpw	WIP