Request: Int 4 GPTQ Autoround
#1
by
jart25
- opened
Hi, would it be possible to publish the model in this format, please? For example: kaitchup/Qwen3-30B-A3B-autoround-4bit-gptq.
It would be really helpful. Thanks a lot!
@jart25 , if you look at https://huggingface.co/Intel/Qwen3-30B-A3B-Instruct-2507-int4-AutoRound/blob/main/quantization_config.json or https://huggingface.co/Intel/Qwen3-30B-A3B-Instruct-2507-int4-mixed-AutoRound/blob/main/quantization_config.json you can see that they are both auto gptq models.
Doit using runpod
jart25
changed discussion status to
closed
jart25
changed discussion status to
open
I’ve uploaded the with quant GPTQ version here: https://huggingface.co/jart25/Qwen3-30B-A3B-Instruct-2507-Autoround-Int-4bit-gptq
The quant autoround is not supported on rocm on vllm