Any chance for more GLM quants?
#4
by
koute
- opened
I'm really enjoying your quant, thanks! So I have two questions:
- Any chance for a GLM-4.5V quant? (https://huggingface.co/zai-org/GLM-4.5V)
- Any chance for an nvfp4 quant? (Assuming llm-compressor supports this already; not sure if it does.)
Thank you! GLM-4.5V quant is in the making, hopefully it will be done today, finger crossed. I am also considering making other quants besides INT4 and INT8, which includes FP4 and INT2 INT3! But I'm not sure which one I should follow.
And yes, llm-compressor does support FP4, but not INT2 and INT3.