Additional quantizations
#2
by
olegshulyakov
- opened
Can you please add additional quantizations like Q4_K_M/Q5_K_M/Q6_K/Q8_0?
Thanks for your request regarding additional quantizations like Q4_K_M, Q5_K_M, Q6_K, and Q8_0. You can actually achieve these quantizations yourself using the llama-quantize tool.