How to create DWQ
#1
by
djosh34
- opened
Awni, idk how you do it. But these DWQs are amazing quants!!
Like how can I make my own dwq quants? Do I need my own datasets?
@awni what is the difference between this one and regular 4bit, you've created?
https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LEARNED_QUANTS.md
DWQ finetunes the parameter quants to perform more like the the non-quantized model