UD-IQ1_M models for distilled R1 versions?
#6
by
SamPurkis
- opened
Is it possible to create UD-IQ1_M models for the distilled versions of R1 as you have for R1?
Might be possible, if the tell us how they did with this one and then we can replicate the process on the distilled models 🤔
Is it possible to create UD-IQ1_M models for the distilled versions of R1 as you have for R1?
Might be possible, if the tell us how they did with this one and then we can replicate the process on the distilled models 🤔
It's possible but to be honest it will not be as efficient as R1 due to their different architectures
You can use the BnB dynamic versions here though but keep in mind this is not a GGUF: https://huggingface.co/collections/unsloth/deepseek-r1-all-versions-678e1c48f5d2fce87892ace5