What's the difference between a model with a name 'unsloth' and a model without it?
#3
by
1sn0treal
- opened
| Unsloth's Dynamic 4-bit Quants selectively avoids quantizing certain parameters, greatly improving accuracy while keeping VRAM usage similar to BnB 4-bit
I watched unsloth for the first time today.
Can I interpret it as meaning that this model used a different quantization method?
| Unsloth's Dynamic 4-bit Quants selectively avoids quantizing certain parameters, greatly improving accuracy while keeping VRAM usage similar to BnB 4-bit
I watched unsloth for the first time today.
Can I interpret it as meaning that this model used a different quantization method?
Yes correct. It uses BnB 4bit but it is much more accurate than standard 4-bit. Read our blog: https://unsloth.ai/blog/dynamic-4bit
Thank you
1sn0treal
changed discussion status to
closed
Thank you