Other quants?
#2
by
vcringer
- opened
Any chance of providing other quants, such as Q4_K_M for example?
We're planning to release some high-quality quantizations (like GGUFs) soon, since the standard quantizations tend to produce noticeably lower quality outputs than the full precision model or Q8.
In the meantime, you can try this Q4_K_M quant
Thanks
vcringer
changed discussion status to
closed