Would There be Dynamic Qunatized Versions like 2.51bit
The 2.51 bit for R1 was real great and I wish there could be one for V3 0324 as well. Much appreciated for your works!
Yes there will be!
Hopefully 2.2Bit as well with imatrix, just like R1.
The 2.51 bit for R1 was real great and I wish there could be one for V3 0324 as well. Much appreciated for your works!
By the way, Unsloth's 2.2Bit is just as good as their 2.51 Bit because 2.2 Bit uses imatrix while 2.51 bit doesn't for some reason.
The 2.51 bit for R1 was real great and I wish there could be one for V3 0324 as well. Much appreciated for your works!
By the way, Unsloth's 2.2Bit is just as good as their 2.51 Bit because 2.2 Bit uses imatrix while 2.51 bit doesn't for some reason.
This is the one to go for I think.
EDIT seems like the new UD-Q2_K_XL is better now if you can run it.
We have just uploaded the 2.51 bit btw. Uploading the rest...there will be 1.58bit dynamic too
We have just uploaded the 2.51 bit btw. Uploading the rest...there will be 1.58bit dynamic too
Thanks for the quick action! I noticed there's an update to the UD_Q2_K_XL model. I wonder if there's a difference between the new one and the prev one? I need to decide if I should download the newly uploaded version or keep using the initial version.
Please use the new one! I made it increase in quite a bit of accuracy albeit it's 20GB to 30GB larger
Please use the new one! I made it increase in quite a bit of accuracy albeit it's 20GB to 30GB larger
That made it impossible to run on my system now (72GB VRAM+128GB RAM, unless there's some config that I might not be aware of). Could you please restore the old 1.58-bit IQ1_S files?