thank you!
#1
by
jacek2024
- opened
could you try also 47B? :)
Started :)
Are the comments about the different quantized files after running benchmarks against each one or they are just generic comments on different quantization levels?
BTW thanks for providing the downloads so fast!
Just generic comments regarding the quantization level
They're especially awkward here because the strange shape of the model resulted in a lot of fallback tensors being used (when quantizing you need a specific original shape for some quantization techniques, for example many require the column count to be divisible by 256. If it's not, it has to fallback to a different quantization)