Questions about GGUF version
#5
by
demo001s
- opened
Which version of Q5 is used in this picture, and what is the merge ratio
The gguf samples are from q4_0, q5_0 and q8_0. Made those first and updated the samples before I had made the new K variants.
The ratio is not constant, tried different ratios for each block while experimenting and found it worked well.
Anibaaal
changed discussion status to
closed