Request to add more GGUF quantization size options
#1
by
makisekurisu-jp
- opened
Iβd like to request the addition of more GGUF quantization size options to better support different hardware setups. Thank you!
sorry for taking so long but uploaded now
ND911
changed discussion status to
closed
Can you PLEASE add smaller Q2 version ?!?!?
At least Q2 versions are @6GB which is similar to SDXL models and can be run on most potato laptops!!!
Also can you make imatrix version for even smaller size 2-4GB ???
PS Q4 version @ 11GB is still fat pig of memory :(
NEED smaller models!
Q2 up and afaik imatrix doesn't work in comfyui
That was fast, looking forward to testing it out, Thanks!