llama.cpp quants
#4
by
engrtipusultan
- opened
Can we have gguf? I believe it is supported by llama.cpp
Not sure what the problem is, mradermacher released ggufs but imatrix failed, and even bartowski has problems with quants: https://old.reddit.com/r/LocalLLaMA/comments/1nqe2wq/support_for_grovemoe_has_been_merged_into_llamacpp/nipo95l/