Please release Q5_K_M and q4_k_m too
Please share other versions like Q5_K_M and q4_k_m too.
Sure thing, I will have those out today.
@Hoioi The Q5_K_M and Q4_K_M quants are available now. I havent tested them yet but im working on that now
Thank you so much. Please verify that they are working fine, that other members and I could download it.
You can also replace the GGUF file in the chat area of this model.
@Hoioi they are tested and the README has been updated.
I am not sure what you mean by
You can also replace the GGUF file in the chat area of this model.
As I understand it, there should be 3 different quantization files that are serving three different needs. Is there a mistake that I have overlooked?
I meant you can use q4_k_m or q5_K_M in this link too if you'd like to :
https://huggingface.co/spaces/macadeliccc/laser-dolphin-mixtral-chat
My bad, I can add a GGUF chat separate from that. Its certainly on my list, but this chat template error is causing an issue for both of the laser-dolphin-mixtral models im hoping to fix that first and then will work on GGUF chat
Thanks a lot for your great works.