When GGUF?

#6
by ChuckMcSneed - opened

Please consider helping adding support for your model to llama.cpp

Yes, supporting ollama can help the model gain better reputation

Confucius say: Man who rush tofu, soon taste regret!
β€”β€” from GLM-4.5

Hey @zai-org-3 , FYI I started on a PR to llama.cpp to try and add your models architecture but it's very much a best effort, if you wish to help out and contribute please do feel free to submit changes to, or comment on my PR that is currently in draft https://github.com/ggml-org/llama.cpp/pull/14939

For anyone else reading this that sees my PR, please do not create GGUFs from it yet, it will have issues as it is not yet finished.

Would really love some folks from @zai-org-3 to help out on https://github.com/ggml-org/llama.cpp/pull/14939 if they could?

Be ready to be happy. Support was merged today into llama.cpp. https://github.com/ggml-org/llama.cpp/commit/ef0144c087b33e5b8da42d529ac71aaf05cb49df

Lets fucking goooo!

ChuckMcSneed changed discussion status to closed

Sign up or log in to comment