I have a draft PR up to llama.cpp, keen for your input
#4
by
smcleod
- opened
Hey @zai-org-3 , FYI I started on a PR to llama.cpp to try and add your models architecture but it's very much a best effort, if you wish to help out and contribute please do feel free to submit changes to, or comment on my PR that is currently in draft https://github.com/ggml-org/llama.cpp/pull/14939
For anyone else reading this that sees my PR, please do not create GGUFs from it yet, it will have issues as it is not yet finished.