Regarding Mistral-based GGUF conversion
#1
by
yuuko-eth
- opened
Dear Audrey,
We were having trouble using llama.cpp (convert.py) to convert from MTK’s provided weights ourselves, but we never had any success. Is it possible for you to share how to convert this kind of model into GGUF, like what patch might have been required for the tool to work. I have attempted token padding option with no avail, though the recent commits seems to claim that the conversion tool is fixed.
Many thanks!! 🥺🤗
Indeed, the current HEAD (36e5a08b as of this writing) is broken w.r.t. this model. The breaking commit is 6efb8eb3 which renamed --padvocab to --pad-vocab.
Here is how to revert to the last-known-good version and convert:
cd llama.cpp
git checkout 6efb8eb3^
python3 convert.py ../Breeze-7B-Instruct-64k-v0.1 --padvocab
audreyt
changed discussion status to
closed