config.json breaks exl quanting
just a heads up that the config.json and the safetensor files are in conflict, which prevents quanting to EXL.
the config.json has Gemma3ForCasualLM, while the safetensors are Gemma3ForConditionalGeneration.
the config is also missing the vision tower portion from the original gemma3 config.json.
I swapped in the original Gemma3 config.json and am attempting to do the conversion that way. here's hoping.
Thanks for letting me know! These are finetuned by Unsloth and I've been having a ton of issues with their software. They keep changing things and each time I run the notebook something new breaks.
The config file is automatically generated by their notebook so I'll have to look into what is going wrong, so thanks for the heads up!
As for vision, these finetuned models were not fine-tuned on vision prompts, so they most likely lost that functionality.
well good news. using the default gemma3 config.json worked, model seems fine. It was quanted by turboderp himself (maker of Exllama)
https://huggingface.co/turboderp/Grok-3-reasoning-gemma3-12B-distilled-HF-exl3/tree/4.0bpw