Text-Only EXL2 Quant of mistralai/Mistral-Small-3.1-24B-Instruct-2503
The following changes were made:
- Vision encoder removed
- Architecture changed to that of mistralai/Mistral-Small-24B-Instruct-2501
- Chat template in tokenizer_config.json was modified (see below).
I was having trouble with the timestamp at the beginning of the system prompt and removed it from tokenizer_config.json.
NOTE Tensor Parallel is not implemented in exllamav2 for both mistralai/Mistral-Small-3.1-24B-Instruct-2503 and mistralai/Mistral-Small-24B-Instruct-2501.
- Downloads last month
- 0
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.
Model tree for gghfez/Mistral-Small-3.1-24B-Instruct-2503-novision-exl2-6bpw
Base model
mistralai/Mistral-Small-3.1-24B-Base-2503