metadata
license: other
license_name: mrl
inference: false
license_link: https://mistral.ai/licenses/MRL-0.1.md
base_model: mistralai/Mistral-Large-Instruct-2407
base_model_relation: quantized
tags:
- exl3
exllamav3 quantizations of Mistral-Large-Instruct-2407
Will update this space with links to the quant branches as they finish uploading. Expect the same sizes as in turboderp's exl3 quant of Mistral-Large-Instruct-2411.
1.40 bpw/H4 20.820 GiB
1.60 bpw/H4 23.670 GiB
1.80 bpw/H5 26.541 GiB
2.00 bpw/H5 29.389 GiB
2.25 bpw/H5 32.927 GiB
2.50 bpw/H5 36.470 GiB
3.00 bpw/H6 43.616 GiB
3.50 bpw/H6 50.697 GiB
4.00 bpw/H6 57.795 GiB
5.00 bpw/H6 71.975 GiB
6.00 bpw/H6 86.155 GiB
8.00 bpw/H8 114.609 GiB