GPTQ INT8 W8A8 quantized mistralai/Devstral-Small-2505
GPTQ INT8 W8A8 quantized mistralai/Devstral-Small-2505 calibrated with a sequence len of 4096 and 128 samples of TokenBender/code_instructions_122k_alpaca_style, glaiveai/glaive-code-assistant-v2, google/code_x_glue_ct_code_to_text for a total sample size of 1024.
Follow the Devstral-Small-2505 docs for running with vllm.
- Downloads last month
- 11
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for zankich/Devstral-Small-2505-INT8
Base model
mistralai/Mistral-Small-3.1-24B-Base-2503
Finetuned
mistralai/Devstral-Small-2505