---
license: apache-2.0
base_model:
- mistralai/Devstral-Small-2505
datasets:
- TokenBender/code_instructions_122k_alpaca_style
- glaiveai/glaive-code-assistant-v2
- google/code_x_glue_ct_code_to_text
pipeline_tag: text2text-generation
tags:
- text-generation-inference
---

# GPTQ INT8 W8A8 quantized mistralai/Devstral-Small-2505 

GPTQ INT8 W8A8 quantized mistralai/Devstral-Small-2505 calibrated with a sequence len of 4096 and 128 samples of `TokenBender/code_instructions_122k_alpaca_style`, `glaiveai/glaive-code-assistant-v2`, `google/code_x_glue_ct_code_to_text` for a total sample size of 1024.

Follow the Devstral-Small-2505 docs for running with vllm.