4-bit Quantization of the Kimi Dev 72B model

Quantized using GPTQModel

quantiziation config:

quant_config = QuantizeConfig(
    bits=4,
    group_size=32,
    sym=True,
    desc_act=False,
    true_sequential=True,
    pack_dtype=torch.int32,
    damp_percent=0.1       
)

Downloads last month: 0

Safetensors

Model size

13.7B params

Tensor type

F16

I32

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for btbtyler09/Kimi-Dev-72B-GPTQ-4bit

Base model

Qwen/Qwen2.5-72B

Finetuned

moonshotai/Kimi-Dev-72B

Quantized

(13)

this model

Collection including btbtyler09/Kimi-Dev-72B-GPTQ-4bit

Kimi Dev

Collection

Collection of Kimi Dev Quantizations • 2 items • Updated about 19 hours ago