Generated with llmcompressor, script included in the repo. I tested it very sparsely to make sure it didn't completely disintegrate, but there's a very good chance that it's less than optimal still.

Downloads last month
630
Safetensors
Model size
16.4B params
Tensor type
BF16
·
F8_E4M3
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for inflatebot/Kimi-VL-A3B-Thinking-2506-FP8-Dynamic