Vision Language Models Quantization - a neuralmagic Collection

neuralmagic 's Collections

DeepSeek-R1-Distill Quantized

Granite 3.1 Quantization

Sparse-Llama-3.1-2of4

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Vision Language Models Quantization

updated Mar 4

Vision Language Models (VLMs) quantized by Neural Magic

RedHatAI/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • 11B • Updated Oct 2, 2024 • 4.53k • 24
RedHatAI/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • 89B • Updated Oct 2, 2024 • 5.29k • 10
RedHatAI/pixtral-12b-FP8-dynamic

Text Generation • 13B • Updated Feb 7 • 49.2k • 10
RedHatAI/Phi-3-vision-128k-instruct-W4A16-G128

Text Generation • 1B • Updated Feb 10 • 39 • 1
RedHatAI/Qwen2-VL-72B-Instruct-quantized.w8a8

Image-Text-to-Text • 73B • Updated May 1 • 14
RedHatAI/Qwen2-VL-72B-Instruct-FP8-dynamic

Image-Text-to-Text • 73B • Updated Mar 31 • 956
RedHatAI/Qwen2-VL-72B-Instruct-quantized.w4a16

Image-Text-to-Text • 13B • Updated Mar 31 • 14
RedHatAI/Qwen2.5-VL-72B-Instruct-FP8-Dynamic

Image-Text-to-Text • 73B • Updated Apr 25 • 2.72k • 4
RedHatAI/Qwen2.5-VL-72B-Instruct-quantized.w8a8

Image-Text-to-Text • 73B • Updated May 1 • 2.07k
RedHatAI/Qwen2.5-VL-72B-Instruct-quantized.w4a16

Image-Text-to-Text • 13B • Updated Apr 3 • 2.19k • 3
RedHatAI/Pixtral-Large-Instruct-2411-hf-quantized.w8a8

Image-Text-to-Text • 124B • Updated Mar 31 • 19
RedHatAI/Pixtral-Large-Instruct-2411-hf-quantized.w4a16

Image-Text-to-Text • 19B • Updated Mar 31 • 1.84k
RedHatAI/pixtral-12b-quantized.w4a16

Image-Text-to-Text • 3B • Updated Feb 26 • 2k • 1
RedHatAI/pixtral-12b-quantized.w8a8

Image-Text-to-Text • 13B • Updated Feb 25 • 1.78k • 1
RedHatAI/Qwen2.5-VL-3B-Instruct-FP8-Dynamic

Image-Text-to-Text • 4B • Updated Apr 22 • 3.11k • 3
RedHatAI/Qwen2.5-VL-3B-Instruct-quantized.w8a8

Image-Text-to-Text • 4B • Updated May 1 • 3.88k • 2
RedHatAI/Qwen2.5-VL-3B-Instruct-quantized.w4a16

Image-Text-to-Text • 2B • Updated Apr 3 • 2.02k • 2
RedHatAI/Qwen2.5-VL-7B-Instruct-FP8-Dynamic

Image-Text-to-Text • 8B • Updated Apr 3 • 44k • 3
RedHatAI/Qwen2.5-VL-7B-Instruct-quantized.w4a16

Image-Text-to-Text • 3B • Updated Apr 3 • 2.06k • 3
RedHatAI/Qwen2.5-VL-7B-Instruct-quantized.w8a8

Image-Text-to-Text • 8B • Updated Apr 3 • 2.85k • 5