Edit Models filters

Inference status

Misc

4-bit precision

AutoTrain Compatible

text-generation-inference

Inference Endpoints

8-bit precision

Mixture of Experts

text-embeddings-inference

Carbon Emissions

Models

21,372

Full-text search

Active filters: 4-bit

unsloth/gemma-2-9b-it-bnb-4bit

Text Generation • Updated Nov 22, 2024 • 70k • 21

TechxGenus/DeepSeek-V2-Lite-Chat-AWQ

Text Generation • Updated Jul 4, 2024 • 23 • 2

MaziyarPanahi/Mistral-Nemo-Instruct-2407-GGUF

Text Generation • Updated Jul 22, 2024 • 1.8M • 40

hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7, 2024 • 206k • 95

unsloth/Meta-Llama-3.1-8B-bnb-4bit

Text Generation • Updated Nov 22, 2024 • 264k • 91

unsloth/Meta-Llama-3.1-70B-bnb-4bit

Text Generation • Updated Nov 22, 2024 • 4.14k • 30

MaziyarPanahi/Meta-Llama-3.1-8B-Instruct-GGUF

Text Generation • Updated Jul 23, 2024 • 1.75M • 15

solidrust/Meta-Llama-3.1-8B-Instruct-abliterated-AWQ

Text Generation • Updated Sep 3, 2024 • 257 • 3

openbmb/MiniCPM-V-2_6-int4

Image-Text-to-Text • Updated 3 days ago • 192k • 70

LoneStriker/Hermes-3-Llama-3.1-8B-4.0bpw-h6-exl2

Updated Aug 15, 2024 • 7 • 1

informatiker/Hermes-3-Llama-3.1-8B-4bit

Updated Aug 15, 2024 • 2 • 2

LoneStriker/Hermes-3-Llama-3.1-70B-4.0bpw-h6-exl2

Updated Aug 16, 2024 • 4 • 1

Statuo/NemoMix-Unleashed-EXL2-4bpw

Text Generation • Updated Aug 23, 2024 • 52 • 7

Qwen/Qwen2-VL-2B-Instruct-AWQ

Image-Text-to-Text • Updated Sep 21, 2024 • 6.85k • 20

Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4

Image-Text-to-Text • Updated Sep 21, 2024 • 652k • 19

MaziyarPanahi/solar-pro-preview-instruct-GGUF

Text Generation • Updated Sep 13, 2024 • 1.74M • 24

Qwen/Qwen2.5-32B-Instruct-GPTQ-Int4

Text Generation • Updated Oct 9, 2024 • 7.88k • 23

Qwen/Qwen2.5-0.5B-Instruct-AWQ

Text Generation • Updated Oct 9, 2024 • 2.4k • 6

Qwen/Qwen2.5-7B-Instruct-AWQ

Text Generation • Updated Oct 9, 2024 • 11.8k • 17

MaziyarPanahi/Qwen2.5-7B-Instruct-GGUF

Text Generation • Updated Sep 18, 2024 • 1.75M • 9

mlx-community/Qwen2.5-72B-Instruct-4bit

Text Generation • Updated Sep 18, 2024 • 125k • 5

unsloth/Qwen2.5-72B-Instruct-bnb-4bit

Text Generation • Updated Sep 23, 2024 • 7.46k • 4

unsloth/Qwen2.5-Coder-7B-bnb-4bit

Text Generation • Updated Nov 12, 2024 • 4.8k • 5

SeanScripts/pixtral-12b-nf4

Image-Text-to-Text • Updated Sep 26, 2024 • 574 • 20

mlx-community/Llama-3.2-3B-Instruct-4bit

Text Generation • Updated Sep 25, 2024 • 940k • 23

mlx-community/Llama-3.2-1B-Instruct-4bit

Text Generation • Updated Oct 7, 2024 • 2.48M • 9

unsloth/Llama-3.2-3B-Instruct-bnb-4bit

Text Generation • Updated 2 days ago • 219k • 15

MaziyarPanahi/Llama-3.2-3B-Instruct-GGUF

Text Generation • Updated Sep 25, 2024 • 1.32M • 9

MaziyarPanahi/Llama-3.2-1B-Instruct-GGUF

Text Generation • Updated Sep 25, 2024 • 1.75M • 7

AMead10/Llama-3.2-1B-Instruct-AWQ

Text Generation • Updated Sep 25, 2024 • 3.06k • 4