hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4
Text Generation
•
59B
•
Updated
•
8.54k
•
36
Llama 3.1 GPTQ, AWQ, and BNB Quants
Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗
-
-
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4
Text Generation • 214B • Updated • 53 • 5 -
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4
Text Generation • 59B • Updated • 1.81k • 16 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Text Generation • 11B • Updated • 154k • 101 -
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit
Text Generation • 37B • Updated • 4.52k • 31 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4
Text Generation • 11B • Updated • 15.2k • 23 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4
Text Generation • 2B • Updated • 280k • 68 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4
Text Generation • 5B • Updated • 2.82k • 8 -
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4
Text Generation • 2B • Updated • 255k • 25