Hugging Face's logo

01-ai
/
Yi-34B-Chat-4bits

Text Generation
Transformers
PyTorch
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
4-bit precision
awq
Model card Files Community
9

4bit GPTQ

#1
by FrankWu - opened
Discussion
FrankWu

can provide 4bit quantized by gptq? My gpu doesn't support awq, thank you

Yhyu13

Checkout TheBloke's Yi-34B Chat GPTQ https://huggingface.co/TheBloke/Yi-34B-Chat-GPTQ

FrankWu changed discussion status to closed

Sign up or log in to comment