Hugging Face's logo

CarperAI
/
stable-vicuna-13b-delta

Text Generation
Transformers
PyTorch
English
llama
causal-lm
text-generation-inference
Inference Endpoints
Model card Files Community
10

Can someone share ggml 4bit version?

#2
by alphaprime90 - opened
Discussion
alphaprime90

For CPU users

TheBloke
•
edited Apr 28, 2023

I have provided them here: https://huggingface.co/TheBloke/stable-vicuna-13B-GGML

I also have 4bit GPTQs for lower-VRAM GPU inference here: https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ

alphaprime90

I have provided them here: https://huggingface.co/TheBloke/stable-vicuna-13B-GGML

I also have 4bit GPTQs for lower-VRAM GPU inference here: https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ

Legend. Thank you.

alphaprime90 changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment