Hugging Face's logo

yunconglong
/
Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B

Text Generation
Transformers
Safetensors
mixtral
Mixture of Experts
DPO
RL-TUNED
text-generation-inference
Model card Files Community
11

GPTQ Quants

#11
by rjmehta - opened
Discussion
rjmehta

Vllm supports gptq. Please gptq quants. @LoneStriker @MaziyarPanahi

MaziyarPanahi

@rjmehta
I don't need one of my A100, so I'll make a GPTQ of this model now.

MaziyarPanahi
•
edited Apr 4, 2024

it's available here: MaziyarPanahi/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B-GPTQ

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment