Spaces:
Paused
Paused
How many gigabytes of VRAM is necessary for this?
#1
by
JLouisBiz
- opened
How many gigabytes of VRAM is necessary for this?
Greater than 30GB VRam
But as you have 24GB VRam you can try Quantized version
Dfloat11: https://github.com/LeanModels/Bagel-DFloat11
or
Int8: https://huggingface.co/Gapeleon/bytedance_BAGEL-7B-MoT-INT8