L3.3-Shakudo-70B-EXL2

Released by Steelskull

Quant(s) by FrenzyBiscuit

Note that while 4.0 BPW is offered, it is not recommended.

4.0 BPW H6 - Fits in 48GB VRAM with 32k FP16 context

4.55 BPW H6 - Fits in 48GB VRAM with 24k FP16 context

5.35 BPW H6 - Fits in 72GB VRAM with 64k FP16 context

6.15 BPW H6 - Fits in 72GB VRAM with 48k FP16 context

6.70 BPW H6 - Fits in 72GB VRAM with 32k FP16 context

7.45 BPW H6 - Fits in 72GB VRAM with 24k FP16 context

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ReadyArt/Steelskull_L3.3-Shakudo-70b-EXL2

Base model

Finetuned

Finetuned

Quantized

(6)

this model