KnutJaegersberg

/

Tess-M-34B-2bit

like 6

https://github.com/kim90000/QTIP-run-but/blob/main/Untitled215%20(1).ipynb

there was an extra library for quip, if I remember correctly. I dont think it was supported by transformers. anyway this model is now outdated, better try newer ones.

Please suggest a modern 70-bit model that works good on 16 gb vram

you might be better off with Qwen2.5 32b Instruct in 4 bit, or gemma3 27b or the new Mistral small