i love you
#1
by
nisten
- opened
my normal awq kept crashing and spent wayy too many hours paying for an A100 80gb to quantize it on and saw this
Glad I could help. ;-)
Took around 50 minutes on a single L40 using a well-aged but up-to-date Ubuntu 22.04 with CUDA 12.6, nvidia-open driver 565.57.01. If this might help for future attempts.
Otherwise drop me a note, if you are in need for quants of a specific model. As long as I've access to those GPU nodes I will try to help.