IQ3_XXS use more VRAM
#5
by
mahdisml
- opened
Hi ! i tried IQ3_XXS but something wrong happened .
Size of IQ3_XXS is 4.78GB but when i use it in lmstudio it takes up 6.5 GB VRAM. I've tried a lot of models before, but I've never seen this issue.
Is this normal ?
before load :
after load :
settings :
Yes, models will use more RAM after being loaded due to a variety of reasons, most notably context and overhead
This is about what you'd expect from a typical model, 2-4GB extra
Thank you so much! 🙏✨