Best model regarding personality but Major flaw
#1
by
Autumnlight
- opened
This is the best model by far when it comes to personality and character IQ.
Issue is that (tested 2q) it suffers from low memory IQ.
Seems like 8-11K is the sweet spot, it has really good backlog attention, follows the scenario very well etc. But after that it breaks totally apart, repeats actions, forgets its goal and becomes basically a character with dementia sadly.
Behemoth can go far past 32K so...
It's probably because you're using a Q2 quant
As per our discord conversation, your problem is likely a combination of using a Q2 quant and 4bit KV. I've had no issues out to 32k, which is as far as I can go on my system with iQ3m and 16bit KV.