New favorite?
Well @mradermacher on occasion I ask you what your new favorite is in hopes that I can somehow run the behemoth (by my standard) of an llm that you recommend. might I ask what your new favorite is thus far?
I can't (personally) run behemoths (by my standards :) for myself, really - my sweet spot is IQ3_M quants of 70B's for a while now. My current favourite is v1 of this model, with StrawberryLemonade in testing. v2.1 and v3 of this model are also good, but I keep going back to the first version when they fail me, and only switch to something else out of curiosity. How about your experiences?
I've been really considering that, my experiences. and no matter how I try to find a new model, i keep coming back to Muse-12B Q8. I've tested a ton of models, Cydonia-24B-v3 (Q4 since thats the best my laptop can handle reasonably), Magnum V4 22b Q4 and various other popular/hyped models and None of them can really compare for me to Muse so far. Using the parameters specified ("temperature": 0.8,"repetition_penalty": 1.05, "min_p": 0.025) always does the trick for me, however recently an odd thing happened where the model started being really un creative but i think its a glitch with my llm host due to a recent update but i fixed it by cranking up the top p and top k for this model. its never done that before. was using backyard for the host originally and always worked great there though.
While Muse cannot do complex calculations like a hefty 70b model can even at low quants, it still shows decent intelligence and overall has been a fine companion for my roleplaying adventures. I've been craving an upgrade though, but none has fit the bill (that my PC can handle) and yeah i tried IQ3_m of your recommendation and my laptop just couldnt handle it xD i can run Q2 though very slowly. Too slowly for me. i need an upgrade xD
@mradermacher