A 12b role play model, outperforming most 24b
For a long time ReadyArt/Broken-Tutu-24B-Transgression-v2.0 was my goto model for rp but i wanted to see how smaller models have progressed. having tested several 8 and 12b variants, i think this one is the best of them all. at q8_0 it has excellent prose, follows instructions well, handles sudden context changes well and at large context windows, doesn't hallucinate. I do have to regenerate sometimes but that's about it.
Congrats and i hope to see more finetunes!
some info about my settings (sillytavern frontend, llama-server backend)
context/response: 32768/1024
Temp: 0.7
Top K: -1
Top P: 1
Min P: 0.03
Rep pen: 1.05
Rep pen range: 64
Freq pen: 0.05
Mistral V7 Tekken context/instruct
Custom, simple system prompt with instructions about formatting and directions to slow the rp pace.
llama-server:
mostly performance/quality settings needed for unified memory and prompt caching on strix halo.
Thank you for the feedback!