Returned to this model again after some time and realized how amazing it
The first time I tried to write stories with this model, I used the chat mode in oobaboga and the standard settings. At first I thought that the model had become dumbed down from RP datasets, since it only returned one phrase per message.
However, when I tried Q4_K_M with Shortwave preset and notebook mode, it started to generate very good texts.
Funny, I had a very similar experience with it.
My first impression of MiquMaid was "meh." I thought the 8x7B Noromaids did better in most things I wanted to try, but ... 10 days later ...
I ran across some EXL2 flavors, and gave it another try. It was spectacular 9/10 gens!
I do think there are some very narrow bands for settings on this one. It's like that bad shower valve that goes from freezing cold to boiling hot in just a tiny twist? This model seems to go from "mind blowing" to "WTF?" with very small tweaks. Once you find a setup that works, save that shit ...
So... what are those settings that worked for both of you ?
So... what are those settings that worked for both of you ?
Shortwave preset in oobabooga textgen webui and q4_k_m
max_new_tokens
512
temperature
1,53
top_p
0,64
min_p
0
top_k
33
repetition_penalty
1,07
presence_penalty
0
frequency_penalty
0
typical_p
1
tfs
1
mirostat_mode
mode=1 is for llama.cpp only.
0
mirostat_tau
5
mirostat_eta
0.1
Thank you !