Any tips on running this? It's just yapping nonsense at me...
#2
by
Varkoyote
- opened
I'm having trouble making this AI do anything efficiently, I don't know why, it's just saying a lot of nonsense, most of it unrelated to any message I send... are there recommended settings to be used please?
Make sure you're using quants with high bitrates (>=4) and that the software you use is up-to-date.
I recommend using these generation parameters: temperature=1.0, top_k=64, top_p=0.95.
Sadly I was using the latest KCPP with q8 :( Maybe it's just not good at what I ask it to do...