recommended sampling settings
#1
by
stockpot
- opened
quadratic sampling ✅
smoothing_factor: 0.18
smoothing_curve: 1
nothing else. really. temp 1, no other samplers, no repetition penalties at all.
also the base mistral nemo template seems to have won out over chatml unexpectedly, i assume due to how i stacked the early and late layers with non-chatml models, so you'll want to fix that. note that most things have an incorrect template for mistral nemo, there should be no whitespace around the [INST] and [/INST] tokens.