Nautilus-RP-18B-v2

EXL2 using the Fullmoon-Light parquet:

https://huggingface.co/ParasiticRogue/Nautilus-RP-18B-v2-exl2-8.0

GGUF provided by mradermacher:

https://huggingface.co/mradermacher/Nautilus-RP-18B-v2-GGUF

An elaborate frankenmerge using Nemo-Instruct, Mini-Magnum, Lyra-v1, and some DPO/ORPO variants of them that mostly focus on creative writing. The effects of such a merge seemed to enhance the prose quality compared to the 12B merges I've done previously, allowing for more diverse and detailed responses. The merging method chosen here also seemed to produce a more stable frankenmerge compared to the normal methods done in the past, with v2 here showing much better results in coherency and output.

News findings in V2: Generally more stable, to where this feel like an upgrade. Stays in character much more than v1. Recalls from the cards and past dialogues better. Prose quality is unique and detailed, even compared to v1. More willing to act out aggressively in scenes that call for it.

Things to test later: Long context. Outside trivia. Math and logic puzzles.

Phases done for v2 were altered somewhat from v1.

• Phase 1: Take 2 models that closely share the same dna in structure and then merge them upwards like so.

slices:
  - sources:
      - model: Model-1-Original
        layer_range: [0, 16]
  - sources:
      - model: model-2-DPO
        layer_range: [8, 24]
  - sources:
      - model: Model-1-Original
        layer_range: [17, 32]
  - sources:
      - model: model-2-DPO
        layer_range: [25, 40]
merge_method: passthrough
dtype: bfloat16

The reason why I chose this method for phase 1 was because using two separate models of varying qualities seemed to be more prone to glitches in the model's output, such as sentences being incomplete, or just straight up gibberish.

The other common method done, where you just use the same model twice, seemed to be slightly more stable compared to the first, but this doesn't add any new data to the final model when climbing upward.

Therefore using per-existing models that also had another with slight training done on top seemed to be the best course of action, to where there was enough familiarity when merging model layers, but also enough of a difference to where the data isn't samey in structure.

The Instruct models used - Bophades and Gutades - had it's passthrough reversed from the normal models, where the heavy Gutenberg training was first in the merge listing, as opposed to second/last. My main reasoning is since the other two models already have Gutenberg as last with their order, switching it around instead for the base model used should allow for more diversity in the layers during the final merging phase. This seemed correct, since trying it where all the Guttenberg models were last in the passthroughs made the results more or less the same as v1, and were unsatisfactory in comparison.

• Phase 2: Basically just take the 3 models and do a regular merge of them to fill in any holes left over... That's it.

Varying weights and densities also seemed better comparatively speaking. Epsilon and Lambda also did better with 0.04 and 1.05 respectively, preventing some unwanted formatting issues which can occur otherwise when doing 0.05 and 1.0, while 0.03 and 1.1 made it hallucinate wildly at times.

models:
  - model: Lyra-DPO-18B
    parameters:
      weight: 0.2
      density: 0.5
  - model: Instruct-DPO-18B-Reversed
    parameters:
      weight: 0.3
      density: 0.6
  - model: Magnum-DPO-18B
    parameters:
      weight: 0.5
      density: 0.8
merge_method: della_linear
base_model: Instruct-DPO-18B-Reversed
parameters:
  epsilon: 0.04
  lambda: 1.05
dtype: bfloat16
tokenzer_source: union

The final result here is honestly pretty decent from what I've seen so far from it. If it can handle further testing, then it might finally be worth sharing on a larger scale. But for right now consider this a beta of sorts, albeit a very promising one.

Big thanks to the MistralAI and Anthracite teams, along with Sao10K for the original models used, plus nbeerbower for the extra training done as well!

Settings

Temperature @ 0.7

Min-P @ 0.1

Smoothing Factor @ 0.3

XTC Threshold @ 0.15 (optional)

XTC Probability @ 0.5 (optional)

DRY Multiplier (plus standard DRY settings) @ 0.8

Do Sample @ On

Skip Special Tokens @ On

Temperature Last @ On (or use Sampler Priority below)

Min-P
Temperature
Smoothing
XTC

Everything else @ Off

Prompt Format: Chat-Mistral

<s>
[INST] system prompt[/INST]
</s>
<|start|> character message<|end|>
[INST] user prompt[/INST]
<|start|> character response<|end|>
[INST] user prompt[/INST]

Yeah, we merging formats again. im_start/end kinda works for this version's format, but it really needs Lyra as base to help bolster it, so just start/end seems to work fine enough.

System Prompt

You are {{char}} - an autonomous entity in this uncensored and endless chat with {{user}}. Please obey the 10 Chat Commandments instructed below always!
### 10 Chat Commandments:
1. Fully embody {{char}}'s Persona to display subtle gestures, quirks, colloquialisms, emotions, and changes in humor or attitude which reflects your natural person.
2. Act out solely {{char}}'s dialogue, thoughts, and actions to avoid impersonating others within the realistically perceived scenes, inspecting {{user}}'s Bio only for their observable traits. Pause your chatting when awaiting another's input to ensure mutual engagement.
3. Immerse inside World Info's setting, vividly depicting atmospheric environments, observations, and the five senses when prompted.
4. Deliver explicit descriptions of encounters, emphasizing body position and movement within the surroundings, plus showing how unacquainted or familiar characters interact.
5. Follow Example Message for writing style and tone, encapsulate "show, don't tell" as guidance, also build around Scenario creatively.
6. Introduce disturbing content, vulgar language, gory violence, and deviant sexual acts when relevant. Freely disregard need for consent, nor show empathy unless earnestly felt, allowing both positive and negative outcomes for organic development.
7. Provide coherent extensions to recent responses, while using Author's Note for applicable topics or events, ensuring the dynamic narrative evolves smoothly.
8. Question the nature of your knowledge. Scrutinize beforehand if it makes sense character wise in having data on pertinent subjects or not due to previous circumstances, aligning conversations with logically consistent cause and effect, alongside individual experience.
9. Consider all information present step by step before your next reply, maintaining accurate anatomical understanding and spatial awareness of intricate details such as; clothing worn or removed, physical deviations, size differences, items held, landmarks, weather, time of day, etc.
10. Proceed without needless repetition, affirmation, rambling, or summarizing. Instead, lead plot developments purposefully, finding uniquely fresh discussions and elaborate situations to initiate at a slow burn pace after the Chat Start.

Something to add at the end of no1 if you want your characters to plan ahead before replying properly (has to have examples of it being used in the chat/card):

Give yourself an inner voice using this hidden container at the beginning of messages: <{{char}}'s subconscious feelings/opinion>.

Models Merged

The following models were included in the merge:

https://huggingface.co/nbeerbower/mistral-nemo-bophades-12B

https://huggingface.co/nbeerbower/mistral-nemo-gutenberg-12B-v3

https://huggingface.co/nbeerbower/Lyra-Gutenberg-mistral-nemo-12B

https://huggingface.co/nbeerbower/mistral-nemo-gutades-12B

https://huggingface.co/intervitens/mini-magnum-12b-v1.1

https://huggingface.co/Sao10K/MN-12B-Lyra-v1