Quantization made by Richard Erkhov.

Mahou-1.3-mistral-nemo-12B - GGUF

Model creator: https://huggingface.co/flammenai/
Original model: https://huggingface.co/flammenai/Mahou-1.3-mistral-nemo-12B/

Name	Quant method	Size
Mahou-1.3-mistral-nemo-12B.Q2_K.gguf	Q2_K	4.46GB
Mahou-1.3-mistral-nemo-12B.IQ3_XS.gguf	IQ3_XS	4.94GB
Mahou-1.3-mistral-nemo-12B.IQ3_S.gguf	IQ3_S	5.18GB
Mahou-1.3-mistral-nemo-12B.Q3_K_S.gguf	Q3_K_S	5.15GB
Mahou-1.3-mistral-nemo-12B.IQ3_M.gguf	IQ3_M	5.33GB
Mahou-1.3-mistral-nemo-12B.Q3_K.gguf	Q3_K	5.67GB
Mahou-1.3-mistral-nemo-12B.Q3_K_M.gguf	Q3_K_M	5.67GB
Mahou-1.3-mistral-nemo-12B.Q3_K_L.gguf	Q3_K_L	6.11GB
Mahou-1.3-mistral-nemo-12B.IQ4_XS.gguf	IQ4_XS	6.33GB
Mahou-1.3-mistral-nemo-12B.Q4_0.gguf	Q4_0	6.59GB
Mahou-1.3-mistral-nemo-12B.IQ4_NL.gguf	IQ4_NL	6.65GB
Mahou-1.3-mistral-nemo-12B.Q4_K_S.gguf	Q4_K_S	6.63GB
Mahou-1.3-mistral-nemo-12B.Q4_K.gguf	Q4_K	6.96GB
Mahou-1.3-mistral-nemo-12B.Q4_K_M.gguf	Q4_K_M	6.96GB
Mahou-1.3-mistral-nemo-12B.Q4_1.gguf	Q4_1	7.26GB
Mahou-1.3-mistral-nemo-12B.Q5_0.gguf	Q5_0	7.93GB
Mahou-1.3-mistral-nemo-12B.Q5_K_S.gguf	Q5_K_S	7.93GB
Mahou-1.3-mistral-nemo-12B.Q5_K.gguf	Q5_K	8.13GB
Mahou-1.3-mistral-nemo-12B.Q5_K_M.gguf	Q5_K_M	8.13GB
Mahou-1.3-mistral-nemo-12B.Q5_1.gguf	Q5_1	8.61GB
Mahou-1.3-mistral-nemo-12B.Q6_K.gguf	Q6_K	9.37GB
Mahou-1.3-mistral-nemo-12B.Q8_0.gguf	Q8_0	12.13GB

Original model description:

library_name: transformers license: apache-2.0 base_model: - mistralai/Mistral-Nemo-Instruct-2407 datasets: - flammenai/MahouMix-v1 - flammenai/FlameMix-DPO-v1

Mahou-1.3-mistral-nemo-12B

Mahou is designed to provide short messages in a conversational context. It is capable of casual conversation and character roleplay.

Chat Format

This model has been trained to use ChatML format.

<|im_start|>system
{{system}}<|im_end|>
<|im_start|>{{char}}
{{message}}<|im_end|>
<|im_start|>{{user}}
{{message}}<|im_end|>

Roleplay Format

Speech without quotes.
Actions in *asterisks*

*leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass.

SillyTavern Settings

Use ChatML for the Context Template.
Enable Instruct Mode.
Use the Mahou ChatML Instruct preset.
Recommended Additonal stopping strings: ["\n", "<|", "</"]
Use the Mahou Sampler preset.

Method

ORPO finetuned on a Google Colab A100 for 1 epoch.

Fine-tune Llama 3 with ORPO

Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.