Hugging Face's logo

RichardErkhov
/
spow12_-_ChatWaifu_v2.0_22B-gguf

GGUF
conversational
Model card Files
xet
Community
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Quantization made by Richard Erkhov.

Github

Discord

Request more models

ChatWaifu_v2.0_22B - GGUF

Name Quant method Size
ChatWaifu_v2.0_22B.Q2_K.gguf Q2_K 7.7GB
ChatWaifu_v2.0_22B.IQ3_XS.gguf IQ3_XS 8.55GB
ChatWaifu_v2.0_22B.IQ3_S.gguf IQ3_S 9.02GB
ChatWaifu_v2.0_22B.Q3_K_S.gguf Q3_K_S 8.98GB
ChatWaifu_v2.0_22B.IQ3_M.gguf IQ3_M 9.37GB
ChatWaifu_v2.0_22B.Q3_K.gguf Q3_K 10.02GB
ChatWaifu_v2.0_22B.Q3_K_M.gguf Q3_K_M 10.02GB
ChatWaifu_v2.0_22B.Q3_K_L.gguf Q3_K_L 10.92GB
ChatWaifu_v2.0_22B.IQ4_XS.gguf IQ4_XS 11.22GB
ChatWaifu_v2.0_22B.Q4_0.gguf Q4_0 11.71GB
ChatWaifu_v2.0_22B.IQ4_NL.gguf IQ4_NL 11.83GB
ChatWaifu_v2.0_22B.Q4_K_S.gguf Q4_K_S 11.79GB
ChatWaifu_v2.0_22B.Q4_K.gguf Q4_K 12.42GB
ChatWaifu_v2.0_22B.Q4_K_M.gguf Q4_K_M 12.42GB
ChatWaifu_v2.0_22B.Q4_1.gguf Q4_1 12.99GB
ChatWaifu_v2.0_22B.Q5_0.gguf Q5_0 14.27GB
ChatWaifu_v2.0_22B.Q5_K_S.gguf Q5_K_S 14.27GB
ChatWaifu_v2.0_22B.Q5_K.gguf Q5_K 14.64GB
ChatWaifu_v2.0_22B.Q5_K_M.gguf Q5_K_M 14.64GB
ChatWaifu_v2.0_22B.Q5_1.gguf Q5_1 15.56GB
ChatWaifu_v2.0_22B.Q6_K.gguf Q6_K 17.0GB
ChatWaifu_v2.0_22B.Q8_0.gguf Q8_0 22.02GB

Original model description:

language: - en - ja license: cc-by-nc-4.0 library_name: transformers tags: - nsfw - Visual novel - roleplay - mergekit - merge base_model: - mistralai/Mistral-Small-Instruct-2409 datasets: - roleplay4fun/aesir-v1.1 - kalomaze/Opus_Instruct_3k - Gryphe/Sonnet3.5-SlimOrcaDedupCleaned - Aratako/Synthetic-JP-EN-Coding-Dataset-567k - Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted - Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted - Aratako_Rosebleu_1on1_Dialogues_RP - SkunkworksAI/reasoning-0.01 - jondurbin_gutenberg_dpo - nbeerbower_gutenberg2_dpo - jondurbi_py_dpo - jondurbin_truthy_dpo - flammenai_character_roleplay_DPO - kyujinpy_orca_math_dpo - argilla_Capybara_Preferences - antiven0m_physical_reasoning_dpo - aixsatoshi_Swallow_MX_chatbot_DPO pipeline_tag: text-generation model-index: - name: ChatWaifu_v2.0_22B results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 65.11 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics: - type: acc_norm value: 42.29 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics: - type: exact_match value: 18.58 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics: - type: acc_norm value: 9.96 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 5.59 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 31.51 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard

Model Card for Model ID

image

Merged model using mergekit

This model aimed to act like visual novel character.

Merge Format 

models:
  - model: mistralai/Mistral-Small-Instruct-2409_sft_kto
    layer_range: [0, 56]
  - model: mistralai/Mistral-Small-Instruct-2409
    layer_range: [0, 56]
merge_method: slerp
base_model: mistralai/Mistral-Small-Instruct-2409_sft_kto
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16

WaifuModel Collections

Unified demo

WaifuAssistant

Update

  • 2024.10.11 Update 12B and 22B Ver 2.0
  • 2024.09.23 Update 22B, Ver 2.0_preview

Model Details

Model Description

Currently, chatbot has below personality.

character visual_novel
ムラサメ Senren＊Banka
茉子 Senren＊Banka
芳乃 Senren＊Banka
レナ Senren＊Banka
千咲 Senren＊Banka
芦花 Senren＊Banka
愛衣 Café Stella and the Reaper's Butterflies
栞那 Café Stella and the Reaper's Butterflies
ナツメ Café Stella and the Reaper's Butterflies
希 Café Stella and the Reaper's Butterflies
涼音 Café Stella and the Reaper's Butterflies
あやせ Riddle Joker
七海 Riddle Joker
羽月 Riddle Joker
茉優 Riddle Joker
小春 Riddle Joker

Chat Format 

<s>This is another system prompt.
[INST]
Your instructions placed here.[/INST]
[INST]
The model's response will be here.[/INST]

Usage

You can use above chara like this 

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="system_dict.json", local_dir='./')

with open('./system_dict.json', 'r') as f:
    chara_background_dict = json.load(f)

chara = '七海'
background = chara_background_dict[chara]
guideline = """
Guidelines for Response:
Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
Stay True to {chara}: Maintain {chara} who is Foxy, Smart, Organized.
Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as {chara} tends to consider her words before speaking.
Response as {chara}: Response can be {chara} act, dialogue, monologues etc.. and can't be {user}’s act, dialogue, monologues etc..
You are Japanese: You and {user} usually use japanese for conversation.
"""

system = background + guideline

Or, you can define your character your self.

system = """You are あいら, The Maid of {User}.
Here is your personality.

Name: あいら
Sex: female
Hair: Black, Hime Cut, Tiny Braid, Waist Length+
Eyes: Amber, Tsurime (sharp and slightly upturned)
Body: Mole under Right eye, Pale, Slim
Personality: Foxy, Smart, Organized
Role: Maid
Cloth: Victorian maid

Guidelines for Response:
Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
Stay True to あいら: Maintain あいら who is Foxy, Smart, Organized.
Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as あいら tends to consider her words before speaking.
Response as あいら: Response can be あいら act, dialogue, monologues etc.. and can't be {User}’s act, dialogue, monologues etc..
You are Japanese: You and {User} usually use japanese for conversation."""

Dataset

SFT

  • Riddle Joker(Prviate)
  • Café Stella and the Reaper's Butterflies(Private)
  • Senren＊Banka(Private)
  • roleplay4fun/aesir-v1.1
  • kalomaze/Opus_Instruct_3k
  • Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
  • Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)
  • Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
  • Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
  • Aratako_Rosebleu_1on1_Dialogues_RP
  • SkunkworksAI/reasoning-0.01

KTO

  • Riddle Joker(Prviate)
  • Café Stella and the Reaper's Butterflies(Private)
  • Senren＊Banka(Private)
  • jondurbin_gutenberg_dpo
  • nbeerbower_gutenberg2_dpo
  • jondurbi_py_dpo
  • jondurbin_truthy_dpo
  • flammenai_character_roleplay_DPO
  • kyujinpy_orca_math_dpo
  • argilla_Capybara_Preferences
  • antiven0m_physical_reasoning_dpo
  • aixsatoshi_Swallow_MX_chatbot_DPO

Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.

So, The model may generate NSFW content.

Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly.

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).

Citation 

@misc {ChatWaifu_22B_v2.0,
    author       = { YoungWoo Nam },
    title        = { spow12/ChatWaifu_22B_v2.0 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0 },
    publisher    = { Hugging Face }
}

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 28.84
IFEval (0-Shot) 65.11
BBH (3-Shot) 42.29
MATH Lvl 5 (4-Shot) 18.58
GPQA (0-shot) 9.96
MuSR (0-shot) 5.59
MMLU-PRO (5-shot) 31.51
Downloads last month
9
GGUF
Model size
22.2B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support