Quantization made by Richard Erkhov.

thea-rp-3b-25r - GGUF

Model creator: https://huggingface.co/piotr25691/
Original model: https://huggingface.co/piotr25691/thea-rp-3b-25r/

Name	Quant method	Size
thea-rp-3b-25r.Q2_K.gguf	Q2_K	1.27GB
thea-rp-3b-25r.IQ3_XS.gguf	IQ3_XS	1.38GB
thea-rp-3b-25r.IQ3_S.gguf	IQ3_S	1.44GB
thea-rp-3b-25r.Q3_K_S.gguf	Q3_K_S	1.44GB
thea-rp-3b-25r.IQ3_M.gguf	IQ3_M	1.49GB
thea-rp-3b-25r.Q3_K.gguf	Q3_K	1.57GB
thea-rp-3b-25r.Q3_K_M.gguf	Q3_K_M	1.57GB
thea-rp-3b-25r.Q3_K_L.gguf	Q3_K_L	1.69GB
thea-rp-3b-25r.IQ4_XS.gguf	IQ4_XS	1.71GB
thea-rp-3b-25r.Q4_0.gguf	Q4_0	1.79GB
thea-rp-3b-25r.IQ4_NL.gguf	IQ4_NL	1.79GB
thea-rp-3b-25r.Q4_K_S.gguf	Q4_K_S	1.8GB
thea-rp-3b-25r.Q4_K.gguf	Q4_K	1.88GB
thea-rp-3b-25r.Q4_K_M.gguf	Q4_K_M	1.88GB
thea-rp-3b-25r.Q4_1.gguf	Q4_1	1.95GB
thea-rp-3b-25r.Q5_0.gguf	Q5_0	2.11GB
thea-rp-3b-25r.Q5_K_S.gguf	Q5_K_S	2.11GB
thea-rp-3b-25r.Q5_K.gguf	Q5_K	2.16GB
thea-rp-3b-25r.Q5_K_M.gguf	Q5_K_M	2.16GB
thea-rp-3b-25r.Q5_1.gguf	Q5_1	2.28GB
thea-rp-3b-25r.Q6_K.gguf	Q6_K	2.46GB
thea-rp-3b-25r.Q8_0.gguf	Q8_0	3.19GB

Original model description:

language: - en license: llama3.2 tags: - text-generation-inference - transformers - llama - trl - sft - reasoning - llama-3 base_model: SicariusSicariiStuff/Impish_LLAMA_3B datasets: - KingNish/reasoning-base-20k - piotr25691/thea-name-overrides model-index: - name: thea-rp-3b-25r results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 65.78 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics: - type: acc_norm value: 20.01 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics: - type: exact_match value: 11.71 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics: - type: acc_norm value: 3.24 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 5.93 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 22.89 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=piotr25691/thea-rp-3b-25r name: Open LLM Leaderboard

Model Description

An uncensored roleplay reasoning Llama 3.2 3B model trained on reasoning data.

It may potentially be a highest scoring RP finetune of Llama 3.2.

It has been trained using improved training code, and gives an improved performance. Here is what inference code you should use:

from transformers import AutoModelForCausalLM, AutoTokenizer

MAX_REASONING_TOKENS = 1024
MAX_RESPONSE_TOKENS = 512

model_name = "piotr25691/thea-rp-3b-25r"

model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Which is greater 9.9 or 9.11 ??"
messages = [
    {"role": "user", "content": prompt}
]

# Generate reasoning
reasoning_template = tokenizer.apply_chat_template(messages, tokenize=False, add_reasoning_prompt=True)
reasoning_inputs = tokenizer(reasoning_template, return_tensors="pt").to(model.device)
reasoning_ids = model.generate(**reasoning_inputs, max_new_tokens=MAX_REASONING_TOKENS)
reasoning_output = tokenizer.decode(reasoning_ids[0, reasoning_inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("REASONING: " + reasoning_output)

# Generate answer
messages.append({"role": "reasoning", "content": reasoning_output})
response_template = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response_inputs = tokenizer(response_template, return_tensors="pt").to(model.device)
response_ids = model.generate(**response_inputs, max_new_tokens=MAX_RESPONSE_TOKENS)
response_output = tokenizer.decode(response_ids[0, response_inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("ANSWER: " + response_output)

Trained by: Piotr Zalewski
License: llama3.2
Finetuned from model: SicariusSicariiStuff/Impish_LLAMA_3B
Dataset used: KingNish/reasoning-base-20k

This Llama model was trained faster than Unsloth using custom training code.

Visit https://www.kaggle.com/code/piotr25691/distributed-llama-training-with-2xt4 to find out how you can finetune your models using BOTH of the Kaggle provided GPUs.

Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.