metadata

base_model: unsloth/Llama-3.2-3B
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft
license: apache-2.0
language:
  - en

GRMR-V3-L3B

GRMR-V3-L3B is a fine-tuned version of unsloth/Llama-3.2-3B specifically optimized for grammar correction tasks.

IMPORTANT: Please ensure you are using the following sampler settings for optimal results:

temperature = 0.7
frequency_penalty = 0.0
presence_penalty = 0.0
min_p = 0.01
top_p = 0.95
top_k = 40

Model description

GRMR-V3-L3B is a grammar correction model built on Meta's Llama 3.2 3B base model. It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.

The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.

Model Details

Developed by: qingy2024

Finetuned from model: unsloth/Llama-3.2-3B

Model type: Llama 3.2

Language(s): English

License: apache-2.0

Training Dataset: qingy2024/grmr-v4-60k

Manual Testing Examples

Here are a few examples of grammar corrections this model can handle:

Original Text	Corrected Text
i dont know weather to bring a umbrella today	I don't know whether to bring an umbrella today.
she go to the store yesterday	She went to the store yesterday.
they is going to be late for the meeting	They are going to be late for the meeting.
the cat laying on the floor all day	The cat is laying on the floor all day.

Training procedure

The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.

Training hyperparameters

Batch size: 8
Gradient accumulation steps: 2
Learning rate: 5e-5
Epochs: 1
Optimizer: AdamW (8-bit)
Weight decay: 0.01
LR scheduler: Cosine
Warmup steps: 180
Max sequence length: 16,384
Training precision: Mixed precision (BF16 where available, FP16 otherwise)

Intended uses & limitations

This model is designed for grammar correction tasks. It can be used to:

Fix grammatical errors in written text
Correct punctuation
Address spelling mistakes
Improve sentence structure and clarity

Limitations

The model may struggle with highly technical or domain-specific content
It may not fully understand context-dependent grammar rules in all cases
Performance may vary for non-standard English or text with multiple errors

How to use

llama.cpp and projects based on it should be able to run this model like any others.

For pure transformers code, you can refer here:

from transformers import AutoModelForCausalLM, AutoTokenizer# Load model and tokenizer
model_name = "qingy2024/GRMR-V3-L3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)# Text with grammar errors to correct
text_to_correct = "i am going to the store tommorow and buy some thing for dinner"# Format as messages
messages = [
    {"role": "user", "content": text_to_correct}
]# Apply the custom chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=512,
    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
    do_sample=True
)
# Decode and print the corrected text
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(corrected_text)

Using with the Hugging Face pipeline

from transformers import pipeline
pipe = pipeline(
    "text-generation",
    model="qingy2024/GRMR-V3-L3B",
    torch_dtype="auto",
    device_map="auto"
)
messages = [
    {"role": "user", "content": "i dont know weather to bring a umbrella today"}
]
result = pipe(
    messages,
    max_new_tokens=100,
    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
    do_sample=True,
    return_full_text=False
)[0]["generated_text"]
print(result)

Note: The Python examples above use temperature=0.1 for reproducibility in quick tests. For optimal grammar correction quality, please use the recommended sampler settings, especially temperature=0.7.

Custom Chat Template

The model uses a custom chat template with special formatting for grammar correction:

User inputs are formatted with <|start_header_id|>text<|end_header_id|> headers
Model outputs are formatted with <|start_header_id|>corrected<|end_header_id|> headers
Messages are separated by <|eot_id|> tokens
The chat template should work without any extra tweaking in vLLM or llama.cpp.

Training Dataset

The model was fine-tuned on the qingy2024/grmr-v4-60k dataset, which contains 60,000 examples of original text and their grammatically corrected versions.

Bias, Risks, and Limitations

The model may reflect biases present in the training data
It may not perform equally well across different writing styles or domains
The model might occasionally introduce errors or change the meaning of text
It focuses on grammatical correctness rather than stylistic improvements

Contact

For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.