|
|
--- |
|
|
base_model: unsloth/Llama-3.2-3B |
|
|
tags: |
|
|
- text-generation-inference |
|
|
- transformers |
|
|
- unsloth |
|
|
- llama |
|
|
- trl |
|
|
- sft |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
<html lang="en"> |
|
|
<head> |
|
|
<meta charset="UTF-8"> |
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|
|
</head> |
|
|
<div class="container"><h1>GRMR-V3-L3B</h1><p>GRMR-V3-L3B is a fine-tuned version of <a href="https://huggingface.co/unsloth/Llama-3.2-3B">unsloth/Llama-3.2-3B</a> specifically optimized for grammar correction tasks.</p><div class="important-note"><p><strong>IMPORTANT:</strong> Please ensure you are using the following sampler settings for optimal results:</p><pre><code>temperature = 0.7 |
|
|
frequency_penalty = 0.0 |
|
|
presence_penalty = 0.0 |
|
|
min_p = 0.01 |
|
|
top_p = 0.95 |
|
|
top_k = 40</code></pre></div><h2>Model description</h2><p>GRMR-V3-L3B is a grammar correction model built on Meta's Llama 3.2 3B base model. It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.</p><p>The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.</p><h3>Model Details</h3><div class="model-meta"><p><strong>Developed by:</strong> qingy2024</p><p><strong>Finetuned from model:</strong> <a href="https://huggingface.co/unsloth/Llama-3.2-3B">unsloth/Llama-3.2-3B</a></p><p><strong>Model type:</strong> Llama 3.2</p><p><strong>Language(s):</strong> English</p><p><strong>License:</strong> apache-2.0</p><p><strong>Training Dataset:</strong> <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a></p></div><hr><h2>Manual Testing Examples</h2><p>Here are a few examples of grammar corrections this model can handle:</p><table><thead><tr><th>Original Text</th><th>Corrected Text</th></tr></thead><tbody><tr><td>i dont know weather to bring a umbrella today</td><td>I don't know whether to bring an umbrella today.</td></tr><tr><td>she go to the store yesterday</td><td>She went to the store yesterday.</td></tr><tr><td>they is going to be late for the meeting</td><td>They are going to be late for the meeting.</td></tr><tr><td>the cat laying on the floor all day</td><td>The cat is laying on the floor all day.</td></tr></tbody></table><hr><h2>Training procedure</h2><p>The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.</p><h3>Training hyperparameters</h3><ul><li><strong>Batch size:</strong> 8</li><li><strong>Gradient accumulation steps:</strong> 2</li><li><strong>Learning rate:</strong> 5e-5</li><li><strong>Epochs:</strong> 1</li><li><strong>Optimizer:</strong> AdamW (8-bit)</li><li><strong>Weight decay:</strong> 0.01</li><li><strong>LR scheduler:</strong> Cosine</li><li><strong>Warmup steps:</strong> 180</li><li><strong>Max sequence length:</strong> 16,384</li><li><strong>Training precision:</strong> Mixed precision (BF16 where available, FP16 otherwise)</li></ul><h2>Intended uses & limitations</h2><p>This model is designed for grammar correction tasks. It can be used to:</p><ul><li>Fix grammatical errors in written text</li><li>Correct punctuation</li><li>Address spelling mistakes</li><li>Improve sentence structure and clarity</li></ul><h3>Limitations</h3><ul><li>The model may struggle with highly technical or domain-specific content</li><li>It may not fully understand context-dependent grammar rules in all cases</li><li>Performance may vary for non-standard English or text with multiple errors</li></ul><h2>How to use</h2><p>llama.cpp and projects based on it should be able to run this model like any others.</p><p>For pure <code>transformers</code> code, you can refer here:</p><pre><code class="language-python">from transformers import AutoModelForCausalLM, AutoTokenizer# Load model and tokenizer |
|
|
model_name = "qingy2024/GRMR-V3-L3B" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_name)# Text with grammar errors to correct |
|
|
text_to_correct = "i am going to the store tommorow and buy some thing for dinner"# Format as messages |
|
|
messages = [ |
|
|
{"role": "user", "content": text_to_correct} |
|
|
]# Apply the custom chat template |
|
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)# Tokenize and generate |
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
|
|
outputs = model.generate( |
|
|
inputs["input_ids"], |
|
|
max_new_tokens=512, |
|
|
temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7 |
|
|
do_sample=True |
|
|
) |
|
|
# Decode and print the corrected text |
|
|
corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
print(corrected_text)</code></pre><h3>Using with the Hugging Face pipeline</h3><pre><code class="language-python">from transformers import pipeline |
|
|
pipe = pipeline( |
|
|
"text-generation", |
|
|
model="qingy2024/GRMR-V3-L3B", |
|
|
torch_dtype="auto", |
|
|
device_map="auto" |
|
|
) |
|
|
messages = [ |
|
|
{"role": "user", "content": "i dont know weather to bring a umbrella today"} |
|
|
] |
|
|
result = pipe( |
|
|
messages, |
|
|
max_new_tokens=100, |
|
|
temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7 |
|
|
do_sample=True, |
|
|
return_full_text=False |
|
|
)[0]["generated_text"] |
|
|
print(result)</code></pre><p><em>Note: The Python examples above use <code>temperature=0.1</code> for reproducibility in quick tests. For optimal grammar correction quality, please use the recommended sampler settings, especially <code>temperature=0.7</code>.</em></p><h2>Custom Chat Template</h2><p class="chat-template-info">The model uses a custom chat template with special formatting for grammar correction:</p><ul><li>User inputs are formatted with <code><|start_header_id|>text<|end_header_id|></code> headers</li><li>Model outputs are formatted with <code><|start_header_id|>corrected<|end_header_id|></code> headers</li><li>Messages are separated by <code><|eot_id|></code> tokens</li><li>The chat template should work without any extra tweaking in vLLM or llama.cpp.</li></ul><h2>Training Dataset</h2><p>The model was fine-tuned on the <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a> dataset, which contains 60,000 examples of original text and their grammatically corrected versions.</p><h2>Bias, Risks, and Limitations</h2><ul><li>The model may reflect biases present in the training data</li><li>It may not perform equally well across different writing styles or domains</li><li>The model might occasionally introduce errors or change the meaning of text</li><li>It focuses on grammatical correctness rather than stylistic improvements</li></ul><h2>Contact</h2><p>For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.</p></div> |
|
|
<style> |
|
|
body { |
|
|
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; |
|
|
line-height: 1.6; |
|
|
margin: 0; |
|
|
padding: 0; |
|
|
background-color: #f8f9fa; |
|
|
color: #333; |
|
|
} |
|
|
.container { |
|
|
max-width: 1200px; |
|
|
margin: 10px auto; |
|
|
padding: 25px; |
|
|
background-color: #ffffff; |
|
|
border-radius: 8px; |
|
|
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08); |
|
|
} |
|
|
h1, h2, h3 { |
|
|
color: #0056b3; /* Primary Blue */ |
|
|
margin-top: 1.5em; |
|
|
margin-bottom: 0.7em; |
|
|
} |
|
|
h1 { |
|
|
text-align: center; |
|
|
font-size: 2.2em; |
|
|
border-bottom: 2px solid #e0e0e0; |
|
|
padding-bottom: 0.5em; |
|
|
margin-top: 0; |
|
|
} |
|
|
h2 { |
|
|
font-size: 1.8em; |
|
|
border-bottom: 1px solid #e9ecef; |
|
|
padding-bottom: 0.3em; |
|
|
} |
|
|
h3 { |
|
|
font-size: 1.4em; |
|
|
color: #007bff; /* Lighter Blue for sub-headings */ |
|
|
} |
|
|
p, li { |
|
|
font-size: 1em; |
|
|
color: #555; |
|
|
} |
|
|
a { |
|
|
color: #007bff; |
|
|
text-decoration: none; |
|
|
} |
|
|
a:hover { |
|
|
text-decoration: underline; |
|
|
color: #0056b3; |
|
|
} |
|
|
.important-note { |
|
|
background-color: #e7f3ff; /* Light blue background */ |
|
|
border-left: 5px solid #007bff; /* Blue accent border */ |
|
|
margin: 20px 0px; |
|
|
border-radius: 5px; |
|
|
} |
|
|
.important-note strong { |
|
|
color: #0056b3; |
|
|
font-weight: 600; |
|
|
} |
|
|
.important-note { |
|
|
background-color: #d0e8ff; |
|
|
padding: 0.05em 1.0em; |
|
|
border-radius: 3px; |
|
|
font-size: 0.9em; |
|
|
} |
|
|
code { |
|
|
padding: 0.1em 0.4em; |
|
|
border-radius: 3px; |
|
|
font-size: 0.9em; |
|
|
} |
|
|
table { |
|
|
width: 100%; |
|
|
border-collapse: collapse; |
|
|
margin: 20px 0; |
|
|
box-shadow: 0 2px 4px rgba(0,0,0,0.05); |
|
|
} |
|
|
th, td { |
|
|
border: 1px solid #dee2e6; |
|
|
padding: 10px 12px; |
|
|
text-align: left; |
|
|
vertical-align: top; |
|
|
} |
|
|
th { |
|
|
background-color: #e9ecef; /* Light gray for headers */ |
|
|
font-weight: 600; |
|
|
color: #212529; |
|
|
} |
|
|
td:first-child { |
|
|
/* font-style: italic; */ |
|
|
color: #444; |
|
|
} |
|
|
pre { |
|
|
background-color: #f1f3f5; |
|
|
padding: 15px; |
|
|
border-radius: 5px; |
|
|
overflow-x: auto; |
|
|
border: 1px solid #ced4da; |
|
|
font-size: 0.9em; |
|
|
} |
|
|
code { |
|
|
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; |
|
|
background-color: #e9ecef; |
|
|
padding: 0.2em 0.4em; |
|
|
border-radius: 3px; |
|
|
font-size: 0.9em; |
|
|
} |
|
|
pre code { |
|
|
background-color: transparent; |
|
|
padding: 0; |
|
|
border-radius: 0; |
|
|
font-size: 1em; |
|
|
} |
|
|
ul { |
|
|
padding-left: 20px; |
|
|
} |
|
|
li { |
|
|
margin-bottom: 0.5em; |
|
|
} |
|
|
hr { |
|
|
border: none; |
|
|
border-top: 1px solid #e0e0e0; |
|
|
margin: 30px 0; |
|
|
} |
|
|
.model-meta { |
|
|
background-color: #f8f9fa; |
|
|
padding: 15px; |
|
|
border-radius: 5px; |
|
|
margin-bottom: 20px; |
|
|
border: 1px solid #e9ecef; |
|
|
} |
|
|
.model-meta p { margin-bottom: 0.5em; } |
|
|
.model-meta strong { color: #333; } |
|
|
/* Specific styling for chat template explanation */ |
|
|
.chat-template-info span { |
|
|
font-weight: bold; |
|
|
color: #0056b3; |
|
|
} |
|
|
</style> |
