qingy2024
/

GRMR-V3-G1B

@@ -4,7 +4,7 @@ tags:
 - text-generation-inference
 - transformers
 - unsloth
-- gemma3_text
 - trl
 - sft
 license: apache-2.0
@@ -12,12 +12,292 @@ language:
 - en
 ---
-# Uploaded  model
-- **Developed by:** qingy2024
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/gemma-3-1b-pt
-This gemma3_text model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

 - text-generation-inference
 - transformers
 - unsloth
+- llama
 - trl
 - sft
 license: apache-2.0
 - en
 ---
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+</head>
+<div class="container">
+    <h1>GRMR-V3-G1B</h1>
+    <p>GRMR-V3-G1B is a fine-tuned version of <a href="https://huggingface.co/unsloth/gemma-3-1b-pt">unsloth/gemma-3-1b-pt</a> specifically optimized for grammar correction tasks.</p>
+    <div class="important-note">
+        <p><strong>IMPORTANT:</strong> Please ensure you are using the following sampler settings for optimal results:</p>
+        <pre><code>temperature = 0.7
+frequency_penalty = 0.0
+presence_penalty = 0.0
+min_p = 0.01
+top_p = 0.95
+top_k = 40</code></pre>
+    </div>
+    <h2>Model description</h2>
+    <p>GRMR-V3-G1B is a grammar correction model built on Gemma 3 1B (base model, no vision). It has been fine-tuned on a large dataset of grammar correction examples to help improve text quality by fixing grammatical errors, punctuation, spelling, and other language issues.</p>
+    <p>The model uses a specialized chat template that structures inputs as "text" and outputs as "corrected" to maintain a clear distinction between original and corrected content.</p>
+    <h3>Model Details</h3>
+    <div class="model-meta">
+        <p><strong>Developed by:</strong> qingy2024</p>
+        <p><strong>Finetuned from model:</strong> <a href="https://huggingface.co/unsloth/gemma-3-1b-pt">unsloth/gemma-3-1b-pt</a>
+        <p><strong>Model type:</strong> Gemma 3</p>
+        <p><strong>Language(s):</strong> English</p>
+        <p><strong>License:</strong> apache-2.0</p>
+        <p><strong>Training Dataset:</strong> <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a></p>
+    </div>
+    <hr>
+    <h2>Manual Testing Examples</h2>
+    <p>Here are a few examples of grammar corrections this model can handle:</p>
+    <table>
+        <thead>
+            <tr>
+                <th>Original Text</th>
+                <th>Corrected Text</th>
+            </tr>
+        </thead>
+        <tbody>
+            <tr>
+                <td>i dont know weather to bring a umbrella today</td>
+                <td>I don't know whether to bring an umbrella today.</td>
+            </tr>
+            <tr>
+                <td>she go to the store yesterday</td>
+                <td>She went to the store yesterday.</td>
+            </tr>
+            <tr>
+                <td>they is going to be late for the meeting</td>
+                <td>They are going to be late for the meeting.</td>
+            </tr>
+            <tr>
+                <td>the cat laying on the floor all day</td>
+                <td>The cat is laying on the floor all day.</td>
+            </tr>
+        </tbody>
+    </table>
+    <hr>
+    <h2>Training procedure</h2>
+    <p>The model was fine-tuned using full parameter fine-tuning (not LoRA) on the GRMR-V4-60K dataset. The training was optimized using the Unsloth framework for efficient training of LLMs.</p>
+    <h3>Training hyperparameters</h3>
+    <ul>
+        <li><strong>Batch size:</strong> 8</li>
+        <li><strong>Gradient accumulation steps:</strong> 2</li>
+        <li><strong>Learning rate:</strong> 5e-5</li>
+        <li><strong>Epochs:</strong> 1</li>
+        <li><strong>Optimizer:</strong> AdamW (8-bit)</li>
+        <li><strong>Weight decay:</strong> 0.01</li>
+        <li><strong>LR scheduler:</strong> Cosine</li>
+        <li><strong>Warmup steps:</strong> 180</li>
+        <li><strong>Max sequence length:</strong> 16,384</li>
+        <li><strong>Training precision:</strong> Mixed precision (BF16 where available, FP16 otherwise)</li>
+    </ul>
+    <h2>Intended uses & limitations</h2>
+    <p>This model is designed for grammar correction tasks. It can be used to:</p>
+    <ul>
+        <li>Fix grammatical errors in written text</li>
+        <li>Correct punctuation</li>
+        <li>Address spelling mistakes</li>
+        <li>Improve sentence structure and clarity</li>
+    </ul>
+    <h3>Limitations</h3>
+    <ul>
+        <li>The model may struggle with highly technical or domain-specific content</li>
+        <li>It may not fully understand context-dependent grammar rules in all cases</li>
+        <li>Performance may vary for non-standard English or text with multiple errors</li>
+    </ul>
+    <h2>How to use</h2>
+    <p><code>llama.cpp</code> and projects based on it should be able to run this model like any others.</p>
+    <p>For pure <code>transformers</code> code, you can refer here:</p>
+    <pre><code class="language-python">from transformers import AutoModelForCausalLM, AutoTokenizer
+# Load model and tokenizer
+model_name = "qingy2024/GRMR-V3-G1B"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+text_to_correct = "i am going to the store tommorow and buy some thing for dinner"
+messages = [
+    {"role": "user", "content": text_to_correct}
+]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    inputs["input_ids"],
+    max_new_tokens=512,
+    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
+    do_sample=True
+)
+corrected_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(corrected_text)</code></pre>
+    <h3>Using with the Hugging Face pipeline</h3><pre><code class="language-python">from transformers import pipeline
+pipe = pipeline(
+    "text-generation",
+    model="qingy2024/GRMR-V3-G1B",
+    torch_dtype="auto",
+    device_map="auto"
+)
+messages = [
+    {"role": "user", "content": "i dont know weather to bring a umbrella today"}
+]
+result = pipe(
+    messages,
+    max_new_tokens=100,
+    temperature=0.1, # NOTE: For best results, use the recommended temperature of 0.7
+    do_sample=True,
+    return_full_text=False
+)[0]["generated_text"]
+print(result)</code></pre><p><em>Note: The Python examples above use <code>temperature=0.1</code> for reproducibility in quick tests. For optimal grammar correction quality, please use the recommended sampler settings, especially <code>temperature=0.7</code>.</em></p><h2>Custom Chat Template</h2><p class="chat-template-info">The model uses a custom chat template with special formatting for grammar correction:</p><ul><li>User inputs are formatted with <code><|text_start|></code> and <code><|text_end|></code> tags</li><li>Model outputs are formatted with <code><|corrected_start|></code> and <code><|corrected_end|></code> tags</li></ul><p>The complete chat template is:</p><pre><code class="language-jinja">{{- bos_token }}
+{#- Process messages with role mapping #}
+{%- for message in messages %}
+{%- if message['role'] == 'user' %}
+    {{- '<start_of_turn>text
+'+ message['content'] | trim + '<end_of_turn>
+' }}
+{%- elif message['role'] == 'assistant' %}
+    {{- '<start_of_turn>corrected
+'+ message['content'] | trim + '<end_of_turn>
+' }}
+{%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+{{- '<start_of_turn>corrected
+' }}
+{%- endif %}</code></pre><h2>Training Dataset</h2><p>The model was fine-tuned on the <a href="https://huggingface.co/datasets/qingy2024/grmr-v4-60k">qingy2024/grmr-v4-60k</a> dataset, which contains 60,000 examples of original text and their grammatically corrected versions.</p><h2>Bias, Risks, and Limitations</h2><ul><li>The model may reflect biases present in the training data</li><li>It may not perform equally well across different writing styles or domains</li><li>The model might occasionally introduce errors or change the meaning of text</li><li>It focuses on grammatical correctness rather than stylistic improvements</li></ul><h2>Citations</h2><pre><code>@article{gemma_2025,
+    title={Gemma 3},
+    url={https://goo.gle/Gemma3Report},
+    publisher={Kaggle},
+    author={Gemma Team},
+    year={2025}
+}</code></pre><h2>Contact</h2><p>For questions or issues related to the model, please reach out via Hugging Face or by creating an issue in the repository.</p></div>
+<style>
+body {
+    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
+    line-height: 1.6;
+    margin: 0;
+    padding: 0;
+    background-color: #f8f9fa;
+    color: #333;
+}
+.container {
+    max-width: 1200px;
+    margin: 10px auto;
+    padding: 25px;
+    background-color: #ffffff;
+    border-radius: 8px;
+    box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08);
+}
+h1, h2, h3 {
+    color: #0056b3; /* Primary Blue */
+    margin-top: 1.5em;
+    margin-bottom: 0.7em;
+}
+h1 {
+    text-align: center;
+    font-size: 2.2em;
+    border-bottom: 2px solid #e0e0e0;
+    padding-bottom: 0.5em;
+    margin-top: 0;
+}
+h2 {
+    font-size: 1.8em;
+    border-bottom: 1px solid #e9ecef;
+    padding-bottom: 0.3em;
+}
+h3 {
+    font-size: 1.4em;
+    color: #007bff; /* Lighter Blue for sub-headings */
+}
+p, li {
+    font-size: 1em;
+    color: #555;
+}
+a {
+    color: #007bff;
+    text-decoration: none;
+}
+a:hover {
+    text-decoration: underline;
+    color: #0056b3;
+}
+.important-note {
+    background-color: #e7f3ff; /* Light blue background */
+    border-left: 5px solid #007bff; /* Blue accent border */
+    margin: 20px 0px;
+    border-radius: 5px;
+}
+.important-note strong {
+    color: #0056b3;
+    font-weight: 600;
+}
+.important-note {
+    background-color: #d0e8ff;
+    padding: 0.05em 1.0em;
+    border-radius: 3px;
+    font-size: 0.9em;
+}
+code {
+    padding: 0.1em 0.4em;
+    border-radius: 3px;
+    font-size: 0.9em;
+}
+table {
+    width: 100%;
+    border-collapse: collapse;
+    margin: 20px 0;
+    box-shadow: 0 2px 4px rgba(0,0,0,0.05);
+}
+th, td {
+    border: 1px solid #dee2e6;
+    padding: 10px 12px;
+    text-align: left;
+    vertical-align: top;
+}
+th {
+    background-color: #e9ecef; /* Light gray for headers */
+    font-weight: 600;
+    color: #212529;
+}
+td:first-child {
+    /* font-style: italic; */
+    color: #444;
+}
+pre {
+    background-color: #f1f3f5;
+    padding: 15px;
+    border-radius: 5px;
+    overflow-x: auto;
+    border: 1px solid #ced4da;
+    font-size: 0.9em;
+}
+code {
+    font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace;
+    background-color: #e9ecef;
+    padding: 0.2em 0.4em;
+    border-radius: 3px;
+    font-size: 0.9em;
+}
+pre code {
+    background-color: transparent;
+    padding: 0;
+    border-radius: 0;
+    font-size: 1em;
+}
+ul {
+    padding-left: 20px;
+}
+li {
+    margin-bottom: 0.5em;
+}
+hr {
+    border: none;
+    border-top: 1px solid #e0e0e0;
+    margin: 30px 0;
+}
+.model-meta {
+    background-color: #f8f9fa;
+    padding: 15px;
+    border-radius: 5px;
+    margin-bottom: 20px;
+    border: 1px solid #e9ecef;
+}
+.model-meta p { margin-bottom: 0.5em; }
+.model-meta strong { color: #333; }
+/* Specific styling for chat template explanation */
+.chat-template-info span {
+    font-weight: bold;
+    color: #0056b3;
+}
+</style>