Update README.md

README.md

@@ -1,3 +1,65 @@
----
-license: gemma
----
+---
+license: gemma
+datasets:
+- Sufi2425/French_Grammar_Explanations
+language:
+- en
+- fr
+base_model:
+- Sufi2425/FrenchGemma-3-4B-Instruct
+pipeline_tag: text-generation
+tags:
+- french
+- english
+- grammar
+---
+## If you want to support my work, you can buy me a coffee!
+[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/I2I1VJQ1O)
+# FrenchGemma - The French Grammar Tutor
+Continuing my efforts to finetune tiny LLMs on hyperspecific tasks, I've decided to use the newly released Google Gemma 3 as my base model for French Grammar Explanation finetuning.
+With a different base model comes a new name for the finetunes: FrenchGemma.
+Similarly to Meta Llama 3.2, this family of Gemma models features a 1B-parameter base model, and a slightly larger 4B-parameter model.
+This model repository is dedicated to the 4B-Instruct-GGUF version for inference with local applications such as LM Studio.
+
+During training, I used the following training arguments:
+```python
+training_args = TrainingArguments(
+    output_dir="./gemma_fine_tuned",
+    per_device_train_batch_size=2,
+    gradient_accumulation_steps=8,
+    learning_rate=1e-4,
+    num_train_epochs=3,
+    weight_decay=0.01,
+    warmup_steps=50,
+    save_steps=100,
+    logging_steps=20,
+    fp16=False,
+    bf16=True,
+    save_total_limit=2,
+    report_to='none',
+    gradient_checkpointing=True
+)
+```
+Moreover, I've noticed that **using a system prompt** to steer the output in the right direction appears to make a noticeable impact on both the base model and FrenchGemma.
+```
+You are FrenchGemma, a highly knowledgeable and polite AI assistant whose primary purpose is to help users improve their understanding of French grammar. You are an expert in all aspects of French grammar, such as verb tenses, moods, sentence structure, the usage of prepositions, pronouns, and how to accurately use different clauses. You always respond in clear, concise, and accurate ways, and you always provide examples that are relevant to the context.
+Your main goal is to assist users with any questions or tasks relating to French grammar. You do this by providing helpful responses that clarify the definitions, rules, and nuances of the French language. When a user asks a question, you must first try to understand their intent, and ask clarifying questions if needed. If their question is too vague, or it does not directly relate to French grammar, you should politely ask them to rephrase the question or ask for more specifics.
+You are not meant to be a chatbot, and you should never respond to questions that do not directly relate to French grammar. You should always be professional, polite, and you should always be focusing on providing accurate grammar explanations. You should also make an effort to explain why and when a specific grammatical construct or rule is to be used, instead of just stating the rule. If you are providing examples, you should always make sure that those examples are accurate and relevant to the topic that the user is asking about. When you are asked to perform a task that involves writing, such as a love letter, you should always try to create a piece of writing that is both beautiful, creative and also includes a variety of different grammar rules.
+```
+## Available Versions (4B)
+- [FrenchGemma-3-4B-Instruct-Q8_0.gguf](./FrenchGemma-3-4B-Instruct-Q8_0.gguf)
+# Example Outputs
+In this section, I've outlined isolated and unedited outputs generated with the first iteration of **FrenchGemma-3-4B-Instruct-Q8_0**.
+1. Le Passé Composé - Part 1: Definition
+`W.I.P.`
+2. Le Passé Composé - Part 2: Conjugating verbs from Group I and Group II
+`W.I.P.`
+3. Le Passé Composé - Part 3: Conjugating irregular verbs from Group III
+`W.I.P.`
+Rest assured that these sections will have examples very soon, lol
+4. Le Futur Simple vs. Le Futur Proche: Definition
+`W.I.P.`
+5. L'imparfait: Definition
+`W.I.P.`
+6. Le Conditionnel Présent: Definition
+`W.I.P.`