Update README.md

README.md

@@ -1,22 +1,151 @@
 ---
 base_model: unsloth/mistral-7b-instruct-v0.2-bnb-4bit
 tags:
-- text-generation-inference
 - transformers
 - unsloth
 - mistral
 - trl
+- text-generation
+- text-generation-inference
 license: apache-2.0
 language:
 - en
+metrics:
+- accuracy
+- bertscore
+- f1
+- recall
+- precision
+library_name: transformers
 ---
+# Mistral 7b instruct 
+This model is a fine-tuned version of [mistral-7b-instruct-v0.2-bnb-4bit](https://huggingface.co/unsloth/mistral-7b-instruct-v0.2-bnb-4bit) on the EngSaf dataset for Automatic Essay Grading.
+
+Robust performance on tasks involving Automatic Essay Grading to give a score and rationale
+
+It achieves the following results on the evaluation set:
+
+  - Loss: 1.3612
+  - Score Precision: 0.333
+  - Score Recall: 0.308
+  - Score F1: 0.3106
+  - Score Accuracy: 0.311
+-------------------------------
+  - Rationale Precision: 0.546
+  - Rationale Recall: 0.549
+  - Rationale F1: 0.5467
+
+## Model Details
+- Base Model: Mistral 7B: https://arxiv.org/abs/2310.06825
+- Fine-tuning Dataset: EngSaf: https://arxiv.org/abs/2407.12818.
+- Task: Automatic Essay Grading
+
+## Training Data
+The model is fine-tuned in the EngSaf dataset, curated for Automatic Essay Grading.
+EngSaf consists of student responses annotated with
+
+- Questions: Typically short-answer or essay-type.
+- Correct Answer: answers provided by teachers.
+- Student Answers: Actual responses written by students.
+- Output Label: The actual student score.
+- Feedback: Explanations justifying the given scores.
+
+## Example Usage
+Below is an example of how to use the model with the Hugging Face Transformers library:
+
+```python
+import torch
+from unsloth import FastLanguageModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model, tokenizer = FastLanguageModel.from_pretrained(model_name="amjad-awad/mistral-7b-instruct-v0.2-bnb-4bit-EngSaf-96K-lr-1e5",max_seq_length=2048,load_in_4bit=True)
+
+model = FastLanguageModel.get_peft_model(
+    model,
+    r=16,
+    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
+    lora_alpha=16,
+    lora_dropout=0,
+    bias="none",
+    use_gradient_checkpointing=True,
+    random_state=3407,
+)
+
+user_content = (
+    "Provide both a score and a rationale by evaluating the student's answer strictly within the mark scheme range, "
+    "grading based on how well it meets the question's requirements by comparing the student answer to the reference answer.\n"
+    "Question: What is photosynthesis?\n"
+    "Reference Answer: Photosynthesis is the process by which green plants and some other organisms use sunlight to synthesize nutrients from carbon dioxide and water. It generally involves the green pigment chlorophyll and generates oxygen as a by-product.\n"
+    "Student Answer: Photosynthesis is how plants make their food using sunlight and carbon dioxide. It also gives off oxygen.\n"
+    "Mark Scheme: {'1':'Mentions use of sunlight', '2':'Mentions carbon dioxide and water', '3':'Mentions production of oxygen', '4':'Explains synthesis of nutrients or food', '5':'Mentions chlorophyll or green pigment'}"
+)
+
+user = [
+    {"role":"system", "content": "You are a grading assistant. Evaluate student answers based on the mark scheme. Respond only in JSON format with keys 'score' (int) and 'rationale' (string)."},
+    {"role":"user", "content": user_content},
+      ]
+
+inputs = tokenizer.apply_chat_template(user, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
+generated_ids = model.generate(**inputs, max_new_tokens=128, temperature=0.2, top_k=5, do_sample=False)[0]
+new_generated_ids = generated_ids[inputs["input_ids"].shape[1]:]
+generated_text = tokenizer.decode(new_generated_ids, skip_special_tokens=True)
+
+print(generated_text)
+```
+
+Results:
+```
+{"score": 5, "rationale": "Your answer is correct. You have accurately described the process of photosynthesis, mentioning the use of sunlight, carbon dioxide, and water, and the production of food and oxygen as by-products. Keep up the good work!"}
+```
+
+
+## Training hyperparameters
+
+The following hyperparameters were used during training:
+- per_device_train_batch_size:1
+- per_device_eval_batch_size:1
+- gradient_accumulation_steps:8
+- eval_strategy:"steps"
+- save_strategy:"steps"
+- eval_steps:10
+- logging_dir:"./logs"
+- logging_steps:10
+- save_total_limit:1
+- learning_rate:1e-5
+- warmup_steps:100
+- weight_decay:0.01
+- num_train_epochs:3
+- load_best_model_at_end:True
+- lr_scheduler_type:"cosine"
+- metric_for_best_model:"eval_loss"
+- greater_is_better:False
 
-# Uploaded  model
+## Training results
 
-- **Developed by:** amjad-awad
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/mistral-7b-instruct-v0.2-bnb-4bit
+| Step | Training Loss | Validation Loss |
+|------|----------------|------------------|
+| 10   | 3.249500       | 3.304678         |
+| 20   | 3.251500       | 3.268980         |
+| 30   | 3.217000       | 3.200722         |
+| 40   | 3.075900       | 3.100004         |
+| 50   | 3.016900       | 2.967428         |
+| 60   | 2.841200       | 2.804593         |
+| 70   | 2.707600       | 2.622378         |
+| 80   | 2.545500       | 2.426775         |
+| 90   | 2.301500       | 2.212558         |
+| 100  | 2.071500       | 1.997747         |
+| 110  | 1.893100       | 1.798315         |
+| 120  | 1.705800       | 1.592137         |
+| 130  | 1.544700       | 1.481935         |
+| 140  | 1.467700       | 1.415517         |
+| 150  | 1.404500       | 1.377060         |
+| 160  | 1.318000       | 1.357312         |
+| 170  | 1.401800       | 1.349880         |
+| 180  | 1.361200       | 1.347993         |
 
-This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Framework versions
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+- Transformers 4.51.3
+- Pytorch  2.7.0
+- Datasets 3.6.0
+- Unsloth 2025.5.6