---
license: apache-2.0
tags:
  - mental-health
  - diagnosis
  - text-generation
  - gemma
  - qlora
  - transformers
  - huggingface
datasets:
  - Jaamie/mental-health-custom-dataset
pipeline_tag: text-generation
language:
  - en
base_model: google/gemma-2-9b-it
library_name: peft
---

# 🧠 Gemma Mental Health QLoRA v2

A fine-tuned version of `google/gemma-2-9b-it` for **mental health diagnosis** using instruction-style QLoRA tuning. This model takes in user statements and predicts the most likely mental disorder in a structured dialogue format.

---

## 🔧 Model Details

- **Base Model**: [`google/gemma-2-9b-it`](https://huggingface.co/google/gemma-2-9b-it)
- **Fine-Tuning Method**: QLoRA (4-bit quantization with `bitsandbytes`)
- **Tokenizer**: ✅ Included
- **LoRA Target Modules**: `["q_proj", "k_proj", "v_proj", "o_proj"]`
- **Sequence Format**:

## Output format

- User: <statement> Diagnosed Mental Disorder: <Predicted_Mental_Health>

---

## 🧪 Use Cases

- 🧠 Mental health Q&A assistant
- 🗨️ Conversational diagnosis suggestion
- 📚 NLP research and experimentation

> ⚠️ **Disclaimer**: This model is for research and educational purposes **only**. It is **not** intended for use in real-world clinical diagnosis without medical supervision.

---

## 💻 How to Use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer and base + adapter model
tokenizer = AutoTokenizer.from_pretrained("Jaamie/gemma_mental_health_qlora_v2")
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "Jaamie/gemma_mental_health_qlora_v2")

# Inference example
prompt = "User: I can't sleep and my thoughts are spiraling out of control.\nDiagnosed Mental Disorder:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
  outputs = model.generate(**inputs, max_new_tokens=30)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))


🏋️ Training Details
Epochs: 2

Batch Size: 4 (with gradient_accumulation_steps = 2)

Max Length: 512

Quantization: 4-bit QLoRA (NF4) with bitsandbytes

Precision: bf16


# Evaluation Results

Metric	         Score
Training Loss	 3.74
Validation Loss	 3.79
Total Examples	 ~22,000


The LLM has been trained on a sample of data from the dataset containing balanced instruction-style dataset with labeled disorders.

Mental Health Class	Sample Count
Depression	4,000
Anxiety	4,000
Suicidal Thoughts	3,000
Personality Disorder	2,000
Bipolar	2,000
Stress	2,000
Normal	5,000

# Contact
Created by Jaamie Maarsh Joy Martin

🌐 https://www.linkedin.com/in/jaamie-maarsh-joy-martin/

📧 jaamiemaarsh@gmail.com