🧠 mindBERT - Mental Health Text Classification

📌 Model Description

mindBERT is a fine-tuned BERT-based model designed for mental health text classification. It can classify text into stress, depression, bipolar disorder, personality disorder, and anxiety with high accuracy. The model was trained on real-world mental health discussions from Reddit.

🔗 Try the Interactive UI: Hugging Face Spaces

---

📊 Training and Evaluation

Training Loss & Learning Rate

Training Summary

Epoch	Training Loss	Validation Loss	Accuracy
1	0.359400	0.285864	89.61%
2	0.210500	0.224632	92.03%
3	0.177800	0.217146	92.83%
4	0.089200	0.249640	93.23%
5	0.087600	0.282782	93.39%

Confusion Matrix

Dataset Label Distribution

Evaluation Metrics (Loss & Accuracy)

🔬 Full Weights & Biases Evaluation

🔗 View Detailed W&B Logs

---

🛠 How to Use

To use this model for inference:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_name = "DrSyedFaizan/mindBERT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I feel so anxious and stressed all the time."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    logits = model(**inputs).logits
    prediction = torch.argmax(logits, dim=1).item()

labels = ["Stress", "Depression", "Bipolar", "Personality Disorder", "Anxiety"]
print(f"Predicted Category: {labels[prediction]}")

---

🔧 Training Parameters

training_args = TrainingArguments(
    output_dir="./results",          # Output directory
    evaluation_strategy="epoch",     # Evaluate once per epoch
    save_strategy="epoch",           # Save at each epoch
    learning_rate=2e-5,               # Learning rate
    per_device_train_batch_size=16,   # Training batch size
    per_device_eval_batch_size=16,    # Evaluation batch size
    num_train_epochs=5,               # Training epochs
    weight_decay=0.01,                # Weight decay
    logging_steps=10,                 # Logging frequency
    lr_scheduler_type="linear",      # Learning rate scheduler
    warmup_steps=500,                 # Warmup steps
    load_best_model_at_end=True,      # Load best model after training
    metric_for_best_model="eval_loss",
    save_total_limit=3,               # Save up to 3 checkpoints
    gradient_accumulation_steps=2,    # Larger batch size simulation
    report_to="wandb"                 # Log to Weights & Biases
)

---

📌 Future Improvements

• Train with larger datasets like CLPsych, eRisk.
• Expand categories for broader mental health classification.
• Deploy as an API for real-world use cases.

💡 mindBERT - Advancing AI for Mental Health Research! 🚀