Hrishikesh4/mental-health-classifier-longformer

Mental Health Sentiment Analyzer

Model Information

• Model Name: Hrishikesh4/mental-health-classifier-longformer
• Base Model: AllenAI/longformer-base-4096
• Model Type: Longformer-4096-based, fine-tuned for mental health classification
• Language: English
• License: MIT

Overview

The Mental Health Sentiment Analyzer is a specialized text classification model designed to detect early signs of depression, anxiety, stress, and neutral mental states from user-provided journals, reflections, or messages. Leveraging Longformer’s extended context window, it handles long, complex narratives for accurate detection of subtle and overlapping emotional states.

The model addresses the challenge of recognizing mental health struggles in digital interactions. It analyzes voluntary text inputs—such as journals, self-reflections, or chatbot conversations—to identify patterns of stress, anxiety, or depression. Depending on user consent, it can provide self-help recommendations, notify a therapist, or alert an emergency contact. The goal is timely intervention with privacy and ethical use.

Model Details

• Fine-tuning Stages: Two-stage process (multi-class → multi-label)
• Purpose: Early detection of mental health risk indicators

Training Process

Stage 1: Multi-class Fine-tuning

• Datasets:
  • Kaggle’s Emotions dataset (sampled 15k)
  • Synthetic mental health journals (20k)
• Task: Single-label emotion classification
• Configuration:
  • Learning rate: 2e-5
  • Weight decay: 0.01
  • Epochs: 3
• Results: ~98% accuracy (train/validation); ~50% accuracy on longer Reddit examples which were multi-class labeled

Stage 2: Multi-label Fine-tuning

• Datasets:
  • Reddit emotional posts (17k, web scraped)
  • PsychForums dataset (2k, web scraped)
  • Synthetic neutral journals (500)
• Task: Multi-label classification (neutral, depressed, anxious, stressed)
• Configuration:
  • Learning rate: 2e-5
  • Weight decay: 0.02
  • Epochs: 2
• Results:
  • Macro F1: 0.8104
  • Weighted F1: 0.8132
  • Adapted test set F1: ~0.89

Model Architecture

Based on Longformer, which uses sparse attention for efficient processing of long texts. The model employs a multi-label classification head to capture overlapping emotional states in long-form entries.

Capabilities

• Classifies text into one or more states: neutral, depression, anxiety, stress
• Handles long journal entries and message histories

Usage

Load and use the model with Hugging Face Transformers:

from transformers import LongformerTokenizerFast, LongformerForSequenceClassification
import torch

model_name = "Hrishikesh4/mental-health-classifier-longformer"
model = LongformerForSequenceClassification.from_pretrained(model_name)
tokenizer = LongformerTokenizerFast.from_pretrained(model_name)

text = "I’ve been feeling overwhelmed with work and can’t sleep properly."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)

with torch.no_grad():
        outputs = model(**inputs)
        predictions = torch.sigmoid(outputs.logits)

print(predictions)

Example Queries

• “I feel like I’m drowning in stress lately.”
• “I’m anxious about meeting new people.”
• “Sometimes I just feel empty and hopeless.”
• “I’m doing okay today.”

Limitations

• Non-clinical tool; not a substitute for professional therapy or diagnosis
• May reflect biases in training data
• Complex emotions may not map cleanly to defined labels

Ethical Considerations

Use responsibly and communicate that this is not a medical tool. Users in distress should be directed to qualified professionals. Integrations should include user consent, privacy protection, and crisis resources.

Citation

If you use this model, please cite:

@misc{kurapati2025MentalHealthSentimentAnalyzer,
    author = {Hrishikesh Kurapati},
    title = {Mental Health Sentiment Analyzer (Longformer-based)},
    year = {2025},
    url = {https://huggingface.co/Hrishikesh4/mental-health-classifier-longformer}
}