|
|
--- |
|
|
language: en |
|
|
license: mit |
|
|
pipeline_tag: text-classification |
|
|
tags: |
|
|
- text-classification |
|
|
- transformers |
|
|
- pytorch |
|
|
- multi-label-classification |
|
|
- multi-class-classification |
|
|
- emotion |
|
|
- bert |
|
|
- go_emotions |
|
|
- emotion-classification |
|
|
datasets: |
|
|
- google-research-datasets/go_emotions |
|
|
metrics: |
|
|
- f1 |
|
|
- precision |
|
|
- recall |
|
|
widget: |
|
|
- text: I’m just chilling today. |
|
|
example_title: Neutral Example |
|
|
- text: Thank you for saving my life! |
|
|
example_title: Gratitude Example |
|
|
- text: I’m nervous about my exam tomorrow. |
|
|
example_title: Nervousness Example |
|
|
base_model: |
|
|
- google-bert/bert-base-uncased |
|
|
--- |
|
|
|
|
|
# GoEmotions BERT Classifier |
|
|
|
|
|
Fine-tuned [BERT-base-uncased](https://huggingface.co/bert-base-uncased) on [go_emotions](https://huggingface.co/datasets/go_emotions) for multi-label classification (28 emotions). |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Architecture**: BERT-base-uncased (110M parameters) |
|
|
- **Training Data**: [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) (58k Reddit comments, 28 emotions) |
|
|
- **Loss Function**: Focal Loss (gamma=2) |
|
|
- **Optimizer**: AdamW (lr=2e-5, weight_decay=0.01) |
|
|
- **Epochs**: 5 |
|
|
- **Hardware**: Kaggle T4 x2 GPUs |
|
|
|
|
|
## Try It Out |
|
|
For accurate predictions with optimized thresholds, use the [Gradio demo](https://logasanjeev-goemotions-bert-demo.hf.space). |
|
|
|
|
|
## Performance |
|
|
|
|
|
- **Micro F1**: 0.6025 (optimized thresholds) |
|
|
- **Macro F1**: 0.5266 |
|
|
- **Precision**: 0.5425 |
|
|
- **Recall**: 0.6775 |
|
|
- **Hamming Loss**: 0.0372 |
|
|
- **Avg Positive Predictions**: 1.4564 |
|
|
|
|
|
### Class-Wise Performance |
|
|
The following table shows per-class metrics on the test set using optimized thresholds (see `thresholds.json`): |
|
|
|
|
|
| Emotion | F1 Score | Precision | Recall | Support | |
|
|
|----------------|----------|-----------|--------|---------| |
|
|
| admiration | 0.7022 | 0.6980 | 0.7063 | 504 | |
|
|
| amusement | 0.8171 | 0.7692 | 0.8712 | 264 | |
|
|
| anger | 0.5123 | 0.5000 | 0.5253 | 198 | |
|
|
| annoyance | 0.3820 | 0.2908 | 0.5563 | 320 | |
|
|
| approval | 0.4112 | 0.3485 | 0.5014 | 351 | |
|
|
| caring | 0.4601 | 0.4045 | 0.5333 | 135 | |
|
|
| confusion | 0.4488 | 0.4533 | 0.4444 | 153 | |
|
|
| curiosity | 0.5721 | 0.4402 | 0.8169 | 284 | |
|
|
| desire | 0.4068 | 0.6857 | 0.2892 | 83 | |
|
|
| disappointment | 0.3476 | 0.3220 | 0.3775 | 151 | |
|
|
| disapproval | 0.4126 | 0.3433 | 0.5169 | 267 | |
|
|
| disgust | 0.4950 | 0.6329 | 0.4065 | 123 | |
|
|
| embarrassment | 0.5000 | 0.7368 | 0.3784 | 37 | |
|
|
| excitement | 0.4084 | 0.4432 | 0.3786 | 103 | |
|
|
| fear | 0.6311 | 0.5078 | 0.8333 | 78 | |
|
|
| gratitude | 0.9173 | 0.9744 | 0.8665 | 352 | |
|
|
| grief | 0.2500 | 0.5000 | 0.1667 | 6 | |
|
|
| joy | 0.6246 | 0.5798 | 0.6770 | 161 | |
|
|
| love | 0.8110 | 0.7630 | 0.8655 | 238 | |
|
|
| nervousness | 0.3830 | 0.3750 | 0.3913 | 23 | |
|
|
| optimism | 0.5777 | 0.5856 | 0.5699 | 186 | |
|
|
| pride | 0.4138 | 0.4615 | 0.3750 | 16 | |
|
|
| realization | 0.2421 | 0.5111 | 0.1586 | 145 | |
|
|
| relief | 0.5385 | 0.4667 | 0.6364 | 11 | |
|
|
| remorse | 0.6797 | 0.5361 | 0.9286 | 56 | |
|
|
| sadness | 0.5391 | 0.6900 | 0.4423 | 156 | |
|
|
| surprise | 0.5724 | 0.5570 | 0.5887 | 141 | |
|
|
| neutral | 0.6895 | 0.5826 | 0.8444 | 1787 | |
|
|
|
|
|
## Usage |
|
|
|
|
|
The model uses optimized thresholds stored in `thresholds.json` for predictions. Example in Python: |
|
|
|
|
|
```python |
|
|
from transformers import BertForSequenceClassification, BertTokenizer |
|
|
import torch |
|
|
import json |
|
|
import requests |
|
|
|
|
|
# Load model and tokenizer |
|
|
repo_id = "logasanjeev/goemotions-bert" |
|
|
model = BertForSequenceClassification.from_pretrained(repo_id) |
|
|
tokenizer = BertTokenizer.from_pretrained(repo_id) |
|
|
|
|
|
# Load thresholds |
|
|
thresholds_url = f"https://huggingface.co/{repo_id}/raw/main/thresholds.json" |
|
|
thresholds_data = json.loads(requests.get(thresholds_url).text) |
|
|
emotion_labels = thresholds_data["emotion_labels"] |
|
|
thresholds = thresholds_data["thresholds"] |
|
|
|
|
|
# Predict |
|
|
text = "I’m just chilling today." |
|
|
encodings = tokenizer(text, padding='max_length', truncation=True, max_length=128, return_tensors='pt') |
|
|
with torch.no_grad(): |
|
|
logits = torch.sigmoid(model(**encodings).logits).numpy()[0] |
|
|
predictions = [(emotion_labels[i], logit) for i, (logit, thresh) in enumerate(zip(logits, thresholds)) if logit >= thresh] |
|
|
print(sorted(predictions, key=lambda x: x[1], reverse=True)) |
|
|
# Output: [('neutral', 0.8147)] |
