|
|
--- |
|
|
language: en |
|
|
datasets: |
|
|
- imdb |
|
|
library_name: transformers |
|
|
tags: |
|
|
- sentiment-analysis |
|
|
- roberta |
|
|
- bilstm |
|
|
- attention |
|
|
- sarcasm |
|
|
license: apache-2.0 |
|
|
pipeline_tag: text-classification |
|
|
model_type: roberta |
|
|
widget: |
|
|
- text: "This movie was a masterpiece!" |
|
|
example_title: Positive Example |
|
|
- text: "If I had a dollar for every cliché..." |
|
|
example_title: Sarcasm Example |
|
|
--- |
|
|
|
|
|
|
|
|
# 🤖 Sentiment Classifier with Sarcasm Detection |
|
|
|
|
|
This model combines `roberta-base` embeddings with a BiLSTM layer and attention mechanism to classify the **sentiment of movie reviews**, even when the language includes sarcasm, irony, or mixed emotional signals. |
|
|
|
|
|
## 🧩 Architecture |
|
|
|
|
|
- **Base Encoder:** RoBERTa (frozen) |
|
|
- **Sequence Modeling:** Bidirectional LSTM (BiLSTM) |
|
|
- **Attention Layer:** Learnable scalar attention weights |
|
|
- **Classifier:** Fully connected + Softmax |
|
|
|
|
|
This model takes the output of RoBERTa, processes it through a BiLSTM, applies attention to focus on important parts of the sentence, and classifies sentiment as **Positive (1)** or **Negative (0)**. |
|
|
|
|
|
## 💡 Why this model? |
|
|
|
|
|
Standard BERT-based sentiment classifiers often fail on subtle language such as: |
|
|
|
|
|
- *"If I had a dollar for every cliché in this film, I’d be rich."* → sarcastic → **Negative** |
|
|
- *"It’s fine. Just fine. Totally… fine."* → sarcastic tone → **Negative** |
|
|
- *"The visuals were stunning, but the plot was as dead as a doorknob."* → contrast → **Negative** |
|
|
|
|
|
This model aims to address that by combining contextual language modeling (RoBERTa) with sequence learning (BiLSTM) and attention-based interpretability. |
|
|
|
|
|
## 📊 Training Details |
|
|
|
|
|
- **Dataset:** IMDB (2-class, 25k reviews) |
|
|
- **Training Subset:** 2,000 samples for prototype |
|
|
- **Loss:** CrossEntropyLoss |
|
|
- **Optimizer:** Adam |
|
|
- **Epochs:** 3 |
|
|
- **Hardware:** Google Colab (T4 GPU) |
|
|
- **Additional Inputs:** Custom sarcastic and ambiguous examples |
|
|
|
|
|
## 🧪 Example Inputs |
|
|
|
|
|
| Input | Prediction | |
|
|
|-------|------------| |
|
|
| "If I had a dollar for every cliché..." | 🔴 Negative | |
|
|
| "It’s fine. Just fine. Totally fine." | 🔴 Negative | |
|
|
| "The acting was great, but the plot..." | 🔴 Negative | |
|
|
| "Brilliant visuals and solid pacing." | 🟢 Positive | |
|
|
|
|
|
## 🚀 How to Use |
|
|
|
|
|
```python |
|
|
from transformers import RobertaTokenizer |
|
|
from my_model import RobertaBiLSTMAttention # Your custom model |
|
|
import torch |
|
|
|
|
|
tokenizer = RobertaTokenizer.from_pretrained("roberta-base") |
|
|
model = RobertaBiLSTMAttention() |
|
|
model.load_state_dict(torch.load("pytorch_model.bin")) |
|
|
model.eval() |
|
|
|
|
|
text = "It’s fine. Just fine. Totally… fine." |
|
|
tokens = tokenizer(text, return_tensors="pt", padding="max_length", truncation=True, max_length=128) |
|
|
logits = model(tokens["input_ids"], tokens["attention_mask"]) |
|
|
pred = torch.argmax(logits, dim=1).item() |
|
|
print("Sentiment:", "Positive" if pred == 1 else "Negative") |
|
|
|
