README.md

---
license: cc-by-4.0
language: ti
widget:
- text: "<text-to-classify>"
datasets:
- fgaim/tigrinya-abusive-language-detection
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: tiroberta-tiald-all-tasks
  results:
  - task:
      name: Text Classification
      type: text-classification
    metrics:
    - name: Abu Accuracy
      type: accuracy
      value: 0.8611111111111112
    - name: F1
      type: f1
      value: 0.8611109396431353
    - name: Precision
      type: precision
      value: 0.8611128943846637
    - name: Recall
      type: recall
      value: 0.8611111111111112
---


# TiRoBERTa Fine-tuned for Multi-task Abusiveness, Sentiment, and Topic Classification

This model is a fine-tuned version of [TiRoBERTa](https://huggingface.co/fgaim/tiroberta-base) on the [TiALD](https://huggingface.co/datasets/fgaim/tigrinya-abusive-language-detection) dataset.

**Tigrinya Abusive Language Detection (TiALD) Dataset** is a large-scale, multi-task benchmark dataset for abusive language detection in the Tigrinya language. It consists of **13,717 YouTube comments** annotated for **abusiveness**, **sentiment**, and **topic** tasks. The dataset includes comments written in both the **Ge’ez script** and prevalent non-standard Latin **transliterations** to mirror real-world usage.

> ⚠️ The dataset contains explicit, obscene, and potentially hateful language. It should be used for research purposes only. ⚠️

This work accompanies the paper ["A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings"](https://arxiv.org/abs/2505.12116).

## Model Usage

```python
from transformers import pipeline

tiald_multitask = pipeline("text-classification", model="fgaim/tiroberta-tiald-all-tasks", top_k=11)
tiald_multitask("<text-to-classify>")
```

### Performance Metrics

This model achieves the following results on the TiALD test set:

```json
"abusiveness_metrics": {
    "accuracy": 0.8611111111111112,
    "macro_f1": 0.8611109396431353,
    "macro_recall": 0.8611111111111112,
    "macro_precision": 0.8611128943846637,
    "weighted_f1": 0.8611109396431355,
    "weighted_recall": 0.8611111111111112,
    "weighted_precision": 0.8611128943846637
},
"topic_metrics": {
    "accuracy": 0.6155555555555555,
    "macro_f1": 0.5491185274678864,
    "macro_recall": 0.5143416011263588,
    "macro_precision": 0.7341640739780486,
    "weighted_f1": 0.5944096153417657,
    "weighted_recall": 0.6155555555555555,
    "weighted_precision": 0.6870800624645906
},
"sentiment_metrics": {
    "accuracy": 0.6533333333333333,
    "macro_f1": 0.5340845253007789,
    "macro_recall": 0.5410170159158625,
    "macro_precision": 0.534652401599494,
    "weighted_f1": 0.6620101614004723,
    "weighted_recall": 0.6533333333333333,
    "weighted_precision": 0.6750245466592532
}
```

## Training Hyperparameters

The following hyperparameters were used during training:

- learning_rate: 3e-05
- train_batch_size: 8
- optimizer: Adam (betas=0.9, 0.999, epsilon=1e-08)
- lr_scheduler_type: linear
- num_epochs: 7.0
- seed: 42

## Intended Usage

The TiALD dataset and models designed to support:

- Research in abusive language detection in low-resource languages
- Context-aware abuse, sentiment, and topic modeling
- Multi-task and transfer learning with digraphic scripts
- Evaluation of multilingual and fine-tuned language models

Researchers and developers should avoid using this dataset for direct moderation or enforcement tasks without human oversight.

## Ethical Considerations

- **Sensitive content**: Contains toxic and offensive language. Use for research purposes only.
- **Cultural sensitivity**: Abuse is context-dependent; annotations were made by native speakers to account for cultural nuance.
- **Bias mitigation**: Data sampling and annotation were carefully designed to minimize reinforcement of stereotypes.
- **Privacy**: All the source content for the dataset is publicly available on YouTube.
- **Respect for expression**: The dataset should not be used for automated censorship without human review.

This research received IRB approval (Ref: KH2022-133) and followed ethical data collection and annotation practices, including informed consent of annotators.

## Citation

If you use this model or the `TiALD` dataset in your work, please cite:

```bibtex
@misc{gaim-etal-2025-tiald-benchmark,
  title         = {A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings},
  author        = {Fitsum Gaim and Hoyun Song and Huije Lee and Changgeon Ko and Eui Jun Hwang and Jong C. Park},
  year          = {2025},
  eprint        = {2505.12116},
  archiveprefix = {arXiv},
  primaryclass  = {cs.CL},
  url           = {https://arxiv.org/abs/2505.12116}
}
```

## License

This dataset is released under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).