Add a multitask trained model and model card

.gitignore

@@ -0,0 +1,2 @@
+.ipynb_checkpoints/
+

README.md

@@ -1,3 +1,138 @@
 ---
 license: cc-by-4.0
+language: ti
+widget:
+- text: "<text-to-classify>"
+datasets:
+- fgaim/tigrinya-abusive-language-detection
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: tiroberta-tiald-all-tasks
+  results:
+  - task:
+      name: Text Classification
+      type: text-classification
+    metrics:
+    - name: Abu Accuracy
+      type: accuracy
+      value: 0.8611111111111112
+    - name: F1
+      type: f1
+      value: 0.8611109396431353
+    - name: Precision
+      type: precision
+      value: 0.8611128943846637
+    - name: Recall
+      type: recall
+      value: 0.8611111111111112
 ---
+
+
+# TiRoBERTa Fine-tuned for Multi-task Abusiveness, Sentiment, and Topic Classification
+
+This model is a fine-tuned version of [TiRoBERTa](https://huggingface.co/fgaim/tiroberta-base) on the [TiALD](https://huggingface.co/datasets/fgaim/tigrinya-abusive-language-detection) dataset.
+
+**Tigrinya Abusive Language Detection (TiALD) Dataset** is a large-scale, multi-task benchmark dataset for abusive language detection in the Tigrinya language. It consists of **13,717 YouTube comments** annotated for **abusiveness**, **sentiment**, and **topic** tasks. The dataset includes comments written in both the **Ge’ez script** and prevalent non-standard Latin **transliterations** to mirror real-world usage.
+
+> ⚠️ The dataset contains explicit, obscene, and potentially hateful language. It should be used for research purposes only. ⚠️
+
+This work accompanies the paper ["A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings"](https://arxiv.org/abs/2505.12116).
+
+## Model Usage
+
+```python
+from transformers import pipeline
+
+tiald_multitask = pipeline("text-classification", model="fgaim/tiroberta-tiald-all-tasks", top_k=11)
+tiald_multitask("<text-to-classify>")
+```
+
+### Performance Metrics
+
+This model achieves the following results on the TiALD test set:
+
+```json
+"abusiveness_metrics": {
+    "accuracy": 0.8611111111111112,
+    "macro_f1": 0.8611109396431353,
+    "macro_recall": 0.8611111111111112,
+    "macro_precision": 0.8611128943846637,
+    "weighted_f1": 0.8611109396431355,
+    "weighted_recall": 0.8611111111111112,
+    "weighted_precision": 0.8611128943846637
+},
+"topic_metrics": {
+    "accuracy": 0.6155555555555555,
+    "macro_f1": 0.5491185274678864,
+    "macro_recall": 0.5143416011263588,
+    "macro_precision": 0.7341640739780486,
+    "weighted_f1": 0.5944096153417657,
+    "weighted_recall": 0.6155555555555555,
+    "weighted_precision": 0.6870800624645906
+},
+"sentiment_metrics": {
+    "accuracy": 0.6533333333333333,
+    "macro_f1": 0.5340845253007789,
+    "macro_recall": 0.5410170159158625,
+    "macro_precision": 0.534652401599494,
+    "weighted_f1": 0.6620101614004723,
+    "weighted_recall": 0.6533333333333333,
+    "weighted_precision": 0.6750245466592532
+}
+```
+
+## Training Hyperparameters
+
+The following hyperparameters were used during training:
+
+- learning_rate: 3e-05
+- train_batch_size: 8
+- optimizer: Adam (betas=0.9, 0.999, epsilon=1e-08)
+- lr_scheduler_type: linear
+- num_epochs: 7.0
+- seed: 42
+
+## Intended Usage
+
+The TiALD dataset and models designed to support:
+
+- Research in abusive language detection in low-resource languages
+- Context-aware abuse, sentiment, and topic modeling
+- Multi-task and transfer learning with digraphic scripts
+- Evaluation of multilingual and fine-tuned language models
+
+Researchers and developers should avoid using this dataset for direct moderation or enforcement tasks without human oversight.
+
+## Ethical Considerations
+
+- **Sensitive content**: Contains toxic and offensive language. Use for research purposes only.
+- **Cultural sensitivity**: Abuse is context-dependent; annotations were made by native speakers to account for cultural nuance.
+- **Bias mitigation**: Data sampling and annotation were carefully designed to minimize reinforcement of stereotypes.
+- **Privacy**: All the source content for the dataset is publicly available on YouTube.
+- **Respect for expression**: The dataset should not be used for automated censorship without human review.
+
+This research received IRB approval (Ref: KH2022-133) and followed ethical data collection and annotation practices, including informed consent of annotators.
+
+## Citation
+
+If you use this model or the `TiALD` dataset in your work, please cite:
+
+```bibtex
+@misc{gaim-etal-2025-tiald-benchmark,
+  title         = {A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings},
+  author        = {Fitsum Gaim and Hoyun Song and Huije Lee and Changgeon Ko and Eui Jun Hwang and Jong C. Park},
+  year          = {2025},
+  eprint        = {2505.12116},
+  archiveprefix = {arXiv},
+  primaryclass  = {cs.CL},
+  url           = {https://arxiv.org/abs/2505.12116}
+}
+```
+
+## License
+
+This dataset is released under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).

best_trial.json

@@ -0,0 +1,5 @@
+{
+  "learning_rate": 3e-05,
+  "train_batch_size": 8,
+  "num_train_epochs": 7
+}

config.json

@@ -0,0 +1,53 @@
+{
+  "architectures": [
+    "RobertaForMultiLabelSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "Abusive",
+    "1": "Not Abusive",
+    "2": "Political",
+    "3": "Racial",
+    "4": "Religious",
+    "5": "Sexist",
+    "6": "Other Topic",
+    "7": "Positive",
+    "8": "Neutral",
+    "9": "Negative",
+    "10": "Mixed Sentiment"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "Abusive": 0,
+    "Not Abusive": 1,
+    "Political": 2,
+    "Racial": 3,
+    "Religious": 4,
+    "Sexist": 5,
+    "Other Topic": 6,
+    "Positive": 7,
+    "Neutral": 8,
+    "Negative": 9,
+    "Mixed Sentiment": 10
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

eval_results.txt

@@ -0,0 +1,2 @@
+LRAP = 0.7497833627278061
+eval_loss = 10.723778989579943

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a87fe5bc9d0030692331fef5155bc2980213d4e1e1a363a86202755c417e3c42
+size 501003076

model_args.json

@@ -0,0 +1 @@
+{"adafactor_beta1": null, "adafactor_clip_threshold": 1.0, "adafactor_decay_rate": -0.8, "adafactor_eps": [1e-30, 0.001], "adafactor_relative_step": true, "adafactor_scale_parameter": true, "adafactor_warmup_init": true, "adam_betas": [0.9, 0.999], "adam_epsilon": 1e-08, "best_model_dir": "outputs/best_model", "cache_dir": "cache_dir/", "config": {}, "cosine_schedule_num_cycles": 0.5, "custom_layer_parameters": [], "custom_parameter_groups": [], "dataloader_num_workers": 0, "do_lower_case": false, "dynamic_quantize": false, "early_stopping_consider_epochs": false, "early_stopping_delta": 0, "early_stopping_metric": "macro_f1", "early_stopping_metric_minimize": false, "early_stopping_patience": 3, "encoding": null, "eval_batch_size": 100, "evaluate_during_training": true, "evaluate_during_training_silent": true, "evaluate_during_training_steps": 2000, "evaluate_during_training_verbose": false, "evaluate_each_epoch": true, "fp16": true, "gradient_accumulation_steps": 1, "learning_rate": 3e-05, "local_rank": -1, "logging_steps": 1, "loss_type": null, "loss_args": {}, "manual_seed": null, "max_grad_norm": 1.0, "max_seq_length": 256, "model_name": "fgaim/tiroberta-base", "model_type": "roberta", "multiprocessing_chunksize": -1, "n_gpu": 1, "no_cache": false, "no_save": false, "not_saved_args": [], "num_train_epochs": 7, "optimizer": "AdamW", "output_dir": "models/tiroberta-base", "overwrite_output_dir": true, "polynomial_decay_schedule_lr_end": 1e-07, "polynomial_decay_schedule_power": 1.0, "process_count": 6, "quantized_model": false, "reprocess_input_data": true, "save_best_model": true, "save_eval_checkpoints": false, "save_model_every_epoch": false, "save_optimizer_and_scheduler": true, "save_steps": 2000, "scheduler": "linear_schedule_with_warmup", "silent": false, "skip_special_tokens": true, "tensorboard_dir": null, "thread_count": null, "tokenizer_name": null, "tokenizer_type": null, "train_batch_size": 8, "train_custom_parameters_only": false, "trust_remote_code": false, "use_cached_eval_features": false, "use_early_stopping": true, "use_hf_datasets": false, "use_multiprocessing": false, "use_multiprocessing_for_evaluation": false, "wandb_kwargs": {"job_type": "training", "name": "tiroberta-base-20250510_063424"}, "wandb_project": "tiald-joint-labels", "warmup_ratio": 0.1, "warmup_steps": 1078, "weight_decay": 0.01, "model_class": "MultiLabelClassificationModel", "sliding_window": false, "stride": 0.8, "threshold": 0.5, "tie_value": 1, "labels_list": [], "labels_map": {}, "lazy_loading": false, "special_tokens_list": []}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "do_lower_case": false,
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "unk_token": "<unk>"
+}

training_progress_scores.csv

@@ -0,0 +1,13 @@
+global_step,train_loss,LRAP,accuracy,macro_f1,weighted_f1,eval_loss
+1540,5.5235595703125,0.7087386163219498,0.0022222222222222222,0.5276624797769354,0.6418452955098061,6.762324757046169
+2000,3.336136817932129,0.713298140131474,0.0033333333333333335,0.5490219749315628,0.6538235528242954,6.732627603742811
+3080,5.402691841125488,0.7355552081663191,0.0011111111111111111,0.5690883792182807,0.6738836283404934,5.939486159218682
+4000,4.94577693939209,0.7516409064186831,0.005555555555555556,0.5857144395278816,0.6937109275167757,6.191740194956462
+4620,4.7006120681762695,0.7349451258617915,0.0022222222222222222,0.5808680848220031,0.6845453635469864,6.53547477722168
+6000,0.8756833076477051,0.7496383277216598,0.028888888888888888,0.5848889297971004,0.6915240625034748,7.2687596744961205
+6160,2.8979897499084473,0.7404720218053544,0.03888888888888889,0.5797768839785963,0.6882614706527342,7.9485422770182295
+7700,0.5499695539474487,0.7383599353321573,0.06444444444444444,0.5910478801573911,0.6963464661450791,8.710189289516872
+8000,0.31079328060150146,0.7527138848805496,0.09222222222222222,0.5948021214697299,0.7003535118782406,9.066656377580431
+9240,0.6766080856323242,0.7457099166265829,0.12555555555555556,0.5990810767553655,0.7036516907901794,10.211644013722738
+10000,1.856939435005188,0.7473248490192927,0.13444444444444445,0.5910801083627714,0.7000364330564518,10.624637179904514
+10780,0.12249794602394104,0.7497833627278061,0.14888888888888888,0.5969603348439917,0.7040227106765218,10.723778989579943