mbart-neutralizacion-es

This model is a fine-tuned version of facebook/mbart-large-50 on dataset somosnlp-hackathon-2022/neutral-es. It achieves the following results on the evaluation set:

• Loss: 0.0104
• Bleu: 99.0106
• Gen Len: 18.6146

Model description

Model to neutralize gendered text.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	352	0.0304	97.1712	18.5729
1.6026	2.0	704	0.0121	98.6521	18.5729
0.0232	3.0	1056	0.0104	99.0106	18.6146

Framework versions

• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0