mt5-small-finetuned-norwegian

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.5625
• Rouge1: 15.482
• Rouge2: 4.6047
• Rougel: 12.9728
• Rougelsum: 14.1462

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.8871	1.0	18	3.6566	10.3421	1.0443	8.0244	8.9345
2.8986	2.0	36	3.6589	14.5319	4.3193	12.1301	13.4717
2.9202	3.0	54	3.6687	17.1853	5.7405	14.1799	16.5628
2.9672	4.0	72	3.6369	16.4974	5.7616	13.6745	15.2608
3.0131	5.0	90	3.6205	13.6482	3.6365	10.9636	12.1308
3.1446	6.0	108	3.6004	14.0123	3.1109	11.0727	12.7497
3.3982	7.0	126	3.5745	16.7604	5.3651	13.9214	15.5052
3.4486	8.0	144	3.5625	15.482	4.6047	12.9728	14.1462

Framework versions

• Transformers 4.51.3
• Pytorch 2.5.1+cu124
• Datasets 3.5.0
• Tokenizers 0.21.0