rlimonta/wav2vec2-large-xlsr-53-coraa-portuguese

Wav2Vec2 Large XLSR-53 CORAA Portuguese - Ricardo Limonta

This model is a fine-tuned version of rlimonta/wav2vec2-large-xlsr-53-cv-portuguese on the CORAA-v1.1 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4876
• Wer: 0.2698

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10000
• num_epochs: 8
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.5629	0.8382	10000	0.8674	0.4162
1.3918	1.6763	20000	0.6962	0.3677
1.2866	2.5145	30000	0.6172	0.3453
1.206	3.3526	40000	0.6028	0.3318
1.1465	4.1908	50000	0.5664	0.3164
1.0939	5.0289	60000	0.5577	0.2996
1.0454	5.8671	70000	0.5243	0.2891
1.0032	6.7052	80000	0.5030	0.2783
0.9684	7.5434	90000	0.4876	0.2698

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1