wav2vec2-xlsr-53-ft-btb-cv-other-cy

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4934
• Wer: 0.3776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 800
• training_steps: 8000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.1054	0.0755	500	2.8161	1.0
1.4196	0.1509	1000	1.1687	0.7941
1.0028	0.2264	1500	0.9890	0.6953
0.8942	0.3019	2000	0.8935	0.6195
0.8285	0.3774	2500	0.8221	0.6075
0.763	0.4528	3000	0.7165	0.5307
0.7203	0.5283	3500	0.6892	0.5054
0.7051	0.6038	4000	0.6848	0.5070
0.6568	0.6792	4500	0.6342	0.4926
0.6315	0.7547	5000	0.5956	0.4494
0.6171	0.8302	5500	0.5533	0.4305
0.5717	0.9057	6000	0.5360	0.4212
0.5699	0.9811	6500	0.5184	0.4040
0.4905	1.0566	7000	0.5081	0.3967
0.4706	1.1321	7500	0.4991	0.3825
0.4614	1.2075	8000	0.4934	0.3776

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1