w2v-bert-2.0-yoruba_naijavoices_250h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the NAIJAVOICES_YORUBA_250H - NA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4563
  • Wer: 0.4444
  • Cer: 0.2656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 160
  • eval_batch_size: 160
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 320
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 100.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.9344 1.2837 1000 0.8891 0.6861 0.3482
0.7957 2.5674 2000 0.6957 0.6096 0.3201
0.6502 3.8511 3000 0.6526 0.5753 0.3087
0.6207 5.1348 4000 0.6331 0.5625 0.3179
0.6032 6.4185 5000 0.6054 0.5553 0.3155
0.7118 7.7022 6000 0.6006 0.5580 0.2983
0.5953 8.9859 7000 0.5827 0.5371 0.2933
0.5799 10.2696 8000 0.5903 0.5310 0.3103
0.5432 11.5533 9000 0.5545 0.5284 0.2891
0.5047 12.8370 10000 0.5423 0.5197 0.2914
0.5812 14.1207 11000 0.5366 0.5127 0.2982
0.5 15.4044 12000 0.5234 0.5072 0.2837
0.5229 16.6881 13000 0.5180 0.5050 0.2851
0.5162 17.9718 14000 0.5127 0.5044 0.2859
0.5709 19.2555 15000 0.5088 0.5009 0.2831
0.4316 20.5392 16000 0.5026 0.4837 0.2828
0.4892 21.8228 17000 0.4984 0.4861 0.2830
0.3795 23.1065 18000 0.5082 0.4826 0.2728
0.3209 24.3902 19000 0.4842 0.4724 0.2712
0.3633 25.6739 20000 0.4732 0.4659 0.2720
0.4002 26.9576 21000 0.4832 0.4618 0.2665
0.2956 28.2413 22000 0.4756 0.4444 0.2646
0.2779 29.5250 23000 0.4837 0.4499 0.2680
0.2933 30.8087 24000 0.4563 0.4453 0.2662
0.2412 32.0924 25000 0.4681 0.4410 0.2622
0.2966 33.3761 26000 0.4804 0.4453 0.2607
0.2494 34.6598 27000 0.4721 0.4418 0.2639
0.2091 35.9435 28000 0.4782 0.4383 0.2615
0.2055 37.2272 29000 0.4915 0.4371 0.2602

Framework versions

  • Transformers 4.48.1
  • Pytorch 2.7.1+cu126
  • Datasets 4.0.0
  • Tokenizers 0.21.2
Downloads last month
71
Safetensors
Model size
606M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for CLEAR-Global/w2v-bert-2.0-yoruba_naijavoices_250h

Finetuned
(339)
this model

Collection including CLEAR-Global/w2v-bert-2.0-yoruba_naijavoices_250h