w2v-bert-2.0-igbo_naijavoices_100h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the NAIJAVOICES_IGBO_100H - NA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3166
  • Wer: 0.3077
  • Cer: 0.1314

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 160
  • eval_batch_size: 160
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 2
  • total_train_batch_size: 320
  • total_eval_batch_size: 320
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 250.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.638 3.4483 1000 0.6548 0.5546 0.2086
0.4035 6.8966 2000 0.4278 0.3949 0.1574
0.5526 10.3448 3000 0.3694 0.3530 0.1461
0.305 13.7931 4000 0.3367 0.3357 0.1392
0.2602 17.2414 5000 0.3230 0.3252 0.1366
0.4161 20.6897 6000 0.3192 0.3226 0.1360
0.2367 24.1379 7000 0.3196 0.3192 0.1345
0.2463 27.5862 8000 0.3153 0.3089 0.1324
0.1819 31.0345 9000 0.3187 0.3034 0.1334
0.1852 34.4828 10000 0.3292 0.3067 0.1326
0.1581 37.9310 11000 0.3510 0.2997 0.1310
0.1455 41.3793 12000 0.3522 0.3080 0.1348
0.0998 44.8276 13000 0.3933 0.3094 0.1373

Framework versions

  • Transformers 4.48.1
  • Pytorch 2.7.1+cu126
  • Datasets 4.0.0
  • Tokenizers 0.21.2
Downloads last month
22
Safetensors
Model size
606M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for CLEAR-Global/w2v-bert-2.0-igbo_naijavoices_100h

Finetuned
(337)
this model

Collection including CLEAR-Global/w2v-bert-2.0-igbo_naijavoices_100h