--- license: mit base_model: facebook/mbart-large-50 tags: - generated_from_trainer metrics: - wer - bleu model-index: - name: Bn_GEDC results: [] --- # Bn_GEDC This model is a fine-tuned version of [facebook/mbart-large-50](https://huggingface.co/facebook/mbart-large-50) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.0461 - Wer: 0.07 - Bleu: 0.847 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine_with_restarts - num_epochs: 2 ### Training results | Training Loss | Epoch | Step | Bleu | Validation Loss | Wer | |:-------------:|:------:|:------:|:-----:|:---------------:|:-----:| | 0.461 | 0.0245 | 2000 | 0.604 | 0.0894 | 0.185 | | 0.0683 | 0.0490 | 4000 | 0.683 | 0.0677 | 0.144 | | 0.052 | 0.0735 | 6000 | 0.71 | 0.0621 | 0.134 | | 0.0427 | 0.0980 | 8000 | 0.732 | 0.0572 | 0.121 | | 0.0373 | 0.1225 | 10000 | 0.749 | 0.0531 | 0.113 | | 0.0335 | 0.1470 | 12000 | 0.759 | 0.0514 | 0.108 | | 0.0397 | 0.1715 | 14000 | 0.77 | 0.0506 | 0.103 | | 0.029 | 0.1960 | 16000 | 0.772 | 0.0508 | 0.103 | | 0.0277 | 0.2205 | 18000 | 0.779 | 0.0496 | 0.099 | | 0.0284 | 0.2450 | 20000 | 0.785 | 0.0468 | 0.096 | | 0.0249 | 0.2695 | 22000 | 0.785 | 0.0479 | 0.097 | | 0.0239 | 0.2940 | 24000 | 0.787 | 0.0481 | 0.095 | | 0.0229 | 0.3185 | 26000 | 0.791 | 0.0473 | 0.094 | | 0.0223 | 0.3430 | 28000 | 0.795 | 0.0461 | 0.092 | | 0.0216 | 0.3674 | 30000 | 0.798 | 0.0471 | 0.091 | | 0.0209 | 0.3919 | 32000 | 0.798 | 0.0467 | 0.091 | | 0.0203 | 0.4164 | 34000 | 0.802 | 0.0464 | 0.089 | | 0.0202 | 0.4409 | 36000 | 0.806 | 0.0454 | 0.087 | | 0.0194 | 0.4654 | 38000 | 0.806 | 0.0462 | 0.087 | | 0.0187 | 0.4899 | 40000 | 0.806 | 0.0471 | 0.087 | | 0.0184 | 0.5144 | 42000 | 0.809 | 0.0462 | 0.086 | | 0.0179 | 0.5389 | 44000 | 0.811 | 0.0444 | 0.085 | | 0.0176 | 0.5634 | 46000 | 0.812 | 0.0460 | 0.085 | | 0.0174 | 0.5879 | 48000 | 0.811 | 0.0469 | 0.086 | | 0.0171 | 0.6124 | 50000 | 0.813 | 0.0465 | 0.084 | | 0.0166 | 0.6369 | 52000 | 0.816 | 0.0446 | 0.083 | | 0.016 | 0.6614 | 54000 | 0.816 | 0.0461 | 0.083 | | 0.0162 | 0.6859 | 56000 | 0.818 | 0.0451 | 0.082 | | 0.0158 | 0.7104 | 58000 | 0.819 | 0.0449 | 0.082 | | 0.0156 | 0.7349 | 60000 | 0.818 | 0.0454 | 0.082 | | 0.0157 | 0.7594 | 62000 | 0.82 | 0.0455 | 0.082 | | 0.015 | 0.7839 | 64000 | 0.822 | 0.0455 | 0.081 | | 0.0148 | 0.8084 | 66000 | 0.822 | 0.0461 | 0.081 | | 0.0146 | 0.8329 | 68000 | 0.823 | 0.0460 | 0.08 | | 0.0145 | 0.8574 | 70000 | 0.824 | 0.0446 | 0.08 | | 0.0144 | 0.8819 | 72000 | 0.824 | 0.0450 | 0.079 | | 0.0141 | 0.9064 | 74000 | 0.822 | 0.0477 | 0.081 | | 0.0139 | 0.9309 | 76000 | 0.826 | 0.0446 | 0.079 | | 0.0137 | 0.9554 | 78000 | 0.827 | 0.0452 | 0.078 | | 0.0136 | 0.9799 | 80000 | 0.827 | 0.0455 | 0.078 | | 0.0128 | 1.0044 | 82000 | 0.829 | 0.0462 | 0.078 | | 0.0104 | 1.0289 | 84000 | 0.829 | 0.0456 | 0.077 | | 0.0105 | 1.0534 | 86000 | 0.829 | 0.0465 | 0.078 | | 0.0103 | 1.0779 | 88000 | 0.831 | 0.0443 | 0.077 | | 0.01 | 1.1023 | 90000 | 0.829 | 0.0456 | 0.077 | | 0.0103 | 1.1268 | 92000 | 0.83 | 0.0466 | 0.077 | | 0.0101 | 1.1513 | 94000 | 0.832 | 0.0462 | 0.076 | | 0.01 | 1.1758 | 96000 | 0.832 | 0.0458 | 0.076 | | 0.01 | 1.2003 | 98000 | 0.834 | 0.0460 | 0.075 | | 0.0098 | 1.2248 | 100000 | 0.834 | 0.0464 | 0.076 | | 0.0098 | 1.2493 | 102000 | 0.834 | 0.0455 | 0.075 | | 0.0096 | 1.2738 | 104000 | 0.836 | 0.0453 | 0.075 | | 0.0099 | 1.2983 | 106000 | 0.835 | 0.0469 | 0.075 | | 0.0095 | 1.3228 | 108000 | 0.836 | 0.0466 | 0.075 | | 0.0094 | 1.3473 | 110000 | 0.836 | 0.0461 | 0.075 | | 0.0094 | 1.3718 | 112000 | 0.837 | 0.0465 | 0.074 | | 0.0093 | 1.3963 | 114000 | 0.838 | 0.0469 | 0.074 | | 0.0092 | 1.4208 | 116000 | 0.838 | 0.0469 | 0.074 | | 0.0092 | 1.4453 | 118000 | 0.838 | 0.0476 | 0.074 | | 0.0092 | 1.4698 | 120000 | 0.839 | 0.0466 | 0.074 | | 0.0091 | 1.4943 | 122000 | 0.841 | 0.0462 | 0.072 | | 0.0089 | 1.5188 | 124000 | 0.839 | 0.0470 | 0.074 | | 0.0088 | 1.5433 | 126000 | 0.839 | 0.0473 | 0.073 | | 0.0087 | 1.5678 | 128000 | 0.841 | 0.0457 | 0.073 | | 0.0086 | 1.5923 | 130000 | 0.843 | 0.0453 | 0.072 | | 0.0085 | 1.6168 | 132000 | 0.841 | 0.0471 | 0.073 | | 0.0086 | 1.6413 | 134000 | 0.842 | 0.0471 | 0.072 | | 0.0086 | 1.6658 | 136000 | 0.844 | 0.0446 | 0.072 | | 0.0082 | 1.6903 | 138000 | 0.844 | 0.0458 | 0.071 | | 0.008 | 1.7148 | 140000 | 0.845 | 0.0460 | 0.071 | | 0.0078 | 1.7393 | 142000 | 0.846 | 0.0460 | 0.071 | | 0.008 | 1.7638 | 144000 | 0.846 | 0.0456 | 0.07 | | 0.0077 | 1.7883 | 146000 | 0.847 | 0.0461 | 0.071 | | 0.0077 | 1.8127 | 148000 | 0.847 | 0.0460 | 0.07 | | 0.0077 | 1.8372 | 150000 | 0.847 | 0.0464 | 0.07 | | 0.0076 | 1.8617 | 152000 | 0.847 | 0.0463 | 0.07 | | 0.0076 | 1.8862 | 154000 | 0.847 | 0.0462 | 0.07 | | 0.0076 | 1.9107 | 156000 | 0.847 | 0.0461 | 0.07 | ### Framework versions - Transformers 4.40.2 - Pytorch 2.3.0+cu121 - Datasets 2.19.1 - Tokenizers 0.19.1