|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
datasets: |
|
|
- ccmatrix |
|
|
metrics: |
|
|
- bleu |
|
|
model-index: |
|
|
- name: t5-small-finetuned-en-to-it-b32 |
|
|
results: |
|
|
- task: |
|
|
name: Sequence-to-sequence Language Modeling |
|
|
type: text2text-generation |
|
|
dataset: |
|
|
name: ccmatrix |
|
|
type: ccmatrix |
|
|
config: en-it |
|
|
split: train[3000:12000] |
|
|
args: en-it |
|
|
metrics: |
|
|
- name: Bleu |
|
|
type: bleu |
|
|
value: 9.6816 |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# t5-small-finetuned-en-to-it-b32 |
|
|
|
|
|
This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the ccmatrix dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 2.1496 |
|
|
- Bleu: 9.6816 |
|
|
- Gen Len: 56.5347 |
|
|
|
|
|
## Model description |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 2e-05 |
|
|
- train_batch_size: 32 |
|
|
- eval_batch_size: 32 |
|
|
- seed: 42 |
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 40 |
|
|
- mixed_precision_training: Native AMP |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len | |
|
|
|:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:| |
|
|
| No log | 1.0 | 282 | 2.9409 | 2.6764 | 69.2487 | |
|
|
| 3.3809 | 2.0 | 564 | 2.8277 | 2.4974 | 87.428 | |
|
|
| 3.3809 | 3.0 | 846 | 2.7483 | 2.6851 | 89.7887 | |
|
|
| 3.1255 | 4.0 | 1128 | 2.6831 | 3.1801 | 85.6927 | |
|
|
| 3.1255 | 5.0 | 1410 | 2.6293 | 3.6949 | 79.9467 | |
|
|
| 2.9965 | 6.0 | 1692 | 2.5809 | 4.0149 | 76.852 | |
|
|
| 2.9965 | 7.0 | 1974 | 2.5403 | 4.3463 | 74.6487 | |
|
|
| 2.9002 | 8.0 | 2256 | 2.5033 | 4.838 | 72.6053 | |
|
|
| 2.8229 | 9.0 | 2538 | 2.4694 | 5.2829 | 67.984 | |
|
|
| 2.8229 | 10.0 | 2820 | 2.4421 | 5.4964 | 68.986 | |
|
|
| 2.76 | 11.0 | 3102 | 2.4135 | 5.8118 | 66.528 | |
|
|
| 2.76 | 12.0 | 3384 | 2.3897 | 6.1966 | 65.052 | |
|
|
| 2.7051 | 13.0 | 3666 | 2.3667 | 6.452 | 64.2273 | |
|
|
| 2.7051 | 14.0 | 3948 | 2.3465 | 6.6428 | 63.516 | |
|
|
| 2.6568 | 15.0 | 4230 | 2.3265 | 6.9467 | 61.8673 | |
|
|
| 2.6183 | 16.0 | 4512 | 2.3101 | 7.2029 | 60.7393 | |
|
|
| 2.6183 | 17.0 | 4794 | 2.2954 | 7.4982 | 60.0327 | |
|
|
| 2.5757 | 18.0 | 5076 | 2.2799 | 7.7555 | 59.968 | |
|
|
| 2.5757 | 19.0 | 5358 | 2.2660 | 7.8406 | 60.0307 | |
|
|
| 2.5534 | 20.0 | 5640 | 2.2558 | 8.0679 | 59.0793 | |
|
|
| 2.5534 | 21.0 | 5922 | 2.2426 | 8.3325 | 58.5367 | |
|
|
| 2.5159 | 22.0 | 6204 | 2.2324 | 8.3538 | 58.6893 | |
|
|
| 2.5159 | 23.0 | 6486 | 2.2217 | 8.5867 | 57.7627 | |
|
|
| 2.4983 | 24.0 | 6768 | 2.2135 | 8.8324 | 56.7367 | |
|
|
| 2.4791 | 25.0 | 7050 | 2.2052 | 8.8113 | 57.4373 | |
|
|
| 2.4791 | 26.0 | 7332 | 2.1981 | 9.0909 | 57.0173 | |
|
|
| 2.4529 | 27.0 | 7614 | 2.1908 | 9.0056 | 57.802 | |
|
|
| 2.4529 | 28.0 | 7896 | 2.1856 | 9.2696 | 56.9773 | |
|
|
| 2.4395 | 29.0 | 8178 | 2.1780 | 9.2824 | 57.0007 | |
|
|
| 2.4395 | 30.0 | 8460 | 2.1722 | 9.2106 | 56.9893 | |
|
|
| 2.4277 | 31.0 | 8742 | 2.1685 | 9.4668 | 56.406 | |
|
|
| 2.4181 | 32.0 | 9024 | 2.1646 | 9.4992 | 56.2327 | |
|
|
| 2.4181 | 33.0 | 9306 | 2.1616 | 9.5054 | 56.3033 | |
|
|
| 2.4071 | 34.0 | 9588 | 2.1578 | 9.5093 | 56.548 | |
|
|
| 2.4071 | 35.0 | 9870 | 2.1554 | 9.5227 | 56.7807 | |
|
|
| 2.3991 | 36.0 | 10152 | 2.1532 | 9.5762 | 56.756 | |
|
|
| 2.3991 | 37.0 | 10434 | 2.1518 | 9.6659 | 56.5913 | |
|
|
| 2.3955 | 38.0 | 10716 | 2.1506 | 9.7199 | 56.5753 | |
|
|
| 2.3955 | 39.0 | 10998 | 2.1498 | 9.6715 | 56.558 | |
|
|
| 2.3913 | 40.0 | 11280 | 2.1496 | 9.6816 | 56.5347 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.22.1 |
|
|
- Pytorch 1.12.1 |
|
|
- Datasets 2.5.1 |
|
|
- Tokenizers 0.11.0 |
|
|
|
