--- license: mit tags: - generated_from_trainer model-index: - name: verdict-classifier-en results: [] --- # verdict-classifier-en This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.1520 - F1 Macro: 0.9013 - F1 Misinformation: 0.9841 - F1 Factual: 0.9697 - F1 Other: 0.75 - Prec Macro: 0.8643 - Prec Misinformation: 0.9954 - Prec Factual: 0.9412 - Prec Other: 0.6562 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - gradient_accumulation_steps: 8 - total_train_batch_size: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 550 - num_epochs: 1000 ### Training results | Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Misinformation | F1 Factual | F1 Other | Prec Macro | Prec Misinformation | Prec Factual | Prec Other | |:-------------:|:-----:|:----:|:---------------:|:--------:|:-----------------:|:----------:|:--------:|:----------:|:-------------------:|:------------:|:----------:| | 1.072 | 0.73 | 50 | 1.0233 | 0.3136 | 0.9408 | 0.0 | 0.0 | 0.2961 | 0.8882 | 0.0 | 0.0 | | 1.0077 | 1.47 | 100 | 0.8870 | 0.3136 | 0.9408 | 0.0 | 0.0 | 0.2961 | 0.8882 | 0.0 | 0.0 | | 0.9439 | 2.2 | 150 | 0.6889 | 0.3136 | 0.9408 | 0.0 | 0.0 | 0.2961 | 0.8882 | 0.0 | 0.0 | | 0.8743 | 2.93 | 200 | 0.3857 | 0.3129 | 0.9386 | 0.0 | 0.0 | 0.2959 | 0.8878 | 0.0 | 0.0 | | 0.7564 | 3.67 | 250 | 0.2474 | 0.4630 | 0.9716 | 0.0 | 0.4176 | 0.4225 | 0.9839 | 0.0 | 0.2836 | | 0.5366 | 4.41 | 300 | 0.1819 | 0.8054 | 0.9713 | 0.8772 | 0.5676 | 0.8043 | 0.9930 | 1.0 | 0.42 | | 0.4043 | 5.15 | 350 | 0.1344 | 0.8425 | 0.9738 | 0.9538 | 0.6 | 0.8093 | 0.9884 | 0.9394 | 0.5 | | 0.3792 | 5.87 | 400 | 0.1259 | 0.8645 | 0.9761 | 0.9841 | 0.6333 | 0.8388 | 0.9885 | 1.0 | 0.5278 | | 0.2756 | 6.61 | 450 | 0.1344 | 0.8576 | 0.9774 | 0.9538 | 0.6415 | 0.8366 | 0.9841 | 0.9394 | 0.5862 | | 0.2589 | 7.35 | 500 | 0.1188 | 0.8738 | 0.9783 | 0.9412 | 0.7018 | 0.8293 | 0.9931 | 0.8889 | 0.6061 | | 0.2175 | 8.09 | 550 | 0.1436 | 0.8573 | 0.9798 | 0.9538 | 0.6383 | 0.8571 | 0.9798 | 0.9394 | 0.6522 | | 0.1888 | 8.81 | 600 | 0.1566 | 0.8613 | 0.9761 | 0.9412 | 0.6667 | 0.8185 | 0.9907 | 0.8889 | 0.5758 | | 0.15 | 9.55 | 650 | 0.1549 | 0.8542 | 0.9773 | 0.9538 | 0.6316 | 0.8245 | 0.9885 | 0.9394 | 0.5455 | | 0.1464 | 10.29 | 700 | 0.1608 | 0.8633 | 0.9773 | 0.9697 | 0.6429 | 0.8307 | 0.9885 | 0.9412 | 0.5625 | | 0.0954 | 11.03 | 750 | 0.1520 | 0.9013 | 0.9841 | 0.9697 | 0.75 | 0.8643 | 0.9954 | 0.9412 | 0.6562 | | 0.1074 | 11.76 | 800 | 0.1655 | 0.8810 | 0.9819 | 0.9552 | 0.7059 | 0.8565 | 0.9886 | 0.9143 | 0.6667 | | 0.1078 | 12.49 | 850 | 0.1937 | 0.8989 | 0.9829 | 0.9552 | 0.7586 | 0.8530 | 0.9977 | 0.9143 | 0.6471 | | 0.098 | 13.23 | 900 | 0.2098 | 0.8767 | 0.9794 | 0.9412 | 0.7097 | 0.8226 | 1.0 | 0.8889 | 0.5789 | | 0.0931 | 13.96 | 950 | 0.1591 | 0.8755 | 0.9819 | 0.9538 | 0.6909 | 0.8477 | 0.9908 | 0.9394 | 0.6129 | | 0.0701 | 14.7 | 1000 | 0.2121 | 0.8926 | 0.9805 | 0.9552 | 0.7419 | 0.8398 | 1.0 | 0.9143 | 0.6053 | | 0.0692 | 15.44 | 1050 | 0.2118 | 0.8989 | 0.9829 | 0.9552 | 0.7586 | 0.8530 | 0.9977 | 0.9143 | 0.6471 | | 0.0848 | 16.17 | 1100 | 0.2094 | 0.8913 | 0.9818 | 0.9552 | 0.7368 | 0.8487 | 0.9954 | 0.9143 | 0.6364 | | 0.0471 | 16.9 | 1150 | 0.2197 | 0.8919 | 0.9818 | 0.9697 | 0.7241 | 0.8514 | 0.9954 | 0.9412 | 0.6176 | | 0.0399 | 17.64 | 1200 | 0.1997 | 0.9019 | 0.9852 | 0.9538 | 0.7667 | 0.8594 | 1.0 | 0.9394 | 0.6389 | | 0.0307 | 18.38 | 1250 | 0.2873 | 0.8830 | 0.9795 | 0.9697 | 0.7000 | 0.8400 | 0.9954 | 0.9412 | 0.5833 | ### Framework versions - Transformers 4.11.3 - Pytorch 1.9.0+cu102 - Datasets 1.9.0 - Tokenizers 0.10.2