N_roberta_agnews_padding70model

This model is a fine-tuned version of roberta-base on the ag_news dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5754
  • Accuracy: 0.9466

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.201 1.0 7500 0.2029 0.9421
0.168 2.0 15000 0.2082 0.945
0.1533 3.0 22500 0.2343 0.9432
0.1208 4.0 30000 0.2381 0.9466
0.1071 5.0 37500 0.2468 0.9464
0.0831 6.0 45000 0.2775 0.9438
0.0758 7.0 52500 0.3080 0.9462
0.056 8.0 60000 0.3970 0.9436
0.0531 9.0 67500 0.3881 0.9401
0.037 10.0 75000 0.3956 0.9443
0.0309 11.0 82500 0.4551 0.9416
0.0257 12.0 90000 0.4521 0.9428
0.0287 13.0 97500 0.4650 0.9413
0.0121 14.0 105000 0.4888 0.9464
0.0116 15.0 112500 0.5071 0.9457
0.0085 16.0 120000 0.5249 0.9449
0.0107 17.0 127500 0.5244 0.9463
0.0031 18.0 135000 0.5597 0.9459
0.0041 19.0 142500 0.5615 0.9476
0.0029 20.0 150000 0.5754 0.9466

Framework versions

  • Transformers 4.33.2
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.5
  • Tokenizers 0.13.3
Downloads last month
8
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Realgon/N_roberta_agnews_padding70model

Finetuned
(1866)
this model

Dataset used to train Realgon/N_roberta_agnews_padding70model

Evaluation results