roberta-base-negcommonsensebalanced-1e-06-64

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3964

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 256
• eval_batch_size: 1024
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
0.6084	1.0	795	0.5485
0.5297	2.0	1590	0.5047
0.5021	3.0	2385	0.4816
0.4891	4.0	3180	0.4691
0.4716	5.0	3975	0.4565
0.4622	6.0	4770	0.4519
0.4537	7.0	5565	0.4408
0.4437	8.0	6360	0.4387
0.4345	9.0	7155	0.4311
0.4295	10.0	7950	0.4286
0.4219	11.0	8745	0.4229
0.4169	12.0	9540	0.4175
0.413	13.0	10335	0.4176
0.4098	14.0	11130	0.4139
0.4046	15.0	11925	0.4096
0.4012	16.0	12720	0.4079
0.3958	17.0	13515	0.4069
0.3915	18.0	14310	0.4066
0.3928	19.0	15105	0.4046
0.3896	20.0	15900	0.4017
0.3856	21.0	16695	0.4004
0.3795	22.0	17490	0.3984
0.3809	23.0	18285	0.3989
0.3796	24.0	19080	0.3983
0.3782	25.0	19875	0.3983
0.3793	26.0	20670	0.3977
0.3747	27.0	21465	0.3959
0.3743	28.0	22260	0.3970
0.3795	29.0	23055	0.3962
0.373	30.0	23850	0.3964

Framework versions

• Transformers 4.48.3
• Pytorch 2.6.0+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0