gpt2_for_whole_train_result_4_2bce

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0368
• Accuracy: 0.9945
• F1: 0.9949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 4096
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.8625	6.8817	50	0.6103	0.7205	0.7164
0.2571	13.7634	100	0.0784	0.972	0.9734
0.0429	20.6452	150	0.0651	0.9805	0.9814
0.0173	27.5269	200	0.0469	0.986	0.9867
0.007	34.4086	250	0.0400	0.9885	0.9891
0.003	41.2903	300	0.0472	0.9885	0.9891
0.0028	48.1720	350	0.0347	0.9945	0.9948
0.0012	55.0538	400	0.0353	0.994	0.9944
0.0009	61.9355	450	0.0433	0.9945	0.9948
0.0006	68.8172	500	0.0366	0.995	0.9953
0.0006	75.6989	550	0.0411	0.995	0.9953
0.0013	82.5806	600	0.0533	0.989	0.9896
0.0006	89.4624	650	0.0357	0.995	0.9953
0.002	96.3441	700	0.0368	0.9945	0.9949

Framework versions

• Transformers 4.40.0
• Pytorch 2.4.1+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1