train_rte_1752826680

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1858
• Num Input Tokens Seen: 3481336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
9.0694	0.5009	281	8.8326	176032
4.1872	1.0018	562	4.3011	349200
2.6065	1.5027	843	2.3399	524208
1.1642	2.0036	1124	1.2671	699264
0.5388	2.5045	1405	0.6599	873600
0.4425	3.0053	1686	0.3937	1048184
0.3118	3.5062	1967	0.2907	1223864
0.2657	4.0071	2248	0.2455	1397624
0.2172	4.5080	2529	0.2220	1570936
0.1999	5.0089	2810	0.2097	1746384
0.1941	5.5098	3091	0.2019	1922384
0.1879	6.0107	3372	0.1951	2092320
0.206	6.5116	3653	0.1921	2267520
0.1597	7.0125	3934	0.1901	2441688
0.1879	7.5134	4215	0.1883	2614936
0.1623	8.0143	4496	0.1862	2790832
0.1826	8.5152	4777	0.1858	2963888
0.1875	9.0160	5058	0.1860	3137352
0.1772	9.5169	5339	0.1858	3312648

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1