fineweb-edu-scorer-xlm-binary

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0544
• Precision: 0.8595
• Recall: 0.6250
• F1 Macro: 0.6742
• Accuracy: 0.9268

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 256
• seed: 0
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	0.1600	0.4549	0.5	0.4764	0.9098
0.0618	0.3042	1000	0.0617	0.8239	0.5514	0.5710	0.9159
0.0586	0.6085	2000	0.0589	0.8438	0.5572	0.5804	0.9173
0.058	0.9127	3000	0.0585	0.8369	0.5772	0.6104	0.9195
0.0575	1.2169	4000	0.0580	0.8538	0.5664	0.5948	0.9188
0.059	1.5211	5000	0.0572	0.8503	0.5856	0.6228	0.9212
0.0577	1.8254	6000	0.0569	0.8602	0.5834	0.6200	0.9213
0.0577	2.1296	7000	0.0586	0.8364	0.6212	0.6676	0.9246
0.0579	2.4338	8000	0.0573	0.8394	0.6178	0.6638	0.9245
0.057	2.7381	9000	0.0562	0.8622	0.5916	0.6317	0.9225
0.0564	3.0423	10000	0.0561	0.8651	0.5879	0.6266	0.9222
0.057	3.3465	11000	0.0560	0.8541	0.6059	0.6501	0.9240
0.0586	3.6507	12000	0.0577	0.8424	0.6307	0.6792	0.9262
0.0569	3.9550	13000	0.0571	0.8407	0.6274	0.6752	0.9257
0.0589	4.2592	14000	0.0565	0.8461	0.6237	0.6714	0.9257
0.0568	4.5634	15000	0.0558	0.8618	0.5970	0.6390	0.9232
0.0566	4.8677	16000	0.0566	0.8688	0.5845	0.6220	0.9219
0.0542	5.1719	17000	0.0558	0.8477	0.6217	0.6692	0.9255
0.0558	5.4761	18000	0.0556	0.8643	0.5990	0.6417	0.9236
0.0569	5.7803	19000	0.0554	0.8647	0.5991	0.6419	0.9237
0.0549	6.0846	20000	0.0568	0.8472	0.6390	0.6890	0.9276
0.0561	6.3888	21000	0.0555	0.8642	0.5998	0.6428	0.9237
0.0564	6.6930	22000	0.0555	0.8536	0.6252	0.6739	0.9264
0.0568	6.9973	23000	0.0559	0.8456	0.6327	0.6818	0.9267
0.0553	7.3015	24000	0.0551	0.8617	0.6083	0.6538	0.9247
0.0556	7.6057	25000	0.0552	0.8626	0.6025	0.6463	0.9240
0.0554	7.9099	26000	0.0549	0.8591	0.6114	0.6575	0.9250
0.054	8.2142	27000	0.0550	0.8559	0.6212	0.6694	0.9260
0.0559	8.5184	28000	0.0559	0.8709	0.5937	0.6349	0.9232
0.0543	8.8226	29000	0.0553	0.8499	0.6312	0.6806	0.9269
0.0563	9.1269	30000	0.0549	0.8545	0.6195	0.6671	0.9257
0.0578	9.4311	31000	0.0567	0.8745	0.5854	0.6234	0.9222
0.0549	9.7353	32000	0.0550	0.8541	0.6268	0.6759	0.9266
0.0589	10.0395	33000	0.0553	0.8686	0.6011	0.6447	0.9241
0.0566	10.3438	34000	0.0548	0.8571	0.6199	0.6679	0.9260
0.0556	10.6480	35000	0.0550	0.8650	0.6053	0.6501	0.9245
0.0553	10.9522	36000	0.0547	0.8609	0.6121	0.6585	0.9252
0.0565	11.2565	37000	0.0548	0.8556	0.6241	0.6728	0.9264
0.0555	11.5607	38000	0.0547	0.8550	0.6209	0.6689	0.9259
0.053	11.8649	39000	0.0555	0.8689	0.5996	0.6428	0.9239
0.0565	12.1692	40000	0.0546	0.8575	0.6167	0.6640	0.9256
0.0548	12.4734	41000	0.0546	0.8599	0.6155	0.6627	0.9256
0.0574	12.7776	42000	0.0552	0.8734	0.6019	0.6461	0.9245
0.056	13.0818	43000	0.0548	0.8519	0.6329	0.6827	0.9272
0.0543	13.3861	44000	0.0545	0.8579	0.6239	0.6727	0.9265
0.0543	13.6903	45000	0.0553	0.8678	0.6001	0.6434	0.9240
0.0559	13.9945	46000	0.0545	0.8585	0.6232	0.6720	0.9265
0.0568	14.2988	47000	0.0549	0.8528	0.6373	0.6878	0.9278
0.0558	14.6030	48000	0.0545	0.8579	0.6183	0.6660	0.9258
0.0543	14.9072	49000	0.0545	0.8578	0.6275	0.6771	0.9270
0.0539	15.2114	50000	0.0546	0.8583	0.6266	0.6761	0.9269
0.0529	15.5157	51000	0.0545	0.8638	0.6153	0.6628	0.9258
0.0527	15.8199	52000	0.0545	0.8623	0.6115	0.6578	0.9252
0.0551	16.1241	53000	0.0545	0.8630	0.6128	0.6595	0.9254
0.0545	16.4284	54000	0.0544	0.8602	0.6175	0.6652	0.9258
0.0565	16.7326	55000	0.0544	0.8609	0.6175	0.6653	0.9259
0.0561	17.0368	56000	0.0544	0.8577	0.6278	0.6773	0.9270
0.0574	17.3410	57000	0.0553	0.8695	0.6012	0.6450	0.9242
0.0562	17.6453	58000	0.0544	0.8594	0.6206	0.6689	0.9262
0.0549	17.9495	59000	0.0544	0.8594	0.6229	0.6717	0.9265
0.0549	18.2537	60000	0.0544	0.8624	0.6168	0.6645	0.9259
0.0541	18.5580	61000	0.0544	0.8625	0.6157	0.6632	0.9258
0.0532	18.8622	62000	0.0544	0.8642	0.6159	0.6636	0.9259
0.0525	19.1664	63000	0.0544	0.8602	0.6251	0.6744	0.9268
0.0537	19.4706	64000	0.0544	0.8604	0.6212	0.6697	0.9263
0.0571	19.7749	65000	0.0544	0.8595	0.6250	0.6742	0.9268

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1