distilbert-base-multilingual-cased-2-contract-sections-classification-v2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3290
• Accuracy: 0.964

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.0275	1.0	1000	0.1685	0.964
0.0281	2.0	2000	0.1838	0.9625
0.0262	3.0	3000	0.1768	0.9675
0.025	4.0	4000	0.1766	0.9643
0.0229	5.0	5000	0.2149	0.958
0.0125	6.0	6000	0.2050	0.9593
0.0157	7.0	7000	0.2108	0.9593
0.0168	8.0	8000	0.2029	0.9625
0.0118	9.0	9000	0.2118	0.9617
0.011	10.0	10000	0.2319	0.9593
0.0103	11.0	11000	0.2175	0.9615
0.0097	12.0	12000	0.2288	0.9625
0.0114	13.0	13000	0.2267	0.9617
0.0064	14.0	14000	0.2401	0.9605
0.0055	15.0	15000	0.2361	0.9607
0.0042	16.0	16000	0.2279	0.9633
0.005	17.0	17000	0.2537	0.96
0.0033	18.0	18000	0.2518	0.9613
0.0052	19.0	19000	0.2680	0.9583
0.0034	20.0	20000	0.2836	0.959
0.0027	21.0	21000	0.2599	0.961
0.0025	22.0	22000	0.2695	0.9587
0.0018	23.0	23000	0.2758	0.959
0.0029	24.0	24000	0.2826	0.9597
0.0032	25.0	25000	0.2645	0.9617
0.002	26.0	26000	0.2856	0.9597
0.0033	27.0	27000	0.2750	0.9595
0.0031	28.0	28000	0.2653	0.9607
0.0021	29.0	29000	0.2687	0.9623
0.004	30.0	30000	0.2878	0.9613
0.0027	31.0	31000	0.2778	0.9625
0.004	32.0	32000	0.2672	0.965
0.005	33.0	33000	0.2771	0.9647
0.0024	34.0	34000	0.2746	0.9663
0.0022	35.0	35000	0.3088	0.9595
0.0001	36.0	36000	0.2909	0.9615
0.0016	37.0	37000	0.2744	0.9645
0.0025	38.0	38000	0.3005	0.9607
0.0006	39.0	39000	0.3034	0.9607
0.0021	40.0	40000	0.3198	0.9607
0.0002	41.0	41000	0.3039	0.9607
0.0005	42.0	42000	0.3338	0.9585
0.001	43.0	43000	0.3179	0.96
0.0016	44.0	44000	0.2949	0.9633
0.0022	45.0	45000	0.3167	0.9597
0.0008	46.0	46000	0.3077	0.9605
0.0028	47.0	47000	0.3055	0.9615
0.0025	48.0	48000	0.2892	0.9643
0.0018	49.0	49000	0.3142	0.9597
0.0013	50.0	50000	0.3204	0.9617
0.003	51.0	51000	0.3505	0.9597
0.0003	52.0	52000	0.3168	0.963
0.0026	53.0	53000	0.3503	0.959
0.0019	54.0	54000	0.3374	0.9633
0.0006	55.0	55000	0.3449	0.96
0.0001	56.0	56000	0.3348	0.9627
0.0027	57.0	57000	0.3310	0.9613
0.0021	58.0	58000	0.3310	0.961
0.0005	59.0	59000	0.3136	0.963
0.0006	60.0	60000	0.3118	0.9637
0.0006	61.0	61000	0.3133	0.9613
0.0013	62.0	62000	0.3058	0.9643
0.0	63.0	63000	0.3053	0.964
0.0008	64.0	64000	0.3016	0.965
0.0008	65.0	65000	0.3109	0.9655
0.0011	66.0	66000	0.3061	0.9647
0.0	67.0	67000	0.3009	0.9665
0.0009	68.0	68000	0.3140	0.9643
0.0006	69.0	69000	0.3105	0.965
0.0007	70.0	70000	0.3120	0.9655
0.0	71.0	71000	0.3334	0.962
0.0018	72.0	72000	0.3361	0.9617
0.0011	73.0	73000	0.3240	0.963
0.0006	74.0	74000	0.3196	0.9637
0.002	75.0	75000	0.3077	0.966
0.0017	76.0	76000	0.3153	0.9633
0.001	77.0	77000	0.3217	0.963
0.001	78.0	78000	0.3192	0.965
0.0	79.0	79000	0.3188	0.9657
0.001	80.0	80000	0.3278	0.9627
0.0016	81.0	81000	0.3189	0.9625
0.0017	82.0	82000	0.3237	0.9625
0.001	83.0	83000	0.3195	0.9635
0.0014	84.0	84000	0.3301	0.9625
0.0	85.0	85000	0.3235	0.9635
0.0017	86.0	86000	0.3313	0.9627
0.0005	87.0	87000	0.3300	0.9625
0.0	88.0	88000	0.3241	0.964
0.0005	89.0	89000	0.3260	0.9637
0.0016	90.0	90000	0.3296	0.9643
0.0006	91.0	91000	0.3302	0.9637
0.0	92.0	92000	0.3283	0.964
0.0016	93.0	93000	0.3250	0.965
0.0007	94.0	94000	0.3260	0.9647
0.0006	95.0	95000	0.3294	0.9637
0.0009	96.0	96000	0.3278	0.9645
0.0016	97.0	97000	0.3277	0.9645
0.0006	98.0	98000	0.3287	0.964
0.0	99.0	99000	0.3290	0.964
0.0005	100.0	100000	0.3290	0.964

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0