lilt-roberta-en-base-finetuned-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5942
• Precision: 0.8678
• Recall: 0.8967
• F1: 0.8820
• Accuracy: 0.8107

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 60
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	5.2632	100	0.7530	0.8051	0.8559	0.8298	0.7870
No log	10.5263	200	0.9869	0.8232	0.8490	0.8359	0.7906
No log	15.7895	300	1.1115	0.8653	0.8773	0.8712	0.8076
No log	21.0526	400	1.2245	0.8313	0.8544	0.8427	0.7940
0.2553	26.3158	500	1.3779	0.8733	0.8798	0.8765	0.8073
0.2553	31.5789	600	1.4417	0.8690	0.8733	0.8712	0.8071
0.2553	36.8421	700	1.6714	0.8571	0.8793	0.8681	0.7929
0.2553	42.1053	800	1.5601	0.8713	0.8977	0.8843	0.8057
0.2553	47.3684	900	1.6547	0.8709	0.8912	0.8809	0.7965
0.0071	52.6316	1000	1.6069	0.8746	0.8872	0.8809	0.8003
0.0071	57.8947	1100	1.6732	0.8451	0.8887	0.8663	0.7912
0.0071	63.1579	1200	1.5469	0.8688	0.8917	0.8801	0.8076
0.0071	68.4211	1300	1.5922	0.8746	0.8972	0.8857	0.8017
0.0071	73.6842	1400	1.5249	0.8694	0.8992	0.8840	0.8120
0.0014	78.9474	1500	1.6120	0.8611	0.9026	0.8814	0.8047
0.0014	84.2105	1600	1.6198	0.8696	0.9041	0.8865	0.8080
0.0014	89.4737	1700	1.6202	0.8694	0.9026	0.8857	0.8136
0.0014	94.7368	1800	1.6261	0.8521	0.8927	0.8719	0.8007
0.0014	100.0	1900	1.5935	0.8689	0.8957	0.8821	0.8110
0.0005	105.2632	2000	1.5942	0.8678	0.8967	0.8820	0.8107

Framework versions

• Transformers 4.50.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0