bert-soccer-qa

This model is a fine-tuned version of deepset/bert-base-uncased-squad2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3943
• Exact: 74.5971
• F1: 80.7187
• Total: 25690
• Hasans Exact: 74.5971
• Hasans F1: 80.7187
• Hasans Total: 25690
• Best Exact: 74.5971
• Best Exact Thresh: 0.0
• Best F1: 80.7187
• Best F1 Thresh: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Exact	F1	Total	Hasans Exact	Hasans F1	Hasans Total	Best Exact	Best Exact Thresh	Best F1	Best F1 Thresh
0.9455	0.0155	100	0.8127	69.8521	77.1360	25690	69.8521	77.1360	25690	69.8521	0.0	77.1360	0.0
0.8743	0.0311	200	0.7383	70.2141	77.3821	25690	70.2141	77.3821	25690	70.2141	0.0	77.3821	0.0
0.7189	0.0466	300	0.7195	71.0393	78.0890	25690	71.0393	78.0890	25690	71.0393	0.0	78.0890	0.0
0.7367	0.0622	400	0.6836	71.0899	78.0062	25690	71.0899	78.0062	25690	71.0899	0.0	78.0062	0.0
0.6469	0.0777	500	0.6646	71.4247	78.2333	25690	71.4247	78.2333	25690	71.4247	0.0	78.2333	0.0
0.6657	0.0932	600	0.6493	70.2842	77.1248	25690	70.2842	77.1248	25690	70.2842	0.0	77.1248	0.0
0.662	0.1088	700	0.6340	72.2265	79.0144	25690	72.2265	79.0144	25690	72.2265	0.0	79.0144	0.0
0.6969	0.1243	800	0.6086	72.1292	78.8389	25690	72.1292	78.8389	25690	72.1292	0.0	78.8389	0.0
0.669	0.1398	900	0.5938	71.8957	78.6137	25690	71.8957	78.6137	25690	71.8957	0.0	78.6137	0.0
0.6676	0.1554	1000	0.5817	72.2849	78.8877	25690	72.2849	78.8877	25690	72.2849	0.0	78.8877	0.0
0.6664	0.1709	1100	0.5696	71.9541	78.6815	25690	71.9541	78.6815	25690	71.9541	0.0	78.6815	0.0
0.6006	0.1865	1200	0.5661	72.0631	78.7534	25690	72.0631	78.7534	25690	72.0631	0.0	78.7534	0.0
0.6111	0.2020	1300	0.5587	72.6586	79.2351	25690	72.6586	79.2351	25690	72.6586	0.0	79.2351	0.0
0.5793	0.2175	1400	0.5600	72.3939	79.0052	25690	72.3939	79.0052	25690	72.3939	0.0	79.0052	0.0
0.6064	0.2331	1500	0.5501	72.7443	79.3616	25690	72.7443	79.3616	25690	72.7443	0.0	79.3616	0.0
0.6314	0.2486	1600	0.5354	72.2772	78.8176	25690	72.2772	78.8176	25690	72.2772	0.0	78.8176	0.0
0.6741	0.2641	1700	0.5330	72.1059	78.6777	25690	72.1059	78.6777	25690	72.1059	0.0	78.6777	0.0
0.5912	0.2797	1800	0.5291	72.2499	78.7891	25690	72.2499	78.7891	25690	72.2499	0.0	78.7891	0.0
0.584	0.2952	1900	0.5198	72.5691	79.1296	25690	72.5691	79.1296	25690	72.5691	0.0	79.1296	0.0
0.64	0.3108	2000	0.5117	72.7910	79.2873	25690	72.7910	79.2873	25690	72.7910	0.0	79.2873	0.0
0.5361	0.3263	2100	0.5079	73.1335	79.6027	25690	73.1335	79.6027	25690	73.1335	0.0	79.6027	0.0
0.5935	0.3418	2200	0.5025	72.9350	79.4924	25690	72.9350	79.4924	25690	72.9350	0.0	79.4924	0.0
0.5198	0.3574	2300	0.4996	72.6975	79.2533	25690	72.6975	79.2533	25690	72.6975	0.0	79.2533	0.0
0.5474	0.3729	2400	0.4912	73.2970	79.7562	25690	73.2970	79.7562	25690	73.2970	0.0	79.7562	0.0
0.5655	0.3884	2500	0.4847	73.4605	79.9581	25690	73.4605	79.9581	25690	73.4605	0.0	79.9581	0.0
0.5652	0.4040	2600	0.4784	73.3320	79.8207	25690	73.3320	79.8207	25690	73.3320	0.0	79.8207	0.0
0.5288	0.4195	2700	0.4846	73.4838	79.9261	25690	73.4838	79.9261	25690	73.4838	0.0	79.9261	0.0
0.539	0.4351	2800	0.4739	73.0790	79.5401	25690	73.0790	79.5401	25690	73.0790	0.0	79.5401	0.0
0.5493	0.4506	2900	0.4694	73.3009	79.6920	25690	73.3009	79.6920	25690	73.3009	0.0	79.6920	0.0
0.4785	0.4661	3000	0.4669	73.9432	80.1966	25690	73.9432	80.1966	25690	73.9432	0.0	80.1966	0.0
0.4979	0.4817	3100	0.4644	73.5189	79.8528	25690	73.5189	79.8528	25690	73.5189	0.0	79.8528	0.0
0.4908	0.4972	3200	0.4576	73.5734	79.9085	25690	73.5734	79.9085	25690	73.5734	0.0	79.9085	0.0
0.4845	0.5127	3300	0.4491	73.8147	80.1410	25690	73.8147	80.1410	25690	73.8147	0.0	80.1410	0.0
0.5234	0.5283	3400	0.4499	73.6006	79.9882	25690	73.6006	79.9882	25690	73.6006	0.0	79.9882	0.0
0.5345	0.5438	3500	0.4415	73.6395	79.9724	25690	73.6395	79.9724	25690	73.6395	0.0	79.9724	0.0
0.5153	0.5594	3600	0.4388	73.5228	79.8471	25690	73.5228	79.8471	25690	73.5228	0.0	79.8471	0.0
0.5113	0.5749	3700	0.4451	73.9393	80.2260	25690	73.9393	80.2260	25690	73.9393	0.0	80.2260	0.0
0.4752	0.5904	3800	0.4427	73.7330	80.0047	25690	73.7330	80.0047	25690	73.7330	0.0	80.0047	0.0
0.5161	0.6060	3900	0.4382	73.7018	79.9274	25690	73.7018	79.9274	25690	73.7018	0.0	79.9274	0.0
0.4734	0.6215	4000	0.4380	73.8887	80.0773	25690	73.8887	80.0773	25690	73.8887	0.0	80.0773	0.0
0.4852	0.6370	4100	0.4334	74.0093	80.1717	25690	74.0093	80.1717	25690	74.0093	0.0	80.1717	0.0
0.5121	0.6526	4200	0.4244	73.9471	80.1321	25690	73.9471	80.1321	25690	73.9471	0.0	80.1321	0.0
0.4475	0.6681	4300	0.4285	73.9782	80.1311	25690	73.9782	80.1311	25690	73.9782	0.0	80.1311	0.0
0.4963	0.6837	4400	0.4176	74.0016	80.1124	25690	74.0016	80.1124	25690	74.0016	0.0	80.1124	0.0
0.478	0.6992	4500	0.4187	74.2857	80.3374	25690	74.2857	80.3374	25690	74.2857	0.0	80.3374	0.0
0.4634	0.7147	4600	0.4158	74.0171	80.1405	25690	74.0171	80.1405	25690	74.0171	0.0	80.1405	0.0
0.4562	0.7303	4700	0.4158	74.0444	80.1174	25690	74.0444	80.1174	25690	74.0444	0.0	80.1174	0.0
0.509	0.7458	4800	0.4160	73.9081	79.9863	25690	73.9081	79.9863	25690	73.9081	0.0	79.9863	0.0
0.5062	0.7613	4900	0.4129	74.0132	80.1207	25690	74.0132	80.1207	25690	74.0132	0.0	80.1207	0.0
0.4404	0.7769	5000	0.4112	74.3363	80.4688	25690	74.3363	80.4688	25690	74.3363	0.0	80.4688	0.0
0.4835	0.7924	5100	0.4058	74.2935	80.4296	25690	74.2935	80.4296	25690	74.2935	0.0	80.4296	0.0
0.5583	0.8080	5200	0.4043	74.3636	80.4871	25690	74.3636	80.4871	25690	74.3636	0.0	80.4871	0.0
0.5009	0.8235	5300	0.4066	74.3519	80.4009	25690	74.3519	80.4009	25690	74.3519	0.0	80.4009	0.0
0.4881	0.8390	5400	0.4030	74.4142	80.4285	25690	74.4142	80.4285	25690	74.4142	0.0	80.4285	0.0
0.4428	0.8546	5500	0.3954	74.2779	80.3785	25690	74.2779	80.3785	25690	74.2779	0.0	80.3785	0.0
0.4651	0.8701	5600	0.3993	74.3246	80.4433	25690	74.3246	80.4433	25690	74.3246	0.0	80.4433	0.0
0.4067	0.8856	5700	0.4031	74.3480	80.5045	25690	74.3480	80.5045	25690	74.3480	0.0	80.5045	0.0
0.4374	0.9012	5800	0.3958	74.6672	80.7476	25690	74.6672	80.7476	25690	74.6672	0.0	80.7476	0.0
0.4645	0.9167	5900	0.3954	74.3986	80.5765	25690	74.3986	80.5765	25690	74.3986	0.0	80.5765	0.0
0.4689	0.9323	6000	0.3971	74.6516	80.6983	25690	74.6516	80.6983	25690	74.6516	0.0	80.6983	0.0
0.4239	0.9478	6100	0.3926	74.7373	80.7854	25690	74.7373	80.7854	25690	74.7373	0.0	80.7854	0.0
0.4344	0.9633	6200	0.3919	74.6555	80.6953	25690	74.6555	80.6953	25690	74.6555	0.0	80.6953	0.0
0.3875	0.9789	6300	0.3989	74.2507	80.3817	25690	74.2507	80.3817	25690	74.2507	0.0	80.3817	0.0
0.4099	0.9944	6400	0.3944	74.4570	80.4510	25690	74.4570	80.4510	25690	74.4570	0.0	80.4510	0.0
0.4074	1.0099	6500	0.3972	74.8151	80.7001	25690	74.8151	80.7001	25690	74.8151	0.0	80.7001	0.0
0.3219	1.0255	6600	0.4074	74.6399	80.6479	25690	74.6399	80.6479	25690	74.6399	0.0	80.6479	0.0
0.3923	1.0410	6700	0.4027	74.5699	80.5256	25690	74.5699	80.5256	25690	74.5699	0.0	80.5256	0.0
0.359	1.0566	6800	0.3987	74.7334	80.6814	25690	74.7334	80.6814	25690	74.7334	0.0	80.6814	0.0
0.3919	1.0721	6900	0.4035	73.9704	79.9235	25690	73.9704	79.9235	25690	73.9704	0.0	79.9235	0.0
0.3784	1.0876	7000	0.3972	74.6283	80.6954	25690	74.6283	80.6954	25690	74.6283	0.0	80.6954	0.0
0.3317	1.1032	7100	0.3943	74.5971	80.7187	25690	74.5971	80.7187	25690	74.5971	0.0	80.7187	0.0

Framework versions

• Transformers 4.50.3
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1