60-tiny_tobacco3482_kd_CEKD_t2.5_a0.5

This model is a fine-tuned version of WinKawaks/vit-tiny-patch16-224 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4631
• Accuracy: 0.81
• Brier Loss: 0.3427
• Nll: 0.9896
• F1 Micro: 0.81
• F1 Macro: 0.7794
• Ece: 0.3002
• Aurc: 0.0534

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
No log	1.0	7	2.0115	0.1	1.0070	8.1872	0.1000	0.0765	0.3197	0.8958
No log	2.0	14	1.3480	0.19	0.8667	5.2268	0.19	0.1599	0.2694	0.7716
No log	3.0	21	1.0745	0.38	0.7608	4.8555	0.38	0.3111	0.2804	0.4226
No log	4.0	28	0.8841	0.57	0.6204	2.8740	0.57	0.4526	0.2904	0.2536
No log	5.0	35	0.7962	0.63	0.5646	2.4082	0.63	0.5462	0.3178	0.2005
No log	6.0	42	0.7065	0.68	0.4894	2.3364	0.68	0.5773	0.2660	0.1461
No log	7.0	49	0.6639	0.7	0.4548	1.8260	0.7	0.6412	0.2748	0.1340
No log	8.0	56	0.6374	0.745	0.4265	1.8720	0.745	0.6620	0.3127	0.1047
No log	9.0	63	0.6184	0.73	0.4136	1.7957	0.7300	0.6650	0.3022	0.0845
No log	10.0	70	0.5569	0.8	0.3838	1.2127	0.8000	0.7683	0.3047	0.0721
No log	11.0	77	0.5225	0.815	0.3514	1.0539	0.815	0.7871	0.2818	0.0625
No log	12.0	84	0.5330	0.795	0.3609	1.0544	0.795	0.7695	0.2723	0.0786
No log	13.0	91	0.5241	0.78	0.3615	1.1338	0.78	0.7599	0.2681	0.0726
No log	14.0	98	0.5345	0.795	0.3640	1.1229	0.795	0.7527	0.2950	0.0661
No log	15.0	105	0.5042	0.8	0.3459	1.1690	0.8000	0.7664	0.2915	0.0664
No log	16.0	112	0.5285	0.795	0.3779	1.1973	0.795	0.7468	0.3056	0.0647
No log	17.0	119	0.5374	0.77	0.3716	1.5324	0.7700	0.7406	0.2658	0.0910
No log	18.0	126	0.5132	0.79	0.3677	1.1252	0.79	0.7699	0.2842	0.0722
No log	19.0	133	0.4961	0.815	0.3558	0.8880	0.815	0.7941	0.2985	0.0611
No log	20.0	140	0.4971	0.82	0.3556	1.0845	0.82	0.8049	0.3045	0.0631
No log	21.0	147	0.4791	0.835	0.3328	0.9109	0.835	0.8168	0.2803	0.0534
No log	22.0	154	0.4953	0.805	0.3531	1.0087	0.805	0.7758	0.3015	0.0642
No log	23.0	161	0.4962	0.825	0.3543	0.8886	0.825	0.8091	0.3085	0.0672
No log	24.0	168	0.4753	0.82	0.3435	1.0609	0.82	0.8025	0.2994	0.0508
No log	25.0	175	0.4856	0.825	0.3495	1.1336	0.825	0.7846	0.3018	0.0539
No log	26.0	182	0.4824	0.805	0.3499	1.0142	0.805	0.7890	0.3000	0.0577
No log	27.0	189	0.4843	0.83	0.3506	0.7550	0.83	0.8090	0.3163	0.0556
No log	28.0	196	0.4728	0.825	0.3451	1.1111	0.825	0.8012	0.3006	0.0554
No log	29.0	203	0.4844	0.8	0.3483	1.2220	0.8000	0.7750	0.2913	0.0609
No log	30.0	210	0.4728	0.815	0.3478	1.1306	0.815	0.7840	0.3051	0.0540
No log	31.0	217	0.4694	0.83	0.3411	0.8267	0.83	0.8156	0.3101	0.0474
No log	32.0	224	0.4685	0.82	0.3406	1.0564	0.82	0.7909	0.3269	0.0529
No log	33.0	231	0.4692	0.81	0.3407	0.8508	0.81	0.7900	0.2838	0.0517
No log	34.0	238	0.4647	0.815	0.3404	1.0430	0.815	0.7894	0.3098	0.0563
No log	35.0	245	0.4761	0.795	0.3503	1.0340	0.795	0.7687	0.2731	0.0594
No log	36.0	252	0.4802	0.83	0.3571	0.8575	0.83	0.8046	0.3275	0.0511
No log	37.0	259	0.4686	0.8	0.3414	0.8818	0.8000	0.7836	0.3020	0.0560
No log	38.0	266	0.4612	0.815	0.3361	0.9901	0.815	0.7807	0.2988	0.0528
No log	39.0	273	0.4721	0.81	0.3475	0.9803	0.81	0.7875	0.3019	0.0576
No log	40.0	280	0.4646	0.83	0.3425	0.8495	0.83	0.8059	0.3063	0.0507
No log	41.0	287	0.4622	0.805	0.3396	1.0087	0.805	0.7688	0.2903	0.0558
No log	42.0	294	0.4599	0.82	0.3375	0.9166	0.82	0.7996	0.3319	0.0533
No log	43.0	301	0.4688	0.805	0.3475	1.0195	0.805	0.7771	0.2994	0.0573
No log	44.0	308	0.4652	0.805	0.3437	0.9095	0.805	0.7809	0.3134	0.0545
No log	45.0	315	0.4641	0.81	0.3418	0.8922	0.81	0.7806	0.3014	0.0550
No log	46.0	322	0.4579	0.825	0.3361	0.9516	0.825	0.7956	0.2971	0.0490
No log	47.0	329	0.4638	0.82	0.3423	0.8830	0.82	0.7961	0.3259	0.0557
No log	48.0	336	0.4643	0.81	0.3434	1.0046	0.81	0.7789	0.3042	0.0541
No log	49.0	343	0.4596	0.81	0.3388	0.9862	0.81	0.7835	0.3170	0.0532
No log	50.0	350	0.4603	0.815	0.3399	0.9288	0.815	0.7963	0.3031	0.0533
No log	51.0	357	0.4610	0.815	0.3403	0.9900	0.815	0.7898	0.3306	0.0546
No log	52.0	364	0.4617	0.81	0.3412	0.9834	0.81	0.7793	0.3079	0.0533
No log	53.0	371	0.4627	0.815	0.3423	0.9901	0.815	0.7898	0.3023	0.0543
No log	54.0	378	0.4612	0.815	0.3415	0.9868	0.815	0.7962	0.3178	0.0534
No log	55.0	385	0.4617	0.815	0.3416	0.9904	0.815	0.7898	0.3117	0.0533
No log	56.0	392	0.4605	0.81	0.3399	0.9845	0.81	0.7793	0.3069	0.0535
No log	57.0	399	0.4606	0.81	0.3405	0.9818	0.81	0.7793	0.3045	0.0531
No log	58.0	406	0.4614	0.81	0.3413	0.9853	0.81	0.7793	0.3114	0.0537
No log	59.0	413	0.4623	0.81	0.3424	0.9848	0.81	0.7793	0.3045	0.0534
No log	60.0	420	0.4621	0.81	0.3421	0.9898	0.81	0.7863	0.3150	0.0536
No log	61.0	427	0.4620	0.81	0.3417	0.9868	0.81	0.7793	0.3060	0.0534
No log	62.0	434	0.4618	0.81	0.3413	0.9843	0.81	0.7793	0.3029	0.0533
No log	63.0	441	0.4622	0.81	0.3419	0.9868	0.81	0.7793	0.2969	0.0535
No log	64.0	448	0.4621	0.81	0.3419	0.9881	0.81	0.7793	0.3070	0.0542
No log	65.0	455	0.4625	0.81	0.3422	0.9871	0.81	0.7794	0.3131	0.0532
No log	66.0	462	0.4626	0.81	0.3423	0.9880	0.81	0.7794	0.3066	0.0533
No log	67.0	469	0.4621	0.81	0.3420	0.9872	0.81	0.7793	0.3066	0.0536
No log	68.0	476	0.4624	0.81	0.3421	0.9882	0.81	0.7794	0.2966	0.0533
No log	69.0	483	0.4627	0.81	0.3425	0.9891	0.81	0.7794	0.3160	0.0534
No log	70.0	490	0.4628	0.81	0.3424	0.9899	0.81	0.7794	0.2970	0.0533
No log	71.0	497	0.4627	0.81	0.3423	0.9890	0.81	0.7794	0.2968	0.0532
0.3139	72.0	504	0.4625	0.81	0.3423	0.9886	0.81	0.7794	0.2971	0.0534
0.3139	73.0	511	0.4625	0.81	0.3423	0.9892	0.81	0.7794	0.3043	0.0535
0.3139	74.0	518	0.4626	0.81	0.3422	0.9881	0.81	0.7794	0.2969	0.0533
0.3139	75.0	525	0.4631	0.81	0.3428	0.9896	0.81	0.7794	0.3142	0.0533
0.3139	76.0	532	0.4628	0.81	0.3425	0.9893	0.81	0.7794	0.3138	0.0532
0.3139	77.0	539	0.4627	0.81	0.3423	0.9889	0.81	0.7794	0.3040	0.0533
0.3139	78.0	546	0.4628	0.81	0.3425	0.9888	0.81	0.7794	0.3138	0.0533
0.3139	79.0	553	0.4629	0.81	0.3426	0.9898	0.81	0.7794	0.3002	0.0535
0.3139	80.0	560	0.4630	0.81	0.3426	0.9892	0.81	0.7794	0.3041	0.0534
0.3139	81.0	567	0.4631	0.81	0.3428	0.9899	0.81	0.7794	0.3042	0.0534
0.3139	82.0	574	0.4628	0.81	0.3424	0.9889	0.81	0.7794	0.3039	0.0532
0.3139	83.0	581	0.4630	0.81	0.3427	0.9893	0.81	0.7794	0.3068	0.0533
0.3139	84.0	588	0.4629	0.81	0.3426	0.9894	0.81	0.7794	0.3069	0.0534
0.3139	85.0	595	0.4629	0.81	0.3425	0.9893	0.81	0.7794	0.3138	0.0535
0.3139	86.0	602	0.4630	0.81	0.3427	0.9896	0.81	0.7794	0.3070	0.0533
0.3139	87.0	609	0.4630	0.81	0.3426	0.9890	0.81	0.7794	0.3069	0.0534
0.3139	88.0	616	0.4630	0.81	0.3426	0.9893	0.81	0.7794	0.3069	0.0533
0.3139	89.0	623	0.4630	0.81	0.3426	0.9897	0.81	0.7794	0.3001	0.0535
0.3139	90.0	630	0.4631	0.81	0.3428	0.9902	0.81	0.7794	0.2904	0.0534
0.3139	91.0	637	0.4631	0.81	0.3427	0.9892	0.81	0.7794	0.3139	0.0533
0.3139	92.0	644	0.4631	0.81	0.3427	0.9894	0.81	0.7794	0.3071	0.0535
0.3139	93.0	651	0.4631	0.81	0.3428	0.9899	0.81	0.7794	0.3001	0.0534
0.3139	94.0	658	0.4630	0.81	0.3427	0.9894	0.81	0.7794	0.3069	0.0534
0.3139	95.0	665	0.4631	0.81	0.3428	0.9896	0.81	0.7794	0.3071	0.0534
0.3139	96.0	672	0.4630	0.81	0.3427	0.9893	0.81	0.7794	0.3070	0.0534
0.3139	97.0	679	0.4631	0.81	0.3427	0.9895	0.81	0.7794	0.3002	0.0535
0.3139	98.0	686	0.4631	0.81	0.3428	0.9899	0.81	0.7794	0.3002	0.0534
0.3139	99.0	693	0.4631	0.81	0.3427	0.9897	0.81	0.7794	0.3002	0.0534
0.3139	100.0	700	0.4631	0.81	0.3427	0.9896	0.81	0.7794	0.3002	0.0534

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1.post200
• Datasets 2.9.0
• Tokenizers 0.13.2