mistral-journal-finetune-aipi

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.7849

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1
• training_steps: 500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.3541	0.6757	25	1.4800
0.8644	1.3514	50	1.4804
0.8449	2.0270	75	1.5426
0.5501	2.7027	100	1.6733
0.2691	3.3784	125	1.8077
0.2187	4.0541	150	1.9603
0.1408	4.7297	175	2.0132
0.0892	5.4054	200	2.2048
0.0891	6.0811	225	2.2487
0.0633	6.7568	250	2.2303
0.0516	7.4324	275	2.2637
0.0479	8.1081	300	2.2366
0.0486	8.7838	325	2.3266
0.0435	9.4595	350	2.5567
0.0389	10.1351	375	2.6020
0.04	10.8108	400	2.7252
0.0341	11.4865	425	2.6801
0.0357	12.1622	450	2.7342
0.0363	12.8378	475	2.7706
0.0342	13.5135	500	2.7849

Framework versions

• PEFT 0.10.1.dev0
• Transformers 4.40.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.19.0