091_train_3-2_3B_lora_sft_003

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct on the german_alpaca_under_512_tokens_001, the identity-chatgulaschpt, the german_alpaca_under_512_tokens_002 and the alpaca_under_512_tokens datasets. It achieves the following results on the evaluation set:

• Loss: 1.8359

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
1.3671	0.1595	100	2.5773
1.4539	0.3190	200	2.3549
1.4112	0.4785	300	2.3948
1.3296	0.6380	400	2.2991
1.3561	0.7974	500	2.2534
1.2712	0.9569	600	2.1885
1.1332	1.1164	700	2.1567
1.2149	1.2759	800	2.1424
1.1982	1.4354	900	2.0592
1.161	1.5949	1000	2.0637
1.1154	1.7544	1100	1.9346
1.1121	1.9139	1200	1.9293
1.0087	2.0734	1300	1.9061
0.948	2.2329	1400	1.8543
0.9588	2.3923	1500	1.8512
1.0303	2.5518	1600	1.8442
1.0075	2.7113	1700	1.8368
0.9631	2.8708	1800	1.8350

Framework versions

• PEFT 0.15.2
• Transformers 4.55.0
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1