---
library_name: peft
license: other
base_model: Qwen/Qwen3-32B
tags:
- llama-factory
- lora
- generated_from_trainer
model-index:
- name: Qwen3-32B-alpaca-th-52k-dolly-th-15k-wangchan-instruct-seed-4201
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Qwen3-32B-alpaca-th-52k-dolly-th-15k-wangchan-instruct-seed-4201

This model is a fine-tuned version of [Qwen/Qwen3-32B](https://huggingface.co//Qwen/Qwen3-32B) on the alpaca-th-52k, the dolly-th-15k and the wangchan-instruct datasets.
It achieves the following results on the evaluation set:
- Loss: 0.6413

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 2
- seed: 4201
- distributed_type: multi-GPU
- num_devices: 32
- gradient_accumulation_steps: 8
- total_train_batch_size: 512
- total_eval_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3.0

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 0.9293        | 0.0575 | 10   | 1.0471          |
| 0.8085        | 0.1149 | 20   | 0.8245          |
| 0.7547        | 0.1724 | 30   | 0.7581          |
| 0.7289        | 0.2299 | 40   | 0.7386          |
| 0.6965        | 0.2874 | 50   | 0.7242          |
| 0.6848        | 0.3448 | 60   | 0.7109          |
| 0.693         | 0.4023 | 70   | 0.7022          |
| 0.7101        | 0.4598 | 80   | 0.6947          |
| 0.7293        | 0.5172 | 90   | 0.6888          |
| 0.6852        | 0.5747 | 100  | 0.6822          |
| 0.7033        | 0.6322 | 110  | 0.6770          |
| 0.6815        | 0.6897 | 120  | 0.6736          |
| 0.679         | 0.7471 | 130  | 0.6707          |
| 0.6571        | 0.8046 | 140  | 0.6682          |
| 0.6491        | 0.8621 | 150  | 0.6660          |
| 0.7015        | 0.9195 | 160  | 0.6636          |
| 0.6523        | 0.9770 | 170  | 0.6619          |
| 0.6672        | 1.0345 | 180  | 0.6602          |
| 0.6862        | 1.0920 | 190  | 0.6588          |
| 0.6755        | 1.1494 | 200  | 0.6577          |
| 0.6279        | 1.2069 | 210  | 0.6563          |
| 0.6622        | 1.2644 | 220  | 0.6551          |
| 0.6329        | 1.3218 | 230  | 0.6542          |
| 0.6559        | 1.3793 | 240  | 0.6528          |
| 0.6389        | 1.4368 | 250  | 0.6517          |
| 0.6476        | 1.4943 | 260  | 0.6506          |
| 0.6412        | 1.5517 | 270  | 0.6497          |
| 0.6232        | 1.6092 | 280  | 0.6485          |
| 0.6243        | 1.6667 | 290  | 0.6478          |
| 0.6467        | 1.7241 | 300  | 0.6469          |
| 0.6146        | 1.7816 | 310  | 0.6460          |
| 0.6386        | 1.8391 | 320  | 0.6450          |
| 0.6456        | 1.8966 | 330  | 0.6443          |
| 0.6402        | 1.9540 | 340  | 0.6437          |
| 0.6455        | 2.0115 | 350  | 0.6434          |
| 0.5888        | 2.0690 | 360  | 0.6437          |
| 0.6267        | 2.1264 | 370  | 0.6435          |
| 0.6292        | 2.1839 | 380  | 0.6434          |
| 0.6058        | 2.2414 | 390  | 0.6432          |
| 0.6221        | 2.2989 | 400  | 0.6427          |
| 0.6254        | 2.3563 | 410  | 0.6428          |
| 0.6178        | 2.4138 | 420  | 0.6423          |
| 0.6161        | 2.4713 | 430  | 0.6420          |
| 0.634         | 2.5287 | 440  | 0.6419          |
| 0.6241        | 2.5862 | 450  | 0.6418          |
| 0.6084        | 2.6437 | 460  | 0.6416          |
| 0.6264        | 2.7011 | 470  | 0.6415          |
| 0.608         | 2.7586 | 480  | 0.6413          |
| 0.6039        | 2.8161 | 490  | 0.6413          |
| 0.6445        | 2.8736 | 500  | 0.6413          |
| 0.6249        | 2.9310 | 510  | 0.6413          |
| 0.6006        | 2.9885 | 520  | 0.6413          |


### Framework versions

- PEFT 0.15.2
- Transformers 4.52.3
- Pytorch 2.7.0+cu126
- Datasets 3.6.0
- Tokenizers 0.21.1