README.md

---
language:
- ms
- en
datasets:
- mesolitica/Malaysian-Reasoning
base_model:
- mesolitica/Malaysian-Qwen2.5-1.5B-Instruct
---

# Malaysian Qwen 2.5 1.5B Instruct Reasoning SFT

Continue finetuning https://huggingface.co/mesolitica/Malaysian-Qwen2.5-1.5B-Instruct on highly curated Malaysian Reasoning dataset.

## Improvement

1. Reasoning on Math, Science, Translation, Dialects, Multiple choices, coding and Maktabah Al Bakri.
2. Warmup reasoning.

## Training session

Finetune on [mesolitica/Malaysian-Reasoning](https://huggingface.co/datasets/mesolitica/Malaysian-Reasoning) to make the model better reasoning on Malaysian context.
  
## How we train

1. Full parameters on 12k context length.
5. WanDB at https://wandb.ai/huseinzol05/fpf-qwen2.5-1.5b-malaysian-12k-reasoning

Source code at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5

### Dialect Translation

All the benchmarks generate using vLLM, evaluation based on sacrebleu CHRF max@5.

Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-dialect

Dialect to standard Malay,

```

```

Standard Malay to dialect,

```

```

### MalayMMLU

Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-malaymmlu

Evaluation based on Accuracy@1,

```

```

Evaluation based on Accuracy@5,

```

```

## Special thanks

Special thanks to https://www.sns.com.my and Nvidia for 8x H100 node!