language:
- ms
- en
datasets:
- mesolitica/Malaysian-Reasoning
base_model:
- mesolitica/Malaysian-Qwen2.5-1.5B-Instruct
Malaysian Qwen 2.5 1.5B Instruct Reasoning SFT
Continue finetuning https://huggingface.co/mesolitica/Malaysian-Qwen2.5-1.5B-Instruct on highly curated Malaysian Reasoning dataset.
Improvement
- Reasoning on Math, Science, Translation, Dialects, Multiple choices, coding and Maktabah Al Bakri.
- Warmup reasoning.
Training session
Finetune on mesolitica/Malaysian-Reasoning to make the model better reasoning on Malaysian context.
How we train
- Full parameters on 12k context length.
- WanDB at https://wandb.ai/huseinzol05/fpf-qwen2.5-1.5b-malaysian-12k-reasoning
Source code at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5
Dialect Translation
All the benchmarks generate using vLLM, evaluation based on sacrebleu CHRF max@5.
Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-dialect
Dialect to standard Malay,
Standard Malay to dialect,
MalayMMLU
Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-malaymmlu
Evaluation based on Accuracy@1,
Evaluation based on Accuracy@5,
Special thanks
Special thanks to https://www.sns.com.my and Nvidia for 8x H100 node!