README.md

metadata

datasets:
  - LLM360/MegaMath
language:
  - en
library_name: transformers
license: llama3.2
pipeline_tag: text-generation
tags:
  - math
  - code
  - cot
  - pal

# MegaMath-Llama-3.2-3B

[Arxiv](arxiv.org/abs/2504.02807) | [Datasets](https://huggingface.co/datasets/LLM360/MegaMath) | [Code](https://github.com/LLM360/MegaMath)

A proof-of-concept model train on [MegaMath](https://huggingface.co/datasets/LLM360/MegaMath) dataset, capable of both Chain-of-Thought and Program-Aided-Language problem solving.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/628f6e5ab90dde28ef57d293/Sw4P-clZhFMxBSNmVAaww.png)

## Performance


![image/png](https://cdn-uploads.huggingface.co/production/uploads/628f6e5ab90dde28ef57d293/nZYsgAj1vhuoKhpJb4ZU7.png)

## Citation
If you find our work useful, please cite
```bibtex
@article{zhou2025megamath,
  title     = {MegaMath: Pushing the Limits of Open Math Corpora},
  author    = {Zhou, Fan and Wang, Zengzhi and Ranjan, Nikhil and Cheng, Zhoujun and Tang, Liping and He, Guowei and Liu, Zhengzhong and Xing, Eric P.},
  journal   = {arXiv preprint arXiv:2504.xxxxx},
  year      = {2025},
  note      = {Preprint}
}