Llama-3.2-3B-NuminaQA

Links

• 📜 Paper
• 💻 GitHub
• 🤗 Oat-Zero Collection

Introduction

This model serves as a 3B base in our minimalist R1-Zero recipe.

Training details:

• Base model: HuggingFaceTB/FineMath-Llama-3B
• Dataset: lkevinzc/numia-1.5-qa-concatenated
• Epochs: 2
• Learning rate: 1e-5

Citation

@misc{liu2025understanding,
  title={Understanding R1-Zero-Like Training: A Critical Perspective},
  author={Zichen Liu and Changyu Chen and Wenjun Li and Penghui Qi and Tianyu Pang and Chao Du and Wee Sun Lee and Min Lin},
  year={2025},
  howpublished={\url{https://github.com/sail-sg/understand-r1-zero}},
}