metadata

base_model: deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
library_name: transformers
model_name: DeepSeek-R1-0528-Qwen3-8B-GRPO-trlv5
tags:
  - generated_from_trainer
  - trl
  - grpo
  - rpt
licence: license
license: apache-2.0
language:
  - en
  - es
  - fr
  - ar

Model Card for RPT-DeepSeek-R1-0528-Qwen3-8B

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-0528-Qwen3-8B. It has been trained using TRL.

This model was trained with GRPO, a method introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. The GRPO training followed the methodology introduced in RPT (Reinforcement Pre-Training Paper ): https://huggingface.co/papers/2506.08007

Framework versions

TRL: 0.19.0
Transformers: 4.52.4
Pytorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.2
vllm: 0.8.5.post1

Citations

Cite GRPO as:

@article{zhihong2024deepseekmath,
    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
    year         = 2024,
    eprint       = {arXiv:2402.03300},
}

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}