---
license: mit
library_name: transformers
pipeline_tag: image-text-to-text
---
[[📖 arXiv Paper](https://arxiv.org/abs/2502.10391)]
[[📊 MM-RLHF Data](https://huggingface.co/datasets/yifanzhang114/MM-RLHF)]
[[📝 Homepage](https://mm-rlhf.github.io/)]
[[🏆 Reward Model](https://huggingface.co/yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen)]
[[🔮 MM-RewardBench](https://huggingface.co/datasets/yifanzhang114/MM-RLHF-RewardBench)]
[[🔮 MM-SafetyBench](https://github.com/yfzhang114/mmrlhf-eval)]
[[📈 Evaluation Suite](https://github.com/yfzhang114/mmrlhf-eval)]
[[📊 Training Code](https://github.com/yfzhang114/MM-RLHF)]
# The Next Step Forward in Multimodal LLM Alignment
**[2025/02/10]** 🔥 We are proud to open-source **MM-RLHF**, a comprehensive project for aligning Multimodal Large Language Models (MLLMs) with human preferences. This release includes:
- A **high-quality MLLM alignment dataset**.
- A **strong Critique-Based MLLM reward model** and its training algorithm.
- A **novel alignment algorithm MM-DPO**.
- **Two new benchmarks**.
Our dataset and algorithms enable consistent performance improvements across **10 dimensions** and **27 benchmarks**.\n
## Use
### Intended use
The model was trained on [MM-RLHF data](https://huggingface.co/datasets/yifanzhang114/MM-RLHF) and have the ability to interact with images, multi-image and videos.
**Feel free to share your generations in the Community tab!**
### Generation
We provide the simple generation process for using our model. For more details, you could refer to [Github](https://github.com/yfzhang114/MM-RLHF).
## Citation
If you find it useful for your research and applications, please cite related papers/blogs using this BibTeX:
```bibtex
@article{zhang2025mm,
title={MM-RLHF: The Next Step Forward in Multimodal LLM Alignment},
author={Zhang, Yi-Fan and Yu, Tao and Tian, Haochen and Fu, Chaoyou and Li, Peiyan and Zeng, Jianshu and Xie, Wulin and Shi, Yang and Zhang, Huanyu and Wu, Junkang and others},
journal={arXiv preprint arXiv:2502.10391},
year={2025}
}
```