EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models

Resources

🐙 GitHub: Explore the project repository to run evaluation script. AgibotTech/EWMBench.
📑 arXiv: Read our paper for detailed methodology and results at arXiv:2505.09694.
🤗 Data: Discover EWMBench Dataset, we sample a diverse dataset from AgiBot World for running EWMBench evaluation.
🤗 Model: Download pretrained weights used for evaluation from EWMBench-model.

For running evaluation script, please download necessary model weights and modify the config.yaml to specify weigthts path, following the instruction in EWMBench github repo.

License and Citation

All the data and code within this repo are under CC BY-NC-SA 4.0. Please consider citing our project if it helps your research.

@article{hu2025ewmbench,
  title={EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models},
  author={Hu, Yue and Huang, Siyuan and Liao, Yue and Chen, Shengcong and Zhou, Pengfei and Chen, Liliang and Yao, Maoqing and Ren, Guanghui},
  journal={arXiv preprint arXiv:2505.09694},
  year={2025}
}