ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation
Salvatore Esposito
Abstract
ROOM is a simulation framework that generates photorealistic bronchoscopy training data using patient CT scans, enabling the development and validation of autonomy algorithms in medical robotics.
Continuum robots are advancing bronchoscopy procedures by accessing complex lung airways and enabling targeted interventions. However, their development is limited by the lack of realistic training and test environments: Real data is difficult to collect due to ethical constraints and patient safety concerns, and developing autonomy algorithms requires realistic imaging and physical feedback. We present ROOM (Realistic Optical Observation in Medicine), a comprehensive simulation framework designed for generating photorealistic bronchoscopy training data. By leveraging patient CT scans, our pipeline renders multi-modal sensor data including RGB images with realistic noise and light specularities, metric depth maps, surface normals, optical flow and point clouds at medically relevant scales. We validate the data generated by ROOM in two canonical tasks for medical robotics -- multi-view pose estimation and monocular depth estimation, demonstrating diverse challenges that state-of-the-art methods must overcome to transfer to these medical settings. Furthermore, we show that the data produced by ROOM can be used to fine-tune existing depth estimation models to overcome these challenges, also enabling other downstream applications such as navigation. We expect that ROOM will enable large-scale data generation across diverse patient anatomies and procedural scenarios that are challenging to capture in clinical settings. Code and data: https://github.com/iamsalvatore/room.
Community
ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation
-> https://arxiv.org/abs/2509.13177
This paper introduces ROOM, a comprehensive simulation framework that addresses a critical challenge in medical robotics: the scarcity of training data for procedures such as bronchoscopy and colonoscopy, due to ethical constraints and patient safety concerns.
Key innovations:
-Photorealistic data generation: Creates multi-modal sensor data (RGB, depth, surface normals, optical flow, point clouds) of patient organs from their CT scans
-Physics-based simulation: Models continuum robot dynamics with realistic friction, actuator noise, and collision detection
-Medical-scale accuracy: Calibrated to millimetre precision for clinical relevance
Impact demonstrated:
-The authors validate their synthetic data on canonical medical robotics tasks, showing that:
-Traditional SLAM methods struggle in bronchoscopy environments (ORB-SLAM3: 25% success)
-Modern learning methods like VGGT achieve 79% pose estimation accuracy
-Fine-tuning depth estimation models with ROOM data significantly improves performance on real bronchoscopy images
Community value:
-Open-source release upon acceptance, enabling the large-scale generation of multimodal data.
-The framework's modular design means researchers can swap CT scans, rendering engines, or robot models, making it broadly applicable beyond bronchoscopy. This could accelerate the development of autonomous navigation systems for minimally invasive procedures.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- BREA-Depth: Bronchoscopy Realistic Airway-geometric Depth Estimation (2025)
- DATR: Diffusion-based 3D Apple Tree Reconstruction Framework with Sparse-View (2025)
- NeeCo: Image Synthesis of Novel Instrument States Based on Dynamic and Deformable 3D Gaussian Reconstruction (2025)
- Unifying Scale-Aware Depth Prediction and Perceptual Priors for Monocular Endoscope Pose Estimation and Tissue Reconstruction (2025)
- Physically-based Lighting Augmentation for Robotic Manipulation (2025)
- EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images (2025)
- Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper