Frywind/REVPT-3B

Reinforced Visual Perception with Tools

This repository contains the official implementation for the paper Reinforced Visual Perception with Tools.

Abstract

Visual reasoning, a cornerstone of human intelligence, encompasses complex perceptual and logical processes essential for solving diverse visual problems. While advances in computer vision have produced powerful models for various perceptual tasks, leveraging these for general visual reasoning remains challenging. Prior work demonstrates that augmenting LLMs with vision models via supervised finetuning improves performance, but faces key limitations such as expensive data generation, reliance on careful data filtering, and poor generalization. To address these issues, we propose ReVPT to enhance multi-modal LLMs' abilities to reason about and use visual tools through reinforcement learning. We introduce a novel RL algorithm based on GRPO, designed to train models to reason with a suite of four visual tools. Through extensive experiments, we show that our method achieves state-of-the-art performance on several perception-heavy benchmarks, including SAT, CV-Bench, BLINK and MMStar, significantly outperforming the supervised and text-based RL finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the community new insights on RL-based visual tool-usage through extensive ablations.

Repository and Code

The official code and further details on the project can be found on the GitHub repository.

You can also find related models and datasets in the Hugging Face collection mentioned in the GitHub README.

Installation, Training, and Evaluation

For detailed instructions on how to install dependencies, set up tool services, train your own models, and run evaluations, please refer to the comprehensive guides provided in the official GitHub repository.