Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
ldwang
Abstract
Vision-Language Models (VLMs) have recently made significant progress, but the limited scale and quality of open-source instruction data hinder their performance compared to closed-source models. In this work, we address this limitation by introducing Infinity-MM, a large-scale multimodal instruction dataset with 40 million samples, enhanced through rigorous quality filtering and deduplication. We also propose a synthetic instruction generation method based on open-source VLMs, using detailed image annotations and diverse question generation. Using this data, we trained a 2-billion-parameter VLM, Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of similar scale. This demonstrates that expanding instruction data and generating synthetic data can significantly improve the performance of open-source models.
Community
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Video Instruction Tuning With Synthetic Data (2024)
- VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment (2024)
- MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2024)
- POINTS: Improving Your Vision-language Model with Affordable Strategies (2024)
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
