Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities
Paper
•
2401.14405
•
Published
•
11
General Multimodal Learning
-
-
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
Paper • 2406.18521 • Published • 28 -
xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
Paper • 2408.12590 • Published • 33 -
Law of Vision Representation in MLLMs
Paper • 2408.16357 • Published • 92 -
CogVLM2: Visual Language Models for Image and Video Understanding
Paper • 2408.16500 • Published • 56 -
Building and better understanding vision-language models: insights and future directions
Paper • 2408.12637 • Published • 117 -
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution
Paper • 2409.12961 • Published • 23 -
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models
Paper • 2410.02740 • Published • 52 -
Video Instruction Tuning With Synthetic Data
Paper • 2410.02713 • Published • 36 -
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
Paper • 2410.03051 • Published • 3 -
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models
Paper • 2410.03290 • Published • 6 -
A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generation
Paper • 2410.01912 • Published • 13 -
MIO: A Foundation Model on Multimodal Tokens
Paper • 2409.17692 • Published • 49 -
Emu3: Next-Token Prediction is All You Need
Paper • 2409.18869 • Published • 90 -
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning
Paper • 2409.20566 • Published • 51 -
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
Paper • 2410.13848 • Published • 27 -
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
Paper • 2408.15998 • Published • 83 -
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos
Paper • 2411.04923 • Published • 20