🖼️📝 OneEncoder: A Unified Text & Image Model
OneEncoder is a lightweight framework for cross-modal alignment, focusing on efficiently integrating text and images (with future extensions to other modalities). Unlike traditional methods relying on massive modality-specific encoders, OneEncoder progressively aligns different data types, making it cost-effective and performant even on small paired datasets.
🚀 Key Features
✅ Multimodal Alignment: Initially supports text & image, with extension to other modalities.
✅ Lightweight & Efficient: Avoids full retraining when adding new modalities.
✅ Superior Performance: Outperforms models that require large specialized datasets.
🎯 Applications
- Visual Question Answering (VQA)
- Image-Text Retrieval
- Multimodal Content Understanding
📝 Authors
📌 Bilal FAYE, Hanane AZZAG, Mustapha LEBBAH, Djamel BOUCHAFFRA
📄 Research Paper
📜 arXiv: OneEncoder: Progressive Cross-Modal Alignment
📌 Resources
🔗 GitHub Repo: OneEncoder
🚀 Hugging Face Demo: OneEncoder Retriever
📓 Demo Notebook: OneEncoder Demos
🔊 OneEncoder for Text, Image & Audio: HF Model
- Downloads last month
- 33
Model tree for bilalfaye/OneEncoder-text-image
Base model
google-bert/bert-base-uncased