Update README.md
Browse files
README.md
CHANGED
@@ -16,18 +16,8 @@ In the v1.5 (08/2024) release, we present a series of XGen-MM models including:
|
|
16 |
- [π€ xGen-MM-instruct](https://huggingface.co/Salesforce/xgen-mm-phi3-mini-instruct-singleimg-r-v1.5): `xgen-mm-phi3-mini-instruct-singleimg-r-v1.5`
|
17 |
- [π€ xGen-MM-instruct-dpo](https://huggingface.co/Salesforce/xgen-mm-phi3-mini-instruct-dpo-r-v1.5): `xgen-mm-phi3-mini-instruct-dpo-r-v1.5`
|
18 |
|
19 |
-
In addition to the models, our team also released a series of datasets for multi-modal pre-training, including:
|
20 |
-
- [π MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens](https://arxiv.org/abs/2406.11271)
|
21 |
-
- [π€ BLIP3-OCR-200M (coming soon)](https://huggingface.co/datasets/Salesforce/blip3-ocr-200m): a dataset with dense OCR annotations.
|
22 |
-
- [π€ BLIP3-GROUNDING-50M (coming soon)](https://huggingface.co/datasets/Salesforce/blip3-grounding-50m): a dataset for enhancing the ability to ground semantic concepts in images.
|
23 |
-
- BLIP3-KALE (stay tuned): a large-scale curated high-quality caption dataset.
|
24 |
-
|
25 |
For more details, check out our [tech report](https://arxiv.org/pdf/2408.08872), [fine-tuning code](https://github.com/salesforce/LAVIS/tree/xgen-mm), and project page (coming soon).
|
26 |
|
27 |
-
# Data
|
28 |
-
The base model is pre-trained on a mixture of data sources described above, with around 100 billion image-text tokens in total.
|
29 |
-
|
30 |
-
|
31 |
# Results
|
32 |
|
33 |
### Few-shot Evaluation on Base model (without instruction tuning)
|
|
|
16 |
- [π€ xGen-MM-instruct](https://huggingface.co/Salesforce/xgen-mm-phi3-mini-instruct-singleimg-r-v1.5): `xgen-mm-phi3-mini-instruct-singleimg-r-v1.5`
|
17 |
- [π€ xGen-MM-instruct-dpo](https://huggingface.co/Salesforce/xgen-mm-phi3-mini-instruct-dpo-r-v1.5): `xgen-mm-phi3-mini-instruct-dpo-r-v1.5`
|
18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
For more details, check out our [tech report](https://arxiv.org/pdf/2408.08872), [fine-tuning code](https://github.com/salesforce/LAVIS/tree/xgen-mm), and project page (coming soon).
|
20 |
|
|
|
|
|
|
|
|
|
21 |
# Results
|
22 |
|
23 |
### Few-shot Evaluation on Base model (without instruction tuning)
|