Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Paper
•
2502.12982
•
Published
•
9
•
4
Longxu Dou PRO
dreamerdeo
AI & ML interests
Natural Language Processing
Recent Activity
reacted
to
their
post
with ➕
1 day ago
🚀 Excited to share our technical report on the Southeast Asian multilingual model Sailor2 and its latest updates!
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
🧭 We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:
💡 More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.
🌈 Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.
⚡️ Faster inference: Achieved 2.5x faster inference speed with speculative decoding.
🌪️ More model sizes: Introduced new sizes of 3B and 14B through model pruning.
🌟 All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
📚 Technical report: https://huggingface.co/papers/2502.12982
🤖️ Models: https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b
💬 Demo: https://huggingface.co/spaces/sail/Sailor2-20B-Chat
📣 Sailor2 community: https://huggingface.co/sailor2
updated
a Space
1 day ago
sailor2/README
new activity
1 day ago
sail/Sailor2-8B-Chat:Fix formatting
Organizations
dreamerdeo's activity
Fix formatting
#3 opened 1 day ago
by
nielsr
Add pipeline tag, library name
#2 opened 3 days ago
by
nielsr
Add Transformers library and pipeline tag
4
#3 opened 3 days ago
by
nielsr
Add metadata
1
#4 opened 2 days ago
by
nielsr
Add `library_name` and `pipeline_tag` metadata
#2 opened 3 days ago
by
nielsr
Add library name and pipeline tag
#3 opened 3 days ago
by
nielsr
Add library_name and pipeline_tag metadata
#3 opened 3 days ago
by
nielsr
Add pipeline tag and library name
#2 opened 3 days ago
by
nielsr
Add library name and pipeline tag
#2 opened 3 days ago
by
nielsr
Add library name and set relevant pipeline tag
#2 opened 3 days ago
by
nielsr
Add library_name and pipeline_tag
#3 opened 3 days ago
by
nielsr
Add library_name and pipeline_tag
#2 opened 3 days ago
by
nielsr
Add library name and pipeline tag
#1 opened 3 days ago
by
nielsr
Adding `safetensors` variant of this model
#1 opened 2 days ago
by
SFconvertbot
Adding `safetensors` variant of this model
#1 opened 2 days ago
by
SFconvertbot
Different between sail/Sailor2-L-20B-Chat and sail/Sailor2-20B-Chat
2
#2 opened 3 days ago
by
thanhvovan265
commented
3
papers
3 days ago
Upload folder using huggingface_hub
2
#1 opened 3 days ago
by
dreamerdeo