Ame Vi's picture

6 13

Ame Vi

Ameeeee

·

AI & ML interests

None yet

Recent Activity

updated a dataset 6 days ago

Ameeeee/Cultural_diversity

published a dataset 6 days ago

Ameeeee/Cultural_diversity

reacted to tomaarsen's post with 🔥 15 days ago

An assembly of 18 European companies, labs, and universities have banded together to launch 🇪🇺 EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc. 🇪🇺 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi 3️⃣ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion ➡️ Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common. ⚙️ Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported. 🔥 A new Pareto frontier (stronger *and* smaller) for multilingual encoder models 📊 Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight. 📝 Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code. Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release * https://huggingface.co/EuroBERT/EuroBERT-210m * https://huggingface.co/EuroBERT/EuroBERT-610m * https://huggingface.co/EuroBERT/EuroBERT-2.1B The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!

View all activity

Organizations

Ameeeee's activity

published an article 3 months ago

Article

Introducing the Synthetic Data Generator - Build Datasets with Natural Language

By

and 5 others •

Dec 16, 2024

• 118

published an article 4 months ago

Article

Open Preference Dataset for Text-to-Image Generation by the 🤗 Community

By

and 6 others •

Dec 9, 2024

• 56

published an article 4 months ago

Article

Let’s make a generation of amazing image generation models

By

and 4 others •

Nov 26, 2024

• 33

published an article 5 months ago

Article

How to optimize your data labelling project with custom interfaces

By

and 9 others •

Oct 16, 2024

• 18