Hugging Face's logo

Sentence Transformers

university
https://www.SBERT.net
nreimers
Activity Feed

AI & ML interests

In the following you find models tuned to be used for sentence / text embedding generation. They can be used with the sentence-transformers package.

Recent Activity

tomaarsen  new activity 7 days ago
sentence-transformers/distiluse-base-multilingual-cased-v2:Update README.md
tomaarsen  updated a dataset 11 days ago
sentence-transformers/msmarco
tomaarsen  updated a dataset 19 days ago
sentence-transformers/msmarco-msmarco-MiniLM-L6-v3
View all activity

sentence-transformers's activity

tomaarsen 
in sentence-transformers/distiluse-base-multilingual-cased-v2 7 days ago

Update README.md

1
#11 opened 8 days ago by
FischerWells
tomaarsen 
updated a dataset 11 days ago
tomaarsen 
posted an update 15 days ago
view post
Post
6508
An assembly of 18 European companies, labs, and universities have banded together to launch 🇪🇺 EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc.

🇪🇺 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi
3️⃣ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion
➡️ Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common.
⚙️ Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported.
🔥 A new Pareto frontier (stronger *and* smaller) for multilingual encoder models
📊 Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight.
📝 Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code.

Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release
* EuroBERT/EuroBERT-210m
* EuroBERT/EuroBERT-610m
* EuroBERT/EuroBERT-2.1B

The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!
  • 1 reply
·
tomaarsen 
updated a dataset 19 days ago
tomaarsen 
updated 16 models 19 days ago