Upload README.md
Browse files
README.md
CHANGED
@@ -24,8 +24,9 @@ language:
|
|
24 |
1. [Introduction](#introduction)
|
25 |
2. [Technical Report](#technical-report)
|
26 |
3. [Highlights](#highlights)
|
27 |
-
4. [
|
28 |
-
5. [
|
|
|
29 |
|
30 |
# Introduction
|
31 |
|
@@ -48,6 +49,24 @@ A technical report detailing our proposed `LEAF` training procedure will be avai
|
|
48 |
* **Flexible Architecture Support**: `mdbr-leaf-mt` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
|
49 |
* **MRL and Quantization Support**: embedding vectors generated by `mdbr-leaf-mt` compress well when truncated (MRL) and can be stored using more efficient types like `int8` and `binary`. [See below](#mrl) for more information.
|
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
# Quickstart
|
52 |
|
53 |
## Sentence Transformers
|
|
|
24 |
1. [Introduction](#introduction)
|
25 |
2. [Technical Report](#technical-report)
|
26 |
3. [Highlights](#highlights)
|
27 |
+
4. [Benchmarks](#benchmark-comparison)
|
28 |
+
5. [Quickstart](#quickstart)
|
29 |
+
6. [Citation](#citation)
|
30 |
|
31 |
# Introduction
|
32 |
|
|
|
49 |
* **Flexible Architecture Support**: `mdbr-leaf-mt` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
|
50 |
* **MRL and Quantization Support**: embedding vectors generated by `mdbr-leaf-mt` compress well when truncated (MRL) and can be stored using more efficient types like `int8` and `binary`. [See below](#mrl) for more information.
|
51 |
|
52 |
+
## Benchmark Comparison
|
53 |
+
|
54 |
+
The table below shows the scores for `mdbr-leaf-mt` on the MTEB v2 (English) benchmark, compared to other retrieval models.
|
55 |
+
|
56 |
+
`mdbr-leaf-mt` ranks #1 on this benchmark for models with <30M parameters.
|
57 |
+
|
58 |
+
| Model | Size | MTEB v2 (Eng) |
|
59 |
+
|------------------------------------|---------|---------------|
|
60 |
+
| OpenAI text-embedding-3-large | Unknown | 66.43 |
|
61 |
+
| OpenAI text-embedding-3-small | Unknown | 64.56 |
|
62 |
+
| **mdbr-leaf-mt** | 23M | **63.97** |
|
63 |
+
| gte-small | 33M | 63.22 |
|
64 |
+
| snowflake-arctic-embed-s | 32M | 61.59 |
|
65 |
+
| e5-small-v2 | 33M | 61.32 |
|
66 |
+
| granite-embedding-small-english-r2 | 47M | 61.07 |
|
67 |
+
| all-MiniLM-L6-v2 | 22M | 59.03 |
|
68 |
+
|
69 |
+
|
70 |
# Quickstart
|
71 |
|
72 |
## Sentence Transformers
|