rvo commited on
Commit
8321e9a
·
verified ·
1 Parent(s): 2993c48

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -2
README.md CHANGED
@@ -24,8 +24,9 @@ language:
24
  1. [Introduction](#introduction)
25
  2. [Technical Report](#technical-report)
26
  3. [Highlights](#highlights)
27
- 4. [Quickstart](#quickstart)
28
- 5. [Citation](#citation)
 
29
 
30
  # Introduction
31
 
@@ -48,6 +49,24 @@ A technical report detailing our proposed `LEAF` training procedure will be avai
48
  * **Flexible Architecture Support**: `mdbr-leaf-mt` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
49
  * **MRL and Quantization Support**: embedding vectors generated by `mdbr-leaf-mt` compress well when truncated (MRL) and can be stored using more efficient types like `int8` and `binary`. [See below](#mrl) for more information.
50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  # Quickstart
52
 
53
  ## Sentence Transformers
 
24
  1. [Introduction](#introduction)
25
  2. [Technical Report](#technical-report)
26
  3. [Highlights](#highlights)
27
+ 4. [Benchmarks](#benchmark-comparison)
28
+ 5. [Quickstart](#quickstart)
29
+ 6. [Citation](#citation)
30
 
31
  # Introduction
32
 
 
49
  * **Flexible Architecture Support**: `mdbr-leaf-mt` supports asymmetric retrieval architectures enabling even greater retrieval results. [See below](#asymmetric-retrieval-setup) for more information.
50
  * **MRL and Quantization Support**: embedding vectors generated by `mdbr-leaf-mt` compress well when truncated (MRL) and can be stored using more efficient types like `int8` and `binary`. [See below](#mrl) for more information.
51
 
52
+ ## Benchmark Comparison
53
+
54
+ The table below shows the scores for `mdbr-leaf-mt` on the MTEB v2 (English) benchmark, compared to other retrieval models.
55
+
56
+ `mdbr-leaf-mt` ranks #1 on this benchmark for models with <30M parameters.
57
+
58
+ | Model | Size | MTEB v2 (Eng) |
59
+ |------------------------------------|---------|---------------|
60
+ | OpenAI text-embedding-3-large | Unknown | 66.43 |
61
+ | OpenAI text-embedding-3-small | Unknown | 64.56 |
62
+ | **mdbr-leaf-mt** | 23M | **63.97** |
63
+ | gte-small | 33M | 63.22 |
64
+ | snowflake-arctic-embed-s | 32M | 61.59 |
65
+ | e5-small-v2 | 33M | 61.32 |
66
+ | granite-embedding-small-english-r2 | 47M | 61.07 |
67
+ | all-MiniLM-L6-v2 | 22M | 59.03 |
68
+
69
+
70
  # Quickstart
71
 
72
  ## Sentence Transformers