Update README.md
Browse files
README.md
CHANGED
@@ -21,6 +21,19 @@ tags:
|
|
21 |
model-index:
|
22 |
- name: CrossEncoder based on jhu-clsp/ettin-encoder-17m
|
23 |
results:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
- task:
|
25 |
type: semantic-similarity
|
26 |
name: Semantic Similarity
|
@@ -28,12 +41,12 @@ model-index:
|
|
28 |
name: sts dev
|
29 |
type: sts-dev
|
30 |
metrics:
|
31 |
-
- type:
|
32 |
-
value: 0.
|
33 |
-
name: Pearson
|
34 |
-
- type:
|
35 |
-
value: 0.
|
36 |
-
name: Spearman
|
37 |
---
|
38 |
|
39 |
# EttinX Cross-Encoder: Semantic Similarity (STS)
|
@@ -49,7 +62,7 @@ which can easily process a few hundred sentence pairs per second on CPU, and a f
|
|
49 |
---
|
50 |
|
51 |
## Features
|
52 |
-
- **High performing:** Achieves **Pearson: 0.
|
53 |
- **Efficient architecture:** Based on the Ettin-encoder design (17M parameters), offering very fast inference speeds.
|
54 |
- **Extended context length:** Processes sequences up to 8192 tokens, great for LLM output evals.
|
55 |
- **Diversified training:** Pretrained on `dleemiller/wiki-sim` and fine-tuned on `sentence-transformers/stsb`.
|
@@ -64,7 +77,7 @@ which can easily process a few hundred sentence pairs per second on CPU, and a f
|
|
64 |
| `ModernCE-base-sts` | **0.9162** | **0.9122** | **8192** | 149M | **Fast** |
|
65 |
| `stsb-roberta-large` | 0.9147 | - | 512 | 355M | Slow |
|
66 |
| `stsb-distilroberta-base` | 0.8792 | - | 512 | 82M | Fast |
|
67 |
-
| `EttinX-sts-xxs` | 0.
|
68 |
|
69 |
|
70 |
---
|
@@ -106,8 +119,8 @@ Fine-tuning was performed on the [`sentence-transformers/stsb`](https://huggingf
|
|
106 |
|
107 |
### Validation Results
|
108 |
The model achieved the following test set performance after fine-tuning:
|
109 |
-
- **Pearson Correlation:** 0.
|
110 |
-
- **Spearman Correlation:** 0.
|
111 |
|
112 |
---
|
113 |
|
|
|
21 |
model-index:
|
22 |
- name: CrossEncoder based on jhu-clsp/ettin-encoder-17m
|
23 |
results:
|
24 |
+
- task:
|
25 |
+
type: semantic-similarity
|
26 |
+
name: Semantic Similarity
|
27 |
+
dataset:
|
28 |
+
name: sts test
|
29 |
+
type: sts-test
|
30 |
+
metrics:
|
31 |
+
- type: pearson_cosine
|
32 |
+
value: 0.8413715686076841
|
33 |
+
name: Pearson Cosine
|
34 |
+
- type: spearman_cosine
|
35 |
+
value: 0.8310895302151975
|
36 |
+
name: Spearman Cosine
|
37 |
- task:
|
38 |
type: semantic-similarity
|
39 |
name: Semantic Similarity
|
|
|
41 |
name: sts dev
|
42 |
type: sts-dev
|
43 |
metrics:
|
44 |
+
- type: pearson_cosine
|
45 |
+
value: 0.8815197312565873
|
46 |
+
name: Pearson Cosine
|
47 |
+
- type: spearman_cosine
|
48 |
+
value: 0.8786002071426082
|
49 |
+
name: Spearman Cosine
|
50 |
---
|
51 |
|
52 |
# EttinX Cross-Encoder: Semantic Similarity (STS)
|
|
|
62 |
---
|
63 |
|
64 |
## Features
|
65 |
+
- **High performing:** Achieves **Pearson: 0.8414** and **Spearman: 0.8311** on the STS-Benchmark test set.
|
66 |
- **Efficient architecture:** Based on the Ettin-encoder design (17M parameters), offering very fast inference speeds.
|
67 |
- **Extended context length:** Processes sequences up to 8192 tokens, great for LLM output evals.
|
68 |
- **Diversified training:** Pretrained on `dleemiller/wiki-sim` and fine-tuned on `sentence-transformers/stsb`.
|
|
|
77 |
| `ModernCE-base-sts` | **0.9162** | **0.9122** | **8192** | 149M | **Fast** |
|
78 |
| `stsb-roberta-large` | 0.9147 | - | 512 | 355M | Slow |
|
79 |
| `stsb-distilroberta-base` | 0.8792 | - | 512 | 82M | Fast |
|
80 |
+
| `EttinX-sts-xxs` | 0.8414 | 0.8311 | **8192** | 17M | **Very Fast** |
|
81 |
|
82 |
|
83 |
---
|
|
|
119 |
|
120 |
### Validation Results
|
121 |
The model achieved the following test set performance after fine-tuning:
|
122 |
+
- **Pearson Correlation:** 0.8414
|
123 |
+
- **Spearman Correlation:** 0.8311
|
124 |
|
125 |
---
|
126 |
|