dleemiller commited on
Commit
bb2939d
·
verified ·
1 Parent(s): a15018e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -10
README.md CHANGED
@@ -21,6 +21,19 @@ tags:
21
  model-index:
22
  - name: CrossEncoder based on jhu-clsp/ettin-encoder-17m
23
  results:
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  - task:
25
  type: semantic-similarity
26
  name: Semantic Similarity
@@ -28,12 +41,12 @@ model-index:
28
  name: sts dev
29
  type: sts-dev
30
  metrics:
31
- - type: pearson
32
- value: 0.8779482261036501
33
- name: Pearson
34
- - type: spearman
35
- value: 0.8762065013920353
36
- name: Spearman
37
  ---
38
 
39
  # EttinX Cross-Encoder: Semantic Similarity (STS)
@@ -49,7 +62,7 @@ which can easily process a few hundred sentence pairs per second on CPU, and a f
49
  ---
50
 
51
  ## Features
52
- - **High performing:** Achieves **Pearson: 0.8316** and **Spearman: 0.8211** on the STS-Benchmark test set.
53
  - **Efficient architecture:** Based on the Ettin-encoder design (17M parameters), offering very fast inference speeds.
54
  - **Extended context length:** Processes sequences up to 8192 tokens, great for LLM output evals.
55
  - **Diversified training:** Pretrained on `dleemiller/wiki-sim` and fine-tuned on `sentence-transformers/stsb`.
@@ -64,7 +77,7 @@ which can easily process a few hundred sentence pairs per second on CPU, and a f
64
  | `ModernCE-base-sts` | **0.9162** | **0.9122** | **8192** | 149M | **Fast** |
65
  | `stsb-roberta-large` | 0.9147 | - | 512 | 355M | Slow |
66
  | `stsb-distilroberta-base` | 0.8792 | - | 512 | 82M | Fast |
67
- | `EttinX-sts-xxs` | 0.8316 | 0.8211 | **8192** | 17M | **Very Fast** |
68
 
69
 
70
  ---
@@ -106,8 +119,8 @@ Fine-tuning was performed on the [`sentence-transformers/stsb`](https://huggingf
106
 
107
  ### Validation Results
108
  The model achieved the following test set performance after fine-tuning:
109
- - **Pearson Correlation:** 0.8316
110
- - **Spearman Correlation:** 0.8211
111
 
112
  ---
113
 
 
21
  model-index:
22
  - name: CrossEncoder based on jhu-clsp/ettin-encoder-17m
23
  results:
24
+ - task:
25
+ type: semantic-similarity
26
+ name: Semantic Similarity
27
+ dataset:
28
+ name: sts test
29
+ type: sts-test
30
+ metrics:
31
+ - type: pearson_cosine
32
+ value: 0.8413715686076841
33
+ name: Pearson Cosine
34
+ - type: spearman_cosine
35
+ value: 0.8310895302151975
36
+ name: Spearman Cosine
37
  - task:
38
  type: semantic-similarity
39
  name: Semantic Similarity
 
41
  name: sts dev
42
  type: sts-dev
43
  metrics:
44
+ - type: pearson_cosine
45
+ value: 0.8815197312565873
46
+ name: Pearson Cosine
47
+ - type: spearman_cosine
48
+ value: 0.8786002071426082
49
+ name: Spearman Cosine
50
  ---
51
 
52
  # EttinX Cross-Encoder: Semantic Similarity (STS)
 
62
  ---
63
 
64
  ## Features
65
+ - **High performing:** Achieves **Pearson: 0.8414** and **Spearman: 0.8311** on the STS-Benchmark test set.
66
  - **Efficient architecture:** Based on the Ettin-encoder design (17M parameters), offering very fast inference speeds.
67
  - **Extended context length:** Processes sequences up to 8192 tokens, great for LLM output evals.
68
  - **Diversified training:** Pretrained on `dleemiller/wiki-sim` and fine-tuned on `sentence-transformers/stsb`.
 
77
  | `ModernCE-base-sts` | **0.9162** | **0.9122** | **8192** | 149M | **Fast** |
78
  | `stsb-roberta-large` | 0.9147 | - | 512 | 355M | Slow |
79
  | `stsb-distilroberta-base` | 0.8792 | - | 512 | 82M | Fast |
80
+ | `EttinX-sts-xxs` | 0.8414 | 0.8311 | **8192** | 17M | **Very Fast** |
81
 
82
 
83
  ---
 
119
 
120
  ### Validation Results
121
  The model achieved the following test set performance after fine-tuning:
122
+ - **Pearson Correlation:** 0.8414
123
+ - **Spearman Correlation:** 0.8311
124
 
125
  ---
126