sergeyzh commited on
Commit
8c35638
·
verified ·
1 Parent(s): 1f6cd66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +28 -3
README.md CHANGED
@@ -79,14 +79,39 @@ print(util.dot_score(embeddings, embeddings))
79
 
80
  ## Быстродействие и размеры
81
 
 
 
82
  | Модель | CPU | GPU | size | dim | n_ctx | n_vocab |
83
  |:---------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
84
  | [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 149.026 | 15.629 | 2136 | 1024 | 514 | 250002 |
85
  | [sergeyzh/LaBSE-ru-sts](https://huggingface.co/sergeyzh/LaBSE-ru-sts) | 42.835 | 8.561 | 490 | 768 | 512 | 55083 |
86
- | **sergeyzh/rubert-tiny-sts** | **3.208** | **2.866** | **111** | **312** | **2048** | **83828** |
87
- | [Tochka-AI/ruRoPEBert-e5-base-512](https://huggingface.co/Tochka-AI/ruRoPEBert-e5-base-512) | 43.314 | 9.338 | 530 | 768 | 512 | 69382 |
88
  | [cointegrated/LaBSE-en-ru](https://huggingface.co/cointegrated/LaBSE-en-ru) | 42.867 | 8.549 | 490 | 768 | 512 | 55083 |
89
- | [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) | 3.212 | 2.850 | 111 | 312 | 2048 | 83828 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
 
92
  ## Связанные ресурсы
 
79
 
80
  ## Быстродействие и размеры
81
 
82
+ На бенчмарке [encodechka](https://github.com/avidale/encodechka):
83
+
84
  | Модель | CPU | GPU | size | dim | n_ctx | n_vocab |
85
  |:---------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
86
  | [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 149.026 | 15.629 | 2136 | 1024 | 514 | 250002 |
87
  | [sergeyzh/LaBSE-ru-sts](https://huggingface.co/sergeyzh/LaBSE-ru-sts) | 42.835 | 8.561 | 490 | 768 | 512 | 55083 |
88
+ | **sergeyzh/rubert-tiny-sts** | **3.208** | **3.379** | **111** | **312** | **2048** | **83828** |
89
+ | [Tochka-AI/ruRoPEBert-e5-base-512](https://huggingface.co/Tochka-AI/ruRoPEBert-e5-base-512) | 43.314 | 9.338 | 532 | 768 | 512 | 69382 |
90
  | [cointegrated/LaBSE-en-ru](https://huggingface.co/cointegrated/LaBSE-en-ru) | 42.867 | 8.549 | 490 | 768 | 512 | 55083 |
91
+ | [cointegrated/rubert-tiny2](https://huggingface.co/cointegrated/rubert-tiny2) | 3.212 | 3.384 | 111 | 312 | 2048 | 83828 |
92
+
93
+
94
+
95
+ При использовании батчей с `sentence_transformers`:
96
+
97
+ ```python
98
+ from sentence_transformers import SentenceTransformer
99
+
100
+ model_name = 'sergeyzh/rubert-tiny-sts'
101
+ model = SentenceTransformer(model_name, device='cpu')
102
+ sentences = ["Тест быстродействия на CPU Ryzen 7 3800X: batch = 1000"] * 1000
103
+ %timeit -n 5 -r 3 model.encode(sentences)
104
+
105
+ # 840 ms ± 8.08 ms per loop (mean ± std. dev. of 3 runs, 5 loops each)
106
+ # 1000/0.840 = 1190 snt/s
107
+
108
+ model = SentenceTransformer(model_name, device='cuda')
109
+ sentences = ["Тест быстродействия на GPU RTX 3060: batch = 8000"] * 8000
110
+ %timeit -n 5 -r 3 model.encode(sentences)
111
+
112
+ # 922 ms ± 29.5 ms per loop (mean ± std. dev. of 3 runs, 5 loops each)
113
+ # 8000/0.922 = 8677 snt/s
114
+ ```
115
 
116
 
117
  ## Связанные ресурсы