Nuf-hugginface commited on
Commit
30ab736
·
verified ·
1 Parent(s): 2aec704

Add new SentenceTransformer model

Browse files
Files changed (2) hide show
  1. README.md +137 -144
  2. model.safetensors +1 -1
README.md CHANGED
@@ -12,82 +12,75 @@ tags:
12
  - loss:MultipleNegativesRankingLoss
13
  base_model: nomic-ai/modernbert-embed-base
14
  widget:
15
- - source_sentence: What does 'multi-modal' refer to in the context of the services
16
- mentioned?
17
  sentences:
18
- - '1. Chatbots and Virtual Assistants
19
-
20
- One of the most visible LLM integrations is in chatbots. Tools like ChatGPT, Claude,
21
- and Bard are themselves chatbot interfaces built on LLMs. Many businesses are
22
- now integrating these models into their websites and customer support systems.'
23
- - For example, e-commerce websites can deploy LLM-powered assistants to help customers
24
- find products, track orders, or get personalized recommendations—much more effectively
25
- than traditional rule-based bots.
26
- - Some services, like ColBERT, Marqo, and ColQwen, specialize in integrating LLMs
27
- into search pipelines for both text and multi-modal (text + image) content.
28
- - source_sentence: What is one method mentioned for deploying LLMs?
29
- sentences:
30
- - However, deploying LLMs effectively in real-world applications often requires
31
- LLM integration. This means embedding these models into systems, workflows, or
32
- products where they can interact with other components like databases, APIs, user
33
- interfaces, or even custom business logic
34
- - Some services, like ColBERT, Marqo, and ColQwen, specialize in integrating LLMs
35
- into search pipelines for both text and multi-modal (text + image) content.
36
- - However, deploying LLMs effectively in real-world applications often requires
37
- LLM integration. This means embedding these models into systems, workflows, or
38
- products where they can interact with other components like databases, APIs, user
39
- interfaces, or even custom business logic
40
- - source_sentence: What will an LLM likely respond with when prompted about the capital
41
- of France?
42
- sentences:
43
- - . For instance, a spam filter doesn’t just block emails with specific keywords—it
44
- learns from thousands of examples what spam typically looks like.
45
  - Over the past few years, the field of ML has advanced rapidly, especially in the
46
  area of Natural Language Processing (NLP)—the ability of machines to understand
47
  and generate human language. At the forefront of this progress are Large Language
48
  Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s
49
  PaLM, and Meta’s LLaMA
50
- - For example, given a prompt like "The capital of France is", an LLM trained on
51
- a wide range of texts will likely respond with "Paris". But beyond trivia, LLMs
52
- can write essays, draft emails, simulate conversations, generate code snippets,
53
- and much more.
54
- - source_sentence: What might an LLM be connected to in a customer support chatbot?
55
- sentences:
56
- - . For instance, a spam filter doesn’t just block emails with specific keywords—it
57
- learns from thousands of examples what spam typically looks like.
58
  - . For example, integrating an LLM into a customer support chatbot might involve
59
  connecting it to a company’s internal knowledge base, enabling it to answer customer
60
  questions using accurate, up-to-date information.
61
- - Large Language Models (LLMs) and Their Integrations
62
- - source_sentence: What type of dialogues can LLMs simulate?
 
 
 
 
63
  sentences:
64
- - 'Hallucinations: LLMs can sometimes generate plausible-sounding but incorrect
65
- or fictional information.
66
-
67
-
68
- Data Privacy: Sending sensitive data to third-party models raises privacy and
69
- compliance concerns.
70
-
71
-
72
- Cost and Latency: Running LLMs, especially large ones, can be computationally
73
- expensive and slow.'
74
- - '6. APIs and Developer Tools
75
-
76
- Developers can integrate LLMs into their own apps using APIs provided by companies
77
- like OpenAI, Anthropic, and Cohere. These APIs allow developers to send prompts
78
- and receive intelligent outputs in return.
79
 
 
 
 
80
 
81
- This enables custom applications like:
82
 
 
83
 
84
- Smart assistants in mobile apps
 
 
 
 
 
85
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86
 
87
- AI-powered research tools
 
 
88
 
89
 
90
- Voice interfaces'
 
 
 
 
 
 
 
 
 
 
 
 
91
  - '5. Education and Learning Platforms
92
 
93
  Educational tools like Khanmigo (from Khan Academy) and other tutoring platforms
@@ -122,49 +115,49 @@ model-index:
122
  type: dim_768
123
  metrics:
124
  - type: cosine_accuracy@1
125
- value: 0.6
126
  name: Cosine Accuracy@1
127
  - type: cosine_accuracy@3
128
- value: 0.8666666666666667
129
  name: Cosine Accuracy@3
130
  - type: cosine_accuracy@5
131
- value: 0.8666666666666667
132
  name: Cosine Accuracy@5
133
  - type: cosine_accuracy@10
134
  value: 1.0
135
  name: Cosine Accuracy@10
136
  - type: cosine_precision@1
137
- value: 0.6
138
  name: Cosine Precision@1
139
  - type: cosine_precision@3
140
- value: 0.28888888888888886
141
  name: Cosine Precision@3
142
  - type: cosine_precision@5
143
- value: 0.17333333333333337
144
  name: Cosine Precision@5
145
  - type: cosine_precision@10
146
  value: 0.10000000000000003
147
  name: Cosine Precision@10
148
  - type: cosine_recall@1
149
- value: 0.6
150
  name: Cosine Recall@1
151
  - type: cosine_recall@3
152
- value: 0.8666666666666667
153
  name: Cosine Recall@3
154
  - type: cosine_recall@5
155
- value: 0.8666666666666667
156
  name: Cosine Recall@5
157
  - type: cosine_recall@10
158
  value: 1.0
159
  name: Cosine Recall@10
160
  - type: cosine_ndcg@10
161
- value: 0.8025374182760189
162
  name: Cosine Ndcg@10
163
  - type: cosine_mrr@10
164
- value: 0.74
165
  name: Cosine Mrr@10
166
  - type: cosine_map@100
167
- value: 0.74
168
  name: Cosine Map@100
169
  - task:
170
  type: information-retrieval
@@ -180,10 +173,10 @@ model-index:
180
  value: 0.8
181
  name: Cosine Accuracy@3
182
  - type: cosine_accuracy@5
183
- value: 0.8
184
  name: Cosine Accuracy@5
185
  - type: cosine_accuracy@10
186
- value: 0.9333333333333333
187
  name: Cosine Accuracy@10
188
  - type: cosine_precision@1
189
  value: 0.6666666666666666
@@ -192,10 +185,10 @@ model-index:
192
  value: 0.2666666666666667
193
  name: Cosine Precision@3
194
  - type: cosine_precision@5
195
- value: 0.16000000000000003
196
  name: Cosine Precision@5
197
  - type: cosine_precision@10
198
- value: 0.09333333333333335
199
  name: Cosine Precision@10
200
  - type: cosine_recall@1
201
  value: 0.6666666666666666
@@ -204,19 +197,19 @@ model-index:
204
  value: 0.8
205
  name: Cosine Recall@3
206
  - type: cosine_recall@5
207
- value: 0.8
208
  name: Cosine Recall@5
209
  - type: cosine_recall@10
210
- value: 0.9333333333333333
211
  name: Cosine Recall@10
212
  - type: cosine_ndcg@10
213
- value: 0.7955687714024445
214
  name: Cosine Ndcg@10
215
  - type: cosine_mrr@10
216
- value: 0.7527777777777779
217
  name: Cosine Mrr@10
218
  - type: cosine_map@100
219
- value: 0.7583333333333333
220
  name: Cosine Map@100
221
  - task:
222
  type: information-retrieval
@@ -229,10 +222,10 @@ model-index:
229
  value: 0.6666666666666666
230
  name: Cosine Accuracy@1
231
  - type: cosine_accuracy@3
232
- value: 0.8
233
  name: Cosine Accuracy@3
234
  - type: cosine_accuracy@5
235
- value: 0.8
236
  name: Cosine Accuracy@5
237
  - type: cosine_accuracy@10
238
  value: 1.0
@@ -241,10 +234,10 @@ model-index:
241
  value: 0.6666666666666666
242
  name: Cosine Precision@1
243
  - type: cosine_precision@3
244
- value: 0.2666666666666667
245
  name: Cosine Precision@3
246
  - type: cosine_precision@5
247
- value: 0.16000000000000003
248
  name: Cosine Precision@5
249
  - type: cosine_precision@10
250
  value: 0.10000000000000003
@@ -253,22 +246,22 @@ model-index:
253
  value: 0.6666666666666666
254
  name: Cosine Recall@1
255
  - type: cosine_recall@3
256
- value: 0.8
257
  name: Cosine Recall@3
258
  - type: cosine_recall@5
259
- value: 0.8
260
  name: Cosine Recall@5
261
  - type: cosine_recall@10
262
  value: 1.0
263
  name: Cosine Recall@10
264
  - type: cosine_ndcg@10
265
- value: 0.7985736897839496
266
  name: Cosine Ndcg@10
267
  - type: cosine_mrr@10
268
- value: 0.7384126984126984
269
  name: Cosine Mrr@10
270
  - type: cosine_map@100
271
- value: 0.7384126984126984
272
  name: Cosine Map@100
273
  - task:
274
  type: information-retrieval
@@ -278,49 +271,49 @@ model-index:
278
  type: dim_128
279
  metrics:
280
  - type: cosine_accuracy@1
281
- value: 0.6666666666666666
282
  name: Cosine Accuracy@1
283
  - type: cosine_accuracy@3
284
- value: 0.8
285
  name: Cosine Accuracy@3
286
  - type: cosine_accuracy@5
287
  value: 0.8
288
  name: Cosine Accuracy@5
289
  - type: cosine_accuracy@10
290
- value: 0.8666666666666667
291
  name: Cosine Accuracy@10
292
  - type: cosine_precision@1
293
- value: 0.6666666666666666
294
  name: Cosine Precision@1
295
  - type: cosine_precision@3
296
- value: 0.2666666666666667
297
  name: Cosine Precision@3
298
  - type: cosine_precision@5
299
  value: 0.16000000000000003
300
  name: Cosine Precision@5
301
  - type: cosine_precision@10
302
- value: 0.08666666666666668
303
  name: Cosine Precision@10
304
  - type: cosine_recall@1
305
- value: 0.6666666666666666
306
  name: Cosine Recall@1
307
  - type: cosine_recall@3
308
- value: 0.8
309
  name: Cosine Recall@3
310
  - type: cosine_recall@5
311
  value: 0.8
312
  name: Cosine Recall@5
313
  - type: cosine_recall@10
314
- value: 0.8666666666666667
315
  name: Cosine Recall@10
316
  - type: cosine_ndcg@10
317
- value: 0.7700616222307202
318
  name: Cosine Ndcg@10
319
  - type: cosine_mrr@10
320
- value: 0.74
321
  name: Cosine Mrr@10
322
  - type: cosine_map@100
323
- value: 0.7479365079365079
324
  name: Cosine Map@100
325
  - task:
326
  type: information-retrieval
@@ -330,49 +323,49 @@ model-index:
330
  type: dim_64
331
  metrics:
332
  - type: cosine_accuracy@1
333
- value: 0.6
334
  name: Cosine Accuracy@1
335
  - type: cosine_accuracy@3
336
- value: 0.8
337
  name: Cosine Accuracy@3
338
  - type: cosine_accuracy@5
339
  value: 0.8
340
  name: Cosine Accuracy@5
341
  - type: cosine_accuracy@10
342
- value: 0.8
343
  name: Cosine Accuracy@10
344
  - type: cosine_precision@1
345
- value: 0.6
346
  name: Cosine Precision@1
347
  - type: cosine_precision@3
348
- value: 0.2666666666666667
349
  name: Cosine Precision@3
350
  - type: cosine_precision@5
351
  value: 0.16000000000000003
352
  name: Cosine Precision@5
353
  - type: cosine_precision@10
354
- value: 0.08000000000000002
355
  name: Cosine Precision@10
356
  - type: cosine_recall@1
357
- value: 0.6
358
  name: Cosine Recall@1
359
  - type: cosine_recall@3
360
- value: 0.8
361
  name: Cosine Recall@3
362
  - type: cosine_recall@5
363
  value: 0.8
364
  name: Cosine Recall@5
365
  - type: cosine_recall@10
366
- value: 0.8
367
  name: Cosine Recall@10
368
  - type: cosine_ndcg@10
369
- value: 0.7174573004761944
370
  name: Cosine Ndcg@10
371
  - type: cosine_mrr@10
372
- value: 0.6888888888888889
373
  name: Cosine Mrr@10
374
  - type: cosine_map@100
375
- value: 0.7003968253968255
376
  name: Cosine Map@100
377
  ---
378
 
@@ -428,7 +421,7 @@ model = SentenceTransformer("Nuf-hugginface/modernbert-embed-quickb")
428
  sentences = [
429
  'What type of dialogues can LLMs simulate?',
430
  '5. Education and Learning Platforms\nEducational tools like Khanmigo (from Khan Academy) and other tutoring platforms are leveraging LLMs to provide real-time help to students. LLMs can break down complex topics, provide feedback on writing, and simulate Socratic-style dialogues.',
431
- '6. APIs and Developer Tools\nDevelopers can integrate LLMs into their own apps using APIs provided by companies like OpenAI, Anthropic, and Cohere. These APIs allow developers to send prompts and receive intelligent outputs in return.\n\nThis enables custom applications like:\n\nSmart assistants in mobile apps\n\nAI-powered research tools\n\nVoice interfaces',
432
  ]
433
  embeddings = model.encode(sentences)
434
  print(embeddings.shape)
@@ -475,21 +468,21 @@ You can finetune this model on your own dataset.
475
 
476
  | Metric | dim_768 | dim_512 | dim_256 | dim_128 | dim_64 |
477
  |:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------|
478
- | cosine_accuracy@1 | 0.6 | 0.6667 | 0.6667 | 0.6667 | 0.6 |
479
- | cosine_accuracy@3 | 0.8667 | 0.8 | 0.8 | 0.8 | 0.8 |
480
- | cosine_accuracy@5 | 0.8667 | 0.8 | 0.8 | 0.8 | 0.8 |
481
- | cosine_accuracy@10 | 1.0 | 0.9333 | 1.0 | 0.8667 | 0.8 |
482
- | cosine_precision@1 | 0.6 | 0.6667 | 0.6667 | 0.6667 | 0.6 |
483
- | cosine_precision@3 | 0.2889 | 0.2667 | 0.2667 | 0.2667 | 0.2667 |
484
- | cosine_precision@5 | 0.1733 | 0.16 | 0.16 | 0.16 | 0.16 |
485
- | cosine_precision@10 | 0.1 | 0.0933 | 0.1 | 0.0867 | 0.08 |
486
- | cosine_recall@1 | 0.6 | 0.6667 | 0.6667 | 0.6667 | 0.6 |
487
- | cosine_recall@3 | 0.8667 | 0.8 | 0.8 | 0.8 | 0.8 |
488
- | cosine_recall@5 | 0.8667 | 0.8 | 0.8 | 0.8 | 0.8 |
489
- | cosine_recall@10 | 1.0 | 0.9333 | 1.0 | 0.8667 | 0.8 |
490
- | **cosine_ndcg@10** | **0.8025** | **0.7956** | **0.7986** | **0.7701** | **0.7175** |
491
- | cosine_mrr@10 | 0.74 | 0.7528 | 0.7384 | 0.74 | 0.6889 |
492
- | cosine_map@100 | 0.74 | 0.7583 | 0.7384 | 0.7479 | 0.7004 |
493
 
494
  <!--
495
  ## Bias, Risks and Limitations
@@ -512,16 +505,16 @@ You can finetune this model on your own dataset.
512
  * Size: 127 training samples
513
  * Columns: <code>anchor</code> and <code>positive</code>
514
  * Approximate statistics based on the first 127 samples:
515
- | | anchor | positive |
516
- |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
517
- | type | string | string |
518
- | details | <ul><li>min: 8 tokens</li><li>mean: 13.2 tokens</li><li>max: 20 tokens</li></ul> | <ul><li>min: 13 tokens</li><li>mean: 53.85 tokens</li><li>max: 86 tokens</li></ul> |
519
  * Samples:
520
- | anchor | positive |
521
- |:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
522
- | <code>What documents could the system retrieve in relation to sick leave?</code> | <code>For instance, in a document management system, a user might type "policies about sick leave", and the system—integrated with an LLM—could retrieve documents discussing "medical leave", "employee absence", and "illness policies", even if those exact words weren’t used.</code> |
523
- | <code>What is one of the most visible integrations of LLM technology?</code> | <code>1. Chatbots and Virtual Assistants<br>One of the most visible LLM integrations is in chatbots. Tools like ChatGPT, Claude, and Bard are themselves chatbot interfaces built on LLMs. Many businesses are now integrating these models into their websites and customer support systems.</code> |
524
- | <code>What does AI stand for?</code> | <code>Introduction to AI, Machine Learning, LLMs, and Their Integration</code> |
525
  * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
526
  ```json
527
  {
@@ -680,13 +673,13 @@ You can finetune this model on your own dataset.
680
  </details>
681
 
682
  ### Training Logs
683
- | Epoch | Step | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
684
- |:-------:|:-----:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
685
- | 1.0 | 4 | - | 0.7853 | 0.8214 | 0.7673 | 0.7586 | 0.6883 |
686
- | **2.0** | **8** | **-** | **0.7764** | **0.7902** | **0.7686** | **0.7701** | **0.7321** |
687
- | 2.5 | 10 | 13.8004 | - | - | - | - | - |
688
- | 3.0 | 12 | - | 0.8028 | 0.7710 | 0.7932 | 0.7701 | 0.7175 |
689
- | 4.0 | 16 | - | 0.8025 | 0.7956 | 0.7986 | 0.7701 | 0.7175 |
690
 
691
  * The bold row denotes the saved checkpoint.
692
 
 
12
  - loss:MultipleNegativesRankingLoss
13
  base_model: nomic-ai/modernbert-embed-base
14
  widget:
15
+ - source_sentence: What is the difference between traditional programming and ML?
 
16
  sentences:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  - Over the past few years, the field of ML has advanced rapidly, especially in the
18
  area of Natural Language Processing (NLP)—the ability of machines to understand
19
  and generate human language. At the forefront of this progress are Large Language
20
  Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformer), Google’s
21
  PaLM, and Meta’s LLaMA
 
 
 
 
 
 
 
 
22
  - . For example, integrating an LLM into a customer support chatbot might involve
23
  connecting it to a company’s internal knowledge base, enabling it to answer customer
24
  questions using accurate, up-to-date information.
25
+ - A major subset of AI is Machine Learning (ML), which involves algorithms that
26
+ learn from data rather than being explicitly programmed. Instead of writing detailed
27
+ instructions for every task, ML models find patterns in large datasets and use
28
+ these patterns to make predictions or decisions
29
+ - source_sentence: What is one of the tasks mentioned that involves creating new written
30
+ content?
31
  sentences:
32
+ - In summary, AI and ML form the foundation for intelligent automation, while LLMs
33
+ represent a breakthrough in language understanding and generation. Integrating
34
+ these models into real-world systems unlocks practical value, turning raw intelligence
35
+ into tangible solutions
36
+ - '8. Security and Compliance Integrations
 
 
 
 
 
 
 
 
 
 
37
 
38
+ Some organizations are integrating LLMs to detect anomalies in text communications
39
+ (e.g., phishing detection or policy violations). LLMs can analyze language usage
40
+ and flag potentially suspicious behavior more flexibly than keyword-based filters.
41
 
 
42
 
43
+ Challenges in LLM Integration
44
 
45
+ Despite their promise, integrating LLMs comes with challenges:'
46
+ - . These include text generation, summarization, translation, question answering,
47
+ code generation, and more.
48
+ - source_sentence: What is one of the components mentioned alongside AI?
49
+ sentences:
50
+ - '2. Search Engines and Semantic Search
51
 
52
+ Traditional keyword-based search systems are being enhanced or replaced by semantic
53
+ search, where LLMs understand the meaning behind queries. Instead of just matching
54
+ words, they interpret intent.'
55
+ - For example, e-commerce websites can deploy LLM-powered assistants to help customers
56
+ find products, track orders, or get personalized recommendations—much more effectively
57
+ than traditional rule-based bots.
58
+ - Introduction to AI, Machine Learning, LLMs, and Their Integration
59
+ - source_sentence: What is required to provide intelligent features within broader
60
+ applications?
61
+ sentences:
62
+ - . For instance, a spam filter doesn’t just block emails with specific keywords—it
63
+ learns from thousands of examples what spam typically looks like.
64
+ - 'The Rise of LLM Integrations
65
 
66
+ While LLMs are powerful on their own, their true potential is unlocked through
67
+ integration—connecting these models with other software, services, or systems
68
+ to provide intelligent features within broader applications.
69
 
70
 
71
+ Here are some key ways LLMs are being integrated into the digital world:'
72
+ - For instance, in a document management system, a user might type "policies about
73
+ sick leave", and the system—integrated with an LLM—could retrieve documents discussing
74
+ "medical leave", "employee absence", and "illness policies", even if those exact
75
+ words weren’t used.
76
+ - source_sentence: What type of dialogues can LLMs simulate?
77
+ sentences:
78
+ - Companies are also experimenting with Retrieval-Augmented Generation (RAG)—a technique
79
+ where LLMs are paired with document databases (e.g., vector stores like Supabase,
80
+ Pinecone, or Weaviate) to answer questions with enterprise-specific knowledge.
81
+ - . For example, integrating an LLM into a customer support chatbot might involve
82
+ connecting it to a company’s internal knowledge base, enabling it to answer customer
83
+ questions using accurate, up-to-date information.
84
  - '5. Education and Learning Platforms
85
 
86
  Educational tools like Khanmigo (from Khan Academy) and other tutoring platforms
 
115
  type: dim_768
116
  metrics:
117
  - type: cosine_accuracy@1
118
+ value: 0.6666666666666666
119
  name: Cosine Accuracy@1
120
  - type: cosine_accuracy@3
121
+ value: 0.8
122
  name: Cosine Accuracy@3
123
  - type: cosine_accuracy@5
124
+ value: 1.0
125
  name: Cosine Accuracy@5
126
  - type: cosine_accuracy@10
127
  value: 1.0
128
  name: Cosine Accuracy@10
129
  - type: cosine_precision@1
130
+ value: 0.6666666666666666
131
  name: Cosine Precision@1
132
  - type: cosine_precision@3
133
+ value: 0.2666666666666667
134
  name: Cosine Precision@3
135
  - type: cosine_precision@5
136
+ value: 0.20000000000000007
137
  name: Cosine Precision@5
138
  - type: cosine_precision@10
139
  value: 0.10000000000000003
140
  name: Cosine Precision@10
141
  - type: cosine_recall@1
142
+ value: 0.6666666666666666
143
  name: Cosine Recall@1
144
  - type: cosine_recall@3
145
+ value: 0.8
146
  name: Cosine Recall@3
147
  - type: cosine_recall@5
148
+ value: 1.0
149
  name: Cosine Recall@5
150
  - type: cosine_recall@10
151
  value: 1.0
152
  name: Cosine Recall@10
153
  - type: cosine_ndcg@10
154
+ value: 0.8310827786456928
155
  name: Cosine Ndcg@10
156
  - type: cosine_mrr@10
157
+ value: 0.7766666666666667
158
  name: Cosine Mrr@10
159
  - type: cosine_map@100
160
+ value: 0.7766666666666667
161
  name: Cosine Map@100
162
  - task:
163
  type: information-retrieval
 
173
  value: 0.8
174
  name: Cosine Accuracy@3
175
  - type: cosine_accuracy@5
176
+ value: 0.8666666666666667
177
  name: Cosine Accuracy@5
178
  - type: cosine_accuracy@10
179
+ value: 1.0
180
  name: Cosine Accuracy@10
181
  - type: cosine_precision@1
182
  value: 0.6666666666666666
 
185
  value: 0.2666666666666667
186
  name: Cosine Precision@3
187
  - type: cosine_precision@5
188
+ value: 0.17333333333333337
189
  name: Cosine Precision@5
190
  - type: cosine_precision@10
191
+ value: 0.10000000000000003
192
  name: Cosine Precision@10
193
  - type: cosine_recall@1
194
  value: 0.6666666666666666
 
197
  value: 0.8
198
  name: Cosine Recall@3
199
  - type: cosine_recall@5
200
+ value: 0.8666666666666667
201
  name: Cosine Recall@5
202
  - type: cosine_recall@10
203
+ value: 1.0
204
  name: Cosine Recall@10
205
  - type: cosine_ndcg@10
206
+ value: 0.8203966331432972
207
  name: Cosine Ndcg@10
208
  - type: cosine_mrr@10
209
+ value: 0.7651851851851852
210
  name: Cosine Mrr@10
211
  - type: cosine_map@100
212
+ value: 0.7651851851851852
213
  name: Cosine Map@100
214
  - task:
215
  type: information-retrieval
 
222
  value: 0.6666666666666666
223
  name: Cosine Accuracy@1
224
  - type: cosine_accuracy@3
225
+ value: 0.8666666666666667
226
  name: Cosine Accuracy@3
227
  - type: cosine_accuracy@5
228
+ value: 1.0
229
  name: Cosine Accuracy@5
230
  - type: cosine_accuracy@10
231
  value: 1.0
 
234
  value: 0.6666666666666666
235
  name: Cosine Precision@1
236
  - type: cosine_precision@3
237
+ value: 0.28888888888888886
238
  name: Cosine Precision@3
239
  - type: cosine_precision@5
240
+ value: 0.20000000000000007
241
  name: Cosine Precision@5
242
  - type: cosine_precision@10
243
  value: 0.10000000000000003
 
246
  value: 0.6666666666666666
247
  name: Cosine Recall@1
248
  - type: cosine_recall@3
249
+ value: 0.8666666666666667
250
  name: Cosine Recall@3
251
  - type: cosine_recall@5
252
+ value: 1.0
253
  name: Cosine Recall@5
254
  - type: cosine_recall@10
255
  value: 1.0
256
  name: Cosine Recall@10
257
  - type: cosine_ndcg@10
258
+ value: 0.8357043414408
259
  name: Cosine Ndcg@10
260
  - type: cosine_mrr@10
261
+ value: 0.7822222222222223
262
  name: Cosine Mrr@10
263
  - type: cosine_map@100
264
+ value: 0.7822222222222223
265
  name: Cosine Map@100
266
  - task:
267
  type: information-retrieval
 
271
  type: dim_128
272
  metrics:
273
  - type: cosine_accuracy@1
274
+ value: 0.5333333333333333
275
  name: Cosine Accuracy@1
276
  - type: cosine_accuracy@3
277
+ value: 0.7333333333333333
278
  name: Cosine Accuracy@3
279
  - type: cosine_accuracy@5
280
  value: 0.8
281
  name: Cosine Accuracy@5
282
  - type: cosine_accuracy@10
283
+ value: 0.9333333333333333
284
  name: Cosine Accuracy@10
285
  - type: cosine_precision@1
286
+ value: 0.5333333333333333
287
  name: Cosine Precision@1
288
  - type: cosine_precision@3
289
+ value: 0.2444444444444445
290
  name: Cosine Precision@3
291
  - type: cosine_precision@5
292
  value: 0.16000000000000003
293
  name: Cosine Precision@5
294
  - type: cosine_precision@10
295
+ value: 0.09333333333333335
296
  name: Cosine Precision@10
297
  - type: cosine_recall@1
298
+ value: 0.5333333333333333
299
  name: Cosine Recall@1
300
  - type: cosine_recall@3
301
+ value: 0.7333333333333333
302
  name: Cosine Recall@3
303
  - type: cosine_recall@5
304
  value: 0.8
305
  name: Cosine Recall@5
306
  - type: cosine_recall@10
307
+ value: 0.9333333333333333
308
  name: Cosine Recall@10
309
  - type: cosine_ndcg@10
310
+ value: 0.7203966331432973
311
  name: Cosine Ndcg@10
312
  - type: cosine_mrr@10
313
+ value: 0.6540740740740741
314
  name: Cosine Mrr@10
315
  - type: cosine_map@100
316
+ value: 0.6592022792022793
317
  name: Cosine Map@100
318
  - task:
319
  type: information-retrieval
 
323
  type: dim_64
324
  metrics:
325
  - type: cosine_accuracy@1
326
+ value: 0.4666666666666667
327
  name: Cosine Accuracy@1
328
  - type: cosine_accuracy@3
329
+ value: 0.6666666666666666
330
  name: Cosine Accuracy@3
331
  - type: cosine_accuracy@5
332
  value: 0.8
333
  name: Cosine Accuracy@5
334
  - type: cosine_accuracy@10
335
+ value: 0.8666666666666667
336
  name: Cosine Accuracy@10
337
  - type: cosine_precision@1
338
+ value: 0.4666666666666667
339
  name: Cosine Precision@1
340
  - type: cosine_precision@3
341
+ value: 0.22222222222222224
342
  name: Cosine Precision@3
343
  - type: cosine_precision@5
344
  value: 0.16000000000000003
345
  name: Cosine Precision@5
346
  - type: cosine_precision@10
347
+ value: 0.08666666666666668
348
  name: Cosine Precision@10
349
  - type: cosine_recall@1
350
+ value: 0.4666666666666667
351
  name: Cosine Recall@1
352
  - type: cosine_recall@3
353
+ value: 0.6666666666666666
354
  name: Cosine Recall@3
355
  - type: cosine_recall@5
356
  value: 0.8
357
  name: Cosine Recall@5
358
  - type: cosine_recall@10
359
+ value: 0.8666666666666667
360
  name: Cosine Recall@10
361
  - type: cosine_ndcg@10
362
+ value: 0.6507228370099043
363
  name: Cosine Ndcg@10
364
  - type: cosine_mrr@10
365
+ value: 0.5822222222222223
366
  name: Cosine Mrr@10
367
  - type: cosine_map@100
368
+ value: 0.58890559732665
369
  name: Cosine Map@100
370
  ---
371
 
 
421
  sentences = [
422
  'What type of dialogues can LLMs simulate?',
423
  '5. Education and Learning Platforms\nEducational tools like Khanmigo (from Khan Academy) and other tutoring platforms are leveraging LLMs to provide real-time help to students. LLMs can break down complex topics, provide feedback on writing, and simulate Socratic-style dialogues.',
424
+ '. For example, integrating an LLM into a customer support chatbot might involve connecting it to a company’s internal knowledge base, enabling it to answer customer questions using accurate, up-to-date information.',
425
  ]
426
  embeddings = model.encode(sentences)
427
  print(embeddings.shape)
 
468
 
469
  | Metric | dim_768 | dim_512 | dim_256 | dim_128 | dim_64 |
470
  |:--------------------|:-----------|:-----------|:-----------|:-----------|:-----------|
471
+ | cosine_accuracy@1 | 0.6667 | 0.6667 | 0.6667 | 0.5333 | 0.4667 |
472
+ | cosine_accuracy@3 | 0.8 | 0.8 | 0.8667 | 0.7333 | 0.6667 |
473
+ | cosine_accuracy@5 | 1.0 | 0.8667 | 1.0 | 0.8 | 0.8 |
474
+ | cosine_accuracy@10 | 1.0 | 1.0 | 1.0 | 0.9333 | 0.8667 |
475
+ | cosine_precision@1 | 0.6667 | 0.6667 | 0.6667 | 0.5333 | 0.4667 |
476
+ | cosine_precision@3 | 0.2667 | 0.2667 | 0.2889 | 0.2444 | 0.2222 |
477
+ | cosine_precision@5 | 0.2 | 0.1733 | 0.2 | 0.16 | 0.16 |
478
+ | cosine_precision@10 | 0.1 | 0.1 | 0.1 | 0.0933 | 0.0867 |
479
+ | cosine_recall@1 | 0.6667 | 0.6667 | 0.6667 | 0.5333 | 0.4667 |
480
+ | cosine_recall@3 | 0.8 | 0.8 | 0.8667 | 0.7333 | 0.6667 |
481
+ | cosine_recall@5 | 1.0 | 0.8667 | 1.0 | 0.8 | 0.8 |
482
+ | cosine_recall@10 | 1.0 | 1.0 | 1.0 | 0.9333 | 0.8667 |
483
+ | **cosine_ndcg@10** | **0.8311** | **0.8204** | **0.8357** | **0.7204** | **0.6507** |
484
+ | cosine_mrr@10 | 0.7767 | 0.7652 | 0.7822 | 0.6541 | 0.5822 |
485
+ | cosine_map@100 | 0.7767 | 0.7652 | 0.7822 | 0.6592 | 0.5889 |
486
 
487
  <!--
488
  ## Bias, Risks and Limitations
 
505
  * Size: 127 training samples
506
  * Columns: <code>anchor</code> and <code>positive</code>
507
  * Approximate statistics based on the first 127 samples:
508
+ | | anchor | positive |
509
+ |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
510
+ | type | string | string |
511
+ | details | <ul><li>min: 8 tokens</li><li>mean: 13.28 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 13 tokens</li><li>mean: 53.34 tokens</li><li>max: 86 tokens</li></ul> |
512
  * Samples:
513
+ | anchor | positive |
514
+ |:----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
515
+ | <code>What task mentioned is related to providing answers to inquiries?</code> | <code>. These include text generation, summarization, translation, question answering, code generation, and more.</code> |
516
+ | <code>What do LLMs learn to work effectively?</code> | <code>LLMs work by learning statistical relationships between words and phrases, allowing them to predict and generate language that feels natural. The power of these models lies not only in their size but also in the diversity of tasks they can perform with little to no task-specific training</code> |
517
+ | <code>In which industries is the generalization ability considered useful?</code> | <code>. This generalization ability makes them incredibly useful across industries—from customer service and education to software development and healthcare.</code> |
518
  * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
519
  ```json
520
  {
 
673
  </details>
674
 
675
  ### Training Logs
676
+ | Epoch | Step | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
677
+ |:-------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
678
+ | 1.0 | 4 | - | 0.7790 | 0.7120 | 0.7474 | 0.6321 | 0.5684 |
679
+ | 2.0 | 8 | - | 0.8275 | 0.7966 | 0.8091 | 0.6904 | 0.6102 |
680
+ | 2.5 | 10 | 13.4453 | - | - | - | - | - |
681
+ | 3.0 | 12 | - | 0.8311 | 0.8204 | 0.8357 | 0.7178 | 0.6557 |
682
+ | **4.0** | **16** | **-** | **0.8311** | **0.8204** | **0.8357** | **0.7204** | **0.6507** |
683
 
684
  * The bold row denotes the saved checkpoint.
685
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e7fa83288c96da5c91a8d0a7f680fa88e3010ec41496910019839fb29c898aa3
3
  size 596070136
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3d2c21c46d543a74109f26cb657b70ff5920256bc2c79d47d87b9086e1fae84
3
  size 596070136