Hugging Face's logo

yosefw
/
colbert-ettin-17m

Sentence Similarity
TensorBoard
Safetensors
sentence-transformers
PyLate
modernbert
ColBERT
feature-extraction
Generated from Trainer
dataset_size:972246
loss:Contrastive
Eval Results
text-embeddings-inference
Model card Files
xet
Metrics Community
yosefw commited on
Commit
1f3b47b
·
verified ·
1 Parent(s): 7190a5f

Add new SentenceTransformer model

Browse files
Files changed (7) hide show
  1. 1_Dense/config.json +1 -0
  2. 1_Dense/model.safetensors +3 -0
  3. README.md +471 -0
  4. config_sentence_transformers.json +49 -0
  5. model.safetensors +1 -1
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
1_Dense/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"in_features": 256, "out_features": 128, "bias": false, "activation_function": "torch.nn.modules.linear.Identity"}
1_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:283321b2bd484d2618bc786cd0ed663e2f31bbd269912a6293307dfe35318941
3
+ size 131160
README.md ADDED
@@ -0,0 +1,471 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - ColBERT
4
+ - PyLate
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - generated_from_trainer
9
+ - dataset_size:850000
10
+ - loss:Contrastive
11
+ base_model: jhu-clsp/ettin-encoder-17m
12
+ pipeline_tag: sentence-similarity
13
+ library_name: PyLate
14
+ metrics:
15
+ - accuracy
16
+ model-index:
17
+ - name: PyLate model based on jhu-clsp/ettin-encoder-17m
18
+ results:
19
+ - task:
20
+ type: col-berttriplet
21
+ name: Col BERTTriplet
22
+ dataset:
23
+ name: Unknown
24
+ type: unknown
25
+ metrics:
26
+ - type: accuracy
27
+ value: 0.6794333457946777
28
+ name: Accuracy
29
+ ---
30
+
31
+ # PyLate model based on jhu-clsp/ettin-encoder-17m
32
+
33
+ This is a [PyLate](https://github.com/lightonai/pylate) model finetuned from [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m). It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
34
+
35
+ ## Model Details
36
+
37
+ ### Model Description
38
+ - **Model Type:** PyLate model
39
+ - **Base model:** [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m) <!-- at revision 987607455c61e7a5bbc85f7758e0512ea6d0ae4c -->
40
+ - **Document Length:** 256 tokens
41
+ - **Query Length:** 32 tokens
42
+ - **Output Dimensionality:** 128 tokens
43
+ - **Similarity Function:** MaxSim
44
+ <!-- - **Training Dataset:** Unknown -->
45
+ <!-- - **Language:** Unknown -->
46
+ <!-- - **License:** Unknown -->
47
+
48
+ ### Model Sources
49
+
50
+ - **Documentation:** [PyLate Documentation](https://lightonai.github.io/pylate/)
51
+ - **Repository:** [PyLate on GitHub](https://github.com/lightonai/pylate)
52
+ - **Hugging Face:** [PyLate models on Hugging Face](https://huggingface.co/models?library=PyLate)
53
+
54
+ ### Full Model Architecture
55
+
56
+ ```
57
+ ColBERT(
58
+ (0): Transformer({'max_seq_length': 255, 'do_lower_case': False}) with Transformer model: ModernBertModel
59
+ (1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
60
+ )
61
+ ```
62
+
63
+ ## Usage
64
+ First install the PyLate library:
65
+
66
+ ```bash
67
+ pip install -U pylate
68
+ ```
69
+
70
+ ### Retrieval
71
+
72
+ PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.
73
+
74
+ #### Indexing documents
75
+
76
+ First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:
77
+
78
+ ```python
79
+ from pylate import indexes, models, retrieve
80
+
81
+ # Step 1: Load the ColBERT model
82
+ model = models.ColBERT(
83
+ model_name_or_path=yosefw/colbert-ettin-17m,
84
+ )
85
+
86
+ # Step 2: Initialize the Voyager index
87
+ index = indexes.Voyager(
88
+ index_folder="pylate-index",
89
+ index_name="index",
90
+ override=True, # This overwrites the existing index if any
91
+ )
92
+
93
+ # Step 3: Encode the documents
94
+ documents_ids = ["1", "2", "3"]
95
+ documents = ["document 1 text", "document 2 text", "document 3 text"]
96
+
97
+ documents_embeddings = model.encode(
98
+ documents,
99
+ batch_size=32,
100
+ is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries
101
+ show_progress_bar=True,
102
+ )
103
+
104
+ # Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
105
+ index.add_documents(
106
+ documents_ids=documents_ids,
107
+ documents_embeddings=documents_embeddings,
108
+ )
109
+ ```
110
+
111
+ Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
112
+
113
+ ```python
114
+ # To load an index, simply instantiate it with the correct folder/name and without overriding it
115
+ index = indexes.Voyager(
116
+ index_folder="pylate-index",
117
+ index_name="index",
118
+ )
119
+ ```
120
+
121
+ #### Retrieving top-k documents for queries
122
+
123
+ Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries.
124
+ To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
125
+
126
+ ```python
127
+ # Step 1: Initialize the ColBERT retriever
128
+ retriever = retrieve.ColBERT(index=index)
129
+
130
+ # Step 2: Encode the queries
131
+ queries_embeddings = model.encode(
132
+ ["query for document 3", "query for document 1"],
133
+ batch_size=32,
134
+ is_query=True, # # Ensure that it is set to False to indicate that these are queries
135
+ show_progress_bar=True,
136
+ )
137
+
138
+ # Step 3: Retrieve top-k documents
139
+ scores = retriever.retrieve(
140
+ queries_embeddings=queries_embeddings,
141
+ k=10, # Retrieve the top 10 matches for each query
142
+ )
143
+ ```
144
+
145
+ ### Reranking
146
+ If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
147
+
148
+ ```python
149
+ from pylate import rank, models
150
+
151
+ queries = [
152
+ "query A",
153
+ "query B",
154
+ ]
155
+
156
+ documents = [
157
+ ["document A", "document B"],
158
+ ["document 1", "document C", "document B"],
159
+ ]
160
+
161
+ documents_ids = [
162
+ [1, 2],
163
+ [1, 3, 2],
164
+ ]
165
+
166
+ model = models.ColBERT(
167
+ model_name_or_path=yosefw/colbert-ettin-17m,
168
+ )
169
+
170
+ queries_embeddings = model.encode(
171
+ queries,
172
+ is_query=True,
173
+ )
174
+
175
+ documents_embeddings = model.encode(
176
+ documents,
177
+ is_query=False,
178
+ )
179
+
180
+ reranked_documents = rank.rerank(
181
+ documents_ids=documents_ids,
182
+ queries_embeddings=queries_embeddings,
183
+ documents_embeddings=documents_embeddings,
184
+ )
185
+ ```
186
+
187
+ <!--
188
+ ### Direct Usage (Transformers)
189
+
190
+ <details><summary>Click to see the direct usage in Transformers</summary>
191
+
192
+ </details>
193
+ -->
194
+
195
+ <!--
196
+ ### Downstream Usage (Sentence Transformers)
197
+
198
+ You can finetune this model on your own dataset.
199
+
200
+ <details><summary>Click to expand</summary>
201
+
202
+ </details>
203
+ -->
204
+
205
+ <!--
206
+ ### Out-of-Scope Use
207
+
208
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
209
+ -->
210
+
211
+ ## Evaluation
212
+
213
+ ### Metrics
214
+
215
+ #### Col BERTTriplet
216
+
217
+ * Evaluated with <code>pylate.evaluation.colbert_triplet.ColBERTTripletEvaluator</code>
218
+
219
+ | Metric | Value |
220
+ |:-------------|:-----------|
221
+ | **accuracy** | **0.6794** |
222
+
223
+ <!--
224
+ ## Bias, Risks and Limitations
225
+
226
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
227
+ -->
228
+
229
+ <!--
230
+ ### Recommendations
231
+
232
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
233
+ -->
234
+
235
+ ## Training Details
236
+
237
+ ### Training Dataset
238
+
239
+ #### Unnamed Dataset
240
+
241
+
242
+ * Size: 850,000 training samples
243
+ * Columns: <code>query</code>, <code>positive</code>, <code>negative_1</code>, <code>negative_2</code>, and <code>negative_3</code>
244
+ * Approximate statistics based on the first 1000 samples:
245
+ | | query | positive | negative_1 | negative_2 | negative_3 |
246
+ |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
247
+ | type | string | string | string | string | string |
248
+ | details | <ul><li>min: 6 tokens</li><li>mean: 10.39 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 31.91 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 24 tokens</li><li>mean: 31.95 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 31.95 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 31.96 tokens</li><li>max: 32 tokens</li></ul> |
249
+ * Samples:
250
+ | query | positive | negative_1 | negative_2 | negative_3 |
251
+ |:----------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
252
+ | <code>are freight charges taxable in arizona</code> | <code>Shipping is Not Taxable if Separately Stated These states say shipping charges are not taxable if you show the charge separately from the selling price of the item. They are taxable if you include the charge as part of the price of the item. 1 Alabama. 2 Arizona.</code> | <code>They have nexus in Arizona, and the merchandise ships from California via a 3rd party shipper. The invoice states a merchandise cost and a freight/handling cost in two separate boxes. But they are charging sales tax on the sum of the merchandise plus the freight/handling cost.</code> | <code>Tax on freight generally follows the transaction the freight relates to. If the goods you sell are exempt and you charge your customer for freight or delivery charges or handling charges, then these charges are exempt.</code> | <code>1 Freight is by common carrier-you should not use your own truck. 2 To do so indicates shipping is a necessary part of the sales transaction and this makes the freight taxable in many states; 3 Show charges for freight separate from everything else-Do not mark it up and do not combine handling charges with freight charges.</code> |
253
+ | <code>are freight charges taxable in arizona</code> | <code>Shipping is Not Taxable if Separately Stated These states say shipping charges are not taxable if you show the charge separately from the selling price of the item. They are taxable if you include the charge as part of the price of the item. 1 Alabama. 2 Arizona.</code> | <code>From the Arizona Department of Revenue website: When included in the retail sales price, a shipping charge from the manufacturer or wholesaler to the retailer is considered a cost of doing business and is included in the taxable proceeds from the sale.</code> | <code>1 Show charges for freight separate from everything else-Do not mark it up and do not combine handling charges with freight charges. 2 Most states charge tax on handling charges, so combining them with freight charges (e.g. S&H) will make both taxable.</code> | <code>Summary of Sales Tax on Shipping in Arizona. Many states are confusing on shipping or even simply don’t take into account, making you play guessing games. Arizona, though, is nice and straightforward. If the shipping costs are separately stated on the invoice, you shouldn’t have to collect.</code> |
254
+ | <code>average gas price oahu</code> | <code>HONOLULU (HawaiiNewsNow) -. Even though gas prices have sunk below $3 at some locations on Oahu for the first time in seven years, the average gas price in Hawaii is still the highest in the country. Hawaii News Now found the lowest-priced gas at Costco in Iwilei Friday, at $2.99 a gallon.ven though gas prices have sunk below $3 at some locations on Oahu for the first time in seven years, the average gas price in Hawaii is still the highest in the country.</code> | <code>Even though gas prices have sunk below $3 at some locations on Oahu for the first time in seven years, the average gas price in Hawaii is still the highest in the country.Hawaii News Now found the lowest-priced gas at Costco in Iwilei Friday, at $2.99 a gallon.ven though gas prices have sunk below $3 at some locations on Oahu for the first time in seven years, the average gas price in Hawaii is still the highest in the country.</code> | <code>yesterday. $2.73update. There are 48 Regular gas price reports in the past 5 days in Honolulu, HI. The average Regular gas price in Honolulu, HI is $2.72, which is $0.07 lower than U.S. national average Regular gas price $2.79.The lowest Regular gas price is $2.43 of CGES located at 400 Sand Island Pkwy, Honolulu, HI 96819.2.73update. There are 48 Regular gas price reports in the past 5 days in Honolulu, HI. The average Regular gas price in Honolulu, HI is $2.72, which is $0.07 lower than U.S. national average Regular gas price $2.79. The lowest Regular gas price is $2.43 of CGES located at 400 Sand Island Pkwy, Honolulu, HI 96819.</code> | <code>Diesel Fuel Prices: The national average price for diesel in September 2015 was $2.521 per gallon which was a decrease of $1.250 (33.1%) from the same month a year ago, and a decrease of 12.4 cents (4.7%) from August 2015.asoline Consumption: Hawaii’s statewide consumption for gasoline in July 2015, as measured by the gasoline tax base, was 39.6 million gallons, which increased 1.0 million gallons (2.5%) from the same month a year ago.</code> |
255
+ * Loss: <code>pylate.losses.contrastive.Contrastive</code>
256
+
257
+ ### Evaluation Dataset
258
+
259
+ #### Unnamed Dataset
260
+
261
+
262
+ * Size: 20,000 evaluation samples
263
+ * Columns: <code>query</code>, <code>positive</code>, <code>negative_1</code>, <code>negative_2</code>, and <code>negative_3</code>
264
+ * Approximate statistics based on the first 1000 samples:
265
+ | | query | positive | negative_1 | negative_2 | negative_3 |
266
+ |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
267
+ | type | string | string | string | string | string |
268
+ | details | <ul><li>min: 6 tokens</li><li>mean: 10.12 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 31.98 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 24 tokens</li><li>mean: 31.94 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 20 tokens</li><li>mean: 31.93 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 18 tokens</li><li>mean: 31.95 tokens</li><li>max: 32 tokens</li></ul> |
269
+ * Samples:
270
+ | query | positive | negative_1 | negative_2 | negative_3 |
271
+ |:------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
272
+ | <code>heart specialists in ridgeland ms</code> | <code>Dr. George Reynolds Jr, MD is a cardiology specialist in Ridgeland, MS and has been practicing for 35 years. He graduated from Vanderbilt University School Of Medicine in 1977 and specializes in cardiology and internal medicine.</code> | <code>Dr. James Kramer is a Internist in Ridgeland, MS. Find Dr. Kramer's phone number, address and more.</code> | <code>General internal medicine physicians, or internists, are primary-care doctors who perform physical exams and treat a wide spectrum of common illnesses in adult men and women. One of every four physicians in the U.S. is an internist, many of whom are certified in one of 19 subspecialties, including cardiology, infectious disease and medical oncology.</code> | <code>Chronic Pulmonary Heart Diseases (incl. Pulmonary Hypertension) Coarctation of the Aorta; Congenital Aortic Valve Disorders; Congenital Heart Defects; Congenital Heart Disease; Congestive Heart Failure; Coronary Artery Disease (CAD) Endocarditis; Heart Attack (Acute Myocardial Infarction) Heart Disease; Heart Murmur; Heart Palpitations; Hyperlipidemia; Hypertension</code> |
273
+ | <code>heart specialists in ridgeland ms</code> | <code>Dr. George Reynolds Jr, MD is a cardiology specialist in Ridgeland, MS and has been practicing for 35 years. He graduated from Vanderbilt University School Of Medicine in 1977 and specializes in cardiology and internal medicine.</code> | <code>Dr. James Kramer is an internist in Ridgeland, Mississippi. He received his medical degree from Loma Linda University School of Medicine and has been in practice for more than 20 years. Dr. James Kramer's Details</code> | <code>Congenital Heart Defects; Congenital Heart Disease; Congestive Heart Failure; Coronary Artery Disease (CAD) Endocarditis; Heart Attack (Acute Myocardial Infarction) Heart Disease; Heart Murmur; Heart Palpitations; Hyperlipidemia; Hypertension; Hypertensive Chronic Kidney Disease; Hypertensive Heart and Chronic Kidney Disease; Hypertensive Heart Disease; Hypotension</code> | <code>Aneurysm and Dissection of Heart; Aneurysm of Heart; Angina and Acute Coronary Syndrome; Aortic Aneurysm; Aortic Dissection; Aortic Ectasia; Aortic Stenosis; Aortic Valve Disease; Aortic Valve Regurgitation; Arrhythmias; Atrial Fibrillation; Atrial Flutter; Autonomic Disorders; Benign Tumor; Cardiomegaly; Cardiomyopathy; Carotid Artery Disease; Chest Pain</code> |
274
+ | <code>does baytril otic require a prescription</code> | <code>Baytril Otic Ear Drops-Enrofloxacin/Silver Sulfadiazine-Prices & Information. A prescription is required for this item. A prescription is required for this item. Brand medication is not available at this time.</code> | <code>RX required for this item. Click here for our full Prescription Policy and Form. Baytril Otic (enrofloxacin/silver sulfadiazine) Emulsion from Bayer is the first fluoroquinolone approved by the Food and Drug Administration for the topical treatment of canine otitis externa.</code> | <code>Baytril Otic is indicated as a treatment for canine otitis externa (ear infection) complicated by certain bacterial and fungal organisms. Baytril Otic is an emulsion containing 5 mg enrofloxacin and 10 mg silver sulfadiazine per ml. Baytril Otic Drops are administered into the outer ear. As a general guide, dogs weighing 35 lbs. or less are given 5-10 drops per treatment; dogs over 35 lbs. are given 10-15 drops per treatment. Following treatment, gently massage the ear to ensure distribution. May be applied twice daily for up to 14 days, or as indicated. Enrofloxacin and silver sulfadiazine.</code> | <code>Baytril for dogs is an antibiotic often prescribed for bacterial infections, particularly those involving the ears. Ear infections are rare in many animals, but quite common in dogs. This is particularly true for dogs with long droopy ears, where it will stay very warm and moist.</code> |
275
+ * Loss: <code>pylate.losses.contrastive.Contrastive</code>
276
+
277
+ ### Training Hyperparameters
278
+ #### Non-Default Hyperparameters
279
+
280
+ - `eval_strategy`: epoch
281
+ - `per_device_train_batch_size`: 32
282
+ - `per_device_eval_batch_size`: 32
283
+ - `learning_rate`: 2e-05
284
+ - `num_train_epochs`: 4
285
+ - `lr_scheduler_type`: cosine
286
+ - `warmup_ratio`: 0.05
287
+ - `fp16`: True
288
+ - `push_to_hub`: True
289
+
290
+ #### All Hyperparameters
291
+ <details><summary>Click to expand</summary>
292
+
293
+ - `overwrite_output_dir`: False
294
+ - `do_predict`: False
295
+ - `eval_strategy`: epoch
296
+ - `prediction_loss_only`: True
297
+ - `per_device_train_batch_size`: 32
298
+ - `per_device_eval_batch_size`: 32
299
+ - `per_gpu_train_batch_size`: None
300
+ - `per_gpu_eval_batch_size`: None
301
+ - `gradient_accumulation_steps`: 1
302
+ - `eval_accumulation_steps`: None
303
+ - `torch_empty_cache_steps`: None
304
+ - `learning_rate`: 2e-05
305
+ - `weight_decay`: 0.0
306
+ - `adam_beta1`: 0.9
307
+ - `adam_beta2`: 0.999
308
+ - `adam_epsilon`: 1e-08
309
+ - `max_grad_norm`: 1.0
310
+ - `num_train_epochs`: 4
311
+ - `max_steps`: -1
312
+ - `lr_scheduler_type`: cosine
313
+ - `lr_scheduler_kwargs`: {}
314
+ - `warmup_ratio`: 0.05
315
+ - `warmup_steps`: 0
316
+ - `log_level`: passive
317
+ - `log_level_replica`: warning
318
+ - `log_on_each_node`: True
319
+ - `logging_nan_inf_filter`: True
320
+ - `save_safetensors`: True
321
+ - `save_on_each_node`: False
322
+ - `save_only_model`: False
323
+ - `restore_callback_states_from_checkpoint`: False
324
+ - `no_cuda`: False
325
+ - `use_cpu`: False
326
+ - `use_mps_device`: False
327
+ - `seed`: 42
328
+ - `data_seed`: None
329
+ - `jit_mode_eval`: False
330
+ - `use_ipex`: False
331
+ - `bf16`: False
332
+ - `fp16`: True
333
+ - `fp16_opt_level`: O1
334
+ - `half_precision_backend`: auto
335
+ - `bf16_full_eval`: False
336
+ - `fp16_full_eval`: False
337
+ - `tf32`: None
338
+ - `local_rank`: 0
339
+ - `ddp_backend`: None
340
+ - `tpu_num_cores`: None
341
+ - `tpu_metrics_debug`: False
342
+ - `debug`: []
343
+ - `dataloader_drop_last`: False
344
+ - `dataloader_num_workers`: 0
345
+ - `dataloader_prefetch_factor`: None
346
+ - `past_index`: -1
347
+ - `disable_tqdm`: False
348
+ - `remove_unused_columns`: True
349
+ - `label_names`: None
350
+ - `load_best_model_at_end`: False
351
+ - `ignore_data_skip`: False
352
+ - `fsdp`: []
353
+ - `fsdp_min_num_params`: 0
354
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
355
+ - `fsdp_transformer_layer_cls_to_wrap`: None
356
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
357
+ - `deepspeed`: None
358
+ - `label_smoothing_factor`: 0.0
359
+ - `optim`: adamw_torch
360
+ - `optim_args`: None
361
+ - `adafactor`: False
362
+ - `group_by_length`: False
363
+ - `length_column_name`: length
364
+ - `ddp_find_unused_parameters`: None
365
+ - `ddp_bucket_cap_mb`: None
366
+ - `ddp_broadcast_buffers`: False
367
+ - `dataloader_pin_memory`: True
368
+ - `dataloader_persistent_workers`: False
369
+ - `skip_memory_metrics`: True
370
+ - `use_legacy_prediction_loop`: False
371
+ - `push_to_hub`: True
372
+ - `resume_from_checkpoint`: None
373
+ - `hub_model_id`: None
374
+ - `hub_strategy`: every_save
375
+ - `hub_private_repo`: None
376
+ - `hub_always_push`: False
377
+ - `gradient_checkpointing`: False
378
+ - `gradient_checkpointing_kwargs`: None
379
+ - `include_inputs_for_metrics`: False
380
+ - `include_for_metrics`: []
381
+ - `eval_do_concat_batches`: True
382
+ - `fp16_backend`: auto
383
+ - `push_to_hub_model_id`: None
384
+ - `push_to_hub_organization`: None
385
+ - `mp_parameters`:
386
+ - `auto_find_batch_size`: False
387
+ - `full_determinism`: False
388
+ - `torchdynamo`: None
389
+ - `ray_scope`: last
390
+ - `ddp_timeout`: 1800
391
+ - `torch_compile`: False
392
+ - `torch_compile_backend`: None
393
+ - `torch_compile_mode`: None
394
+ - `dispatch_batches`: None
395
+ - `split_batches`: None
396
+ - `include_tokens_per_second`: False
397
+ - `include_num_input_tokens_seen`: False
398
+ - `neftune_noise_alpha`: None
399
+ - `optim_target_modules`: None
400
+ - `batch_eval_metrics`: False
401
+ - `eval_on_start`: False
402
+ - `use_liger_kernel`: False
403
+ - `eval_use_gather_object`: False
404
+ - `average_tokens_across_devices`: False
405
+ - `prompts`: None
406
+ - `batch_sampler`: batch_sampler
407
+ - `multi_dataset_batch_sampler`: proportional
408
+
409
+ </details>
410
+
411
+ ### Training Logs
412
+ | Epoch | Step | accuracy |
413
+ |:-----:|:----:|:--------:|
414
+ | 0 | 0 | 0.6794 |
415
+
416
+
417
+ ### Framework Versions
418
+ - Python: 3.11.11
419
+ - Sentence Transformers: 4.0.2
420
+ - PyLate: 1.2.0
421
+ - Transformers: 4.48.2
422
+ - PyTorch: 2.6.0+cu124
423
+ - Accelerate: 1.5.2
424
+ - Datasets: 3.6.0
425
+ - Tokenizers: 0.21.1
426
+
427
+
428
+ ## Citation
429
+
430
+ ### BibTeX
431
+
432
+ #### Sentence Transformers
433
+ ```bibtex
434
+ @inproceedings{reimers-2019-sentence-bert,
435
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
436
+ author = "Reimers, Nils and Gurevych, Iryna",
437
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
438
+ month = "11",
439
+ year = "2019",
440
+ publisher = "Association for Computational Linguistics",
441
+ url = "https://arxiv.org/abs/1908.10084"
442
+ }
443
+ ```
444
+
445
+ #### PyLate
446
+ ```bibtex
447
+ @misc{PyLate,
448
+ title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
449
+ author={Chaffin, Antoine and Sourty, Raphaël},
450
+ url={https://github.com/lightonai/pylate},
451
+ year={2024}
452
+ }
453
+ ```
454
+
455
+ <!--
456
+ ## Glossary
457
+
458
+ *Clearly define terms in order to be accessible across audiences.*
459
+ -->
460
+
461
+ <!--
462
+ ## Model Card Authors
463
+
464
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
465
+ -->
466
+
467
+ <!--
468
+ ## Model Card Contact
469
+
470
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
471
+ -->
config_sentence_transformers.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "4.0.2",
4
+ "transformers": "4.48.2",
5
+ "pytorch": "2.6.0+cu124"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "MaxSim",
10
+ "query_prefix": "[Q] ",
11
+ "document_prefix": "[D] ",
12
+ "query_length": 32,
13
+ "document_length": 256,
14
+ "attend_to_expansion_tokens": false,
15
+ "skiplist_words": [
16
+ "!",
17
+ "\"",
18
+ "#",
19
+ "$",
20
+ "%",
21
+ "&",
22
+ "'",
23
+ "(",
24
+ ")",
25
+ "*",
26
+ "+",
27
+ ",",
28
+ "-",
29
+ ".",
30
+ "/",
31
+ ":",
32
+ ";",
33
+ "<",
34
+ "=",
35
+ ">",
36
+ "?",
37
+ "@",
38
+ "[",
39
+ "\\",
40
+ "]",
41
+ "^",
42
+ "_",
43
+ "`",
44
+ "{",
45
+ "|",
46
+ "}",
47
+ "~"
48
+ ]
49
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f35757612a2654512ffa56a538a11a14404d4015b614e2646e02a3ba2847c37
3
  size 67195976
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c3929cd462a740a9abfeffe9b1799cd5ca32eb9d481b7f0dad8c8c0f1544693
3
  size 67195976
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Dense",
12
+ "type": "pylate.models.Dense.Dense"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 255,
3
+ "do_lower_case": false
4
+ }