Hugging Face's logo

tomaarsen
/
reranker-msmarco-MiniLM-L12-H384-uncased-plistmle-sum-to-1

Text Ranking
sentence-transformers
Safetensors
English
bert
cross-encoder
Generated from Trainer
dataset_size:399282
loss:PListMLELoss
Eval Results
Model card Files Community
tomaarsen HF staff commited on
Commit
176e945
·
verified ·
1 Parent(s): 88fe7e4

Add new CrossEncoder model

Browse files
Files changed (7) hide show
  1. README.md +633 -0
  2. config.json +34 -0
  3. model.safetensors +3 -0
  4. special_tokens_map.json +7 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +58 -0
  7. vocab.txt +0 -0
README.md ADDED
@@ -0,0 +1,633 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - sentence-transformers
6
+ - cross-encoder
7
+ - generated_from_trainer
8
+ - dataset_size:399282
9
+ - loss:PListMLELoss
10
+ base_model: microsoft/MiniLM-L12-H384-uncased
11
+ datasets:
12
+ - sentence-transformers/msmarco
13
+ pipeline_tag: text-ranking
14
+ library_name: sentence-transformers
15
+ metrics:
16
+ - map
17
+ - mrr@10
18
+ - ndcg@10
19
+ co2_eq_emissions:
20
+ emissions: 913.3735420464258
21
+ energy_consumed: 2.3498040711044084
22
+ source: codecarbon
23
+ training_type: fine-tuning
24
+ on_cloud: false
25
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
26
+ ram_total_size: 31.777088165283203
27
+ hours_used: 8.397
28
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
29
+ model-index:
30
+ - name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
31
+ results:
32
+ - task:
33
+ type: cross-encoder-reranking
34
+ name: Cross Encoder Reranking
35
+ dataset:
36
+ name: NanoMSMARCO R100
37
+ type: NanoMSMARCO_R100
38
+ metrics:
39
+ - type: map
40
+ value: 0.5687
41
+ name: Map
42
+ - type: mrr@10
43
+ value: 0.5602
44
+ name: Mrr@10
45
+ - type: ndcg@10
46
+ value: 0.6369
47
+ name: Ndcg@10
48
+ - task:
49
+ type: cross-encoder-reranking
50
+ name: Cross Encoder Reranking
51
+ dataset:
52
+ name: NanoNFCorpus R100
53
+ type: NanoNFCorpus_R100
54
+ metrics:
55
+ - type: map
56
+ value: 0.3577
57
+ name: Map
58
+ - type: mrr@10
59
+ value: 0.5777
60
+ name: Mrr@10
61
+ - type: ndcg@10
62
+ value: 0.4077
63
+ name: Ndcg@10
64
+ - task:
65
+ type: cross-encoder-reranking
66
+ name: Cross Encoder Reranking
67
+ dataset:
68
+ name: NanoNQ R100
69
+ type: NanoNQ_R100
70
+ metrics:
71
+ - type: map
72
+ value: 0.7267
73
+ name: Map
74
+ - type: mrr@10
75
+ value: 0.7479
76
+ name: Mrr@10
77
+ - type: ndcg@10
78
+ value: 0.7642
79
+ name: Ndcg@10
80
+ - task:
81
+ type: cross-encoder-nano-beir
82
+ name: Cross Encoder Nano BEIR
83
+ dataset:
84
+ name: NanoBEIR R100 mean
85
+ type: NanoBEIR_R100_mean
86
+ metrics:
87
+ - type: map
88
+ value: 0.5511
89
+ name: Map
90
+ - type: mrr@10
91
+ value: 0.6286
92
+ name: Mrr@10
93
+ - type: ndcg@10
94
+ value: 0.603
95
+ name: Ndcg@10
96
+ ---
97
+
98
+ # CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
99
+
100
+ This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
101
+
102
+ ## Model Details
103
+
104
+ ### Model Description
105
+ - **Model Type:** Cross Encoder
106
+ - **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
107
+ - **Maximum Sequence Length:** 512 tokens
108
+ - **Number of Output Labels:** 1 label
109
+ - **Training Dataset:**
110
+ - [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco)
111
+ - **Language:** en
112
+ <!-- - **License:** Unknown -->
113
+
114
+ ### Model Sources
115
+
116
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
117
+ - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
118
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
119
+ - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
120
+
121
+ ## Usage
122
+
123
+ ### Direct Usage (Sentence Transformers)
124
+
125
+ First install the Sentence Transformers library:
126
+
127
+ ```bash
128
+ pip install -U sentence-transformers
129
+ ```
130
+
131
+ Then you can load this model and run inference.
132
+ ```python
133
+ from sentence_transformers import CrossEncoder
134
+
135
+ # Download from the 🤗 Hub
136
+ model = CrossEncoder("tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-plistmle-sum-to-1")
137
+ # Get scores for pairs of texts
138
+ pairs = [
139
+ ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
140
+ ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
141
+ ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
142
+ ]
143
+ scores = model.predict(pairs)
144
+ print(scores.shape)
145
+ # (3,)
146
+
147
+ # Or rank different texts based on similarity to a single text
148
+ ranks = model.rank(
149
+ 'How many calories in an egg',
150
+ [
151
+ 'There are on average between 55 and 80 calories in an egg depending on its size.',
152
+ 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
153
+ 'Most of the calories in an egg come from the yellow yolk in the center.',
154
+ ]
155
+ )
156
+ # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
157
+ ```
158
+
159
+ <!--
160
+ ### Direct Usage (Transformers)
161
+
162
+ <details><summary>Click to see the direct usage in Transformers</summary>
163
+
164
+ </details>
165
+ -->
166
+
167
+ <!--
168
+ ### Downstream Usage (Sentence Transformers)
169
+
170
+ You can finetune this model on your own dataset.
171
+
172
+ <details><summary>Click to expand</summary>
173
+
174
+ </details>
175
+ -->
176
+
177
+ <!--
178
+ ### Out-of-Scope Use
179
+
180
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
181
+ -->
182
+
183
+ ## Evaluation
184
+
185
+ ### Metrics
186
+
187
+ #### Cross Encoder Reranking
188
+
189
+ * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
190
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
191
+ ```json
192
+ {
193
+ "at_k": 10,
194
+ "always_rerank_positives": true
195
+ }
196
+ ```
197
+
198
+ | Metric | NanoMSMARCO_R100 | NanoNFCorpus_R100 | NanoNQ_R100 |
199
+ |:------------|:---------------------|:---------------------|:---------------------|
200
+ | map | 0.5687 (+0.0792) | 0.3577 (+0.0967) | 0.7267 (+0.3071) |
201
+ | mrr@10 | 0.5602 (+0.0827) | 0.5777 (+0.0779) | 0.7479 (+0.3212) |
202
+ | **ndcg@10** | **0.6369 (+0.0965)** | **0.4077 (+0.0827)** | **0.7642 (+0.2636)** |
203
+
204
+ #### Cross Encoder Nano BEIR
205
+
206
+ * Dataset: `NanoBEIR_R100_mean`
207
+ * Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
208
+ ```json
209
+ {
210
+ "dataset_names": [
211
+ "msmarco",
212
+ "nfcorpus",
213
+ "nq"
214
+ ],
215
+ "rerank_k": 100,
216
+ "at_k": 10,
217
+ "always_rerank_positives": true
218
+ }
219
+ ```
220
+
221
+ | Metric | Value |
222
+ |:------------|:---------------------|
223
+ | map | 0.5511 (+0.1610) |
224
+ | mrr@10 | 0.6286 (+0.1606) |
225
+ | **ndcg@10** | **0.6030 (+0.1476)** |
226
+
227
+ <!--
228
+ ## Bias, Risks and Limitations
229
+
230
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
231
+ -->
232
+
233
+ <!--
234
+ ### Recommendations
235
+
236
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
237
+ -->
238
+
239
+ ## Training Details
240
+
241
+ ### Training Dataset
242
+
243
+ #### msmarco
244
+
245
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [a0537b6](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/a0537b6c8669051b215b020183c276a1eb2027d5)
246
+ * Size: 399,282 training samples
247
+ * Columns: <code>query_id</code>, <code>doc_ids</code>, and <code>labels</code>
248
+ * Approximate statistics based on the first 1000 samples:
249
+ | | query_id | doc_ids | labels |
250
+ |:--------|:----------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|
251
+ | type | string | list | list |
252
+ | details | <ul><li>min: 6 characters</li><li>mean: 33.0 characters</li><li>max: 154 characters</li></ul> | <ul><li>min: 6 elements</li><li>mean: 13.23 elements</li><li>max: 20 elements</li></ul> | <ul><li>min: 6 elements</li><li>mean: 13.23 elements</li><li>max: 20 elements</li></ul> |
253
+ * Samples:
254
+ | query_id | doc_ids | labels |
255
+ |:-----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
256
+ | <code>intel current gen core processors</code> | <code>["Identical or more capable versions of Core processors are also sold as Xeon processors for the server and workstation markets. As of 2017 the current lineup of Core processors included the Intel Core i7, Intel Core i5, and Intel Core i3, along with the Y - Series Intel Core CPU's.", "Most noticeably that Panasonic switched from Intel Core 2 Duo power to the latest Intel Core i3 and i5 processors. The three processors available in the new Toughbook 31, together with the new Mobile Intel QM57 Express chipset, are all part of Intel's Calpella platform.", 'The new 7th Gen Intel Core i7-7700HQ processor gives the 14-inch Razer Blade 2.8GHz of quad-core processing power and Turbo Boost speeds, which automatically increases the speed of active cores â\x80\x93 up to 3.8GHz.', 'Key difference: Intel Core i3 is a type of dual-core processor. i5 processors have 2 to 4 cores. A dual-core processor is a type of a central processing unit (CPU) that has two complete execution cores. Hence, it has t...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
257
+ | <code>renovation definition</code> | <code>['Renovation is the act of renewing or restoring something. If your kitchen is undergoing a renovation, thereâ\x80\x99s probably plaster and paint all over the place and you should probably get take-out.', 'NEW GALLERY SPACES OPENING IN 2017. In early 2017, our fourth floor will be transformed into a new destination for historical education and innovation. During the current renovation, objects from our permanent collection are on view throughout the Museum.', 'A same level house extension in Australia will cost approximately $60,000 to $200,000+. Adding a room or extending your living area on the ground floor are affordable ways of creating more space.Here are some key points to consider that will help you keep your renovation costs in check.RTICLE Stephanie Matheson. A same level house extension in Australia will cost approximately $60,000 to $200,000+. Adding a room or extending your living area on the ground floor are affordable ways of creating more space. Here are some key points...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
258
+ | <code>what is a girasol</code> | <code>['Girasol definition, an opal that reflects light in a bright luminous glow. See more.', 'Also, a type of opal from Mexico, referred to as Mexican water opal, is a colorless opal which exhibits either a bluish or golden internal sheen. Girasol opal is a term sometimes mistakenly and improperly used to refer to fire opals, as well as a type of transparent to semitransparent type milky quartz from Madagascar which displays an asterism, or star effect, when cut properly.', 'What is the meaning of Girasol? How popular is the baby name Girasol? Learn the origin and popularity plus how to pronounce Girasol', 'There are 5 basic types of opal. These types are Peruvian Opal, Fire Opal, Girasol Opal, Common opal and Precious Opal. There are 5 basic types of opal. These types are Peruvian Opal, Fire Opal, Girasol Opal, Common opal and Precious Opal.', 'girasol (Ë\x88dÊ\x92ɪrÉ\x99Ë\x8csÉ\x92l; -Ë\x8csÉ\x99Ê\x8al) , girosol or girasole n (Jewellery) a type of opal that has a red or pink glow in br...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
259
+ * Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
260
+ ```json
261
+ {
262
+ "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
263
+ "activation_fct": "torch.nn.modules.linear.Identity",
264
+ "mini_batch_size": 16,
265
+ "respect_input_order": true
266
+ }
267
+ ```
268
+
269
+ ### Evaluation Dataset
270
+
271
+ #### msmarco
272
+
273
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [a0537b6](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/a0537b6c8669051b215b020183c276a1eb2027d5)
274
+ * Size: 1,000 evaluation samples
275
+ * Columns: <code>query_id</code>, <code>doc_ids</code>, and <code>labels</code>
276
+ * Approximate statistics based on the first 1000 samples:
277
+ | | query_id | doc_ids | labels |
278
+ |:--------|:------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------|
279
+ | type | string | list | list |
280
+ | details | <ul><li>min: 10 characters</li><li>mean: 33.63 characters</li><li>max: 137 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 12.50 elements</li><li>max: 20 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 12.50 elements</li><li>max: 20 elements</li></ul> |
281
+ * Samples:
282
+ | query_id | doc_ids | labels |
283
+ |:----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
284
+ | <code>can marijuana help dementia</code> | <code>["Cannabis 'could stop dementia in its tracks'. Cannabis may help keep Alzheimer's disease at bay. In experiments, a marijuana-based medicine triggered the formation of new brain cells and cut inflammation linked to dementia. The researchers say that using the information to create a pill suitable for people could help prevent or delay the onset of Alzheimer's.", 'Marijuana (cannabis): Marijuana in any form is not allowed on aircraft and is not allowed in the secure part of the airport (beyond the TSA screening areas). In addition it is illegal to import marijuana or marijuana-related items into the US.', 'Depakote and dementia - Can dementia be cured? Unfortunately, no. Dementia is a progressive disease. Even available treatments only slow progression or tame symptoms.', 'Marijuana Prices. The price of marijuana listed below is the typical price to buy marijuana on the black market in U.S. dollars. How much marijuana cost and the sale price of marijuana are based upon the United Natio...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
285
+ | <code>what are carcinogen</code> | <code>['Written By: Carcinogen, any of a number of agents that can cause cancer in humans. They can be divided into three major categories: chemical carcinogens (including those from biological sources), physical carcinogens, and oncogenic (cancer-causing) viruses. 1 Most carcinogens, singly or in combination, produce cancer by interacting with DNA in cells and thereby interfering with normal cellular function.', 'Tarragon (Artemisia dracunculus) is a species of perennial herb in the sunflower family. It is widespread in the wild across much of Eurasia and North America, and is cultivated for culinary and medicinal purposes in many lands.One sub-species, Artemisia dracunculus var. sativa, is cultivated for use of the leaves as an aromatic culinary herb.arragon has an aromatic property reminiscent of anise, due to the presence of estragole, a known carcinogen and teratogen in mice. The European Union investigation revealed that the danger of estragole is minimal even at 100â\x80\x931,000 tim...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
286
+ | <code>who played ben geller in friends</code> | <code>["Noelle and Cali aren't the only twins to have played one child character in Friends. Double vision: Ross' cheeky son Ben (pictured), from his first marriage to Carol, was also played by twins, Dylan and Cole Sprouse, who are now 22.", 'Update 7/29/06: There are now three â\x80\x9cTeaching Pastorsâ\x80\x9d at Applegate Christian Fellowship, according to their web site. Jon Courson is now back at Applegate. The other two listed as Teaching Pastors are Jonâ\x80\x99s two sons: Peter John and Ben Courson.on Courson has been appreciated over the years by many people who are my friends and whom I respect. I believe that he preaches the real Jesus and the true Gospel, for which I rejoice. I also believe that his ministry and church organization is a reasonable example with which to examine important issues together.', 'Ben 10 (Reboot) Ben 10: Omniverse is the fourth iteration of the Ben 10 franchise, and it is the sequel of Ben 10: Ultimate Alien. Ben was all set to be a solo hero with his n...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
287
+ * Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
288
+ ```json
289
+ {
290
+ "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
291
+ "activation_fct": "torch.nn.modules.linear.Identity",
292
+ "mini_batch_size": 16,
293
+ "respect_input_order": true
294
+ }
295
+ ```
296
+
297
+ ### Training Hyperparameters
298
+ #### Non-Default Hyperparameters
299
+
300
+ - `eval_strategy`: steps
301
+ - `per_device_train_batch_size`: 16
302
+ - `per_device_eval_batch_size`: 16
303
+ - `learning_rate`: 2e-05
304
+ - `num_train_epochs`: 1
305
+ - `warmup_ratio`: 0.1
306
+ - `seed`: 12
307
+ - `bf16`: True
308
+ - `load_best_model_at_end`: True
309
+
310
+ #### All Hyperparameters
311
+ <details><summary>Click to expand</summary>
312
+
313
+ - `overwrite_output_dir`: False
314
+ - `do_predict`: False
315
+ - `eval_strategy`: steps
316
+ - `prediction_loss_only`: True
317
+ - `per_device_train_batch_size`: 16
318
+ - `per_device_eval_batch_size`: 16
319
+ - `per_gpu_train_batch_size`: None
320
+ - `per_gpu_eval_batch_size`: None
321
+ - `gradient_accumulation_steps`: 1
322
+ - `eval_accumulation_steps`: None
323
+ - `torch_empty_cache_steps`: None
324
+ - `learning_rate`: 2e-05
325
+ - `weight_decay`: 0.0
326
+ - `adam_beta1`: 0.9
327
+ - `adam_beta2`: 0.999
328
+ - `adam_epsilon`: 1e-08
329
+ - `max_grad_norm`: 1.0
330
+ - `num_train_epochs`: 1
331
+ - `max_steps`: -1
332
+ - `lr_scheduler_type`: linear
333
+ - `lr_scheduler_kwargs`: {}
334
+ - `warmup_ratio`: 0.1
335
+ - `warmup_steps`: 0
336
+ - `log_level`: passive
337
+ - `log_level_replica`: warning
338
+ - `log_on_each_node`: True
339
+ - `logging_nan_inf_filter`: True
340
+ - `save_safetensors`: True
341
+ - `save_on_each_node`: False
342
+ - `save_only_model`: False
343
+ - `restore_callback_states_from_checkpoint`: False
344
+ - `no_cuda`: False
345
+ - `use_cpu`: False
346
+ - `use_mps_device`: False
347
+ - `seed`: 12
348
+ - `data_seed`: None
349
+ - `jit_mode_eval`: False
350
+ - `use_ipex`: False
351
+ - `bf16`: True
352
+ - `fp16`: False
353
+ - `fp16_opt_level`: O1
354
+ - `half_precision_backend`: auto
355
+ - `bf16_full_eval`: False
356
+ - `fp16_full_eval`: False
357
+ - `tf32`: None
358
+ - `local_rank`: 0
359
+ - `ddp_backend`: None
360
+ - `tpu_num_cores`: None
361
+ - `tpu_metrics_debug`: False
362
+ - `debug`: []
363
+ - `dataloader_drop_last`: False
364
+ - `dataloader_num_workers`: 0
365
+ - `dataloader_prefetch_factor`: None
366
+ - `past_index`: -1
367
+ - `disable_tqdm`: False
368
+ - `remove_unused_columns`: True
369
+ - `label_names`: None
370
+ - `load_best_model_at_end`: True
371
+ - `ignore_data_skip`: False
372
+ - `fsdp`: []
373
+ - `fsdp_min_num_params`: 0
374
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
375
+ - `fsdp_transformer_layer_cls_to_wrap`: None
376
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
377
+ - `deepspeed`: None
378
+ - `label_smoothing_factor`: 0.0
379
+ - `optim`: adamw_torch
380
+ - `optim_args`: None
381
+ - `adafactor`: False
382
+ - `group_by_length`: False
383
+ - `length_column_name`: length
384
+ - `ddp_find_unused_parameters`: None
385
+ - `ddp_bucket_cap_mb`: None
386
+ - `ddp_broadcast_buffers`: False
387
+ - `dataloader_pin_memory`: True
388
+ - `dataloader_persistent_workers`: False
389
+ - `skip_memory_metrics`: True
390
+ - `use_legacy_prediction_loop`: False
391
+ - `push_to_hub`: False
392
+ - `resume_from_checkpoint`: None
393
+ - `hub_model_id`: None
394
+ - `hub_strategy`: every_save
395
+ - `hub_private_repo`: None
396
+ - `hub_always_push`: False
397
+ - `gradient_checkpointing`: False
398
+ - `gradient_checkpointing_kwargs`: None
399
+ - `include_inputs_for_metrics`: False
400
+ - `include_for_metrics`: []
401
+ - `eval_do_concat_batches`: True
402
+ - `fp16_backend`: auto
403
+ - `push_to_hub_model_id`: None
404
+ - `push_to_hub_organization`: None
405
+ - `mp_parameters`:
406
+ - `auto_find_batch_size`: False
407
+ - `full_determinism`: False
408
+ - `torchdynamo`: None
409
+ - `ray_scope`: last
410
+ - `ddp_timeout`: 1800
411
+ - `torch_compile`: False
412
+ - `torch_compile_backend`: None
413
+ - `torch_compile_mode`: None
414
+ - `dispatch_batches`: None
415
+ - `split_batches`: None
416
+ - `include_tokens_per_second`: False
417
+ - `include_num_input_tokens_seen`: False
418
+ - `neftune_noise_alpha`: None
419
+ - `optim_target_modules`: None
420
+ - `batch_eval_metrics`: False
421
+ - `eval_on_start`: False
422
+ - `use_liger_kernel`: False
423
+ - `eval_use_gather_object`: False
424
+ - `average_tokens_across_devices`: False
425
+ - `prompts`: None
426
+ - `batch_sampler`: batch_sampler
427
+ - `multi_dataset_batch_sampler`: proportional
428
+
429
+ </details>
430
+
431
+ ### Training Logs
432
+ <details><summary>Click to expand</summary>
433
+
434
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10 | NanoBEIR_R100_mean_ndcg@10 |
435
+ |:----------:|:---------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
436
+ | -1 | -1 | - | - | 0.0450 (-0.4954) | 0.2481 (-0.0770) | 0.0060 (-0.4946) | 0.0997 (-0.3557) |
437
+ | 0.0000 | 1 | 2.9434 | - | - | - | - | - |
438
+ | 0.0080 | 200 | 2.9411 | - | - | - | - | - |
439
+ | 0.0160 | 400 | 2.9116 | - | - | - | - | - |
440
+ | 0.0240 | 600 | 2.4206 | - | - | - | - | - |
441
+ | 0.0321 | 800 | 2.2645 | - | - | - | - | - |
442
+ | 0.0401 | 1000 | 2.2051 | 2.1925 | 0.6174 (+0.0769) | 0.3682 (+0.0432) | 0.7195 (+0.2188) | 0.5683 (+0.1130) |
443
+ | 0.0481 | 1200 | 2.1685 | - | - | - | - | - |
444
+ | 0.0561 | 1400 | 2.1532 | - | - | - | - | - |
445
+ | 0.0641 | 1600 | 2.1569 | - | - | - | - | - |
446
+ | 0.0721 | 1800 | 2.1282 | - | - | - | - | - |
447
+ | 0.0801 | 2000 | 2.0972 | 2.0817 | 0.6131 (+0.0727) | 0.4047 (+0.0796) | 0.7181 (+0.2174) | 0.5786 (+0.1233) |
448
+ | 0.0882 | 2200 | 2.1016 | - | - | - | - | - |
449
+ | 0.0962 | 2400 | 2.1249 | - | - | - | - | - |
450
+ | 0.1042 | 2600 | 2.109 | - | - | - | - | - |
451
+ | 0.1122 | 2800 | 2.0946 | - | - | - | - | - |
452
+ | 0.1202 | 3000 | 2.1034 | 2.0547 | 0.6288 (+0.0884) | 0.4109 (+0.0858) | 0.6923 (+0.1916) | 0.5773 (+0.1220) |
453
+ | 0.1282 | 3200 | 2.0827 | - | - | - | - | - |
454
+ | 0.1362 | 3400 | 2.0947 | - | - | - | - | - |
455
+ | 0.1443 | 3600 | 2.0836 | - | - | - | - | - |
456
+ | 0.1523 | 3800 | 2.0703 | - | - | - | - | - |
457
+ | 0.1603 | 4000 | 2.0699 | 2.0256 | 0.6420 (+0.1016) | 0.4175 (+0.0925) | 0.6758 (+0.1751) | 0.5784 (+0.1231) |
458
+ | 0.1683 | 4200 | 2.0725 | - | - | - | - | - |
459
+ | 0.1763 | 4400 | 2.0883 | - | - | - | - | - |
460
+ | 0.1843 | 4600 | 2.0497 | - | - | - | - | - |
461
+ | 0.1923 | 4800 | 2.0776 | - | - | - | - | - |
462
+ | 0.2004 | 5000 | 2.0525 | 2.0560 | 0.6122 (+0.0718) | 0.4081 (+0.0831) | 0.7000 (+0.1993) | 0.5734 (+0.1181) |
463
+ | 0.2084 | 5200 | 2.0453 | - | - | - | - | - |
464
+ | 0.2164 | 5400 | 2.0658 | - | - | - | - | - |
465
+ | 0.2244 | 5600 | 2.048 | - | - | - | - | - |
466
+ | 0.2324 | 5800 | 2.0611 | - | - | - | - | - |
467
+ | 0.2404 | 6000 | 2.0315 | 2.0254 | 0.6147 (+0.0742) | 0.4107 (+0.0857) | 0.7001 (+0.1995) | 0.5752 (+0.1198) |
468
+ | 0.2484 | 6200 | 2.0503 | - | - | - | - | - |
469
+ | 0.2565 | 6400 | 2.031 | - | - | - | - | - |
470
+ | 0.2645 | 6600 | 2.0414 | - | - | - | - | - |
471
+ | 0.2725 | 6800 | 2.0555 | - | - | - | - | - |
472
+ | 0.2805 | 7000 | 2.0661 | 1.9993 | 0.6386 (+0.0982) | 0.4072 (+0.0821) | 0.7242 (+0.2235) | 0.5900 (+0.1346) |
473
+ | 0.2885 | 7200 | 2.0651 | - | - | - | - | - |
474
+ | 0.2965 | 7400 | 2.0353 | - | - | - | - | - |
475
+ | 0.3045 | 7600 | 2.0476 | - | - | - | - | - |
476
+ | 0.3126 | 7800 | 2.0381 | - | - | - | - | - |
477
+ | 0.3206 | 8000 | 2.0335 | 1.9981 | 0.5942 (+0.0538) | 0.3988 (+0.0737) | 0.7395 (+0.2388) | 0.5775 (+0.1221) |
478
+ | 0.3286 | 8200 | 2.0277 | - | - | - | - | - |
479
+ | 0.3366 | 8400 | 2.0438 | - | - | - | - | - |
480
+ | 0.3446 | 8600 | 2.0432 | - | - | - | - | - |
481
+ | 0.3526 | 8800 | 2.0434 | - | - | - | - | - |
482
+ | 0.3606 | 9000 | 2.0324 | 1.9997 | 0.6263 (+0.0859) | 0.4034 (+0.0783) | 0.7200 (+0.2194) | 0.5832 (+0.1279) |
483
+ | 0.3686 | 9200 | 2.0254 | - | - | - | - | - |
484
+ | 0.3767 | 9400 | 2.0366 | - | - | - | - | - |
485
+ | 0.3847 | 9600 | 2.0327 | - | - | - | - | - |
486
+ | 0.3927 | 9800 | 2.041 | - | - | - | - | - |
487
+ | 0.4007 | 10000 | 2.0329 | 1.9951 | 0.6376 (+0.0972) | 0.3915 (+0.0665) | 0.7179 (+0.2173) | 0.5823 (+0.1270) |
488
+ | 0.4087 | 10200 | 2.0196 | - | - | - | - | - |
489
+ | 0.4167 | 10400 | 2.0331 | - | - | - | - | - |
490
+ | 0.4247 | 10600 | 2.0394 | - | - | - | - | - |
491
+ | 0.4328 | 10800 | 2.0314 | - | - | - | - | - |
492
+ | 0.4408 | 11000 | 2.0344 | 1.9906 | 0.6161 (+0.0757) | 0.3955 (+0.0704) | 0.7287 (+0.2280) | 0.5801 (+0.1247) |
493
+ | 0.4488 | 11200 | 2.0403 | - | - | - | - | - |
494
+ | 0.4568 | 11400 | 2.0394 | - | - | - | - | - |
495
+ | 0.4648 | 11600 | 2.013 | - | - | - | - | - |
496
+ | 0.4728 | 11800 | 2.0279 | - | - | - | - | - |
497
+ | 0.4808 | 12000 | 2.0251 | 1.9968 | 0.6142 (+0.0738) | 0.3985 (+0.0734) | 0.7230 (+0.2224) | 0.5786 (+0.1232) |
498
+ | 0.4889 | 12200 | 2.0252 | - | - | - | - | - |
499
+ | 0.4969 | 12400 | 2.0401 | - | - | - | - | - |
500
+ | 0.5049 | 12600 | 2.0149 | - | - | - | - | - |
501
+ | 0.5129 | 12800 | 2.0293 | - | - | - | - | - |
502
+ | 0.5209 | 13000 | 2.0261 | 1.9864 | 0.6238 (+0.0834) | 0.4051 (+0.0801) | 0.7441 (+0.2434) | 0.5910 (+0.1356) |
503
+ | 0.5289 | 13200 | 2.0079 | - | - | - | - | - |
504
+ | 0.5369 | 13400 | 2.0309 | - | - | - | - | - |
505
+ | 0.5450 | 13600 | 2.0273 | - | - | - | - | - |
506
+ | 0.5530 | 13800 | 2.037 | - | - | - | - | - |
507
+ | 0.5610 | 14000 | 2.038 | 1.9820 | 0.6181 (+0.0777) | 0.4075 (+0.0824) | 0.7417 (+0.2411) | 0.5891 (+0.1337) |
508
+ | 0.5690 | 14200 | 2.0205 | - | - | - | - | - |
509
+ | 0.5770 | 14400 | 2.0116 | - | - | - | - | - |
510
+ | 0.5850 | 14600 | 1.9943 | - | - | - | - | - |
511
+ | 0.5930 | 14800 | 2.0315 | - | - | - | - | - |
512
+ | 0.6011 | 15000 | 2.0179 | 1.9784 | 0.6340 (+0.0935) | 0.4052 (+0.0802) | 0.7390 (+0.2384) | 0.5927 (+0.1374) |
513
+ | 0.6091 | 15200 | 2.022 | - | - | - | - | - |
514
+ | 0.6171 | 15400 | 2.02 | - | - | - | - | - |
515
+ | 0.6251 | 15600 | 2.0152 | - | - | - | - | - |
516
+ | 0.6331 | 15800 | 2.0312 | - | - | - | - | - |
517
+ | 0.6411 | 16000 | 2.0274 | 1.9822 | 0.6473 (+0.1068) | 0.3929 (+0.0678) | 0.7578 (+0.2571) | 0.5993 (+0.1439) |
518
+ | 0.6491 | 16200 | 2.0045 | - | - | - | - | - |
519
+ | 0.6572 | 16400 | 2.0015 | - | - | - | - | - |
520
+ | 0.6652 | 16600 | 2.0162 | - | - | - | - | - |
521
+ | 0.6732 | 16800 | 1.9984 | - | - | - | - | - |
522
+ | 0.6812 | 17000 | 2.0037 | 1.9781 | 0.6431 (+0.1026) | 0.4003 (+0.0753) | 0.7625 (+0.2619) | 0.6020 (+0.1466) |
523
+ | 0.6892 | 17200 | 2.0268 | - | - | - | - | - |
524
+ | 0.6972 | 17400 | 2.0036 | - | - | - | - | - |
525
+ | 0.7052 | 17600 | 2.0034 | - | - | - | - | - |
526
+ | 0.7133 | 17800 | 2.0179 | - | - | - | - | - |
527
+ | 0.7213 | 18000 | 2.0259 | 1.9782 | 0.6291 (+0.0887) | 0.4046 (+0.0795) | 0.7505 (+0.2499) | 0.5948 (+0.1394) |
528
+ | 0.7293 | 18200 | 2.0111 | - | - | - | - | - |
529
+ | 0.7373 | 18400 | 2.0051 | - | - | - | - | - |
530
+ | 0.7453 | 18600 | 2.0028 | - | - | - | - | - |
531
+ | 0.7533 | 18800 | 2.0049 | - | - | - | - | - |
532
+ | 0.7613 | 19000 | 2.0125 | 1.9758 | 0.6410 (+0.1006) | 0.4035 (+0.0784) | 0.7569 (+0.2563) | 0.6005 (+0.1451) |
533
+ | 0.7694 | 19200 | 2.0104 | - | - | - | - | - |
534
+ | 0.7774 | 19400 | 2.0008 | - | - | - | - | - |
535
+ | 0.7854 | 19600 | 1.9954 | - | - | - | - | - |
536
+ | 0.7934 | 19800 | 2.0061 | - | - | - | - | - |
537
+ | 0.8014 | 20000 | 2.0141 | 1.9821 | 0.6344 (+0.0940) | 0.4089 (+0.0839) | 0.7459 (+0.2453) | 0.5964 (+0.1410) |
538
+ | 0.8094 | 20200 | 2.0137 | - | - | - | - | - |
539
+ | 0.8174 | 20400 | 1.9882 | - | - | - | - | - |
540
+ | 0.8255 | 20600 | 2.0107 | - | - | - | - | - |
541
+ | 0.8335 | 20800 | 2.009 | - | - | - | - | - |
542
+ | 0.8415 | 21000 | 2.0053 | 1.9745 | 0.6308 (+0.0903) | 0.4087 (+0.0836) | 0.7682 (+0.2676) | 0.6025 (+0.1472) |
543
+ | 0.8495 | 21200 | 2.0082 | - | - | - | - | - |
544
+ | 0.8575 | 21400 | 2.0092 | - | - | - | - | - |
545
+ | 0.8655 | 21600 | 2.0008 | - | - | - | - | - |
546
+ | 0.8735 | 21800 | 1.9981 | - | - | - | - | - |
547
+ | 0.8816 | 22000 | 2.0058 | 1.9737 | 0.6393 (+0.0989) | 0.4031 (+0.0781) | 0.7596 (+0.2590) | 0.6007 (+0.1453) |
548
+ | 0.8896 | 22200 | 2.0288 | - | - | - | - | - |
549
+ | 0.8976 | 22400 | 2.0172 | - | - | - | - | - |
550
+ | 0.9056 | 22600 | 2.0098 | - | - | - | - | - |
551
+ | 0.9136 | 22800 | 1.9982 | - | - | - | - | - |
552
+ | 0.9216 | 23000 | 2.0185 | 1.9719 | 0.6373 (+0.0969) | 0.4008 (+0.0758) | 0.7596 (+0.2589) | 0.5992 (+0.1439) |
553
+ | 0.9296 | 23200 | 2.0093 | - | - | - | - | - |
554
+ | 0.9377 | 23400 | 1.9904 | - | - | - | - | - |
555
+ | 0.9457 | 23600 | 1.9884 | - | - | - | - | - |
556
+ | 0.9537 | 23800 | 2.0174 | - | - | - | - | - |
557
+ | **0.9617** | **24000** | **1.9845** | **1.9739** | **0.6369 (+0.0965)** | **0.4077 (+0.0827)** | **0.7642 (+0.2636)** | **0.6030 (+0.1476)** |
558
+ | 0.9697 | 24200 | 1.9943 | - | - | - | - | - |
559
+ | 0.9777 | 24400 | 2.0084 | - | - | - | - | - |
560
+ | 0.9857 | 24600 | 2.0001 | - | - | - | - | - |
561
+ | 0.9937 | 24800 | 2.0168 | - | - | - | - | - |
562
+ | -1 | -1 | - | - | 0.6369 (+0.0965) | 0.4077 (+0.0827) | 0.7642 (+0.2636) | 0.6030 (+0.1476) |
563
+
564
+ * The bold row denotes the saved checkpoint.
565
+ </details>
566
+
567
+ ### Environmental Impact
568
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
569
+ - **Energy Consumed**: 2.350 kWh
570
+ - **Carbon Emitted**: 0.913 kg of CO2
571
+ - **Hours Used**: 8.397 hours
572
+
573
+ ### Training Hardware
574
+ - **On Cloud**: No
575
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
576
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
577
+ - **RAM Size**: 31.78 GB
578
+
579
+ ### Framework Versions
580
+ - Python: 3.11.6
581
+ - Sentence Transformers: 3.5.0.dev0
582
+ - Transformers: 4.49.0
583
+ - PyTorch: 2.6.0+cu124
584
+ - Accelerate: 1.5.1
585
+ - Datasets: 3.3.2
586
+ - Tokenizers: 0.21.0
587
+
588
+ ## Citation
589
+
590
+ ### BibTeX
591
+
592
+ #### Sentence Transformers
593
+ ```bibtex
594
+ @inproceedings{reimers-2019-sentence-bert,
595
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
596
+ author = "Reimers, Nils and Gurevych, Iryna",
597
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
598
+ month = "11",
599
+ year = "2019",
600
+ publisher = "Association for Computational Linguistics",
601
+ url = "https://arxiv.org/abs/1908.10084",
602
+ }
603
+ ```
604
+
605
+ #### PListMLELoss
606
+ ```bibtex
607
+ @inproceedings{lan2014position,
608
+ title={Position-Aware ListMLE: A Sequential Learning Process for Ranking.},
609
+ author={Lan, Yanyan and Zhu, Yadong and Guo, Jiafeng and Niu, Shuzi and Cheng, Xueqi},
610
+ booktitle={UAI},
611
+ volume={14},
612
+ pages={449--458},
613
+ year={2014}
614
+ }
615
+ ```
616
+
617
+ <!--
618
+ ## Glossary
619
+
620
+ *Clearly define terms in order to be accessible across audiences.*
621
+ -->
622
+
623
+ <!--
624
+ ## Model Card Authors
625
+
626
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
627
+ -->
628
+
629
+ <!--
630
+ ## Model Card Contact
631
+
632
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
633
+ -->
config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "microsoft/MiniLM-L12-H384-uncased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 384,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 1536,
16
+ "label2id": {
17
+ "LABEL_0": 0
18
+ },
19
+ "layer_norm_eps": 1e-12,
20
+ "max_position_embeddings": 512,
21
+ "model_type": "bert",
22
+ "num_attention_heads": 12,
23
+ "num_hidden_layers": 12,
24
+ "pad_token_id": 0,
25
+ "position_embedding_type": "absolute",
26
+ "sentence_transformers": {
27
+ "activation_fn": "torch.nn.modules.activation.Sigmoid"
28
+ },
29
+ "torch_dtype": "float32",
30
+ "transformers_version": "4.49.0",
31
+ "type_vocab_size": 2,
32
+ "use_cache": true,
33
+ "vocab_size": 30522
34
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bea5a6d836dffbdcd5452d526afe0918f8b65ea9b9b4937327d083833ed900cb
3
+ size 133464836
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "extra_special_tokens": {},
49
+ "mask_token": "[MASK]",
50
+ "model_max_length": 512,
51
+ "never_split": null,
52
+ "pad_token": "[PAD]",
53
+ "sep_token": "[SEP]",
54
+ "strip_accents": null,
55
+ "tokenize_chinese_chars": true,
56
+ "tokenizer_class": "BertTokenizer",
57
+ "unk_token": "[UNK]"
58
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff