Hugging Face's logo

rahulseetharaman
/
reranker-ettin-encoder-68m-msmarco-bce-1m

Text Ranking
sentence-transformers
Safetensors
English
modernbert
cross-encoder
reranker
Generated from Trainer
dataset_size:990000
loss:BinaryCrossEntropyLoss
Eval Results
Model card Files Community
rahulseetharaman commited on
Commit
851406a
·
verified ·
1 Parent(s): 9b0c2c2

Add new CrossEncoder model

Browse files
Files changed (6) hide show
  1. README.md +726 -0
  2. config.json +57 -0
  3. model.safetensors +3 -0
  4. special_tokens_map.json +37 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +945 -0
README.md ADDED
@@ -0,0 +1,726 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - sentence-transformers
6
+ - cross-encoder
7
+ - reranker
8
+ - generated_from_trainer
9
+ - dataset_size:990000
10
+ - loss:BinaryCrossEntropyLoss
11
+ base_model: jhu-clsp/ettin-encoder-68m
12
+ datasets:
13
+ - sentence-transformers/msmarco
14
+ pipeline_tag: text-ranking
15
+ library_name: sentence-transformers
16
+ metrics:
17
+ - map
18
+ - mrr@10
19
+ - ndcg@10
20
+ model-index:
21
+ - name: CrossEncoder based on jhu-clsp/ettin-encoder-68m
22
+ results:
23
+ - task:
24
+ type: cross-encoder-reranking
25
+ name: Cross Encoder Reranking
26
+ dataset:
27
+ name: NanoMSMARCO R100
28
+ type: NanoMSMARCO_R100
29
+ metrics:
30
+ - type: map
31
+ value: 0.6278
32
+ name: Map
33
+ - type: mrr@10
34
+ value: 0.6237
35
+ name: Mrr@10
36
+ - type: ndcg@10
37
+ value: 0.6994
38
+ name: Ndcg@10
39
+ - task:
40
+ type: cross-encoder-reranking
41
+ name: Cross Encoder Reranking
42
+ dataset:
43
+ name: NanoNFCorpus R100
44
+ type: NanoNFCorpus_R100
45
+ metrics:
46
+ - type: map
47
+ value: 0.3505
48
+ name: Map
49
+ - type: mrr@10
50
+ value: 0.5448
51
+ name: Mrr@10
52
+ - type: ndcg@10
53
+ value: 0.4143
54
+ name: Ndcg@10
55
+ - task:
56
+ type: cross-encoder-reranking
57
+ name: Cross Encoder Reranking
58
+ dataset:
59
+ name: NanoNQ R100
60
+ type: NanoNQ_R100
61
+ metrics:
62
+ - type: map
63
+ value: 0.6751
64
+ name: Map
65
+ - type: mrr@10
66
+ value: 0.6943
67
+ name: Mrr@10
68
+ - type: ndcg@10
69
+ value: 0.72
70
+ name: Ndcg@10
71
+ - task:
72
+ type: cross-encoder-nano-beir
73
+ name: Cross Encoder Nano BEIR
74
+ dataset:
75
+ name: NanoBEIR R100 mean
76
+ type: NanoBEIR_R100_mean
77
+ metrics:
78
+ - type: map
79
+ value: 0.5511
80
+ name: Map
81
+ - type: mrr@10
82
+ value: 0.6209
83
+ name: Mrr@10
84
+ - type: ndcg@10
85
+ value: 0.6112
86
+ name: Ndcg@10
87
+ ---
88
+
89
+ # CrossEncoder based on jhu-clsp/ettin-encoder-68m
90
+
91
+ This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [jhu-clsp/ettin-encoder-68m](https://huggingface.co/jhu-clsp/ettin-encoder-68m) on the [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
92
+
93
+ ## Model Details
94
+
95
+ ### Model Description
96
+ - **Model Type:** Cross Encoder
97
+ - **Base model:** [jhu-clsp/ettin-encoder-68m](https://huggingface.co/jhu-clsp/ettin-encoder-68m) <!-- at revision ac19ae4bc51093b31c475665ac872a936d056cc2 -->
98
+ - **Maximum Sequence Length:** 7999 tokens
99
+ - **Number of Output Labels:** 1 label
100
+ - **Training Dataset:**
101
+ - [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco)
102
+ - **Language:** en
103
+ <!-- - **License:** Unknown -->
104
+
105
+ ### Model Sources
106
+
107
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
108
+ - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
109
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
110
+ - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
111
+
112
+ ## Usage
113
+
114
+ ### Direct Usage (Sentence Transformers)
115
+
116
+ First install the Sentence Transformers library:
117
+
118
+ ```bash
119
+ pip install -U sentence-transformers
120
+ ```
121
+
122
+ Then you can load this model and run inference.
123
+ ```python
124
+ from sentence_transformers import CrossEncoder
125
+
126
+ # Download from the 🤗 Hub
127
+ model = CrossEncoder("rahulseetharaman/reranker-ettin-encoder-68m-msmarco-bce-1m")
128
+ # Get scores for pairs of texts
129
+ pairs = [
130
+ ['star voyager cast', 'On August 25, 2012, data from Voyager 1 indicated that it had become the first human-made object to enter interstellar space, traveling further than anyone, or anything, in history. As of 2013, Voyager 1 was moving with a velocity of 17 kilometers per second (11 mi/s) relative to the Sun. Voyager 2 is expected to enter interstellar space by 2016, and its plasma spectrometer should provide the first direct measurements of the density and temperature of the interstellar plasma.'],
131
+ ['physiologist who conducted the classical conditioning studies with dogs', "Classical Conditioning. The people who fed Pavlov's dogs wore lab coats. Pavlov noticed that the dogs began to drool whenever they saw lab coats, even if there was no food in sight. Pavlov wondered why the dogs salivated at lab coats, and not just at food."],
132
+ ['is a written check considered a contract', 'If you bought a flat screen TV and are paying it off monthly, thatâ\x80\x99s considered recurring debt because you canâ\x80\x99t easily cancel your payments. If you subscribe to a magazine or have an Internet or phone contract, those obligations are not considered recurring debt because there is no fixed debt amount you are paying down and you can cancel your contract.'],
133
+ ['definition of an actuator', 'An actuator is a type of motor that is responsible for moving or controlling a mechanism or system. It is operated by a source of energy, typically electric current, hydraulic fluid pressure, or pneumatic pressure, and converts that energy into motion. An actuator is the mechanism by which a control system acts upon an environment. The control system can be simple (a fixed mechanical or electronic system), software-based (e.g. a printer driver, robot control system), a human, or any other input.'],
134
+ ['what are rheumatoid arthritis symptoms', 'While early RA symptoms can be mimicked by other diseases, the symptoms and signs are very characteristic of rheumatoid disease. The 15 early rheumatoid arthritis symptoms and signs discussed in this article include the following: Fatigue. Joint pain. Joint tenderness. Joint swelling. Joint redness. Joint warmth.'],
135
+ ]
136
+ scores = model.predict(pairs)
137
+ print(scores.shape)
138
+ # (5,)
139
+
140
+ # Or rank different texts based on similarity to a single text
141
+ ranks = model.rank(
142
+ 'star voyager cast',
143
+ [
144
+ 'On August 25, 2012, data from Voyager 1 indicated that it had become the first human-made object to enter interstellar space, traveling further than anyone, or anything, in history. As of 2013, Voyager 1 was moving with a velocity of 17 kilometers per second (11 mi/s) relative to the Sun. Voyager 2 is expected to enter interstellar space by 2016, and its plasma spectrometer should provide the first direct measurements of the density and temperature of the interstellar plasma.',
145
+ "Classical Conditioning. The people who fed Pavlov's dogs wore lab coats. Pavlov noticed that the dogs began to drool whenever they saw lab coats, even if there was no food in sight. Pavlov wondered why the dogs salivated at lab coats, and not just at food.",
146
+ 'If you bought a flat screen TV and are paying it off monthly, thatâ\x80\x99s considered recurring debt because you canâ\x80\x99t easily cancel your payments. If you subscribe to a magazine or have an Internet or phone contract, those obligations are not considered recurring debt because there is no fixed debt amount you are paying down and you can cancel your contract.',
147
+ 'An actuator is a type of motor that is responsible for moving or controlling a mechanism or system. It is operated by a source of energy, typically electric current, hydraulic fluid pressure, or pneumatic pressure, and converts that energy into motion. An actuator is the mechanism by which a control system acts upon an environment. The control system can be simple (a fixed mechanical or electronic system), software-based (e.g. a printer driver, robot control system), a human, or any other input.',
148
+ 'While early RA symptoms can be mimicked by other diseases, the symptoms and signs are very characteristic of rheumatoid disease. The 15 early rheumatoid arthritis symptoms and signs discussed in this article include the following: Fatigue. Joint pain. Joint tenderness. Joint swelling. Joint redness. Joint warmth.',
149
+ ]
150
+ )
151
+ # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
152
+ ```
153
+
154
+ <!--
155
+ ### Direct Usage (Transformers)
156
+
157
+ <details><summary>Click to see the direct usage in Transformers</summary>
158
+
159
+ </details>
160
+ -->
161
+
162
+ <!--
163
+ ### Downstream Usage (Sentence Transformers)
164
+
165
+ You can finetune this model on your own dataset.
166
+
167
+ <details><summary>Click to expand</summary>
168
+
169
+ </details>
170
+ -->
171
+
172
+ <!--
173
+ ### Out-of-Scope Use
174
+
175
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
176
+ -->
177
+
178
+ ## Evaluation
179
+
180
+ ### Metrics
181
+
182
+ #### Cross Encoder Reranking
183
+
184
+ * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
185
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
186
+ ```json
187
+ {
188
+ "at_k": 10,
189
+ "always_rerank_positives": true
190
+ }
191
+ ```
192
+
193
+ | Metric | NanoMSMARCO_R100 | NanoNFCorpus_R100 | NanoNQ_R100 |
194
+ |:------------|:---------------------|:---------------------|:---------------------|
195
+ | map | 0.6278 (+0.1382) | 0.3505 (+0.0895) | 0.6751 (+0.2555) |
196
+ | mrr@10 | 0.6237 (+0.1462) | 0.5448 (+0.0450) | 0.6943 (+0.2676) |
197
+ | **ndcg@10** | **0.6994 (+0.1590)** | **0.4143 (+0.0892)** | **0.7200 (+0.2193)** |
198
+
199
+ #### Cross Encoder Nano BEIR
200
+
201
+ * Dataset: `NanoBEIR_R100_mean`
202
+ * Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
203
+ ```json
204
+ {
205
+ "dataset_names": [
206
+ "msmarco",
207
+ "nfcorpus",
208
+ "nq"
209
+ ],
210
+ "rerank_k": 100,
211
+ "at_k": 10,
212
+ "always_rerank_positives": true
213
+ }
214
+ ```
215
+
216
+ | Metric | Value |
217
+ |:------------|:---------------------|
218
+ | map | 0.5511 (+0.1611) |
219
+ | mrr@10 | 0.6209 (+0.1529) |
220
+ | **ndcg@10** | **0.6112 (+0.1558)** |
221
+
222
+ <!--
223
+ ## Bias, Risks and Limitations
224
+
225
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
226
+ -->
227
+
228
+ <!--
229
+ ### Recommendations
230
+
231
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
232
+ -->
233
+
234
+ ## Training Details
235
+
236
+ ### Training Dataset
237
+
238
+ #### msmarco
239
+
240
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [9e329ed](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/9e329ed2e649c9d37b0d91dd6b764ff6fe671d83)
241
+ * Size: 990,000 training samples
242
+ * Columns: <code>query</code>, <code>passage</code>, and <code>score</code>
243
+ * Approximate statistics based on the first 1000 samples:
244
+ | | query | passage | score |
245
+ |:--------|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:--------------------------------------------------------------|
246
+ | type | string | string | float |
247
+ | details | <ul><li>min: 10 characters</li><li>mean: 34.06 characters</li><li>max: 103 characters</li></ul> | <ul><li>min: 55 characters</li><li>mean: 341.02 characters</li><li>max: 943 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
248
+ * Samples:
249
+ | query | passage | score |
250
+ |:----------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
251
+ | <code>can a urinalysis detect kidney disease</code> | <code>A urinalysis is a simple, inexpensive test that can help to detect problems in many parts of the body, including the kidneys, urinary tract, heart, and liver. A urinalysis can include a visual examination of a urine sample, microscopic examination, and a dipstick test.</code> | <code>1.0</code> |
252
+ | <code>what is the hamsa hand</code> | <code>Answer by Mikereptile. Confidence votes 60. It takes about 6-8 weeks to heal, depending on the type of injury sustained. (Keep the finger as still as you can !!!!). When broken place the hand in ice cold water for about 5 min.s then take it and wrap the whole hand in a soft wrap (anything that is a soft colth) .Take the wraped hand and hold it above or upright to the head.hen broken place the hand in ice cold water for about 5 min.s then take it and wrap the whole hand in a soft wrap (anything that is a soft colth) . Take the wraped hand and hold it above or upright to the head.</code> | <code>0.0</code> |
253
+ | <code>was white tiger in the us zoo killed?</code> | <code>Cubby is a male American black bear born at the Chahinkapa Zoo in North Dakota. He was transferred to the Hogle Zoo (Salt Lake City, Utah) in 2003, and arrived at the Oregon Zoo in May 2010.</code> | <code>0.0</code> |
254
+ * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
255
+ ```json
256
+ {
257
+ "activation_fn": "torch.nn.modules.linear.Identity",
258
+ "pos_weight": null
259
+ }
260
+ ```
261
+
262
+ ### Evaluation Dataset
263
+
264
+ #### msmarco
265
+
266
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [9e329ed](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/9e329ed2e649c9d37b0d91dd6b764ff6fe671d83)
267
+ * Size: 10,000 evaluation samples
268
+ * Columns: <code>query</code>, <code>passage</code>, and <code>score</code>
269
+ * Approximate statistics based on the first 1000 samples:
270
+ | | query | passage | score |
271
+ |:--------|:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
272
+ | type | string | string | float |
273
+ | details | <ul><li>min: 10 characters</li><li>mean: 33.97 characters</li><li>max: 121 characters</li></ul> | <ul><li>min: 70 characters</li><li>mean: 345.9 characters</li><li>max: 946 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.48</li><li>max: 1.0</li></ul> |
274
+ * Samples:
275
+ | query | passage | score |
276
+ |:-------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
277
+ | <code>star voyager cast</code> | <code>On August 25, 2012, data from Voyager 1 indicated that it had become the first human-made object to enter interstellar space, traveling further than anyone, or anything, in history. As of 2013, Voyager 1 was moving with a velocity of 17 kilometers per second (11 mi/s) relative to the Sun. Voyager 2 is expected to enter interstellar space by 2016, and its plasma spectrometer should provide the first direct measurements of the density and temperature of the interstellar plasma.</code> | <code>0.0</code> |
278
+ | <code>physiologist who conducted the classical conditioning studies with dogs</code> | <code>Classical Conditioning. The people who fed Pavlov's dogs wore lab coats. Pavlov noticed that the dogs began to drool whenever they saw lab coats, even if there was no food in sight. Pavlov wondered why the dogs salivated at lab coats, and not just at food.</code> | <code>0.0</code> |
279
+ | <code>is a written check considered a contract</code> | <code>If you bought a flat screen TV and are paying it off monthly, that’s considered recurring debt because you can’t easily cancel your payments. If you subscribe to a magazine or have an Internet or phone contract, those obligations are not considered recurring debt because there is no fixed debt amount you are paying down and you can cancel your contract.</code> | <code>0.0</code> |
280
+ * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
281
+ ```json
282
+ {
283
+ "activation_fn": "torch.nn.modules.linear.Identity",
284
+ "pos_weight": null
285
+ }
286
+ ```
287
+
288
+ ### Training Hyperparameters
289
+ #### Non-Default Hyperparameters
290
+
291
+ - `eval_strategy`: steps
292
+ - `per_device_train_batch_size`: 16
293
+ - `per_device_eval_batch_size`: 16
294
+ - `learning_rate`: 2e-05
295
+ - `num_train_epochs`: 4
296
+ - `warmup_ratio`: 0.1
297
+ - `seed`: 12
298
+ - `bf16`: True
299
+ - `dataloader_num_workers`: 4
300
+ - `load_best_model_at_end`: True
301
+
302
+ #### All Hyperparameters
303
+ <details><summary>Click to expand</summary>
304
+
305
+ - `overwrite_output_dir`: False
306
+ - `do_predict`: False
307
+ - `eval_strategy`: steps
308
+ - `prediction_loss_only`: True
309
+ - `per_device_train_batch_size`: 16
310
+ - `per_device_eval_batch_size`: 16
311
+ - `per_gpu_train_batch_size`: None
312
+ - `per_gpu_eval_batch_size`: None
313
+ - `gradient_accumulation_steps`: 1
314
+ - `eval_accumulation_steps`: None
315
+ - `torch_empty_cache_steps`: None
316
+ - `learning_rate`: 2e-05
317
+ - `weight_decay`: 0.0
318
+ - `adam_beta1`: 0.9
319
+ - `adam_beta2`: 0.999
320
+ - `adam_epsilon`: 1e-08
321
+ - `max_grad_norm`: 1.0
322
+ - `num_train_epochs`: 4
323
+ - `max_steps`: -1
324
+ - `lr_scheduler_type`: linear
325
+ - `lr_scheduler_kwargs`: {}
326
+ - `warmup_ratio`: 0.1
327
+ - `warmup_steps`: 0
328
+ - `log_level`: passive
329
+ - `log_level_replica`: warning
330
+ - `log_on_each_node`: True
331
+ - `logging_nan_inf_filter`: True
332
+ - `save_safetensors`: True
333
+ - `save_on_each_node`: False
334
+ - `save_only_model`: False
335
+ - `restore_callback_states_from_checkpoint`: False
336
+ - `no_cuda`: False
337
+ - `use_cpu`: False
338
+ - `use_mps_device`: False
339
+ - `seed`: 12
340
+ - `data_seed`: None
341
+ - `jit_mode_eval`: False
342
+ - `use_ipex`: False
343
+ - `bf16`: True
344
+ - `fp16`: False
345
+ - `fp16_opt_level`: O1
346
+ - `half_precision_backend`: auto
347
+ - `bf16_full_eval`: False
348
+ - `fp16_full_eval`: False
349
+ - `tf32`: None
350
+ - `local_rank`: 0
351
+ - `ddp_backend`: None
352
+ - `tpu_num_cores`: None
353
+ - `tpu_metrics_debug`: False
354
+ - `debug`: []
355
+ - `dataloader_drop_last`: False
356
+ - `dataloader_num_workers`: 4
357
+ - `dataloader_prefetch_factor`: None
358
+ - `past_index`: -1
359
+ - `disable_tqdm`: False
360
+ - `remove_unused_columns`: True
361
+ - `label_names`: None
362
+ - `load_best_model_at_end`: True
363
+ - `ignore_data_skip`: False
364
+ - `fsdp`: []
365
+ - `fsdp_min_num_params`: 0
366
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
367
+ - `fsdp_transformer_layer_cls_to_wrap`: None
368
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
369
+ - `deepspeed`: None
370
+ - `label_smoothing_factor`: 0.0
371
+ - `optim`: adamw_torch
372
+ - `optim_args`: None
373
+ - `adafactor`: False
374
+ - `group_by_length`: False
375
+ - `length_column_name`: length
376
+ - `ddp_find_unused_parameters`: None
377
+ - `ddp_bucket_cap_mb`: None
378
+ - `ddp_broadcast_buffers`: False
379
+ - `dataloader_pin_memory`: True
380
+ - `dataloader_persistent_workers`: False
381
+ - `skip_memory_metrics`: True
382
+ - `use_legacy_prediction_loop`: False
383
+ - `push_to_hub`: False
384
+ - `resume_from_checkpoint`: None
385
+ - `hub_model_id`: None
386
+ - `hub_strategy`: every_save
387
+ - `hub_private_repo`: None
388
+ - `hub_always_push`: False
389
+ - `hub_revision`: None
390
+ - `gradient_checkpointing`: False
391
+ - `gradient_checkpointing_kwargs`: None
392
+ - `include_inputs_for_metrics`: False
393
+ - `include_for_metrics`: []
394
+ - `eval_do_concat_batches`: True
395
+ - `fp16_backend`: auto
396
+ - `push_to_hub_model_id`: None
397
+ - `push_to_hub_organization`: None
398
+ - `mp_parameters`:
399
+ - `auto_find_batch_size`: False
400
+ - `full_determinism`: False
401
+ - `torchdynamo`: None
402
+ - `ray_scope`: last
403
+ - `ddp_timeout`: 1800
404
+ - `torch_compile`: False
405
+ - `torch_compile_backend`: None
406
+ - `torch_compile_mode`: None
407
+ - `include_tokens_per_second`: False
408
+ - `include_num_input_tokens_seen`: False
409
+ - `neftune_noise_alpha`: None
410
+ - `optim_target_modules`: None
411
+ - `batch_eval_metrics`: False
412
+ - `eval_on_start`: False
413
+ - `use_liger_kernel`: False
414
+ - `liger_kernel_config`: None
415
+ - `eval_use_gather_object`: False
416
+ - `average_tokens_across_devices`: False
417
+ - `prompts`: None
418
+ - `batch_sampler`: batch_sampler
419
+ - `multi_dataset_batch_sampler`: proportional
420
+ - `router_mapping`: {}
421
+ - `learning_rate_mapping`: {}
422
+
423
+ </details>
424
+
425
+ ### Training Logs
426
+ <details><summary>Click to expand</summary>
427
+
428
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10 | NanoBEIR_R100_mean_ndcg@10 |
429
+ |:----------:|:---------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
430
+ | -1 | -1 | - | - | 0.0442 (-0.4962) | 0.2554 (-0.0696) | 0.0464 (-0.4542) | 0.1154 (-0.3400) |
431
+ | 0.0000 | 1 | 0.8582 | - | - | - | - | - |
432
+ | 0.0162 | 1000 | 0.8813 | 0.6855 | 0.0988 (-0.4416) | 0.2787 (-0.0463) | 0.0682 (-0.4325) | 0.1486 (-0.3068) |
433
+ | 0.0323 | 2000 | 0.6341 | 0.5531 | 0.2748 (-0.2656) | 0.2585 (-0.0666) | 0.2394 (-0.2613) | 0.2576 (-0.1978) |
434
+ | 0.0485 | 3000 | 0.3973 | 0.3257 | 0.6044 (+0.0640) | 0.3607 (+0.0356) | 0.5467 (+0.0461) | 0.5039 (+0.0486) |
435
+ | 0.0646 | 4000 | 0.2955 | 0.2749 | 0.6877 (+0.1473) | 0.3659 (+0.0409) | 0.5707 (+0.0701) | 0.5414 (+0.0861) |
436
+ | 0.0808 | 5000 | 0.2628 | 0.2559 | 0.6262 (+0.0858) | 0.3703 (+0.0452) | 0.6061 (+0.1055) | 0.5342 (+0.0788) |
437
+ | 0.0970 | 6000 | 0.2452 | 0.2660 | 0.6171 (+0.0767) | 0.3491 (+0.0241) | 0.6228 (+0.1222) | 0.5297 (+0.0743) |
438
+ | 0.1131 | 7000 | 0.2348 | 0.2279 | 0.6279 (+0.0875) | 0.3449 (+0.0198) | 0.6281 (+0.1274) | 0.5336 (+0.0782) |
439
+ | 0.1293 | 8000 | 0.233 | 0.2411 | 0.6620 (+0.1216) | 0.3405 (+0.0154) | 0.6720 (+0.1714) | 0.5582 (+0.1028) |
440
+ | 0.1455 | 9000 | 0.2297 | 0.2208 | 0.6575 (+0.1171) | 0.3829 (+0.0578) | 0.6887 (+0.1880) | 0.5764 (+0.1210) |
441
+ | 0.1616 | 10000 | 0.2231 | 0.2299 | 0.6276 (+0.0872) | 0.3485 (+0.0235) | 0.6504 (+0.1497) | 0.5422 (+0.0868) |
442
+ | 0.1778 | 11000 | 0.2213 | 0.1970 | 0.6582 (+0.1178) | 0.3905 (+0.0655) | 0.6928 (+0.1922) | 0.5805 (+0.1252) |
443
+ | 0.1939 | 12000 | 0.2228 | 0.1994 | 0.6317 (+0.0913) | 0.3916 (+0.0666) | 0.7004 (+0.1997) | 0.5746 (+0.1192) |
444
+ | 0.2101 | 13000 | 0.2103 | 0.2019 | 0.6522 (+0.1118) | 0.3785 (+0.0535) | 0.7015 (+0.2009) | 0.5774 (+0.1221) |
445
+ | 0.2263 | 14000 | 0.2101 | 0.1949 | 0.6154 (+0.0750) | 0.3966 (+0.0716) | 0.6737 (+0.1731) | 0.5619 (+0.1065) |
446
+ | 0.2424 | 15000 | 0.2165 | 0.2119 | 0.6331 (+0.0927) | 0.3808 (+0.0557) | 0.6563 (+0.1556) | 0.5567 (+0.1013) |
447
+ | 0.2586 | 16000 | 0.2062 | 0.1969 | 0.6668 (+0.1264) | 0.3767 (+0.0517) | 0.6760 (+0.1754) | 0.5732 (+0.1178) |
448
+ | 0.2747 | 17000 | 0.2073 | 0.2004 | 0.6499 (+0.1095) | 0.3966 (+0.0716) | 0.6764 (+0.1757) | 0.5743 (+0.1189) |
449
+ | 0.2909 | 18000 | 0.2056 | 0.2156 | 0.6453 (+0.1049) | 0.3838 (+0.0588) | 0.6732 (+0.1725) | 0.5674 (+0.1121) |
450
+ | 0.3071 | 19000 | 0.2077 | 0.1798 | 0.6705 (+0.1301) | 0.3686 (+0.0436) | 0.6431 (+0.1424) | 0.5607 (+0.1054) |
451
+ | 0.3232 | 20000 | 0.2031 | 0.2161 | 0.6396 (+0.0992) | 0.3968 (+0.0717) | 0.6529 (+0.1523) | 0.5631 (+0.1077) |
452
+ | 0.3394 | 21000 | 0.2048 | 0.1824 | 0.6991 (+0.1587) | 0.3996 (+0.0746) | 0.7202 (+0.2195) | 0.6063 (+0.1509) |
453
+ | 0.3556 | 22000 | 0.1982 | 0.1891 | 0.6636 (+0.1231) | 0.3903 (+0.0652) | 0.6701 (+0.1694) | 0.5746 (+0.1193) |
454
+ | 0.3717 | 23000 | 0.2046 | 0.1991 | 0.6518 (+0.1114) | 0.4406 (+0.1156) | 0.7082 (+0.2075) | 0.6002 (+0.1448) |
455
+ | 0.3879 | 24000 | 0.2045 | 0.2121 | 0.6599 (+0.1195) | 0.3811 (+0.0560) | 0.6846 (+0.1839) | 0.5752 (+0.1198) |
456
+ | 0.4040 | 25000 | 0.1953 | 0.2050 | 0.6862 (+0.1458) | 0.3974 (+0.0724) | 0.7090 (+0.2084) | 0.5976 (+0.1422) |
457
+ | 0.4202 | 26000 | 0.2044 | 0.1930 | 0.6985 (+0.1580) | 0.4194 (+0.0944) | 0.6969 (+0.1962) | 0.6049 (+0.1495) |
458
+ | 0.4364 | 27000 | 0.2086 | 0.2190 | 0.6806 (+0.1401) | 0.4171 (+0.0921) | 0.6990 (+0.1984) | 0.5989 (+0.1435) |
459
+ | 0.4525 | 28000 | 0.2021 | 0.1756 | 0.6567 (+0.1163) | 0.4132 (+0.0881) | 0.6949 (+0.1943) | 0.5883 (+0.1329) |
460
+ | 0.4687 | 29000 | 0.1984 | 0.1873 | 0.6669 (+0.1265) | 0.4209 (+0.0959) | 0.6937 (+0.1930) | 0.5938 (+0.1385) |
461
+ | 0.4848 | 30000 | 0.1897 | 0.1893 | 0.6303 (+0.0898) | 0.3925 (+0.0674) | 0.7209 (+0.2202) | 0.5812 (+0.1258) |
462
+ | 0.5010 | 31000 | 0.1971 | 0.1738 | 0.6811 (+0.1407) | 0.3956 (+0.0705) | 0.6962 (+0.1955) | 0.5910 (+0.1356) |
463
+ | 0.5172 | 32000 | 0.1867 | 0.1682 | 0.6683 (+0.1279) | 0.3985 (+0.0734) | 0.6988 (+0.1982) | 0.5885 (+0.1331) |
464
+ | **0.5333** | **33000** | **0.1908** | **0.1686** | **0.6994 (+0.1590)** | **0.4143 (+0.0892)** | **0.7200 (+0.2193)** | **0.6112 (+0.1558)** |
465
+ | 0.5495 | 34000 | 0.1907 | 0.1696 | 0.6615 (+0.1211) | 0.4073 (+0.0822) | 0.7228 (+0.2222) | 0.5972 (+0.1418) |
466
+ | 0.5657 | 35000 | 0.1924 | 0.1709 | 0.6682 (+0.1277) | 0.3833 (+0.0583) | 0.7057 (+0.2050) | 0.5857 (+0.1304) |
467
+ | 0.5818 | 36000 | 0.1824 | 0.1722 | 0.6754 (+0.1350) | 0.3804 (+0.0553) | 0.6967 (+0.1960) | 0.5841 (+0.1288) |
468
+ | 0.5980 | 37000 | 0.1823 | 0.1648 | 0.6836 (+0.1432) | 0.4001 (+0.0751) | 0.7194 (+0.2188) | 0.6010 (+0.1457) |
469
+ | 0.6141 | 38000 | 0.1812 | 0.1824 | 0.7006 (+0.1602) | 0.3977 (+0.0726) | 0.7305 (+0.2298) | 0.6096 (+0.1542) |
470
+ | 0.6303 | 39000 | 0.1833 | 0.2108 | 0.6801 (+0.1397) | 0.3958 (+0.0707) | 0.7334 (+0.2328) | 0.6031 (+0.1477) |
471
+ | 0.6465 | 40000 | 0.1894 | 0.1922 | 0.6729 (+0.1325) | 0.3917 (+0.0667) | 0.7289 (+0.2282) | 0.5978 (+0.1425) |
472
+ | 0.6626 | 41000 | 0.1776 | 0.1924 | 0.6831 (+0.1426) | 0.4006 (+0.0755) | 0.7028 (+0.2022) | 0.5955 (+0.1401) |
473
+ | 0.6788 | 42000 | 0.1782 | 0.1827 | 0.6938 (+0.1534) | 0.4114 (+0.0863) | 0.7168 (+0.2162) | 0.6074 (+0.1520) |
474
+ | 0.6949 | 43000 | 0.1781 | 0.1807 | 0.6567 (+0.1163) | 0.3841 (+0.0591) | 0.6840 (+0.1833) | 0.5750 (+0.1196) |
475
+ | 0.7111 | 44000 | 0.1788 | 0.1774 | 0.6768 (+0.1364) | 0.3791 (+0.0540) | 0.7113 (+0.2106) | 0.5891 (+0.1337) |
476
+ | 0.7273 | 45000 | 0.1788 | 0.1733 | 0.6723 (+0.1318) | 0.4118 (+0.0867) | 0.6907 (+0.1901) | 0.5916 (+0.1362) |
477
+ | 0.7434 | 46000 | 0.1755 | 0.1733 | 0.6659 (+0.1255) | 0.3771 (+0.0520) | 0.6894 (+0.1887) | 0.5775 (+0.1221) |
478
+ | 0.7596 | 47000 | 0.1768 | 0.1680 | 0.6754 (+0.1350) | 0.3799 (+0.0549) | 0.7056 (+0.2049) | 0.5870 (+0.1316) |
479
+ | 0.7758 | 48000 | 0.1802 | 0.1714 | 0.6599 (+0.1194) | 0.3945 (+0.0694) | 0.7129 (+0.2123) | 0.5891 (+0.1337) |
480
+ | 0.7919 | 49000 | 0.1734 | 0.1776 | 0.6676 (+0.1272) | 0.3856 (+0.0606) | 0.6738 (+0.1732) | 0.5757 (+0.1203) |
481
+ | 0.8081 | 50000 | 0.1815 | 0.1718 | 0.6963 (+0.1559) | 0.3726 (+0.0475) | 0.7085 (+0.2078) | 0.5924 (+0.1371) |
482
+ | 0.8242 | 51000 | 0.1817 | 0.1634 | 0.6598 (+0.1193) | 0.3782 (+0.0532) | 0.7116 (+0.2110) | 0.5832 (+0.1278) |
483
+ | 0.8404 | 52000 | 0.171 | 0.1625 | 0.6656 (+0.1252) | 0.3962 (+0.0712) | 0.7207 (+0.2200) | 0.5942 (+0.1388) |
484
+ | 0.8566 | 53000 | 0.1706 | 0.1685 | 0.6678 (+0.1274) | 0.3916 (+0.0665) | 0.6894 (+0.1887) | 0.5829 (+0.1276) |
485
+ | 0.8727 | 54000 | 0.1761 | 0.1658 | 0.6606 (+0.1202) | 0.3947 (+0.0697) | 0.7075 (+0.2068) | 0.5876 (+0.1322) |
486
+ | 0.8889 | 55000 | 0.1788 | 0.1807 | 0.6686 (+0.1282) | 0.4136 (+0.0886) | 0.7070 (+0.2063) | 0.5964 (+0.1410) |
487
+ | 0.9051 | 56000 | 0.1756 | 0.1608 | 0.6815 (+0.1411) | 0.3716 (+0.0466) | 0.7229 (+0.2223) | 0.5920 (+0.1367) |
488
+ | 0.9212 | 57000 | 0.1699 | 0.1650 | 0.7170 (+0.1766) | 0.3921 (+0.0671) | 0.7228 (+0.2222) | 0.6106 (+0.1553) |
489
+ | 0.9374 | 58000 | 0.1702 | 0.1530 | 0.6728 (+0.1324) | 0.3823 (+0.0572) | 0.7181 (+0.2175) | 0.5911 (+0.1357) |
490
+ | 0.9535 | 59000 | 0.1594 | 0.1781 | 0.6997 (+0.1593) | 0.3841 (+0.0591) | 0.7000 (+0.1993) | 0.5946 (+0.1392) |
491
+ | 0.9697 | 60000 | 0.175 | 0.1599 | 0.6977 (+0.1573) | 0.3931 (+0.0680) | 0.7151 (+0.2145) | 0.6020 (+0.1466) |
492
+ | 0.9859 | 61000 | 0.1622 | 0.1697 | 0.6967 (+0.1563) | 0.3916 (+0.0665) | 0.7134 (+0.2127) | 0.6006 (+0.1452) |
493
+ | 1.0020 | 62000 | 0.1649 | 0.1829 | 0.6746 (+0.1342) | 0.3945 (+0.0695) | 0.6802 (+0.1795) | 0.5831 (+0.1277) |
494
+ | 1.0182 | 63000 | 0.1315 | 0.2055 | 0.6766 (+0.1362) | 0.4009 (+0.0759) | 0.6925 (+0.1919) | 0.5900 (+0.1346) |
495
+ | 1.0343 | 64000 | 0.1294 | 0.1927 | 0.6732 (+0.1327) | 0.3788 (+0.0537) | 0.6872 (+0.1866) | 0.5797 (+0.1243) |
496
+ | 1.0505 | 65000 | 0.1378 | 0.1855 | 0.6685 (+0.1281) | 0.3704 (+0.0453) | 0.7186 (+0.2179) | 0.5858 (+0.1305) |
497
+ | 1.0667 | 66000 | 0.1259 | 0.1942 | 0.6561 (+0.1156) | 0.3842 (+0.0591) | 0.7172 (+0.2165) | 0.5858 (+0.1304) |
498
+ | 1.0828 | 67000 | 0.1261 | 0.1980 | 0.6495 (+0.1091) | 0.3799 (+0.0549) | 0.7163 (+0.2156) | 0.5819 (+0.1265) |
499
+ | 1.0990 | 68000 | 0.1327 | 0.2081 | 0.6585 (+0.1181) | 0.3908 (+0.0657) | 0.7170 (+0.2164) | 0.5888 (+0.1334) |
500
+ | 1.1152 | 69000 | 0.1313 | 0.2056 | 0.6589 (+0.1184) | 0.3798 (+0.0548) | 0.7366 (+0.2359) | 0.5918 (+0.1364) |
501
+ | 1.1313 | 70000 | 0.1355 | 0.1886 | 0.6942 (+0.1537) | 0.3901 (+0.0651) | 0.7236 (+0.2230) | 0.6026 (+0.1473) |
502
+ | 1.1475 | 71000 | 0.1262 | 0.2172 | 0.6698 (+0.1294) | 0.3680 (+0.0430) | 0.7188 (+0.2181) | 0.5855 (+0.1302) |
503
+ | 1.1636 | 72000 | 0.1287 | 0.2012 | 0.6682 (+0.1278) | 0.3716 (+0.0466) | 0.7242 (+0.2235) | 0.5880 (+0.1326) |
504
+ | 1.1798 | 73000 | 0.1223 | 0.1959 | 0.6713 (+0.1309) | 0.3622 (+0.0371) | 0.7266 (+0.2259) | 0.5867 (+0.1313) |
505
+ | 1.1960 | 74000 | 0.1281 | 0.1956 | 0.6972 (+0.1568) | 0.3733 (+0.0482) | 0.7444 (+0.2437) | 0.6049 (+0.1496) |
506
+ | 1.2121 | 75000 | 0.1257 | 0.1753 | 0.6213 (+0.0808) | 0.3519 (+0.0269) | 0.7319 (+0.2313) | 0.5684 (+0.1130) |
507
+ | 1.2283 | 76000 | 0.1308 | 0.1695 | 0.6710 (+0.1305) | 0.3714 (+0.0464) | 0.7302 (+0.2295) | 0.5908 (+0.1355) |
508
+ | 1.2444 | 77000 | 0.1337 | 0.1809 | 0.6471 (+0.1066) | 0.3719 (+0.0469) | 0.7496 (+0.2489) | 0.5895 (+0.1342) |
509
+ | 1.2606 | 78000 | 0.1286 | 0.2105 | 0.6844 (+0.1440) | 0.3541 (+0.0291) | 0.6970 (+0.1964) | 0.5785 (+0.1231) |
510
+ | 1.2768 | 79000 | 0.132 | 0.1695 | 0.6933 (+0.1529) | 0.3440 (+0.0189) | 0.7140 (+0.2134) | 0.5838 (+0.1284) |
511
+ | 1.2929 | 80000 | 0.1332 | 0.1799 | 0.6935 (+0.1530) | 0.3584 (+0.0334) | 0.7163 (+0.2157) | 0.5894 (+0.1340) |
512
+ | 1.3091 | 81000 | 0.1325 | 0.1919 | 0.6586 (+0.1182) | 0.3504 (+0.0253) | 0.7213 (+0.2207) | 0.5768 (+0.1214) |
513
+ | 1.3253 | 82000 | 0.1297 | 0.1633 | 0.6573 (+0.1169) | 0.3528 (+0.0278) | 0.7064 (+0.2057) | 0.5722 (+0.1168) |
514
+ | 1.3414 | 83000 | 0.1291 | 0.1800 | 0.6918 (+0.1514) | 0.3690 (+0.0440) | 0.6908 (+0.1902) | 0.5839 (+0.1285) |
515
+ | 1.3576 | 84000 | 0.126 | 0.1758 | 0.6393 (+0.0989) | 0.3710 (+0.0460) | 0.7475 (+0.2468) | 0.5860 (+0.1306) |
516
+ | 1.3737 | 85000 | 0.1199 | 0.1790 | 0.6343 (+0.0939) | 0.3662 (+0.0412) | 0.7255 (+0.2248) | 0.5753 (+0.1200) |
517
+ | 1.3899 | 86000 | 0.1318 | 0.1765 | 0.6880 (+0.1475) | 0.3577 (+0.0327) | 0.7199 (+0.2192) | 0.5885 (+0.1332) |
518
+ | 1.4061 | 87000 | 0.1287 | 0.1701 | 0.6375 (+0.0971) | 0.3680 (+0.0430) | 0.6969 (+0.1962) | 0.5675 (+0.1121) |
519
+ | 1.4222 | 88000 | 0.1285 | 0.1778 | 0.6375 (+0.0971) | 0.3565 (+0.0315) | 0.7073 (+0.2067) | 0.5671 (+0.1117) |
520
+ | 1.4384 | 89000 | 0.1218 | 0.1738 | 0.6486 (+0.1082) | 0.3621 (+0.0370) | 0.6965 (+0.1958) | 0.5690 (+0.1137) |
521
+ | 1.4545 | 90000 | 0.1198 | 0.1975 | 0.6675 (+0.1271) | 0.3696 (+0.0445) | 0.7042 (+0.2035) | 0.5804 (+0.1250) |
522
+ | 1.4707 | 91000 | 0.1303 | 0.1824 | 0.6493 (+0.1089) | 0.3760 (+0.0510) | 0.7107 (+0.2100) | 0.5787 (+0.1233) |
523
+ | 1.4869 | 92000 | 0.1332 | 0.1830 | 0.6515 (+0.1111) | 0.3890 (+0.0639) | 0.7356 (+0.2350) | 0.5920 (+0.1367) |
524
+ | 1.5030 | 93000 | 0.1232 | 0.1876 | 0.6860 (+0.1455) | 0.3531 (+0.0280) | 0.7223 (+0.2216) | 0.5871 (+0.1317) |
525
+ | 1.5192 | 94000 | 0.1305 | 0.1737 | 0.6526 (+0.1122) | 0.3638 (+0.0387) | 0.6858 (+0.1852) | 0.5674 (+0.1120) |
526
+ | 1.5354 | 95000 | 0.1266 | 0.1949 | 0.6643 (+0.1239) | 0.3820 (+0.0569) | 0.7024 (+0.2018) | 0.5829 (+0.1275) |
527
+ | 1.5515 | 96000 | 0.1277 | 0.1750 | 0.6556 (+0.1152) | 0.3523 (+0.0272) | 0.7095 (+0.2088) | 0.5725 (+0.1171) |
528
+ | 1.5677 | 97000 | 0.1303 | 0.1791 | 0.6714 (+0.1310) | 0.3846 (+0.0595) | 0.7299 (+0.2293) | 0.5953 (+0.1399) |
529
+ | 1.5838 | 98000 | 0.1242 | 0.1712 | 0.6802 (+0.1397) | 0.3667 (+0.0416) | 0.6927 (+0.1921) | 0.5798 (+0.1245) |
530
+ | 1.6 | 99000 | 0.117 | 0.1832 | 0.6672 (+0.1267) | 0.3755 (+0.0505) | 0.6992 (+0.1986) | 0.5806 (+0.1253) |
531
+ | 1.6162 | 100000 | 0.1343 | 0.1756 | 0.6694 (+0.1290) | 0.3808 (+0.0558) | 0.7187 (+0.2180) | 0.5896 (+0.1343) |
532
+ | 1.6323 | 101000 | 0.1299 | 0.1576 | 0.6574 (+0.1170) | 0.3975 (+0.0725) | 0.7343 (+0.2336) | 0.5964 (+0.1410) |
533
+ | 1.6485 | 102000 | 0.1236 | 0.1809 | 0.6572 (+0.1167) | 0.4059 (+0.0808) | 0.7055 (+0.2049) | 0.5895 (+0.1342) |
534
+ | 1.6646 | 103000 | 0.1178 | 0.1752 | 0.6691 (+0.1287) | 0.3793 (+0.0542) | 0.6799 (+0.1792) | 0.5761 (+0.1207) |
535
+ | 1.6808 | 104000 | 0.1224 | 0.1616 | 0.6552 (+0.1148) | 0.3849 (+0.0599) | 0.6964 (+0.1957) | 0.5788 (+0.1235) |
536
+ | 1.6970 | 105000 | 0.128 | 0.1675 | 0.6605 (+0.1200) | 0.3935 (+0.0685) | 0.6985 (+0.1979) | 0.5842 (+0.1288) |
537
+ | 1.7131 | 106000 | 0.1251 | 0.1816 | 0.6400 (+0.0996) | 0.3626 (+0.0375) | 0.7083 (+0.2077) | 0.5703 (+0.1149) |
538
+ | 1.7293 | 107000 | 0.1237 | 0.1787 | 0.6630 (+0.1226) | 0.3667 (+0.0416) | 0.7172 (+0.2166) | 0.5823 (+0.1269) |
539
+ | 1.7455 | 108000 | 0.1253 | 0.1750 | 0.6738 (+0.1334) | 0.3604 (+0.0353) | 0.7365 (+0.2359) | 0.5902 (+0.1349) |
540
+ | 1.7616 | 109000 | 0.1194 | 0.1670 | 0.6693 (+0.1289) | 0.3840 (+0.0589) | 0.7042 (+0.2035) | 0.5858 (+0.1305) |
541
+ | 1.7778 | 110000 | 0.1179 | 0.1693 | 0.6791 (+0.1386) | 0.3797 (+0.0546) | 0.7206 (+0.2200) | 0.5931 (+0.1377) |
542
+ | 1.7939 | 111000 | 0.1169 | 0.1843 | 0.6485 (+0.1081) | 0.3857 (+0.0606) | 0.6966 (+0.1959) | 0.5769 (+0.1215) |
543
+ | 1.8101 | 112000 | 0.1145 | 0.1738 | 0.6808 (+0.1404) | 0.4073 (+0.0822) | 0.7073 (+0.2067) | 0.5985 (+0.1431) |
544
+ | 1.8263 | 113000 | 0.1208 | 0.1660 | 0.6568 (+0.1163) | 0.3836 (+0.0586) | 0.7119 (+0.2113) | 0.5841 (+0.1287) |
545
+ | 1.8424 | 114000 | 0.1238 | 0.1635 | 0.6558 (+0.1154) | 0.3865 (+0.0615) | 0.7004 (+0.1997) | 0.5809 (+0.1255) |
546
+ | 1.8586 | 115000 | 0.1184 | 0.1622 | 0.6629 (+0.1225) | 0.3818 (+0.0568) | 0.6845 (+0.1838) | 0.5764 (+0.1210) |
547
+ | 1.8747 | 116000 | 0.1178 | 0.1668 | 0.6836 (+0.1432) | 0.3722 (+0.0472) | 0.6920 (+0.1914) | 0.5826 (+0.1272) |
548
+ | 1.8909 | 117000 | 0.1231 | 0.1602 | 0.6597 (+0.1193) | 0.3742 (+0.0491) | 0.6989 (+0.1982) | 0.5776 (+0.1222) |
549
+ | 1.9071 | 118000 | 0.1186 | 0.1685 | 0.6930 (+0.1526) | 0.3564 (+0.0314) | 0.7211 (+0.2204) | 0.5902 (+0.1348) |
550
+ | 1.9232 | 119000 | 0.1308 | 0.1559 | 0.6775 (+0.1370) | 0.3650 (+0.0400) | 0.7130 (+0.2123) | 0.5851 (+0.1298) |
551
+ | 1.9394 | 120000 | 0.1149 | 0.1746 | 0.6697 (+0.1293) | 0.3600 (+0.0350) | 0.7052 (+0.2046) | 0.5783 (+0.1229) |
552
+ | 1.9556 | 121000 | 0.121 | 0.1746 | 0.7010 (+0.1605) | 0.3552 (+0.0301) | 0.7425 (+0.2418) | 0.5995 (+0.1442) |
553
+ | 1.9717 | 122000 | 0.1233 | 0.1747 | 0.6423 (+0.1019) | 0.3614 (+0.0363) | 0.7171 (+0.2164) | 0.5736 (+0.1182) |
554
+ | 1.9879 | 123000 | 0.1213 | 0.1623 | 0.6751 (+0.1347) | 0.3660 (+0.0410) | 0.7177 (+0.2170) | 0.5863 (+0.1309) |
555
+ | 2.0040 | 124000 | 0.1033 | 0.2043 | 0.6870 (+0.1466) | 0.3696 (+0.0445) | 0.7150 (+0.2144) | 0.5905 (+0.1352) |
556
+ | 2.0202 | 125000 | 0.0581 | 0.2077 | 0.6855 (+0.1450) | 0.3623 (+0.0373) | 0.7018 (+0.2011) | 0.5832 (+0.1278) |
557
+ | 2.0364 | 126000 | 0.0596 | 0.2153 | 0.6705 (+0.1301) | 0.3622 (+0.0371) | 0.6990 (+0.1984) | 0.5772 (+0.1219) |
558
+ | 2.0525 | 127000 | 0.0657 | 0.2094 | 0.6472 (+0.1068) | 0.3608 (+0.0357) | 0.6908 (+0.1901) | 0.5662 (+0.1109) |
559
+ | 2.0687 | 128000 | 0.0654 | 0.2032 | 0.6292 (+0.0888) | 0.3619 (+0.0368) | 0.6823 (+0.1816) | 0.5578 (+0.1024) |
560
+ | 2.0848 | 129000 | 0.0641 | 0.2047 | 0.6441 (+0.1036) | 0.3455 (+0.0204) | 0.6656 (+0.1650) | 0.5517 (+0.0963) |
561
+ | 2.1010 | 130000 | 0.0649 | 0.2266 | 0.6336 (+0.0932) | 0.3537 (+0.0286) | 0.7088 (+0.2082) | 0.5654 (+0.1100) |
562
+ | 2.1172 | 131000 | 0.0652 | 0.2230 | 0.6732 (+0.1327) | 0.3503 (+0.0252) | 0.7125 (+0.2118) | 0.5786 (+0.1233) |
563
+ | 2.1333 | 132000 | 0.0667 | 0.2212 | 0.6691 (+0.1286) | 0.3784 (+0.0534) | 0.7351 (+0.2344) | 0.5942 (+0.1388) |
564
+ | 2.1495 | 133000 | 0.0621 | 0.2197 | 0.6287 (+0.0883) | 0.3677 (+0.0427) | 0.7299 (+0.2293) | 0.5754 (+0.1201) |
565
+ | 2.1657 | 134000 | 0.064 | 0.2114 | 0.6214 (+0.0810) | 0.3789 (+0.0539) | 0.7025 (+0.2018) | 0.5676 (+0.1122) |
566
+ | 2.1818 | 135000 | 0.0643 | 0.2179 | 0.6294 (+0.0889) | 0.3802 (+0.0551) | 0.6886 (+0.1880) | 0.5661 (+0.1107) |
567
+ | 2.1980 | 136000 | 0.0646 | 0.2182 | 0.6115 (+0.0711) | 0.3906 (+0.0656) | 0.6789 (+0.1782) | 0.5604 (+0.1050) |
568
+ | 2.2141 | 137000 | 0.0658 | 0.2111 | 0.6142 (+0.0738) | 0.3764 (+0.0513) | 0.6759 (+0.1753) | 0.5555 (+0.1001) |
569
+ | 2.2303 | 138000 | 0.0689 | 0.2199 | 0.6312 (+0.0908) | 0.3888 (+0.0637) | 0.6798 (+0.1792) | 0.5666 (+0.1112) |
570
+ | 2.2465 | 139000 | 0.0678 | 0.1933 | 0.6024 (+0.0620) | 0.3862 (+0.0611) | 0.6701 (+0.1695) | 0.5529 (+0.0975) |
571
+ | 2.2626 | 140000 | 0.0574 | 0.2291 | 0.6235 (+0.0831) | 0.3767 (+0.0517) | 0.6815 (+0.1808) | 0.5606 (+0.1052) |
572
+ | 2.2788 | 141000 | 0.0601 | 0.2326 | 0.6066 (+0.0662) | 0.3779 (+0.0528) | 0.6758 (+0.1752) | 0.5534 (+0.0980) |
573
+ | 2.2949 | 142000 | 0.0684 | 0.2350 | 0.5871 (+0.0467) | 0.3811 (+0.0561) | 0.6886 (+0.1880) | 0.5523 (+0.0969) |
574
+ | 2.3111 | 143000 | 0.0725 | 0.2061 | 0.5982 (+0.0578) | 0.3831 (+0.0581) | 0.6947 (+0.1941) | 0.5587 (+0.1033) |
575
+ | 2.3273 | 144000 | 0.0618 | 0.2182 | 0.5728 (+0.0323) | 0.3759 (+0.0508) | 0.6799 (+0.1792) | 0.5428 (+0.0875) |
576
+ | 2.3434 | 145000 | 0.0637 | 0.2103 | 0.6153 (+0.0749) | 0.3753 (+0.0502) | 0.7090 (+0.2084) | 0.5666 (+0.1112) |
577
+ | 2.3596 | 146000 | 0.0649 | 0.2253 | 0.5799 (+0.0394) | 0.3762 (+0.0512) | 0.6841 (+0.1835) | 0.5467 (+0.0914) |
578
+ | 2.3758 | 147000 | 0.0701 | 0.2109 | 0.5991 (+0.0587) | 0.3601 (+0.0350) | 0.6712 (+0.1706) | 0.5435 (+0.0881) |
579
+ | 2.3919 | 148000 | 0.0606 | 0.2276 | 0.6070 (+0.0666) | 0.3618 (+0.0368) | 0.6445 (+0.1438) | 0.5378 (+0.0824) |
580
+ | 2.4081 | 149000 | 0.0594 | 0.2216 | 0.5898 (+0.0493) | 0.3714 (+0.0464) | 0.6759 (+0.1752) | 0.5457 (+0.0903) |
581
+ | 2.4242 | 150000 | 0.062 | 0.2190 | 0.6299 (+0.0894) | 0.3600 (+0.0350) | 0.6959 (+0.1953) | 0.5619 (+0.1066) |
582
+ | 2.4404 | 151000 | 0.06 | 0.2158 | 0.6214 (+0.0810) | 0.3836 (+0.0585) | 0.6661 (+0.1654) | 0.5570 (+0.1017) |
583
+ | 2.4566 | 152000 | 0.0642 | 0.2122 | 0.6063 (+0.0659) | 0.3846 (+0.0595) | 0.6770 (+0.1764) | 0.5560 (+0.1006) |
584
+ | 2.4727 | 153000 | 0.0624 | 0.2263 | 0.6179 (+0.0775) | 0.3579 (+0.0328) | 0.6561 (+0.1554) | 0.5440 (+0.0886) |
585
+ | 2.4889 | 154000 | 0.0655 | 0.2343 | 0.6263 (+0.0858) | 0.3802 (+0.0552) | 0.6726 (+0.1720) | 0.5597 (+0.1043) |
586
+ | 2.5051 | 155000 | 0.0618 | 0.2169 | 0.6074 (+0.0670) | 0.3540 (+0.0290) | 0.7061 (+0.2055) | 0.5559 (+0.1005) |
587
+ | 2.5212 | 156000 | 0.0615 | 0.2205 | 0.6423 (+0.1019) | 0.3535 (+0.0285) | 0.6761 (+0.1755) | 0.5573 (+0.1019) |
588
+ | 2.5374 | 157000 | 0.063 | 0.2090 | 0.6079 (+0.0674) | 0.3581 (+0.0330) | 0.6628 (+0.1622) | 0.5429 (+0.0875) |
589
+ | 2.5535 | 158000 | 0.0623 | 0.2165 | 0.6105 (+0.0701) | 0.3642 (+0.0391) | 0.6704 (+0.1697) | 0.5484 (+0.0930) |
590
+ | 2.5697 | 159000 | 0.0605 | 0.2310 | 0.6332 (+0.0928) | 0.3578 (+0.0328) | 0.6811 (+0.1804) | 0.5574 (+0.1020) |
591
+ | 2.5859 | 160000 | 0.0648 | 0.2284 | 0.6276 (+0.0872) | 0.3621 (+0.0370) | 0.6808 (+0.1802) | 0.5568 (+0.1015) |
592
+ | 2.6020 | 161000 | 0.0634 | 0.2171 | 0.6349 (+0.0945) | 0.3615 (+0.0364) | 0.6858 (+0.1851) | 0.5607 (+0.1053) |
593
+ | 2.6182 | 162000 | 0.063 | 0.2160 | 0.6227 (+0.0823) | 0.3569 (+0.0319) | 0.6810 (+0.1804) | 0.5535 (+0.0982) |
594
+ | 2.6343 | 163000 | 0.0691 | 0.2129 | 0.6242 (+0.0838) | 0.3657 (+0.0406) | 0.6679 (+0.1672) | 0.5526 (+0.0972) |
595
+ | 2.6505 | 164000 | 0.0601 | 0.2197 | 0.6206 (+0.0801) | 0.3495 (+0.0244) | 0.6777 (+0.1771) | 0.5492 (+0.0939) |
596
+ | 2.6667 | 165000 | 0.0591 | 0.2285 | 0.6202 (+0.0798) | 0.3518 (+0.0268) | 0.6838 (+0.1831) | 0.5519 (+0.0966) |
597
+ | 2.6828 | 166000 | 0.0713 | 0.2150 | 0.6220 (+0.0815) | 0.3461 (+0.0211) | 0.6729 (+0.1723) | 0.5470 (+0.0916) |
598
+ | 2.6990 | 167000 | 0.0643 | 0.2201 | 0.6382 (+0.0978) | 0.3471 (+0.0221) | 0.6941 (+0.1935) | 0.5598 (+0.1045) |
599
+ | 2.7152 | 168000 | 0.06 | 0.2270 | 0.6371 (+0.0967) | 0.3626 (+0.0376) | 0.6774 (+0.1768) | 0.5590 (+0.1037) |
600
+ | 2.7313 | 169000 | 0.0627 | 0.2273 | 0.6033 (+0.0628) | 0.3478 (+0.0228) | 0.6815 (+0.1809) | 0.5442 (+0.0888) |
601
+ | 2.7475 | 170000 | 0.0623 | 0.2228 | 0.6128 (+0.0724) | 0.3551 (+0.0301) | 0.6769 (+0.1763) | 0.5483 (+0.0929) |
602
+ | 2.7636 | 171000 | 0.0666 | 0.2230 | 0.6051 (+0.0646) | 0.3438 (+0.0188) | 0.6675 (+0.1668) | 0.5388 (+0.0834) |
603
+ | 2.7798 | 172000 | 0.0584 | 0.2169 | 0.6340 (+0.0935) | 0.3562 (+0.0311) | 0.6592 (+0.1585) | 0.5498 (+0.0944) |
604
+ | 2.7960 | 173000 | 0.0653 | 0.2304 | 0.6129 (+0.0725) | 0.3552 (+0.0301) | 0.6386 (+0.1379) | 0.5356 (+0.0802) |
605
+ | 2.8121 | 174000 | 0.0643 | 0.2300 | 0.6020 (+0.0616) | 0.3655 (+0.0404) | 0.6570 (+0.1563) | 0.5415 (+0.0861) |
606
+ | 2.8283 | 175000 | 0.0644 | 0.2349 | 0.5966 (+0.0562) | 0.3627 (+0.0377) | 0.6338 (+0.1332) | 0.5311 (+0.0757) |
607
+ | 2.8444 | 176000 | 0.0631 | 0.2109 | 0.6282 (+0.0878) | 0.3681 (+0.0431) | 0.6587 (+0.1580) | 0.5517 (+0.0963) |
608
+ | 2.8606 | 177000 | 0.0621 | 0.2055 | 0.6093 (+0.0688) | 0.3769 (+0.0519) | 0.6570 (+0.1563) | 0.5477 (+0.0924) |
609
+ | 2.8768 | 178000 | 0.0669 | 0.2091 | 0.6465 (+0.1061) | 0.3656 (+0.0406) | 0.6568 (+0.1562) | 0.5563 (+0.1009) |
610
+ | 2.8929 | 179000 | 0.0604 | 0.2175 | 0.6075 (+0.0671) | 0.3573 (+0.0323) | 0.6504 (+0.1498) | 0.5384 (+0.0831) |
611
+ | 2.9091 | 180000 | 0.0631 | 0.2237 | 0.6071 (+0.0667) | 0.3631 (+0.0381) | 0.6537 (+0.1530) | 0.5413 (+0.0859) |
612
+ | 2.9253 | 181000 | 0.0636 | 0.2130 | 0.6445 (+0.1041) | 0.3487 (+0.0236) | 0.6615 (+0.1608) | 0.5516 (+0.0962) |
613
+ | 2.9414 | 182000 | 0.0622 | 0.2130 | 0.6210 (+0.0806) | 0.3541 (+0.0291) | 0.6636 (+0.1629) | 0.5462 (+0.0909) |
614
+ | 2.9576 | 183000 | 0.0657 | 0.2088 | 0.6195 (+0.0791) | 0.3485 (+0.0234) | 0.6716 (+0.1710) | 0.5465 (+0.0912) |
615
+ | 2.9737 | 184000 | 0.0539 | 0.2289 | 0.6285 (+0.0881) | 0.3433 (+0.0183) | 0.6649 (+0.1643) | 0.5456 (+0.0902) |
616
+ | 2.9899 | 185000 | 0.0601 | 0.2201 | 0.6409 (+0.1005) | 0.3528 (+0.0278) | 0.6615 (+0.1608) | 0.5517 (+0.0964) |
617
+ | 3.0061 | 186000 | 0.0541 | 0.2443 | 0.6192 (+0.0788) | 0.3516 (+0.0266) | 0.6574 (+0.1568) | 0.5428 (+0.0874) |
618
+ | 3.0222 | 187000 | 0.0199 | 0.2788 | 0.6011 (+0.0607) | 0.3463 (+0.0213) | 0.6435 (+0.1428) | 0.5303 (+0.0749) |
619
+ | 3.0384 | 188000 | 0.0213 | 0.2885 | 0.5926 (+0.0521) | 0.3408 (+0.0157) | 0.6478 (+0.1471) | 0.5270 (+0.0717) |
620
+ | 3.0545 | 189000 | 0.0207 | 0.3149 | 0.6097 (+0.0693) | 0.3494 (+0.0244) | 0.6297 (+0.1290) | 0.5296 (+0.0742) |
621
+ | 3.0707 | 190000 | 0.0204 | 0.3161 | 0.6025 (+0.0621) | 0.3396 (+0.0146) | 0.6136 (+0.1129) | 0.5186 (+0.0632) |
622
+ | 3.0869 | 191000 | 0.0192 | 0.3174 | 0.6023 (+0.0618) | 0.3481 (+0.0230) | 0.6316 (+0.1309) | 0.5273 (+0.0719) |
623
+ | 3.1030 | 192000 | 0.0225 | 0.2876 | 0.5816 (+0.0412) | 0.3530 (+0.0280) | 0.6044 (+0.1038) | 0.5130 (+0.0577) |
624
+ | 3.1192 | 193000 | 0.0182 | 0.2983 | 0.5972 (+0.0568) | 0.3341 (+0.0091) | 0.6309 (+0.1303) | 0.5208 (+0.0654) |
625
+ | 3.1354 | 194000 | 0.02 | 0.3006 | 0.6241 (+0.0836) | 0.3423 (+0.0172) | 0.6247 (+0.1241) | 0.5304 (+0.0750) |
626
+ | 3.1515 | 195000 | 0.0213 | 0.2805 | 0.6061 (+0.0656) | 0.3630 (+0.0380) | 0.6098 (+0.1091) | 0.5263 (+0.0709) |
627
+ | 3.1677 | 196000 | 0.0166 | 0.3318 | 0.5571 (+0.0167) | 0.3507 (+0.0257) | 0.6066 (+0.1060) | 0.5048 (+0.0494) |
628
+ | 3.1838 | 197000 | 0.0218 | 0.2959 | 0.5685 (+0.0280) | 0.3454 (+0.0203) | 0.5889 (+0.0883) | 0.5009 (+0.0455) |
629
+ | 3.2 | 198000 | 0.0203 | 0.3050 | 0.5784 (+0.0379) | 0.3533 (+0.0282) | 0.5979 (+0.0972) | 0.5098 (+0.0545) |
630
+ | 3.2162 | 199000 | 0.0211 | 0.3288 | 0.5967 (+0.0563) | 0.3538 (+0.0288) | 0.6098 (+0.1091) | 0.5201 (+0.0647) |
631
+ | 3.2323 | 200000 | 0.0219 | 0.3194 | 0.6186 (+0.0782) | 0.3399 (+0.0149) | 0.6343 (+0.1336) | 0.5309 (+0.0756) |
632
+ | 3.2485 | 201000 | 0.02 | 0.3244 | 0.5620 (+0.0216) | 0.3414 (+0.0163) | 0.5896 (+0.0889) | 0.4977 (+0.0423) |
633
+ | 3.2646 | 202000 | 0.0191 | 0.3081 | 0.6053 (+0.0648) | 0.3423 (+0.0173) | 0.6130 (+0.1124) | 0.5202 (+0.0648) |
634
+ | 3.2808 | 203000 | 0.0195 | 0.3090 | 0.5822 (+0.0418) | 0.3434 (+0.0183) | 0.6045 (+0.1038) | 0.5100 (+0.0547) |
635
+ | 3.2970 | 204000 | 0.0178 | 0.3366 | 0.6128 (+0.0724) | 0.3378 (+0.0128) | 0.6155 (+0.1148) | 0.5220 (+0.0667) |
636
+ | 3.3131 | 205000 | 0.0167 | 0.3068 | 0.5770 (+0.0365) | 0.3403 (+0.0153) | 0.6340 (+0.1334) | 0.5171 (+0.0617) |
637
+ | 3.3293 | 206000 | 0.0177 | 0.3284 | 0.5781 (+0.0377) | 0.3417 (+0.0167) | 0.6392 (+0.1386) | 0.5197 (+0.0643) |
638
+ | 3.3455 | 207000 | 0.0202 | 0.3067 | 0.5642 (+0.0238) | 0.3445 (+0.0194) | 0.6170 (+0.1164) | 0.5086 (+0.0532) |
639
+ | 3.3616 | 208000 | 0.0198 | 0.3222 | 0.5513 (+0.0109) | 0.3559 (+0.0308) | 0.6096 (+0.1090) | 0.5056 (+0.0502) |
640
+ | 3.3778 | 209000 | 0.0252 | 0.3080 | 0.5533 (+0.0128) | 0.3428 (+0.0178) | 0.6151 (+0.1144) | 0.5037 (+0.0483) |
641
+ | 3.3939 | 210000 | 0.0201 | 0.3329 | 0.5452 (+0.0048) | 0.3413 (+0.0163) | 0.6225 (+0.1219) | 0.5030 (+0.0477) |
642
+ | 3.4101 | 211000 | 0.0248 | 0.3061 | 0.5435 (+0.0031) | 0.3404 (+0.0154) | 0.6199 (+0.1193) | 0.5013 (+0.0459) |
643
+ | 3.4263 | 212000 | 0.0146 | 0.3362 | 0.5417 (+0.0013) | 0.3503 (+0.0253) | 0.6250 (+0.1244) | 0.5057 (+0.0503) |
644
+ | 3.4424 | 213000 | 0.0205 | 0.3205 | 0.5481 (+0.0077) | 0.3349 (+0.0099) | 0.5996 (+0.0989) | 0.4942 (+0.0388) |
645
+ | 3.4586 | 214000 | 0.0235 | 0.2910 | 0.5516 (+0.0112) | 0.3353 (+0.0103) | 0.6187 (+0.1180) | 0.5019 (+0.0465) |
646
+ | 3.4747 | 215000 | 0.0188 | 0.3055 | 0.5626 (+0.0221) | 0.3380 (+0.0129) | 0.6133 (+0.1127) | 0.5046 (+0.0492) |
647
+ | 3.4909 | 216000 | 0.0205 | 0.3015 | 0.5533 (+0.0128) | 0.3311 (+0.0061) | 0.6239 (+0.1232) | 0.5027 (+0.0474) |
648
+ | 3.5071 | 217000 | 0.0209 | 0.3043 | 0.5659 (+0.0255) | 0.3426 (+0.0175) | 0.6259 (+0.1253) | 0.5115 (+0.0561) |
649
+ | 3.5232 | 218000 | 0.0256 | 0.2940 | 0.5562 (+0.0158) | 0.3387 (+0.0137) | 0.6291 (+0.1285) | 0.5080 (+0.0526) |
650
+ | 3.5394 | 219000 | 0.0186 | 0.3211 | 0.5480 (+0.0076) | 0.3448 (+0.0198) | 0.6296 (+0.1289) | 0.5075 (+0.0521) |
651
+ | 3.5556 | 220000 | 0.0177 | 0.3184 | 0.5417 (+0.0013) | 0.3387 (+0.0137) | 0.6311 (+0.1305) | 0.5039 (+0.0485) |
652
+ | 3.5717 | 221000 | 0.0173 | 0.3247 | 0.5302 (-0.0103) | 0.3342 (+0.0092) | 0.6252 (+0.1245) | 0.4965 (+0.0411) |
653
+ | 3.5879 | 222000 | 0.0212 | 0.3252 | 0.5374 (-0.0030) | 0.3401 (+0.0151) | 0.6229 (+0.1222) | 0.5001 (+0.0448) |
654
+ | 3.6040 | 223000 | 0.0185 | 0.3113 | 0.5327 (-0.0077) | 0.3401 (+0.0151) | 0.6381 (+0.1374) | 0.5036 (+0.0483) |
655
+ | 3.6202 | 224000 | 0.0209 | 0.3209 | 0.5366 (-0.0038) | 0.3426 (+0.0176) | 0.6172 (+0.1166) | 0.4988 (+0.0434) |
656
+ | 3.6364 | 225000 | 0.0183 | 0.3158 | 0.5281 (-0.0123) | 0.3467 (+0.0217) | 0.6416 (+0.1410) | 0.5055 (+0.0501) |
657
+ | 3.6525 | 226000 | 0.0197 | 0.3195 | 0.5273 (-0.0131) | 0.3437 (+0.0187) | 0.6176 (+0.1170) | 0.4962 (+0.0409) |
658
+ | 3.6687 | 227000 | 0.0195 | 0.3172 | 0.5197 (-0.0207) | 0.3432 (+0.0181) | 0.6214 (+0.1207) | 0.4948 (+0.0394) |
659
+ | 3.6848 | 228000 | 0.0229 | 0.2948 | 0.5299 (-0.0105) | 0.3411 (+0.0161) | 0.6261 (+0.1255) | 0.4991 (+0.0437) |
660
+ | 3.7010 | 229000 | 0.0185 | 0.3077 | 0.5250 (-0.0154) | 0.3422 (+0.0171) | 0.6319 (+0.1313) | 0.4997 (+0.0443) |
661
+ | 3.7172 | 230000 | 0.0195 | 0.3281 | 0.5244 (-0.0160) | 0.3396 (+0.0145) | 0.6174 (+0.1168) | 0.4938 (+0.0384) |
662
+ | 3.7333 | 231000 | 0.019 | 0.3120 | 0.5182 (-0.0222) | 0.3438 (+0.0188) | 0.6419 (+0.1413) | 0.5013 (+0.0459) |
663
+ | 3.7495 | 232000 | 0.0164 | 0.3206 | 0.5158 (-0.0247) | 0.3397 (+0.0147) | 0.6312 (+0.1306) | 0.4956 (+0.0402) |
664
+ | 3.7657 | 233000 | 0.0191 | 0.3244 | 0.5132 (-0.0273) | 0.3362 (+0.0112) | 0.6312 (+0.1305) | 0.4935 (+0.0382) |
665
+ | 3.7818 | 234000 | 0.0138 | 0.3341 | 0.5121 (-0.0284) | 0.3393 (+0.0142) | 0.6343 (+0.1337) | 0.4952 (+0.0399) |
666
+ | 3.7980 | 235000 | 0.0205 | 0.3316 | 0.5099 (-0.0305) | 0.3436 (+0.0186) | 0.6377 (+0.1371) | 0.4971 (+0.0417) |
667
+ | 3.8141 | 236000 | 0.0164 | 0.3268 | 0.5131 (-0.0273) | 0.3438 (+0.0187) | 0.6327 (+0.1321) | 0.4965 (+0.0412) |
668
+ | 3.8303 | 237000 | 0.0227 | 0.3103 | 0.5171 (-0.0233) | 0.3407 (+0.0156) | 0.6308 (+0.1301) | 0.4962 (+0.0408) |
669
+ | 3.8465 | 238000 | 0.0211 | 0.3047 | 0.5146 (-0.0258) | 0.3432 (+0.0182) | 0.6306 (+0.1300) | 0.4961 (+0.0408) |
670
+ | 3.8626 | 239000 | 0.0147 | 0.3180 | 0.5088 (-0.0316) | 0.3410 (+0.0159) | 0.6294 (+0.1287) | 0.4931 (+0.0377) |
671
+ | 3.8788 | 240000 | 0.0187 | 0.3149 | 0.5085 (-0.0319) | 0.3410 (+0.0160) | 0.6295 (+0.1289) | 0.4930 (+0.0377) |
672
+ | 3.8949 | 241000 | 0.0182 | 0.3085 | 0.5201 (-0.0203) | 0.3425 (+0.0175) | 0.6300 (+0.1294) | 0.4975 (+0.0422) |
673
+ | 3.9111 | 242000 | 0.0157 | 0.3114 | 0.5205 (-0.0199) | 0.3446 (+0.0196) | 0.6297 (+0.1291) | 0.4983 (+0.0429) |
674
+ | 3.9273 | 243000 | 0.021 | 0.3062 | 0.5221 (-0.0183) | 0.3453 (+0.0203) | 0.6324 (+0.1317) | 0.4999 (+0.0446) |
675
+ | 3.9434 | 244000 | 0.0183 | 0.3046 | 0.5212 (-0.0192) | 0.3480 (+0.0230) | 0.6286 (+0.1280) | 0.4993 (+0.0439) |
676
+ | 3.9596 | 245000 | 0.0184 | 0.3046 | 0.5221 (-0.0183) | 0.3467 (+0.0217) | 0.6290 (+0.1283) | 0.4992 (+0.0439) |
677
+ | 3.9758 | 246000 | 0.0204 | 0.3019 | 0.5207 (-0.0197) | 0.3441 (+0.0191) | 0.6290 (+0.1283) | 0.4979 (+0.0426) |
678
+ | 3.9919 | 247000 | 0.0191 | 0.3023 | 0.5214 (-0.0190) | 0.3458 (+0.0207) | 0.6291 (+0.1285) | 0.4988 (+0.0434) |
679
+ | -1 | -1 | - | - | 0.6994 (+0.1590) | 0.4143 (+0.0892) | 0.7200 (+0.2193) | 0.6112 (+0.1558) |
680
+
681
+ * The bold row denotes the saved checkpoint.
682
+ </details>
683
+
684
+ ### Framework Versions
685
+ - Python: 3.10.18
686
+ - Sentence Transformers: 5.0.0
687
+ - Transformers: 4.56.0.dev0
688
+ - PyTorch: 2.7.1+cu126
689
+ - Accelerate: 1.9.0
690
+ - Datasets: 4.0.0
691
+ - Tokenizers: 0.21.4
692
+
693
+ ## Citation
694
+
695
+ ### BibTeX
696
+
697
+ #### Sentence Transformers
698
+ ```bibtex
699
+ @inproceedings{reimers-2019-sentence-bert,
700
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
701
+ author = "Reimers, Nils and Gurevych, Iryna",
702
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
703
+ month = "11",
704
+ year = "2019",
705
+ publisher = "Association for Computational Linguistics",
706
+ url = "https://arxiv.org/abs/1908.10084",
707
+ }
708
+ ```
709
+
710
+ <!--
711
+ ## Glossary
712
+
713
+ *Clearly define terms in order to be accessible across audiences.*
714
+ -->
715
+
716
+ <!--
717
+ ## Model Card Authors
718
+
719
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
720
+ -->
721
+
722
+ <!--
723
+ ## Model Card Contact
724
+
725
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
726
+ -->
config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ModernBertForSequenceClassification"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 50281,
8
+ "causal_mask": false,
9
+ "classifier_activation": "gelu",
10
+ "classifier_bias": false,
11
+ "classifier_dropout": 0.0,
12
+ "classifier_pooling": "mean",
13
+ "cls_token_id": 50281,
14
+ "decoder_bias": true,
15
+ "deterministic_flash_attn": false,
16
+ "embedding_dropout": 0.0,
17
+ "eos_token_id": 50282,
18
+ "global_attn_every_n_layers": 3,
19
+ "global_rope_theta": 160000.0,
20
+ "gradient_checkpointing": false,
21
+ "hidden_activation": "gelu",
22
+ "hidden_size": 512,
23
+ "id2label": {
24
+ "0": "LABEL_0"
25
+ },
26
+ "initializer_cutoff_factor": 2.0,
27
+ "initializer_range": 0.02,
28
+ "intermediate_size": 768,
29
+ "is_causal": false,
30
+ "label2id": {
31
+ "LABEL_0": 0
32
+ },
33
+ "layer_norm_eps": 1e-05,
34
+ "local_attention": 128,
35
+ "local_rope_theta": 160000.0,
36
+ "max_position_embeddings": 7999,
37
+ "mlp_bias": false,
38
+ "mlp_dropout": 0.0,
39
+ "model_type": "modernbert",
40
+ "norm_bias": false,
41
+ "norm_eps": 1e-05,
42
+ "num_attention_heads": 8,
43
+ "num_hidden_layers": 19,
44
+ "pad_token_id": 50283,
45
+ "position_embedding_type": "sans_pos",
46
+ "repad_logits_with_grad": false,
47
+ "sentence_transformers": {
48
+ "activation_fn": "torch.nn.modules.activation.Sigmoid",
49
+ "version": "5.0.0"
50
+ },
51
+ "sep_token_id": 50282,
52
+ "sparse_pred_ignore_index": -100,
53
+ "sparse_prediction": false,
54
+ "torch_dtype": "float32",
55
+ "transformers_version": "4.56.0.dev0",
56
+ "vocab_size": 50368
57
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79428213a6eab3301fc2774a6c57ff6c20a0048e3e28b68f00690e80791922d2
3
+ size 273643524
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": true,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,945 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "|||IP_ADDRESS|||",
5
+ "lstrip": false,
6
+ "normalized": true,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": false
10
+ },
11
+ "1": {
12
+ "content": "<|padding|>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "50254": {
20
+ "content": " ",
21
+ "lstrip": false,
22
+ "normalized": true,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": false
26
+ },
27
+ "50255": {
28
+ "content": " ",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": false
34
+ },
35
+ "50256": {
36
+ "content": " ",
37
+ "lstrip": false,
38
+ "normalized": true,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": false
42
+ },
43
+ "50257": {
44
+ "content": " ",
45
+ "lstrip": false,
46
+ "normalized": true,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": false
50
+ },
51
+ "50258": {
52
+ "content": " ",
53
+ "lstrip": false,
54
+ "normalized": true,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": false
58
+ },
59
+ "50259": {
60
+ "content": " ",
61
+ "lstrip": false,
62
+ "normalized": true,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": false
66
+ },
67
+ "50260": {
68
+ "content": " ",
69
+ "lstrip": false,
70
+ "normalized": true,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": false
74
+ },
75
+ "50261": {
76
+ "content": " ",
77
+ "lstrip": false,
78
+ "normalized": true,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": false
82
+ },
83
+ "50262": {
84
+ "content": " ",
85
+ "lstrip": false,
86
+ "normalized": true,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": false
90
+ },
91
+ "50263": {
92
+ "content": " ",
93
+ "lstrip": false,
94
+ "normalized": true,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": false
98
+ },
99
+ "50264": {
100
+ "content": " ",
101
+ "lstrip": false,
102
+ "normalized": true,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": false
106
+ },
107
+ "50265": {
108
+ "content": " ",
109
+ "lstrip": false,
110
+ "normalized": true,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": false
114
+ },
115
+ "50266": {
116
+ "content": " ",
117
+ "lstrip": false,
118
+ "normalized": true,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": false
122
+ },
123
+ "50267": {
124
+ "content": " ",
125
+ "lstrip": false,
126
+ "normalized": true,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": false
130
+ },
131
+ "50268": {
132
+ "content": " ",
133
+ "lstrip": false,
134
+ "normalized": true,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": false
138
+ },
139
+ "50269": {
140
+ "content": " ",
141
+ "lstrip": false,
142
+ "normalized": true,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": false
146
+ },
147
+ "50270": {
148
+ "content": " ",
149
+ "lstrip": false,
150
+ "normalized": true,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": false
154
+ },
155
+ "50271": {
156
+ "content": " ",
157
+ "lstrip": false,
158
+ "normalized": true,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": false
162
+ },
163
+ "50272": {
164
+ "content": " ",
165
+ "lstrip": false,
166
+ "normalized": true,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": false
170
+ },
171
+ "50273": {
172
+ "content": " ",
173
+ "lstrip": false,
174
+ "normalized": true,
175
+ "rstrip": false,
176
+ "single_word": false,
177
+ "special": false
178
+ },
179
+ "50274": {
180
+ "content": " ",
181
+ "lstrip": false,
182
+ "normalized": true,
183
+ "rstrip": false,
184
+ "single_word": false,
185
+ "special": false
186
+ },
187
+ "50275": {
188
+ "content": " ",
189
+ "lstrip": false,
190
+ "normalized": true,
191
+ "rstrip": false,
192
+ "single_word": false,
193
+ "special": false
194
+ },
195
+ "50276": {
196
+ "content": " ",
197
+ "lstrip": false,
198
+ "normalized": true,
199
+ "rstrip": false,
200
+ "single_word": false,
201
+ "special": false
202
+ },
203
+ "50277": {
204
+ "content": "|||EMAIL_ADDRESS|||",
205
+ "lstrip": false,
206
+ "normalized": true,
207
+ "rstrip": false,
208
+ "single_word": false,
209
+ "special": false
210
+ },
211
+ "50278": {
212
+ "content": "|||PHONE_NUMBER|||",
213
+ "lstrip": false,
214
+ "normalized": true,
215
+ "rstrip": false,
216
+ "single_word": false,
217
+ "special": false
218
+ },
219
+ "50279": {
220
+ "content": "<|endoftext|>",
221
+ "lstrip": false,
222
+ "normalized": false,
223
+ "rstrip": false,
224
+ "single_word": false,
225
+ "special": true
226
+ },
227
+ "50280": {
228
+ "content": "[UNK]",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false,
233
+ "special": true
234
+ },
235
+ "50281": {
236
+ "content": "[CLS]",
237
+ "lstrip": false,
238
+ "normalized": false,
239
+ "rstrip": false,
240
+ "single_word": false,
241
+ "special": true
242
+ },
243
+ "50282": {
244
+ "content": "[SEP]",
245
+ "lstrip": false,
246
+ "normalized": false,
247
+ "rstrip": false,
248
+ "single_word": false,
249
+ "special": true
250
+ },
251
+ "50283": {
252
+ "content": "[PAD]",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "50284": {
260
+ "content": "[MASK]",
261
+ "lstrip": true,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "50285": {
268
+ "content": "[unused0]",
269
+ "lstrip": false,
270
+ "normalized": true,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": false
274
+ },
275
+ "50286": {
276
+ "content": "[unused1]",
277
+ "lstrip": false,
278
+ "normalized": true,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": false
282
+ },
283
+ "50287": {
284
+ "content": "[unused2]",
285
+ "lstrip": false,
286
+ "normalized": true,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": false
290
+ },
291
+ "50288": {
292
+ "content": "[unused3]",
293
+ "lstrip": false,
294
+ "normalized": true,
295
+ "rstrip": false,
296
+ "single_word": false,
297
+ "special": false
298
+ },
299
+ "50289": {
300
+ "content": "[unused4]",
301
+ "lstrip": false,
302
+ "normalized": true,
303
+ "rstrip": false,
304
+ "single_word": false,
305
+ "special": false
306
+ },
307
+ "50290": {
308
+ "content": "[unused5]",
309
+ "lstrip": false,
310
+ "normalized": true,
311
+ "rstrip": false,
312
+ "single_word": false,
313
+ "special": false
314
+ },
315
+ "50291": {
316
+ "content": "[unused6]",
317
+ "lstrip": false,
318
+ "normalized": true,
319
+ "rstrip": false,
320
+ "single_word": false,
321
+ "special": false
322
+ },
323
+ "50292": {
324
+ "content": "[unused7]",
325
+ "lstrip": false,
326
+ "normalized": true,
327
+ "rstrip": false,
328
+ "single_word": false,
329
+ "special": false
330
+ },
331
+ "50293": {
332
+ "content": "[unused8]",
333
+ "lstrip": false,
334
+ "normalized": true,
335
+ "rstrip": false,
336
+ "single_word": false,
337
+ "special": false
338
+ },
339
+ "50294": {
340
+ "content": "[unused9]",
341
+ "lstrip": false,
342
+ "normalized": true,
343
+ "rstrip": false,
344
+ "single_word": false,
345
+ "special": false
346
+ },
347
+ "50295": {
348
+ "content": "[unused10]",
349
+ "lstrip": false,
350
+ "normalized": true,
351
+ "rstrip": false,
352
+ "single_word": false,
353
+ "special": false
354
+ },
355
+ "50296": {
356
+ "content": "[unused11]",
357
+ "lstrip": false,
358
+ "normalized": true,
359
+ "rstrip": false,
360
+ "single_word": false,
361
+ "special": false
362
+ },
363
+ "50297": {
364
+ "content": "[unused12]",
365
+ "lstrip": false,
366
+ "normalized": true,
367
+ "rstrip": false,
368
+ "single_word": false,
369
+ "special": false
370
+ },
371
+ "50298": {
372
+ "content": "[unused13]",
373
+ "lstrip": false,
374
+ "normalized": true,
375
+ "rstrip": false,
376
+ "single_word": false,
377
+ "special": false
378
+ },
379
+ "50299": {
380
+ "content": "[unused14]",
381
+ "lstrip": false,
382
+ "normalized": true,
383
+ "rstrip": false,
384
+ "single_word": false,
385
+ "special": false
386
+ },
387
+ "50300": {
388
+ "content": "[unused15]",
389
+ "lstrip": false,
390
+ "normalized": true,
391
+ "rstrip": false,
392
+ "single_word": false,
393
+ "special": false
394
+ },
395
+ "50301": {
396
+ "content": "[unused16]",
397
+ "lstrip": false,
398
+ "normalized": true,
399
+ "rstrip": false,
400
+ "single_word": false,
401
+ "special": false
402
+ },
403
+ "50302": {
404
+ "content": "[unused17]",
405
+ "lstrip": false,
406
+ "normalized": true,
407
+ "rstrip": false,
408
+ "single_word": false,
409
+ "special": false
410
+ },
411
+ "50303": {
412
+ "content": "[unused18]",
413
+ "lstrip": false,
414
+ "normalized": true,
415
+ "rstrip": false,
416
+ "single_word": false,
417
+ "special": false
418
+ },
419
+ "50304": {
420
+ "content": "[unused19]",
421
+ "lstrip": false,
422
+ "normalized": true,
423
+ "rstrip": false,
424
+ "single_word": false,
425
+ "special": false
426
+ },
427
+ "50305": {
428
+ "content": "[unused20]",
429
+ "lstrip": false,
430
+ "normalized": true,
431
+ "rstrip": false,
432
+ "single_word": false,
433
+ "special": false
434
+ },
435
+ "50306": {
436
+ "content": "[unused21]",
437
+ "lstrip": false,
438
+ "normalized": true,
439
+ "rstrip": false,
440
+ "single_word": false,
441
+ "special": false
442
+ },
443
+ "50307": {
444
+ "content": "[unused22]",
445
+ "lstrip": false,
446
+ "normalized": true,
447
+ "rstrip": false,
448
+ "single_word": false,
449
+ "special": false
450
+ },
451
+ "50308": {
452
+ "content": "[unused23]",
453
+ "lstrip": false,
454
+ "normalized": true,
455
+ "rstrip": false,
456
+ "single_word": false,
457
+ "special": false
458
+ },
459
+ "50309": {
460
+ "content": "[unused24]",
461
+ "lstrip": false,
462
+ "normalized": true,
463
+ "rstrip": false,
464
+ "single_word": false,
465
+ "special": false
466
+ },
467
+ "50310": {
468
+ "content": "[unused25]",
469
+ "lstrip": false,
470
+ "normalized": true,
471
+ "rstrip": false,
472
+ "single_word": false,
473
+ "special": false
474
+ },
475
+ "50311": {
476
+ "content": "[unused26]",
477
+ "lstrip": false,
478
+ "normalized": true,
479
+ "rstrip": false,
480
+ "single_word": false,
481
+ "special": false
482
+ },
483
+ "50312": {
484
+ "content": "[unused27]",
485
+ "lstrip": false,
486
+ "normalized": true,
487
+ "rstrip": false,
488
+ "single_word": false,
489
+ "special": false
490
+ },
491
+ "50313": {
492
+ "content": "[unused28]",
493
+ "lstrip": false,
494
+ "normalized": true,
495
+ "rstrip": false,
496
+ "single_word": false,
497
+ "special": false
498
+ },
499
+ "50314": {
500
+ "content": "[unused29]",
501
+ "lstrip": false,
502
+ "normalized": true,
503
+ "rstrip": false,
504
+ "single_word": false,
505
+ "special": false
506
+ },
507
+ "50315": {
508
+ "content": "[unused30]",
509
+ "lstrip": false,
510
+ "normalized": true,
511
+ "rstrip": false,
512
+ "single_word": false,
513
+ "special": false
514
+ },
515
+ "50316": {
516
+ "content": "[unused31]",
517
+ "lstrip": false,
518
+ "normalized": true,
519
+ "rstrip": false,
520
+ "single_word": false,
521
+ "special": false
522
+ },
523
+ "50317": {
524
+ "content": "[unused32]",
525
+ "lstrip": false,
526
+ "normalized": true,
527
+ "rstrip": false,
528
+ "single_word": false,
529
+ "special": false
530
+ },
531
+ "50318": {
532
+ "content": "[unused33]",
533
+ "lstrip": false,
534
+ "normalized": true,
535
+ "rstrip": false,
536
+ "single_word": false,
537
+ "special": false
538
+ },
539
+ "50319": {
540
+ "content": "[unused34]",
541
+ "lstrip": false,
542
+ "normalized": true,
543
+ "rstrip": false,
544
+ "single_word": false,
545
+ "special": false
546
+ },
547
+ "50320": {
548
+ "content": "[unused35]",
549
+ "lstrip": false,
550
+ "normalized": true,
551
+ "rstrip": false,
552
+ "single_word": false,
553
+ "special": false
554
+ },
555
+ "50321": {
556
+ "content": "[unused36]",
557
+ "lstrip": false,
558
+ "normalized": true,
559
+ "rstrip": false,
560
+ "single_word": false,
561
+ "special": false
562
+ },
563
+ "50322": {
564
+ "content": "[unused37]",
565
+ "lstrip": false,
566
+ "normalized": true,
567
+ "rstrip": false,
568
+ "single_word": false,
569
+ "special": false
570
+ },
571
+ "50323": {
572
+ "content": "[unused38]",
573
+ "lstrip": false,
574
+ "normalized": true,
575
+ "rstrip": false,
576
+ "single_word": false,
577
+ "special": false
578
+ },
579
+ "50324": {
580
+ "content": "[unused39]",
581
+ "lstrip": false,
582
+ "normalized": true,
583
+ "rstrip": false,
584
+ "single_word": false,
585
+ "special": false
586
+ },
587
+ "50325": {
588
+ "content": "[unused40]",
589
+ "lstrip": false,
590
+ "normalized": true,
591
+ "rstrip": false,
592
+ "single_word": false,
593
+ "special": false
594
+ },
595
+ "50326": {
596
+ "content": "[unused41]",
597
+ "lstrip": false,
598
+ "normalized": true,
599
+ "rstrip": false,
600
+ "single_word": false,
601
+ "special": false
602
+ },
603
+ "50327": {
604
+ "content": "[unused42]",
605
+ "lstrip": false,
606
+ "normalized": true,
607
+ "rstrip": false,
608
+ "single_word": false,
609
+ "special": false
610
+ },
611
+ "50328": {
612
+ "content": "[unused43]",
613
+ "lstrip": false,
614
+ "normalized": true,
615
+ "rstrip": false,
616
+ "single_word": false,
617
+ "special": false
618
+ },
619
+ "50329": {
620
+ "content": "[unused44]",
621
+ "lstrip": false,
622
+ "normalized": true,
623
+ "rstrip": false,
624
+ "single_word": false,
625
+ "special": false
626
+ },
627
+ "50330": {
628
+ "content": "[unused45]",
629
+ "lstrip": false,
630
+ "normalized": true,
631
+ "rstrip": false,
632
+ "single_word": false,
633
+ "special": false
634
+ },
635
+ "50331": {
636
+ "content": "[unused46]",
637
+ "lstrip": false,
638
+ "normalized": true,
639
+ "rstrip": false,
640
+ "single_word": false,
641
+ "special": false
642
+ },
643
+ "50332": {
644
+ "content": "[unused47]",
645
+ "lstrip": false,
646
+ "normalized": true,
647
+ "rstrip": false,
648
+ "single_word": false,
649
+ "special": false
650
+ },
651
+ "50333": {
652
+ "content": "[unused48]",
653
+ "lstrip": false,
654
+ "normalized": true,
655
+ "rstrip": false,
656
+ "single_word": false,
657
+ "special": false
658
+ },
659
+ "50334": {
660
+ "content": "[unused49]",
661
+ "lstrip": false,
662
+ "normalized": true,
663
+ "rstrip": false,
664
+ "single_word": false,
665
+ "special": false
666
+ },
667
+ "50335": {
668
+ "content": "[unused50]",
669
+ "lstrip": false,
670
+ "normalized": true,
671
+ "rstrip": false,
672
+ "single_word": false,
673
+ "special": false
674
+ },
675
+ "50336": {
676
+ "content": "[unused51]",
677
+ "lstrip": false,
678
+ "normalized": true,
679
+ "rstrip": false,
680
+ "single_word": false,
681
+ "special": false
682
+ },
683
+ "50337": {
684
+ "content": "[unused52]",
685
+ "lstrip": false,
686
+ "normalized": true,
687
+ "rstrip": false,
688
+ "single_word": false,
689
+ "special": false
690
+ },
691
+ "50338": {
692
+ "content": "[unused53]",
693
+ "lstrip": false,
694
+ "normalized": true,
695
+ "rstrip": false,
696
+ "single_word": false,
697
+ "special": false
698
+ },
699
+ "50339": {
700
+ "content": "[unused54]",
701
+ "lstrip": false,
702
+ "normalized": true,
703
+ "rstrip": false,
704
+ "single_word": false,
705
+ "special": false
706
+ },
707
+ "50340": {
708
+ "content": "[unused55]",
709
+ "lstrip": false,
710
+ "normalized": true,
711
+ "rstrip": false,
712
+ "single_word": false,
713
+ "special": false
714
+ },
715
+ "50341": {
716
+ "content": "[unused56]",
717
+ "lstrip": false,
718
+ "normalized": true,
719
+ "rstrip": false,
720
+ "single_word": false,
721
+ "special": false
722
+ },
723
+ "50342": {
724
+ "content": "[unused57]",
725
+ "lstrip": false,
726
+ "normalized": true,
727
+ "rstrip": false,
728
+ "single_word": false,
729
+ "special": false
730
+ },
731
+ "50343": {
732
+ "content": "[unused58]",
733
+ "lstrip": false,
734
+ "normalized": true,
735
+ "rstrip": false,
736
+ "single_word": false,
737
+ "special": false
738
+ },
739
+ "50344": {
740
+ "content": "[unused59]",
741
+ "lstrip": false,
742
+ "normalized": true,
743
+ "rstrip": false,
744
+ "single_word": false,
745
+ "special": false
746
+ },
747
+ "50345": {
748
+ "content": "[unused60]",
749
+ "lstrip": false,
750
+ "normalized": true,
751
+ "rstrip": false,
752
+ "single_word": false,
753
+ "special": false
754
+ },
755
+ "50346": {
756
+ "content": "[unused61]",
757
+ "lstrip": false,
758
+ "normalized": true,
759
+ "rstrip": false,
760
+ "single_word": false,
761
+ "special": false
762
+ },
763
+ "50347": {
764
+ "content": "[unused62]",
765
+ "lstrip": false,
766
+ "normalized": true,
767
+ "rstrip": false,
768
+ "single_word": false,
769
+ "special": false
770
+ },
771
+ "50348": {
772
+ "content": "[unused63]",
773
+ "lstrip": false,
774
+ "normalized": true,
775
+ "rstrip": false,
776
+ "single_word": false,
777
+ "special": false
778
+ },
779
+ "50349": {
780
+ "content": "[unused64]",
781
+ "lstrip": false,
782
+ "normalized": true,
783
+ "rstrip": false,
784
+ "single_word": false,
785
+ "special": false
786
+ },
787
+ "50350": {
788
+ "content": "[unused65]",
789
+ "lstrip": false,
790
+ "normalized": true,
791
+ "rstrip": false,
792
+ "single_word": false,
793
+ "special": false
794
+ },
795
+ "50351": {
796
+ "content": "[unused66]",
797
+ "lstrip": false,
798
+ "normalized": true,
799
+ "rstrip": false,
800
+ "single_word": false,
801
+ "special": false
802
+ },
803
+ "50352": {
804
+ "content": "[unused67]",
805
+ "lstrip": false,
806
+ "normalized": true,
807
+ "rstrip": false,
808
+ "single_word": false,
809
+ "special": false
810
+ },
811
+ "50353": {
812
+ "content": "[unused68]",
813
+ "lstrip": false,
814
+ "normalized": true,
815
+ "rstrip": false,
816
+ "single_word": false,
817
+ "special": false
818
+ },
819
+ "50354": {
820
+ "content": "[unused69]",
821
+ "lstrip": false,
822
+ "normalized": true,
823
+ "rstrip": false,
824
+ "single_word": false,
825
+ "special": false
826
+ },
827
+ "50355": {
828
+ "content": "[unused70]",
829
+ "lstrip": false,
830
+ "normalized": true,
831
+ "rstrip": false,
832
+ "single_word": false,
833
+ "special": false
834
+ },
835
+ "50356": {
836
+ "content": "[unused71]",
837
+ "lstrip": false,
838
+ "normalized": true,
839
+ "rstrip": false,
840
+ "single_word": false,
841
+ "special": false
842
+ },
843
+ "50357": {
844
+ "content": "[unused72]",
845
+ "lstrip": false,
846
+ "normalized": true,
847
+ "rstrip": false,
848
+ "single_word": false,
849
+ "special": false
850
+ },
851
+ "50358": {
852
+ "content": "[unused73]",
853
+ "lstrip": false,
854
+ "normalized": true,
855
+ "rstrip": false,
856
+ "single_word": false,
857
+ "special": false
858
+ },
859
+ "50359": {
860
+ "content": "[unused74]",
861
+ "lstrip": false,
862
+ "normalized": true,
863
+ "rstrip": false,
864
+ "single_word": false,
865
+ "special": false
866
+ },
867
+ "50360": {
868
+ "content": "[unused75]",
869
+ "lstrip": false,
870
+ "normalized": true,
871
+ "rstrip": false,
872
+ "single_word": false,
873
+ "special": false
874
+ },
875
+ "50361": {
876
+ "content": "[unused76]",
877
+ "lstrip": false,
878
+ "normalized": true,
879
+ "rstrip": false,
880
+ "single_word": false,
881
+ "special": false
882
+ },
883
+ "50362": {
884
+ "content": "[unused77]",
885
+ "lstrip": false,
886
+ "normalized": true,
887
+ "rstrip": false,
888
+ "single_word": false,
889
+ "special": false
890
+ },
891
+ "50363": {
892
+ "content": "[unused78]",
893
+ "lstrip": false,
894
+ "normalized": true,
895
+ "rstrip": false,
896
+ "single_word": false,
897
+ "special": false
898
+ },
899
+ "50364": {
900
+ "content": "[unused79]",
901
+ "lstrip": false,
902
+ "normalized": true,
903
+ "rstrip": false,
904
+ "single_word": false,
905
+ "special": false
906
+ },
907
+ "50365": {
908
+ "content": "[unused80]",
909
+ "lstrip": false,
910
+ "normalized": true,
911
+ "rstrip": false,
912
+ "single_word": false,
913
+ "special": false
914
+ },
915
+ "50366": {
916
+ "content": "[unused81]",
917
+ "lstrip": false,
918
+ "normalized": true,
919
+ "rstrip": false,
920
+ "single_word": false,
921
+ "special": false
922
+ },
923
+ "50367": {
924
+ "content": "[unused82]",
925
+ "lstrip": false,
926
+ "normalized": true,
927
+ "rstrip": false,
928
+ "single_word": false,
929
+ "special": false
930
+ }
931
+ },
932
+ "clean_up_tokenization_spaces": true,
933
+ "cls_token": "[CLS]",
934
+ "extra_special_tokens": {},
935
+ "mask_token": "[MASK]",
936
+ "model_input_names": [
937
+ "input_ids",
938
+ "attention_mask"
939
+ ],
940
+ "model_max_length": 7999,
941
+ "pad_token": "[PAD]",
942
+ "sep_token": "[SEP]",
943
+ "tokenizer_class": "PreTrainedTokenizerFast",
944
+ "unk_token": "[UNK]"
945
+ }