File size: 29,083 Bytes
e33d632
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
---
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: clinico-finetuned-augmented1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# clinico-finetuned-augmented1

This model is a fine-tuned version of [joheras/distilbert-base-spanish-uncased-finetuned-clinais](https://huggingface.co/joheras/distilbert-base-spanish-uncased-finetuned-clinais) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2935
- Precision: 0.3471
- Recall: 0.55
- F1: 0.4256
- Accuracy: 0.8319

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 300

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log        | 1.0   | 23   | 1.3471          | 0.0013    | 0.0022 | 0.0016 | 0.5548   |
| No log        | 2.0   | 46   | 1.0386          | 0.0120    | 0.0344 | 0.0179 | 0.6527   |
| No log        | 3.0   | 69   | 0.9088          | 0.0410    | 0.0933 | 0.0569 | 0.7125   |
| No log        | 4.0   | 92   | 0.8189          | 0.0604    | 0.1356 | 0.0835 | 0.7446   |
| No log        | 5.0   | 115  | 0.7484          | 0.0943    | 0.1844 | 0.1248 | 0.7704   |
| No log        | 6.0   | 138  | 0.7094          | 0.1132    | 0.2178 | 0.1489 | 0.7772   |
| No log        | 7.0   | 161  | 0.7022          | 0.1169    | 0.2211 | 0.1530 | 0.7800   |
| No log        | 8.0   | 184  | 0.6776          | 0.1575    | 0.2878 | 0.2036 | 0.7944   |
| No log        | 9.0   | 207  | 0.6983          | 0.1493    | 0.2911 | 0.1974 | 0.7964   |
| No log        | 10.0  | 230  | 0.6821          | 0.1772    | 0.33   | 0.2306 | 0.8021   |
| No log        | 11.0  | 253  | 0.6845          | 0.1861    | 0.3511 | 0.2433 | 0.8053   |
| No log        | 12.0  | 276  | 0.6874          | 0.2020    | 0.3756 | 0.2627 | 0.8106   |
| No log        | 13.0  | 299  | 0.6973          | 0.2098    | 0.3967 | 0.2744 | 0.8102   |
| No log        | 14.0  | 322  | 0.7230          | 0.2140    | 0.41   | 0.2812 | 0.8140   |
| No log        | 15.0  | 345  | 0.7448          | 0.2243    | 0.4244 | 0.2935 | 0.8100   |
| No log        | 16.0  | 368  | 0.7519          | 0.2378    | 0.4278 | 0.3057 | 0.8111   |
| No log        | 17.0  | 391  | 0.7503          | 0.2326    | 0.4378 | 0.3038 | 0.8119   |
| No log        | 18.0  | 414  | 0.7573          | 0.2494    | 0.4456 | 0.3198 | 0.8139   |
| No log        | 19.0  | 437  | 0.7970          | 0.2435    | 0.4544 | 0.3171 | 0.8137   |
| No log        | 20.0  | 460  | 0.7990          | 0.2523    | 0.4511 | 0.3236 | 0.8170   |
| No log        | 21.0  | 483  | 0.8115          | 0.2484    | 0.4633 | 0.3234 | 0.8139   |
| 0.4675        | 22.0  | 506  | 0.8135          | 0.2572    | 0.4667 | 0.3316 | 0.8157   |
| 0.4675        | 23.0  | 529  | 0.8509          | 0.2644    | 0.4656 | 0.3372 | 0.8108   |
| 0.4675        | 24.0  | 552  | 0.8450          | 0.2600    | 0.4789 | 0.3370 | 0.8143   |
| 0.4675        | 25.0  | 575  | 0.8667          | 0.2777    | 0.4811 | 0.3522 | 0.8220   |
| 0.4675        | 26.0  | 598  | 0.8639          | 0.2634    | 0.4767 | 0.3393 | 0.8152   |
| 0.4675        | 27.0  | 621  | 0.8903          | 0.2790    | 0.4811 | 0.3532 | 0.8154   |
| 0.4675        | 28.0  | 644  | 0.8697          | 0.2885    | 0.4978 | 0.3653 | 0.8212   |
| 0.4675        | 29.0  | 667  | 0.8856          | 0.2851    | 0.4856 | 0.3592 | 0.8197   |
| 0.4675        | 30.0  | 690  | 0.9332          | 0.2670    | 0.4933 | 0.3465 | 0.8141   |
| 0.4675        | 31.0  | 713  | 0.9064          | 0.2929    | 0.5078 | 0.3715 | 0.8201   |
| 0.4675        | 32.0  | 736  | 0.9256          | 0.2915    | 0.5089 | 0.3707 | 0.8222   |
| 0.4675        | 33.0  | 759  | 0.9447          | 0.2959    | 0.51   | 0.3745 | 0.8175   |
| 0.4675        | 34.0  | 782  | 0.9387          | 0.2998    | 0.5133 | 0.3785 | 0.8199   |
| 0.4675        | 35.0  | 805  | 0.9545          | 0.3124    | 0.52   | 0.3903 | 0.8225   |
| 0.4675        | 36.0  | 828  | 0.9785          | 0.2962    | 0.5056 | 0.3736 | 0.8179   |
| 0.4675        | 37.0  | 851  | 0.9635          | 0.3004    | 0.5133 | 0.3790 | 0.8201   |
| 0.4675        | 38.0  | 874  | 0.9749          | 0.2810    | 0.5011 | 0.3601 | 0.8229   |
| 0.4675        | 39.0  | 897  | 0.9753          | 0.3018    | 0.5289 | 0.3843 | 0.8201   |
| 0.4675        | 40.0  | 920  | 0.9735          | 0.3001    | 0.5122 | 0.3785 | 0.8221   |
| 0.4675        | 41.0  | 943  | 1.0196          | 0.3033    | 0.5078 | 0.3797 | 0.8179   |
| 0.4675        | 42.0  | 966  | 0.9775          | 0.3037    | 0.5267 | 0.3852 | 0.8254   |
| 0.4675        | 43.0  | 989  | 1.0051          | 0.3028    | 0.5133 | 0.3809 | 0.8221   |
| 0.0384        | 44.0  | 1012 | 1.0053          | 0.3067    | 0.5256 | 0.3874 | 0.8250   |
| 0.0384        | 45.0  | 1035 | 1.0145          | 0.3090    | 0.5356 | 0.3919 | 0.8225   |
| 0.0384        | 46.0  | 1058 | 1.0299          | 0.3081    | 0.5244 | 0.3882 | 0.8212   |
| 0.0384        | 47.0  | 1081 | 1.0265          | 0.3231    | 0.5256 | 0.4002 | 0.8239   |
| 0.0384        | 48.0  | 1104 | 1.0151          | 0.3154    | 0.5278 | 0.3948 | 0.8248   |
| 0.0384        | 49.0  | 1127 | 1.0384          | 0.3229    | 0.5378 | 0.4035 | 0.8234   |
| 0.0384        | 50.0  | 1150 | 1.0641          | 0.3159    | 0.5311 | 0.3962 | 0.8207   |
| 0.0384        | 51.0  | 1173 | 1.0592          | 0.3123    | 0.5233 | 0.3912 | 0.8218   |
| 0.0384        | 52.0  | 1196 | 1.0314          | 0.3131    | 0.5378 | 0.3957 | 0.8238   |
| 0.0384        | 53.0  | 1219 | 1.0466          | 0.3169    | 0.5344 | 0.3978 | 0.8235   |
| 0.0384        | 54.0  | 1242 | 1.0402          | 0.3005    | 0.5156 | 0.3797 | 0.8229   |
| 0.0384        | 55.0  | 1265 | 1.0384          | 0.3149    | 0.5311 | 0.3954 | 0.8239   |
| 0.0384        | 56.0  | 1288 | 1.0401          | 0.3282    | 0.5456 | 0.4098 | 0.8234   |
| 0.0384        | 57.0  | 1311 | 1.0584          | 0.3221    | 0.5322 | 0.4013 | 0.8232   |
| 0.0384        | 58.0  | 1334 | 1.0665          | 0.3308    | 0.5411 | 0.4106 | 0.8256   |
| 0.0384        | 59.0  | 1357 | 1.0774          | 0.3201    | 0.5267 | 0.3982 | 0.8228   |
| 0.0384        | 60.0  | 1380 | 1.0800          | 0.3144    | 0.5344 | 0.3959 | 0.8201   |
| 0.0384        | 61.0  | 1403 | 1.1017          | 0.3071    | 0.52   | 0.3861 | 0.8197   |
| 0.0384        | 62.0  | 1426 | 1.1059          | 0.3218    | 0.5289 | 0.4002 | 0.8192   |
| 0.0384        | 63.0  | 1449 | 1.0926          | 0.3199    | 0.5367 | 0.4008 | 0.8216   |
| 0.0384        | 64.0  | 1472 | 1.0825          | 0.3135    | 0.5267 | 0.3930 | 0.8262   |
| 0.0384        | 65.0  | 1495 | 1.1075          | 0.3190    | 0.5344 | 0.3995 | 0.8213   |
| 0.0109        | 66.0  | 1518 | 1.1220          | 0.3174    | 0.5233 | 0.3951 | 0.8201   |
| 0.0109        | 67.0  | 1541 | 1.1047          | 0.3233    | 0.5367 | 0.4035 | 0.8221   |
| 0.0109        | 68.0  | 1564 | 1.1506          | 0.3203    | 0.5367 | 0.4012 | 0.8226   |
| 0.0109        | 69.0  | 1587 | 1.1263          | 0.3188    | 0.5289 | 0.3978 | 0.8229   |
| 0.0109        | 70.0  | 1610 | 1.1126          | 0.3150    | 0.5344 | 0.3964 | 0.8217   |
| 0.0109        | 71.0  | 1633 | 1.1014          | 0.3210    | 0.5389 | 0.4023 | 0.8239   |
| 0.0109        | 72.0  | 1656 | 1.1620          | 0.3213    | 0.5444 | 0.4041 | 0.8206   |
| 0.0109        | 73.0  | 1679 | 1.1406          | 0.3400    | 0.55   | 0.4202 | 0.8229   |
| 0.0109        | 74.0  | 1702 | 1.1173          | 0.3276    | 0.5467 | 0.4097 | 0.8224   |
| 0.0109        | 75.0  | 1725 | 1.1185          | 0.3274    | 0.5533 | 0.4114 | 0.8256   |
| 0.0109        | 76.0  | 1748 | 1.1058          | 0.3215    | 0.5422 | 0.4036 | 0.8235   |
| 0.0109        | 77.0  | 1771 | 1.1212          | 0.3250    | 0.5356 | 0.4045 | 0.8269   |
| 0.0109        | 78.0  | 1794 | 1.1542          | 0.3185    | 0.5267 | 0.3970 | 0.8237   |
| 0.0109        | 79.0  | 1817 | 1.1348          | 0.3299    | 0.54   | 0.4096 | 0.8237   |
| 0.0109        | 80.0  | 1840 | 1.1355          | 0.3282    | 0.5411 | 0.4086 | 0.8244   |
| 0.0109        | 81.0  | 1863 | 1.1255          | 0.3172    | 0.5456 | 0.4011 | 0.8238   |
| 0.0109        | 82.0  | 1886 | 1.1328          | 0.3322    | 0.5467 | 0.4133 | 0.8232   |
| 0.0109        | 83.0  | 1909 | 1.1444          | 0.3359    | 0.54   | 0.4141 | 0.8232   |
| 0.0109        | 84.0  | 1932 | 1.1474          | 0.3395    | 0.5467 | 0.4189 | 0.8231   |
| 0.0109        | 85.0  | 1955 | 1.1526          | 0.3336    | 0.5411 | 0.4127 | 0.8225   |
| 0.0109        | 86.0  | 1978 | 1.1408          | 0.3327    | 0.5411 | 0.4120 | 0.8262   |
| 0.0055        | 87.0  | 2001 | 1.1414          | 0.3269    | 0.5433 | 0.4082 | 0.8232   |
| 0.0055        | 88.0  | 2024 | 1.1626          | 0.3354    | 0.54   | 0.4138 | 0.8230   |
| 0.0055        | 89.0  | 2047 | 1.1622          | 0.3394    | 0.5378 | 0.4162 | 0.8251   |
| 0.0055        | 90.0  | 2070 | 1.1423          | 0.3251    | 0.5244 | 0.4014 | 0.8264   |
| 0.0055        | 91.0  | 2093 | 1.1629          | 0.3290    | 0.5322 | 0.4066 | 0.8252   |
| 0.0055        | 92.0  | 2116 | 1.1478          | 0.3372    | 0.5544 | 0.4193 | 0.8238   |
| 0.0055        | 93.0  | 2139 | 1.1847          | 0.3280    | 0.5267 | 0.4043 | 0.8252   |
| 0.0055        | 94.0  | 2162 | 1.1801          | 0.3447    | 0.5411 | 0.4211 | 0.8204   |
| 0.0055        | 95.0  | 2185 | 1.1526          | 0.3297    | 0.5378 | 0.4088 | 0.8258   |
| 0.0055        | 96.0  | 2208 | 1.1786          | 0.3475    | 0.5433 | 0.4239 | 0.8230   |
| 0.0055        | 97.0  | 2231 | 1.1672          | 0.3347    | 0.5411 | 0.4136 | 0.8247   |
| 0.0055        | 98.0  | 2254 | 1.2031          | 0.3230    | 0.5422 | 0.4048 | 0.8214   |
| 0.0055        | 99.0  | 2277 | 1.1589          | 0.3347    | 0.5478 | 0.4155 | 0.8280   |
| 0.0055        | 100.0 | 2300 | 1.2049          | 0.3276    | 0.5533 | 0.4116 | 0.8214   |
| 0.0055        | 101.0 | 2323 | 1.1695          | 0.3304    | 0.5422 | 0.4106 | 0.8248   |
| 0.0055        | 102.0 | 2346 | 1.1933          | 0.3406    | 0.5522 | 0.4214 | 0.8218   |
| 0.0055        | 103.0 | 2369 | 1.1821          | 0.3351    | 0.5544 | 0.4177 | 0.8231   |
| 0.0055        | 104.0 | 2392 | 1.2068          | 0.3369    | 0.5311 | 0.4122 | 0.8207   |
| 0.0055        | 105.0 | 2415 | 1.1866          | 0.3384    | 0.5478 | 0.4183 | 0.8214   |
| 0.0055        | 106.0 | 2438 | 1.1917          | 0.3347    | 0.5544 | 0.4174 | 0.8243   |
| 0.0055        | 107.0 | 2461 | 1.2080          | 0.3304    | 0.5456 | 0.4116 | 0.8232   |
| 0.0055        | 108.0 | 2484 | 1.1625          | 0.3351    | 0.5544 | 0.4177 | 0.8258   |
| 0.0034        | 109.0 | 2507 | 1.2035          | 0.3251    | 0.5378 | 0.4052 | 0.8238   |
| 0.0034        | 110.0 | 2530 | 1.1610          | 0.3340    | 0.5456 | 0.4143 | 0.8276   |
| 0.0034        | 111.0 | 2553 | 1.1941          | 0.3522    | 0.5533 | 0.4304 | 0.8267   |
| 0.0034        | 112.0 | 2576 | 1.1784          | 0.3424    | 0.5467 | 0.4211 | 0.8284   |
| 0.0034        | 113.0 | 2599 | 1.1831          | 0.3351    | 0.5478 | 0.4159 | 0.8301   |
| 0.0034        | 114.0 | 2622 | 1.1999          | 0.3368    | 0.5467 | 0.4168 | 0.8265   |
| 0.0034        | 115.0 | 2645 | 1.1896          | 0.3407    | 0.5467 | 0.4198 | 0.8277   |
| 0.0034        | 116.0 | 2668 | 1.1865          | 0.3451    | 0.5433 | 0.4221 | 0.8291   |
| 0.0034        | 117.0 | 2691 | 1.2529          | 0.3498    | 0.55   | 0.4276 | 0.8228   |
| 0.0034        | 118.0 | 2714 | 1.2159          | 0.3356    | 0.5478 | 0.4162 | 0.8267   |
| 0.0034        | 119.0 | 2737 | 1.1932          | 0.3401    | 0.5556 | 0.4219 | 0.8283   |
| 0.0034        | 120.0 | 2760 | 1.1842          | 0.3419    | 0.5467 | 0.4207 | 0.8287   |
| 0.0034        | 121.0 | 2783 | 1.2179          | 0.3407    | 0.5456 | 0.4195 | 0.8270   |
| 0.0034        | 122.0 | 2806 | 1.2617          | 0.3322    | 0.5489 | 0.4139 | 0.8233   |
| 0.0034        | 123.0 | 2829 | 1.1982          | 0.3356    | 0.5544 | 0.4181 | 0.8237   |
| 0.0034        | 124.0 | 2852 | 1.1664          | 0.3379    | 0.5533 | 0.4195 | 0.8297   |
| 0.0034        | 125.0 | 2875 | 1.1780          | 0.3599    | 0.5578 | 0.4375 | 0.8299   |
| 0.0034        | 126.0 | 2898 | 1.2004          | 0.3515    | 0.5578 | 0.4313 | 0.8310   |
| 0.0034        | 127.0 | 2921 | 1.1824          | 0.3405    | 0.5478 | 0.4199 | 0.8325   |
| 0.0034        | 128.0 | 2944 | 1.1934          | 0.3390    | 0.5533 | 0.4204 | 0.8277   |
| 0.0034        | 129.0 | 2967 | 1.1975          | 0.3409    | 0.5511 | 0.4212 | 0.8288   |
| 0.0034        | 130.0 | 2990 | 1.1867          | 0.3313    | 0.5567 | 0.4154 | 0.8266   |
| 0.0025        | 131.0 | 3013 | 1.2359          | 0.3336    | 0.5567 | 0.4172 | 0.8191   |
| 0.0025        | 132.0 | 3036 | 1.2057          | 0.3302    | 0.5544 | 0.4139 | 0.8263   |
| 0.0025        | 133.0 | 3059 | 1.1999          | 0.3480    | 0.5544 | 0.4276 | 0.8304   |
| 0.0025        | 134.0 | 3082 | 1.2098          | 0.3230    | 0.5544 | 0.4082 | 0.8246   |
| 0.0025        | 135.0 | 3105 | 1.2305          | 0.3255    | 0.54   | 0.4062 | 0.8227   |
| 0.0025        | 136.0 | 3128 | 1.2238          | 0.3290    | 0.5356 | 0.4076 | 0.8255   |
| 0.0025        | 137.0 | 3151 | 1.2014          | 0.3530    | 0.5456 | 0.4286 | 0.8258   |
| 0.0025        | 138.0 | 3174 | 1.2373          | 0.3536    | 0.5611 | 0.4338 | 0.8248   |
| 0.0025        | 139.0 | 3197 | 1.2234          | 0.3333    | 0.5544 | 0.4164 | 0.8256   |
| 0.0025        | 140.0 | 3220 | 1.2205          | 0.3345    | 0.5433 | 0.4141 | 0.8278   |
| 0.0025        | 141.0 | 3243 | 1.2107          | 0.3406    | 0.5522 | 0.4214 | 0.8275   |
| 0.0025        | 142.0 | 3266 | 1.2316          | 0.3467    | 0.5478 | 0.4246 | 0.8233   |
| 0.0025        | 143.0 | 3289 | 1.2420          | 0.3431    | 0.5489 | 0.4222 | 0.8232   |
| 0.0025        | 144.0 | 3312 | 1.2324          | 0.3430    | 0.5533 | 0.4235 | 0.8247   |
| 0.0025        | 145.0 | 3335 | 1.2120          | 0.3391    | 0.5489 | 0.4192 | 0.8274   |
| 0.0025        | 146.0 | 3358 | 1.2396          | 0.3520    | 0.5589 | 0.4319 | 0.8249   |
| 0.0025        | 147.0 | 3381 | 1.2337          | 0.3502    | 0.5622 | 0.4316 | 0.8252   |
| 0.0025        | 148.0 | 3404 | 1.2046          | 0.3436    | 0.5556 | 0.4246 | 0.8305   |
| 0.0025        | 149.0 | 3427 | 1.2149          | 0.3406    | 0.5589 | 0.4232 | 0.8262   |
| 0.0025        | 150.0 | 3450 | 1.2147          | 0.3469    | 0.5589 | 0.4281 | 0.8276   |
| 0.0025        | 151.0 | 3473 | 1.2293          | 0.3486    | 0.5578 | 0.4291 | 0.8271   |
| 0.0025        | 152.0 | 3496 | 1.2431          | 0.3596    | 0.5622 | 0.4387 | 0.8274   |
| 0.0019        | 153.0 | 3519 | 1.2356          | 0.3603    | 0.5533 | 0.4365 | 0.8270   |
| 0.0019        | 154.0 | 3542 | 1.2459          | 0.3600    | 0.5656 | 0.4399 | 0.8266   |
| 0.0019        | 155.0 | 3565 | 1.2203          | 0.3524    | 0.5556 | 0.4312 | 0.8312   |
| 0.0019        | 156.0 | 3588 | 1.2337          | 0.3653    | 0.5633 | 0.4432 | 0.8304   |
| 0.0019        | 157.0 | 3611 | 1.2318          | 0.3572    | 0.5644 | 0.4376 | 0.8290   |
| 0.0019        | 158.0 | 3634 | 1.2456          | 0.3542    | 0.5589 | 0.4336 | 0.8270   |
| 0.0019        | 159.0 | 3657 | 1.2533          | 0.3474    | 0.5567 | 0.4278 | 0.8252   |
| 0.0019        | 160.0 | 3680 | 1.2369          | 0.3522    | 0.5544 | 0.4307 | 0.8305   |
| 0.0019        | 161.0 | 3703 | 1.2516          | 0.3645    | 0.5589 | 0.4412 | 0.8292   |
| 0.0019        | 162.0 | 3726 | 1.2369          | 0.3448    | 0.5567 | 0.4258 | 0.8296   |
| 0.0019        | 163.0 | 3749 | 1.2350          | 0.3544    | 0.56   | 0.4341 | 0.8316   |
| 0.0019        | 164.0 | 3772 | 1.2407          | 0.3493    | 0.5589 | 0.4299 | 0.8280   |
| 0.0019        | 165.0 | 3795 | 1.2450          | 0.3431    | 0.5578 | 0.4249 | 0.8287   |
| 0.0019        | 166.0 | 3818 | 1.2396          | 0.3593    | 0.5633 | 0.4388 | 0.8312   |
| 0.0019        | 167.0 | 3841 | 1.2487          | 0.3568    | 0.5689 | 0.4385 | 0.8283   |
| 0.0019        | 168.0 | 3864 | 1.2995          | 0.3504    | 0.5622 | 0.4317 | 0.8231   |
| 0.0019        | 169.0 | 3887 | 1.2728          | 0.3502    | 0.57   | 0.4338 | 0.8261   |
| 0.0019        | 170.0 | 3910 | 1.2672          | 0.3605    | 0.5644 | 0.4400 | 0.8263   |
| 0.0019        | 171.0 | 3933 | 1.2394          | 0.3580    | 0.5489 | 0.4333 | 0.8301   |
| 0.0019        | 172.0 | 3956 | 1.2566          | 0.3539    | 0.5556 | 0.4323 | 0.8279   |
| 0.0019        | 173.0 | 3979 | 1.2587          | 0.3514    | 0.5533 | 0.4299 | 0.8281   |
| 0.0013        | 174.0 | 4002 | 1.2540          | 0.3489    | 0.5467 | 0.4260 | 0.8307   |
| 0.0013        | 175.0 | 4025 | 1.2817          | 0.3495    | 0.5511 | 0.4278 | 0.8231   |
| 0.0013        | 176.0 | 4048 | 1.2569          | 0.3543    | 0.5578 | 0.4333 | 0.8275   |
| 0.0013        | 177.0 | 4071 | 1.3119          | 0.3562    | 0.5544 | 0.4337 | 0.8245   |
| 0.0013        | 178.0 | 4094 | 1.3102          | 0.3423    | 0.5522 | 0.4226 | 0.8213   |
| 0.0013        | 179.0 | 4117 | 1.2313          | 0.3407    | 0.5489 | 0.4204 | 0.8320   |
| 0.0013        | 180.0 | 4140 | 1.2375          | 0.3506    | 0.5567 | 0.4302 | 0.8272   |
| 0.0013        | 181.0 | 4163 | 1.2360          | 0.3409    | 0.5489 | 0.4206 | 0.8289   |
| 0.0013        | 182.0 | 4186 | 1.2685          | 0.3465    | 0.5544 | 0.4265 | 0.8281   |
| 0.0013        | 183.0 | 4209 | 1.3033          | 0.3510    | 0.5511 | 0.4289 | 0.8238   |
| 0.0013        | 184.0 | 4232 | 1.2407          | 0.3626    | 0.5511 | 0.4374 | 0.8319   |
| 0.0013        | 185.0 | 4255 | 1.2542          | 0.3605    | 0.5644 | 0.4400 | 0.8284   |
| 0.0013        | 186.0 | 4278 | 1.2654          | 0.3637    | 0.5633 | 0.4420 | 0.8278   |
| 0.0013        | 187.0 | 4301 | 1.2667          | 0.3516    | 0.5644 | 0.4333 | 0.8280   |
| 0.0013        | 188.0 | 4324 | 1.2661          | 0.3542    | 0.5667 | 0.4359 | 0.8293   |
| 0.0013        | 189.0 | 4347 | 1.2831          | 0.3524    | 0.5533 | 0.4306 | 0.8257   |
| 0.0013        | 190.0 | 4370 | 1.2593          | 0.3603    | 0.55   | 0.4354 | 0.8288   |
| 0.0013        | 191.0 | 4393 | 1.2186          | 0.3501    | 0.5711 | 0.4341 | 0.8284   |
| 0.0013        | 192.0 | 4416 | 1.2797          | 0.3576    | 0.5622 | 0.4371 | 0.8222   |
| 0.0013        | 193.0 | 4439 | 1.2491          | 0.3501    | 0.5656 | 0.4325 | 0.8264   |
| 0.0013        | 194.0 | 4462 | 1.2456          | 0.3580    | 0.5544 | 0.4350 | 0.8305   |
| 0.0013        | 195.0 | 4485 | 1.2567          | 0.3542    | 0.5478 | 0.4302 | 0.8275   |
| 0.0014        | 196.0 | 4508 | 1.2551          | 0.3512    | 0.5611 | 0.4320 | 0.8274   |
| 0.0014        | 197.0 | 4531 | 1.2512          | 0.3468    | 0.5611 | 0.4287 | 0.8295   |
| 0.0014        | 198.0 | 4554 | 1.2607          | 0.3465    | 0.5556 | 0.4268 | 0.8279   |
| 0.0014        | 199.0 | 4577 | 1.2703          | 0.3501    | 0.5556 | 0.4296 | 0.8264   |
| 0.0014        | 200.0 | 4600 | 1.2536          | 0.3421    | 0.5511 | 0.4221 | 0.8289   |
| 0.0014        | 201.0 | 4623 | 1.2978          | 0.3302    | 0.5467 | 0.4117 | 0.8242   |
| 0.0014        | 202.0 | 4646 | 1.2775          | 0.3495    | 0.5456 | 0.4260 | 0.8279   |
| 0.0014        | 203.0 | 4669 | 1.2697          | 0.3467    | 0.5556 | 0.4270 | 0.8274   |
| 0.0014        | 204.0 | 4692 | 1.2595          | 0.3525    | 0.5511 | 0.4300 | 0.8295   |
| 0.0014        | 205.0 | 4715 | 1.2565          | 0.3479    | 0.5578 | 0.4285 | 0.8291   |
| 0.0014        | 206.0 | 4738 | 1.2547          | 0.3592    | 0.5611 | 0.4380 | 0.8316   |
| 0.0014        | 207.0 | 4761 | 1.2585          | 0.3537    | 0.5533 | 0.4315 | 0.8303   |
| 0.0014        | 208.0 | 4784 | 1.2788          | 0.3543    | 0.5622 | 0.4347 | 0.8295   |
| 0.0014        | 209.0 | 4807 | 1.2566          | 0.3655    | 0.5556 | 0.4409 | 0.8327   |
| 0.0014        | 210.0 | 4830 | 1.2612          | 0.3655    | 0.5633 | 0.4434 | 0.8326   |
| 0.0014        | 211.0 | 4853 | 1.2631          | 0.3532    | 0.56   | 0.4332 | 0.8321   |
| 0.0014        | 212.0 | 4876 | 1.2867          | 0.3563    | 0.5467 | 0.4314 | 0.8304   |
| 0.0014        | 213.0 | 4899 | 1.2959          | 0.3417    | 0.5444 | 0.4199 | 0.8272   |
| 0.0014        | 214.0 | 4922 | 1.2547          | 0.3577    | 0.5556 | 0.4352 | 0.8339   |
| 0.0014        | 215.0 | 4945 | 1.2812          | 0.3491    | 0.5567 | 0.4291 | 0.8284   |
| 0.0014        | 216.0 | 4968 | 1.2780          | 0.3455    | 0.5489 | 0.4240 | 0.8313   |
| 0.0014        | 217.0 | 4991 | 1.2361          | 0.3500    | 0.5511 | 0.4281 | 0.8349   |
| 0.0011        | 218.0 | 5014 | 1.2627          | 0.3324    | 0.5433 | 0.4125 | 0.8304   |
| 0.0011        | 219.0 | 5037 | 1.2884          | 0.3547    | 0.5544 | 0.4326 | 0.8282   |
| 0.0011        | 220.0 | 5060 | 1.2828          | 0.3320    | 0.5456 | 0.4128 | 0.8270   |
| 0.0011        | 221.0 | 5083 | 1.3193          | 0.3527    | 0.5467 | 0.4288 | 0.8259   |
| 0.0011        | 222.0 | 5106 | 1.3023          | 0.3555    | 0.5522 | 0.4326 | 0.8246   |
| 0.0011        | 223.0 | 5129 | 1.2836          | 0.3354    | 0.5511 | 0.4170 | 0.8305   |
| 0.0011        | 224.0 | 5152 | 1.3094          | 0.3469    | 0.5511 | 0.4258 | 0.8271   |
| 0.0011        | 225.0 | 5175 | 1.2760          | 0.3538    | 0.5511 | 0.4309 | 0.8338   |
| 0.0011        | 226.0 | 5198 | 1.2786          | 0.3494    | 0.5567 | 0.4293 | 0.8336   |
| 0.0011        | 227.0 | 5221 | 1.2766          | 0.3564    | 0.5556 | 0.4342 | 0.8340   |
| 0.0011        | 228.0 | 5244 | 1.2738          | 0.3555    | 0.5533 | 0.4329 | 0.8336   |
| 0.0011        | 229.0 | 5267 | 1.2871          | 0.3538    | 0.5578 | 0.4329 | 0.8292   |
| 0.0011        | 230.0 | 5290 | 1.2836          | 0.3568    | 0.5578 | 0.4352 | 0.8321   |
| 0.0011        | 231.0 | 5313 | 1.2787          | 0.3534    | 0.5533 | 0.4314 | 0.8300   |
| 0.0011        | 232.0 | 5336 | 1.2907          | 0.3538    | 0.5578 | 0.4329 | 0.8327   |
| 0.0011        | 233.0 | 5359 | 1.2812          | 0.3544    | 0.5556 | 0.4327 | 0.8316   |
| 0.0011        | 234.0 | 5382 | 1.2802          | 0.3543    | 0.5567 | 0.4330 | 0.8303   |
| 0.0011        | 235.0 | 5405 | 1.3033          | 0.3484    | 0.5567 | 0.4286 | 0.8284   |
| 0.0011        | 236.0 | 5428 | 1.2852          | 0.3491    | 0.5578 | 0.4294 | 0.8301   |
| 0.0011        | 237.0 | 5451 | 1.2866          | 0.3481    | 0.5511 | 0.4267 | 0.8291   |
| 0.0011        | 238.0 | 5474 | 1.2914          | 0.3524    | 0.5544 | 0.4309 | 0.8297   |
| 0.0011        | 239.0 | 5497 | 1.2944          | 0.3432    | 0.5522 | 0.4233 | 0.8278   |
| 0.0009        | 240.0 | 5520 | 1.2860          | 0.3470    | 0.5567 | 0.4275 | 0.8308   |
| 0.0009        | 241.0 | 5543 | 1.2786          | 0.3516    | 0.5489 | 0.4286 | 0.8309   |
| 0.0009        | 242.0 | 5566 | 1.2743          | 0.3425    | 0.5511 | 0.4225 | 0.8307   |
| 0.0009        | 243.0 | 5589 | 1.2843          | 0.3467    | 0.5489 | 0.4249 | 0.8285   |
| 0.0009        | 244.0 | 5612 | 1.3106          | 0.3372    | 0.5522 | 0.4187 | 0.8266   |
| 0.0009        | 245.0 | 5635 | 1.2952          | 0.3537    | 0.5456 | 0.4292 | 0.8306   |
| 0.0009        | 246.0 | 5658 | 1.2716          | 0.3503    | 0.5511 | 0.4283 | 0.8340   |
| 0.0009        | 247.0 | 5681 | 1.3108          | 0.3452    | 0.5511 | 0.4245 | 0.8254   |
| 0.0009        | 248.0 | 5704 | 1.3098          | 0.3322    | 0.54   | 0.4113 | 0.8283   |
| 0.0009        | 249.0 | 5727 | 1.2782          | 0.3605    | 0.5544 | 0.4370 | 0.8325   |
| 0.0009        | 250.0 | 5750 | 1.2979          | 0.3449    | 0.5522 | 0.4246 | 0.8294   |
| 0.0009        | 251.0 | 5773 | 1.2919          | 0.3492    | 0.5544 | 0.4285 | 0.8313   |
| 0.0009        | 252.0 | 5796 | 1.2946          | 0.3385    | 0.5533 | 0.4201 | 0.8313   |
| 0.0009        | 253.0 | 5819 | 1.2994          | 0.3439    | 0.5544 | 0.4245 | 0.8312   |
| 0.0009        | 254.0 | 5842 | 1.2962          | 0.3522    | 0.5544 | 0.4307 | 0.8312   |
| 0.0009        | 255.0 | 5865 | 1.3038          | 0.3577    | 0.5556 | 0.4352 | 0.8300   |
| 0.0009        | 256.0 | 5888 | 1.3091          | 0.3492    | 0.5556 | 0.4288 | 0.8315   |
| 0.0009        | 257.0 | 5911 | 1.2943          | 0.3477    | 0.5556 | 0.4277 | 0.8315   |
| 0.0009        | 258.0 | 5934 | 1.3091          | 0.3427    | 0.5556 | 0.4239 | 0.8282   |
| 0.0009        | 259.0 | 5957 | 1.2957          | 0.3391    | 0.5478 | 0.4189 | 0.8325   |
| 0.0009        | 260.0 | 5980 | 1.2927          | 0.3367    | 0.55   | 0.4177 | 0.8293   |
| 0.0008        | 261.0 | 6003 | 1.2926          | 0.3363    | 0.55   | 0.4174 | 0.8309   |
| 0.0008        | 262.0 | 6026 | 1.2889          | 0.3449    | 0.5533 | 0.4249 | 0.8326   |
| 0.0008        | 263.0 | 6049 | 1.3226          | 0.3413    | 0.5522 | 0.4219 | 0.8260   |
| 0.0008        | 264.0 | 6072 | 1.2979          | 0.3550    | 0.5578 | 0.4339 | 0.8284   |
| 0.0008        | 265.0 | 6095 | 1.3032          | 0.3493    | 0.5511 | 0.4276 | 0.8289   |
| 0.0008        | 266.0 | 6118 | 1.3065          | 0.3495    | 0.5533 | 0.4284 | 0.8300   |
| 0.0008        | 267.0 | 6141 | 1.2992          | 0.3524    | 0.5467 | 0.4286 | 0.8316   |
| 0.0008        | 268.0 | 6164 | 1.2990          | 0.3560    | 0.5522 | 0.4329 | 0.8310   |
| 0.0008        | 269.0 | 6187 | 1.3041          | 0.3506    | 0.5489 | 0.4279 | 0.8305   |
| 0.0008        | 270.0 | 6210 | 1.3046          | 0.3468    | 0.5444 | 0.4237 | 0.8306   |
| 0.0008        | 271.0 | 6233 | 1.3001          | 0.3626    | 0.5556 | 0.4388 | 0.8317   |
| 0.0008        | 272.0 | 6256 | 1.3011          | 0.3611    | 0.5533 | 0.4370 | 0.8304   |
| 0.0008        | 273.0 | 6279 | 1.3057          | 0.3454    | 0.5522 | 0.4250 | 0.8297   |
| 0.0008        | 274.0 | 6302 | 1.3142          | 0.3416    | 0.5478 | 0.4208 | 0.8280   |
| 0.0008        | 275.0 | 6325 | 1.2959          | 0.3463    | 0.5544 | 0.4263 | 0.8316   |
| 0.0008        | 276.0 | 6348 | 1.2905          | 0.3512    | 0.5544 | 0.4300 | 0.8324   |
| 0.0008        | 277.0 | 6371 | 1.3094          | 0.3531    | 0.5489 | 0.4298 | 0.8301   |
| 0.0008        | 278.0 | 6394 | 1.3291          | 0.3472    | 0.5456 | 0.4244 | 0.8275   |
| 0.0008        | 279.0 | 6417 | 1.2861          | 0.3545    | 0.5522 | 0.4318 | 0.8332   |
| 0.0008        | 280.0 | 6440 | 1.2926          | 0.3549    | 0.5544 | 0.4328 | 0.8307   |
| 0.0008        | 281.0 | 6463 | 1.2926          | 0.3395    | 0.55   | 0.4198 | 0.8313   |
| 0.0008        | 282.0 | 6486 | 1.2915          | 0.3447    | 0.5511 | 0.4241 | 0.8320   |
| 0.0007        | 283.0 | 6509 | 1.2848          | 0.3524    | 0.5544 | 0.4309 | 0.8327   |
| 0.0007        | 284.0 | 6532 | 1.2948          | 0.3491    | 0.55   | 0.4271 | 0.8301   |
| 0.0007        | 285.0 | 6555 | 1.2945          | 0.3492    | 0.5533 | 0.4282 | 0.8311   |
| 0.0007        | 286.0 | 6578 | 1.2921          | 0.3580    | 0.5544 | 0.4350 | 0.8316   |
| 0.0007        | 287.0 | 6601 | 1.2980          | 0.3462    | 0.5489 | 0.4246 | 0.8305   |
| 0.0007        | 288.0 | 6624 | 1.2962          | 0.3388    | 0.5478 | 0.4187 | 0.8310   |
| 0.0007        | 289.0 | 6647 | 1.2928          | 0.3416    | 0.5478 | 0.4208 | 0.8315   |
| 0.0007        | 290.0 | 6670 | 1.2891          | 0.3503    | 0.55   | 0.4280 | 0.8318   |
| 0.0007        | 291.0 | 6693 | 1.2905          | 0.3587    | 0.5544 | 0.4356 | 0.8317   |
| 0.0007        | 292.0 | 6716 | 1.2954          | 0.3559    | 0.55   | 0.4321 | 0.8312   |
| 0.0007        | 293.0 | 6739 | 1.2948          | 0.3510    | 0.5522 | 0.4292 | 0.8311   |
| 0.0007        | 294.0 | 6762 | 1.2944          | 0.3520    | 0.5522 | 0.4299 | 0.8311   |
| 0.0007        | 295.0 | 6785 | 1.2957          | 0.3499    | 0.5489 | 0.4273 | 0.8310   |
| 0.0007        | 296.0 | 6808 | 1.2960          | 0.3489    | 0.5478 | 0.4263 | 0.8311   |
| 0.0007        | 297.0 | 6831 | 1.2938          | 0.3481    | 0.55   | 0.4264 | 0.8319   |
| 0.0007        | 298.0 | 6854 | 1.2933          | 0.3447    | 0.55   | 0.4238 | 0.8317   |
| 0.0007        | 299.0 | 6877 | 1.2933          | 0.3456    | 0.5511 | 0.4248 | 0.8319   |
| 0.0007        | 300.0 | 6900 | 1.2935          | 0.3471    | 0.55   | 0.4256 | 0.8319   |


### Framework versions

- Transformers 4.27.0.dev0
- Pytorch 1.13.0
- Datasets 2.8.0
- Tokenizers 0.12.1