Hugging Face's logo

ZMC2019
/
DeepScaleR-1.5B-L36-RollE3

Text Generation
Transformers
Safetensors
qwen2
Generated from Trainer
open-r1
trl
sft
conversational
text-generation-inference
Model card Files Community
ZMC2019 commited on
Commit
8c7403e
·
verified ·
1 Parent(s): 2833297

Model save

Browse files
Files changed (5) hide show
  1. README.md +58 -0
  2. all_results.json +8 -0
  3. generation_config.json +9 -0
  4. train_results.json +8 -0
  5. trainer_state.json +539 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: agentica-org/DeepScaleR-1.5B-Preview
3
+ library_name: transformers
4
+ model_name: DeepScaleR-1.5B-L36-RollE3
5
+ tags:
6
+ - generated_from_trainer
7
+ - trl
8
+ - sft
9
+ licence: license
10
+ ---
11
+
12
+ # Model Card for DeepScaleR-1.5B-L36-RollE3
13
+
14
+ This model is a fine-tuned version of [agentica-org/DeepScaleR-1.5B-Preview](https://huggingface.co/agentica-org/DeepScaleR-1.5B-Preview).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl).
16
+
17
+ ## Quick start
18
+
19
+ ```python
20
+ from transformers import pipeline
21
+
22
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="ZMC2019/DeepScaleR-1.5B-L36-RollE3", device="cuda")
24
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
+ print(output["generated_text"])
26
+ ```
27
+
28
+ ## Training procedure
29
+
30
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/chenzhuoming911/huggingface/runs/reb9ifw3)
31
+
32
+
33
+ This model was trained with SFT.
34
+
35
+ ### Framework versions
36
+
37
+ - TRL: 0.16.0.dev0
38
+ - Transformers: 4.49.0
39
+ - Pytorch: 2.5.1
40
+ - Datasets: 3.4.1
41
+ - Tokenizers: 0.21.1
42
+
43
+ ## Citations
44
+
45
+
46
+
47
+ Cite TRL as:
48
+
49
+ ```bibtex
50
+ @misc{vonwerra2022trl,
51
+ title = {{TRL: Transformer Reinforcement Learning}},
52
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
53
+ year = 2020,
54
+ journal = {GitHub repository},
55
+ publisher = {GitHub},
56
+ howpublished = {\url{https://github.com/huggingface/trl}}
57
+ }
58
+ ```
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 1.2726481677031834e+17,
3
+ "train_loss": 0.3811905702956918,
4
+ "train_runtime": 3910.3734,
5
+ "train_samples": 93733,
6
+ "train_samples_per_second": 1.455,
7
+ "train_steps_per_second": 0.091
8
+ }
generation_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 151646,
4
+ "do_sample": true,
5
+ "eos_token_id": 151643,
6
+ "temperature": 0.6,
7
+ "top_p": 0.95,
8
+ "transformers_version": "4.49.0"
9
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 1.2726481677031834e+17,
3
+ "train_loss": 0.3811905702956918,
4
+ "train_runtime": 3910.3734,
5
+ "train_samples": 93733,
6
+ "train_samples_per_second": 1.455,
7
+ "train_steps_per_second": 0.091
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,539 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 3.0,
5
+ "eval_steps": 500,
6
+ "global_step": 357,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.04201680672268908,
13
+ "grad_norm": 4.70634130932114,
14
+ "learning_rate": 6.944444444444445e-06,
15
+ "loss": 0.9136,
16
+ "step": 5
17
+ },
18
+ {
19
+ "epoch": 0.08403361344537816,
20
+ "grad_norm": 1.8906913115297757,
21
+ "learning_rate": 1.388888888888889e-05,
22
+ "loss": 0.766,
23
+ "step": 10
24
+ },
25
+ {
26
+ "epoch": 0.12605042016806722,
27
+ "grad_norm": 0.6826181059075677,
28
+ "learning_rate": 2.0833333333333336e-05,
29
+ "loss": 0.6325,
30
+ "step": 15
31
+ },
32
+ {
33
+ "epoch": 0.16806722689075632,
34
+ "grad_norm": 0.33363228311628446,
35
+ "learning_rate": 2.777777777777778e-05,
36
+ "loss": 0.5511,
37
+ "step": 20
38
+ },
39
+ {
40
+ "epoch": 0.21008403361344538,
41
+ "grad_norm": 0.2907608801770254,
42
+ "learning_rate": 3.472222222222222e-05,
43
+ "loss": 0.5244,
44
+ "step": 25
45
+ },
46
+ {
47
+ "epoch": 0.25210084033613445,
48
+ "grad_norm": 0.2733826152956853,
49
+ "learning_rate": 4.166666666666667e-05,
50
+ "loss": 0.4913,
51
+ "step": 30
52
+ },
53
+ {
54
+ "epoch": 0.29411764705882354,
55
+ "grad_norm": 0.2732334536953152,
56
+ "learning_rate": 4.8611111111111115e-05,
57
+ "loss": 0.4751,
58
+ "step": 35
59
+ },
60
+ {
61
+ "epoch": 0.33613445378151263,
62
+ "grad_norm": 0.2585751886114213,
63
+ "learning_rate": 4.937694704049845e-05,
64
+ "loss": 0.4596,
65
+ "step": 40
66
+ },
67
+ {
68
+ "epoch": 0.37815126050420167,
69
+ "grad_norm": 0.2397885612111248,
70
+ "learning_rate": 4.85981308411215e-05,
71
+ "loss": 0.4349,
72
+ "step": 45
73
+ },
74
+ {
75
+ "epoch": 0.42016806722689076,
76
+ "grad_norm": 0.21116743482172115,
77
+ "learning_rate": 4.781931464174455e-05,
78
+ "loss": 0.4326,
79
+ "step": 50
80
+ },
81
+ {
82
+ "epoch": 0.46218487394957986,
83
+ "grad_norm": 0.14451656745477998,
84
+ "learning_rate": 4.7040498442367604e-05,
85
+ "loss": 0.4157,
86
+ "step": 55
87
+ },
88
+ {
89
+ "epoch": 0.5042016806722689,
90
+ "grad_norm": 0.14127541762263016,
91
+ "learning_rate": 4.6261682242990654e-05,
92
+ "loss": 0.4122,
93
+ "step": 60
94
+ },
95
+ {
96
+ "epoch": 0.5462184873949579,
97
+ "grad_norm": 0.671598848272524,
98
+ "learning_rate": 4.548286604361371e-05,
99
+ "loss": 0.4149,
100
+ "step": 65
101
+ },
102
+ {
103
+ "epoch": 0.5882352941176471,
104
+ "grad_norm": 0.13463214509481336,
105
+ "learning_rate": 4.470404984423676e-05,
106
+ "loss": 0.3959,
107
+ "step": 70
108
+ },
109
+ {
110
+ "epoch": 0.6302521008403361,
111
+ "grad_norm": 0.1561144783705333,
112
+ "learning_rate": 4.392523364485982e-05,
113
+ "loss": 0.4038,
114
+ "step": 75
115
+ },
116
+ {
117
+ "epoch": 0.6722689075630253,
118
+ "grad_norm": 0.135660513281463,
119
+ "learning_rate": 4.314641744548287e-05,
120
+ "loss": 0.4012,
121
+ "step": 80
122
+ },
123
+ {
124
+ "epoch": 0.7142857142857143,
125
+ "grad_norm": 0.1329272951870047,
126
+ "learning_rate": 4.236760124610592e-05,
127
+ "loss": 0.3952,
128
+ "step": 85
129
+ },
130
+ {
131
+ "epoch": 0.7563025210084033,
132
+ "grad_norm": 0.12595448081633448,
133
+ "learning_rate": 4.1588785046728974e-05,
134
+ "loss": 0.3893,
135
+ "step": 90
136
+ },
137
+ {
138
+ "epoch": 0.7983193277310925,
139
+ "grad_norm": 0.1293818852034363,
140
+ "learning_rate": 4.0809968847352024e-05,
141
+ "loss": 0.4,
142
+ "step": 95
143
+ },
144
+ {
145
+ "epoch": 0.8403361344537815,
146
+ "grad_norm": 0.1469536727483663,
147
+ "learning_rate": 4.003115264797508e-05,
148
+ "loss": 0.3929,
149
+ "step": 100
150
+ },
151
+ {
152
+ "epoch": 0.8823529411764706,
153
+ "grad_norm": 0.12410046786523299,
154
+ "learning_rate": 3.925233644859813e-05,
155
+ "loss": 0.388,
156
+ "step": 105
157
+ },
158
+ {
159
+ "epoch": 0.9243697478991597,
160
+ "grad_norm": 0.13619892881270407,
161
+ "learning_rate": 3.847352024922119e-05,
162
+ "loss": 0.3821,
163
+ "step": 110
164
+ },
165
+ {
166
+ "epoch": 0.9663865546218487,
167
+ "grad_norm": 0.11885167396488784,
168
+ "learning_rate": 3.769470404984424e-05,
169
+ "loss": 0.3946,
170
+ "step": 115
171
+ },
172
+ {
173
+ "epoch": 1.0084033613445378,
174
+ "grad_norm": 0.14287918831630414,
175
+ "learning_rate": 3.691588785046729e-05,
176
+ "loss": 0.3801,
177
+ "step": 120
178
+ },
179
+ {
180
+ "epoch": 1.050420168067227,
181
+ "grad_norm": 0.1285290919486453,
182
+ "learning_rate": 3.6137071651090344e-05,
183
+ "loss": 0.3652,
184
+ "step": 125
185
+ },
186
+ {
187
+ "epoch": 1.092436974789916,
188
+ "grad_norm": 0.12470234407748469,
189
+ "learning_rate": 3.5358255451713394e-05,
190
+ "loss": 0.3587,
191
+ "step": 130
192
+ },
193
+ {
194
+ "epoch": 1.134453781512605,
195
+ "grad_norm": 0.11633316420488911,
196
+ "learning_rate": 3.457943925233645e-05,
197
+ "loss": 0.357,
198
+ "step": 135
199
+ },
200
+ {
201
+ "epoch": 1.1764705882352942,
202
+ "grad_norm": 0.12243208920648133,
203
+ "learning_rate": 3.38006230529595e-05,
204
+ "loss": 0.3534,
205
+ "step": 140
206
+ },
207
+ {
208
+ "epoch": 1.2184873949579833,
209
+ "grad_norm": 0.12086774297684207,
210
+ "learning_rate": 3.302180685358255e-05,
211
+ "loss": 0.3603,
212
+ "step": 145
213
+ },
214
+ {
215
+ "epoch": 1.2605042016806722,
216
+ "grad_norm": 0.12226080249060899,
217
+ "learning_rate": 3.224299065420561e-05,
218
+ "loss": 0.355,
219
+ "step": 150
220
+ },
221
+ {
222
+ "epoch": 1.3025210084033614,
223
+ "grad_norm": 0.11411438458299594,
224
+ "learning_rate": 3.146417445482866e-05,
225
+ "loss": 0.3602,
226
+ "step": 155
227
+ },
228
+ {
229
+ "epoch": 1.3445378151260505,
230
+ "grad_norm": 0.11464926522938973,
231
+ "learning_rate": 3.0685358255451714e-05,
232
+ "loss": 0.3518,
233
+ "step": 160
234
+ },
235
+ {
236
+ "epoch": 1.3865546218487395,
237
+ "grad_norm": 0.12001429741902232,
238
+ "learning_rate": 2.9906542056074764e-05,
239
+ "loss": 0.355,
240
+ "step": 165
241
+ },
242
+ {
243
+ "epoch": 1.4285714285714286,
244
+ "grad_norm": 0.11341054683006553,
245
+ "learning_rate": 2.9127725856697818e-05,
246
+ "loss": 0.3546,
247
+ "step": 170
248
+ },
249
+ {
250
+ "epoch": 1.4705882352941178,
251
+ "grad_norm": 0.11534598808870551,
252
+ "learning_rate": 2.834890965732087e-05,
253
+ "loss": 0.3481,
254
+ "step": 175
255
+ },
256
+ {
257
+ "epoch": 1.5126050420168067,
258
+ "grad_norm": 0.11555410704104117,
259
+ "learning_rate": 2.7570093457943924e-05,
260
+ "loss": 0.3566,
261
+ "step": 180
262
+ },
263
+ {
264
+ "epoch": 1.5546218487394958,
265
+ "grad_norm": 0.11882277392875285,
266
+ "learning_rate": 2.6791277258566978e-05,
267
+ "loss": 0.3503,
268
+ "step": 185
269
+ },
270
+ {
271
+ "epoch": 1.596638655462185,
272
+ "grad_norm": 0.10912064576299817,
273
+ "learning_rate": 2.601246105919003e-05,
274
+ "loss": 0.3467,
275
+ "step": 190
276
+ },
277
+ {
278
+ "epoch": 1.638655462184874,
279
+ "grad_norm": 0.11671548577730292,
280
+ "learning_rate": 2.5233644859813084e-05,
281
+ "loss": 0.3518,
282
+ "step": 195
283
+ },
284
+ {
285
+ "epoch": 1.680672268907563,
286
+ "grad_norm": 0.11709095347100043,
287
+ "learning_rate": 2.4454828660436138e-05,
288
+ "loss": 0.3514,
289
+ "step": 200
290
+ },
291
+ {
292
+ "epoch": 1.7226890756302522,
293
+ "grad_norm": 0.4547280409942813,
294
+ "learning_rate": 2.367601246105919e-05,
295
+ "loss": 0.3545,
296
+ "step": 205
297
+ },
298
+ {
299
+ "epoch": 1.7647058823529411,
300
+ "grad_norm": 0.12116945588708594,
301
+ "learning_rate": 2.2897196261682244e-05,
302
+ "loss": 0.3471,
303
+ "step": 210
304
+ },
305
+ {
306
+ "epoch": 1.8067226890756303,
307
+ "grad_norm": 0.11202328680044989,
308
+ "learning_rate": 2.2118380062305298e-05,
309
+ "loss": 0.3501,
310
+ "step": 215
311
+ },
312
+ {
313
+ "epoch": 1.8487394957983194,
314
+ "grad_norm": 0.11989297797083326,
315
+ "learning_rate": 2.133956386292835e-05,
316
+ "loss": 0.3511,
317
+ "step": 220
318
+ },
319
+ {
320
+ "epoch": 1.8907563025210083,
321
+ "grad_norm": 0.11344656402005421,
322
+ "learning_rate": 2.05607476635514e-05,
323
+ "loss": 0.3427,
324
+ "step": 225
325
+ },
326
+ {
327
+ "epoch": 1.9327731092436975,
328
+ "grad_norm": 0.11459966187379855,
329
+ "learning_rate": 1.9781931464174454e-05,
330
+ "loss": 0.35,
331
+ "step": 230
332
+ },
333
+ {
334
+ "epoch": 1.9747899159663866,
335
+ "grad_norm": 0.10470258572360276,
336
+ "learning_rate": 1.9003115264797507e-05,
337
+ "loss": 0.3459,
338
+ "step": 235
339
+ },
340
+ {
341
+ "epoch": 2.0168067226890756,
342
+ "grad_norm": 0.11916183310406858,
343
+ "learning_rate": 1.822429906542056e-05,
344
+ "loss": 0.3366,
345
+ "step": 240
346
+ },
347
+ {
348
+ "epoch": 2.0588235294117645,
349
+ "grad_norm": 0.1385981052541961,
350
+ "learning_rate": 1.7445482866043614e-05,
351
+ "loss": 0.3251,
352
+ "step": 245
353
+ },
354
+ {
355
+ "epoch": 2.100840336134454,
356
+ "grad_norm": 0.1235803846831308,
357
+ "learning_rate": 1.6666666666666667e-05,
358
+ "loss": 0.3237,
359
+ "step": 250
360
+ },
361
+ {
362
+ "epoch": 2.142857142857143,
363
+ "grad_norm": 0.11353299337591315,
364
+ "learning_rate": 1.588785046728972e-05,
365
+ "loss": 0.3198,
366
+ "step": 255
367
+ },
368
+ {
369
+ "epoch": 2.184873949579832,
370
+ "grad_norm": 0.11174702397768707,
371
+ "learning_rate": 1.5109034267912772e-05,
372
+ "loss": 0.3191,
373
+ "step": 260
374
+ },
375
+ {
376
+ "epoch": 2.226890756302521,
377
+ "grad_norm": 0.1137863668401699,
378
+ "learning_rate": 1.4330218068535826e-05,
379
+ "loss": 0.3229,
380
+ "step": 265
381
+ },
382
+ {
383
+ "epoch": 2.26890756302521,
384
+ "grad_norm": 0.12072471085734401,
385
+ "learning_rate": 1.3551401869158877e-05,
386
+ "loss": 0.324,
387
+ "step": 270
388
+ },
389
+ {
390
+ "epoch": 2.310924369747899,
391
+ "grad_norm": 0.11123684767070798,
392
+ "learning_rate": 1.277258566978193e-05,
393
+ "loss": 0.3214,
394
+ "step": 275
395
+ },
396
+ {
397
+ "epoch": 2.3529411764705883,
398
+ "grad_norm": 0.11295726491194695,
399
+ "learning_rate": 1.1993769470404986e-05,
400
+ "loss": 0.318,
401
+ "step": 280
402
+ },
403
+ {
404
+ "epoch": 2.3949579831932772,
405
+ "grad_norm": 0.1106796291907045,
406
+ "learning_rate": 1.1214953271028037e-05,
407
+ "loss": 0.3235,
408
+ "step": 285
409
+ },
410
+ {
411
+ "epoch": 2.4369747899159666,
412
+ "grad_norm": 0.11033037272448094,
413
+ "learning_rate": 1.043613707165109e-05,
414
+ "loss": 0.3227,
415
+ "step": 290
416
+ },
417
+ {
418
+ "epoch": 2.4789915966386555,
419
+ "grad_norm": 0.10873033497026395,
420
+ "learning_rate": 9.657320872274144e-06,
421
+ "loss": 0.3215,
422
+ "step": 295
423
+ },
424
+ {
425
+ "epoch": 2.5210084033613445,
426
+ "grad_norm": 0.11267308869650684,
427
+ "learning_rate": 8.878504672897196e-06,
428
+ "loss": 0.3236,
429
+ "step": 300
430
+ },
431
+ {
432
+ "epoch": 2.5630252100840334,
433
+ "grad_norm": 0.10662033804378954,
434
+ "learning_rate": 8.099688473520249e-06,
435
+ "loss": 0.3185,
436
+ "step": 305
437
+ },
438
+ {
439
+ "epoch": 2.6050420168067228,
440
+ "grad_norm": 0.10419898622414428,
441
+ "learning_rate": 7.3208722741433015e-06,
442
+ "loss": 0.3201,
443
+ "step": 310
444
+ },
445
+ {
446
+ "epoch": 2.6470588235294117,
447
+ "grad_norm": 0.1194071123415722,
448
+ "learning_rate": 6.542056074766355e-06,
449
+ "loss": 0.3181,
450
+ "step": 315
451
+ },
452
+ {
453
+ "epoch": 2.689075630252101,
454
+ "grad_norm": 0.10315644198712422,
455
+ "learning_rate": 5.763239875389408e-06,
456
+ "loss": 0.319,
457
+ "step": 320
458
+ },
459
+ {
460
+ "epoch": 2.73109243697479,
461
+ "grad_norm": 0.10169341589102616,
462
+ "learning_rate": 4.9844236760124615e-06,
463
+ "loss": 0.3155,
464
+ "step": 325
465
+ },
466
+ {
467
+ "epoch": 2.773109243697479,
468
+ "grad_norm": 0.10841897463334976,
469
+ "learning_rate": 4.205607476635514e-06,
470
+ "loss": 0.3221,
471
+ "step": 330
472
+ },
473
+ {
474
+ "epoch": 2.815126050420168,
475
+ "grad_norm": 0.11304762877258454,
476
+ "learning_rate": 3.426791277258567e-06,
477
+ "loss": 0.3224,
478
+ "step": 335
479
+ },
480
+ {
481
+ "epoch": 2.857142857142857,
482
+ "grad_norm": 0.10152209760807467,
483
+ "learning_rate": 2.64797507788162e-06,
484
+ "loss": 0.3117,
485
+ "step": 340
486
+ },
487
+ {
488
+ "epoch": 2.899159663865546,
489
+ "grad_norm": 0.10952489428700839,
490
+ "learning_rate": 1.8691588785046728e-06,
491
+ "loss": 0.3218,
492
+ "step": 345
493
+ },
494
+ {
495
+ "epoch": 2.9411764705882355,
496
+ "grad_norm": 0.1024965907199079,
497
+ "learning_rate": 1.0903426791277259e-06,
498
+ "loss": 0.3206,
499
+ "step": 350
500
+ },
501
+ {
502
+ "epoch": 2.9831932773109244,
503
+ "grad_norm": 0.1048143744280903,
504
+ "learning_rate": 3.1152647975077885e-07,
505
+ "loss": 0.3265,
506
+ "step": 355
507
+ },
508
+ {
509
+ "epoch": 3.0,
510
+ "step": 357,
511
+ "total_flos": 1.2726481677031834e+17,
512
+ "train_loss": 0.3811905702956918,
513
+ "train_runtime": 3910.3734,
514
+ "train_samples_per_second": 1.455,
515
+ "train_steps_per_second": 0.091
516
+ }
517
+ ],
518
+ "logging_steps": 5,
519
+ "max_steps": 357,
520
+ "num_input_tokens_seen": 0,
521
+ "num_train_epochs": 3,
522
+ "save_steps": 500,
523
+ "stateful_callbacks": {
524
+ "TrainerControl": {
525
+ "args": {
526
+ "should_epoch_stop": false,
527
+ "should_evaluate": false,
528
+ "should_log": false,
529
+ "should_save": true,
530
+ "should_training_stop": true
531
+ },
532
+ "attributes": {}
533
+ }
534
+ },
535
+ "total_flos": 1.2726481677031834e+17,
536
+ "train_batch_size": 1,
537
+ "trial_name": null,
538
+ "trial_params": null
539
+ }