Hugging Face's logo

a1024053774
/
rl_course_vizdoom_health_gathering_supreme

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files
xet
Metrics Community
a1024053774 commited on
Commit
8ef344d
·
verified ·
1 Parent(s): 011f9b5

Upload folder using huggingface_hub

Browse files
Files changed (10) hide show
  1. .gitattributes +1 -0
  2. .summary/0/events.out.tfevents.1756031555.LuckyE +3 -0
  3. README.md +56 -0
  4. checkpoint_p0/best_000000290_1187840_reward_4.752.pth +3 -0
  5. checkpoint_p0/checkpoint_000000760_3112960.pth +3 -0
  6. checkpoint_p0/checkpoint_000000978_4005888.pth +3 -0
  7. config.json +142 -0
  8. git.diff +209 -0
  9. replay.mp4 +3 -0
  10. sf_log.txt +971 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1756031555.LuckyE ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ca457ca4550aa4877a494d9fc267ea29fafbc011629eb7d649d773936e2c745
3
+ size 431539
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: doom_health_gathering_supreme
15
+ type: doom_health_gathering_supreme
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 3.94 +/- 0.20
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ A(n) **APPO** model trained on the **doom_health_gathering_supreme** environment.
24
+
25
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
26
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
27
+
28
+
29
+ ## Downloading the model
30
+
31
+ After installing Sample-Factory, download the model with:
32
+ ```
33
+ python -m sample_factory.huggingface.load_from_hub -r a1024053774/rl_course_vizdoom_health_gathering_supreme
34
+ ```
35
+
36
+
37
+ ## Using the model
38
+
39
+ To run the model after download, use the `enjoy` script corresponding to this environment:
40
+ ```
41
+ python -m <path.to.enjoy.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme
42
+ ```
43
+
44
+
45
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
46
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
47
+
48
+ ## Training with this model
49
+
50
+ To continue training with this model, use the `train` script corresponding to this environment:
51
+ ```
52
+ python -m <path.to.train.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme --restart_behavior=resume --train_for_env_steps=10000000000
53
+ ```
54
+
55
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
56
+
checkpoint_p0/best_000000290_1187840_reward_4.752.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab4d619c55c6edce4fcbe2411f70da37dd13db3cc2262c65bf03c7b6a283ca08
3
+ size 34929516
checkpoint_p0/checkpoint_000000760_3112960.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a36ead2a6276c2493b0920039b5d39584070a15ba4d3e04aab7e4894da9186fd
3
+ size 34929954
checkpoint_p0/checkpoint_000000978_4005888.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28384ae2f280107259038516e8d7b8a95bf73d167736c353306e26c975f91f57
3
+ size 34929954
config.json ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "help": false,
3
+ "algo": "APPO",
4
+ "env": "doom_health_gathering_supreme",
5
+ "experiment": "default_experiment",
6
+ "train_dir": "/home/luckye/_DevelopmentCode/RL_Boot/train_dir",
7
+ "restart_behavior": "resume",
8
+ "device": "gpu",
9
+ "seed": null,
10
+ "num_policies": 1,
11
+ "async_rl": true,
12
+ "serial_mode": false,
13
+ "batched_sampling": false,
14
+ "num_batches_to_accumulate": 2,
15
+ "worker_num_splits": 2,
16
+ "policy_workers_per_policy": 1,
17
+ "max_policy_lag": 1000,
18
+ "num_workers": 8,
19
+ "num_envs_per_worker": 4,
20
+ "batch_size": 1024,
21
+ "num_batches_per_epoch": 1,
22
+ "num_epochs": 1,
23
+ "rollout": 32,
24
+ "recurrence": 32,
25
+ "shuffle_minibatches": false,
26
+ "gamma": 0.99,
27
+ "reward_scale": 1.0,
28
+ "reward_clip": 1000.0,
29
+ "value_bootstrap": false,
30
+ "normalize_returns": true,
31
+ "exploration_loss_coeff": 0.001,
32
+ "value_loss_coeff": 0.5,
33
+ "kl_loss_coeff": 0.0,
34
+ "exploration_loss": "symmetric_kl",
35
+ "gae_lambda": 0.95,
36
+ "ppo_clip_ratio": 0.1,
37
+ "ppo_clip_value": 0.2,
38
+ "with_vtrace": false,
39
+ "vtrace_rho": 1.0,
40
+ "vtrace_c": 1.0,
41
+ "optimizer": "adam",
42
+ "adam_eps": 1e-06,
43
+ "adam_beta1": 0.9,
44
+ "adam_beta2": 0.999,
45
+ "max_grad_norm": 4.0,
46
+ "learning_rate": 0.0001,
47
+ "lr_schedule": "constant",
48
+ "lr_schedule_kl_threshold": 0.008,
49
+ "lr_adaptive_min": 1e-06,
50
+ "lr_adaptive_max": 0.01,
51
+ "obs_subtract_mean": 0.0,
52
+ "obs_scale": 255.0,
53
+ "normalize_input": true,
54
+ "normalize_input_keys": null,
55
+ "decorrelate_experience_max_seconds": 0,
56
+ "decorrelate_envs_on_one_worker": true,
57
+ "actor_worker_gpus": [],
58
+ "set_workers_cpu_affinity": true,
59
+ "force_envs_single_thread": false,
60
+ "default_niceness": 0,
61
+ "log_to_file": true,
62
+ "experiment_summaries_interval": 10,
63
+ "flush_summaries_interval": 30,
64
+ "stats_avg": 100,
65
+ "summaries_use_frameskip": true,
66
+ "heartbeat_interval": 20,
67
+ "heartbeat_reporting_interval": 600,
68
+ "train_for_env_steps": 4000000,
69
+ "train_for_seconds": 10000000000,
70
+ "save_every_sec": 120,
71
+ "keep_checkpoints": 2,
72
+ "load_checkpoint_kind": "latest",
73
+ "save_milestones_sec": -1,
74
+ "save_best_every_sec": 5,
75
+ "save_best_metric": "reward",
76
+ "save_best_after": 100000,
77
+ "benchmark": false,
78
+ "encoder_mlp_layers": [
79
+ 512,
80
+ 512
81
+ ],
82
+ "encoder_conv_architecture": "convnet_simple",
83
+ "encoder_conv_mlp_layers": [
84
+ 512
85
+ ],
86
+ "use_rnn": true,
87
+ "rnn_size": 512,
88
+ "rnn_type": "gru",
89
+ "rnn_num_layers": 1,
90
+ "decoder_mlp_layers": [],
91
+ "nonlinearity": "elu",
92
+ "policy_initialization": "orthogonal",
93
+ "policy_init_gain": 1.0,
94
+ "actor_critic_share_weights": true,
95
+ "adaptive_stddev": true,
96
+ "continuous_tanh_scale": 0.0,
97
+ "initial_stddev": 1.0,
98
+ "use_env_info_cache": false,
99
+ "env_gpu_actions": false,
100
+ "env_gpu_observations": true,
101
+ "env_frameskip": 4,
102
+ "env_framestack": 1,
103
+ "pixel_format": "CHW",
104
+ "use_record_episode_statistics": false,
105
+ "with_wandb": false,
106
+ "wandb_user": null,
107
+ "wandb_project": "sample_factory",
108
+ "wandb_group": null,
109
+ "wandb_job_type": "SF",
110
+ "wandb_tags": [],
111
+ "with_pbt": false,
112
+ "pbt_mix_policies_in_one_env": true,
113
+ "pbt_period_env_steps": 5000000,
114
+ "pbt_start_mutation": 20000000,
115
+ "pbt_replace_fraction": 0.3,
116
+ "pbt_mutation_rate": 0.15,
117
+ "pbt_replace_reward_gap": 0.1,
118
+ "pbt_replace_reward_gap_absolute": 1e-06,
119
+ "pbt_optimize_gamma": false,
120
+ "pbt_target_objective": "true_objective",
121
+ "pbt_perturb_min": 1.1,
122
+ "pbt_perturb_max": 1.5,
123
+ "num_agents": -1,
124
+ "num_humans": 0,
125
+ "num_bots": -1,
126
+ "start_bot_difficulty": null,
127
+ "timelimit": null,
128
+ "res_w": 128,
129
+ "res_h": 72,
130
+ "wide_aspect_ratio": false,
131
+ "eval_env_frameskip": 1,
132
+ "fps": 35,
133
+ "command_line": "--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000",
134
+ "cli_args": {
135
+ "env": "doom_health_gathering_supreme",
136
+ "num_workers": 8,
137
+ "num_envs_per_worker": 4,
138
+ "train_for_env_steps": 4000000
139
+ },
140
+ "git_hash": "d8b66389a2d55c2e8ac16cf6f769f9a366ee5a94",
141
+ "git_repo_name": "https://github.com/a1024053774/RL_Boot.git"
142
+ }
git.diff ADDED
@@ -0,0 +1,209 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ diff --git a/Hugging_Face/notebooks/unit8-part2-Doom/unit8_part2.ipynb b/Hugging_Face/notebooks/unit8-part2-Doom/unit8_part2.ipynb
2
+ index bda29bc..562cfe3 100644
3
+ --- a/Hugging_Face/notebooks/unit8-part2-Doom/unit8_part2.ipynb
4
+ +++ b/Hugging_Face/notebooks/unit8-part2-Doom/unit8_part2.ipynb
5
+ @@ -240,14 +240,14 @@
6
+ },
7
+ {
8
+ "cell_type": "code",
9
+ - "source": [
10
+ - "!pip install sample-factory==2.1.1"
11
+ - ],
12
+ + "execution_count": null,
13
+ "metadata": {
14
+ "id": "alxUt7Au-O8e"
15
+ },
16
+ - "execution_count": null,
17
+ - "outputs": []
18
+ + "outputs": [],
19
+ + "source": [
20
+ + "!pip install sample-factory==2.1.1"
21
+ + ]
22
+ },
23
+ {
24
+ "cell_type": "markdown",
25
+ @@ -260,7 +260,7 @@
26
+ },
27
+ {
28
+ "cell_type": "code",
29
+ - "execution_count": null,
30
+ + "execution_count": 1,
31
+ "metadata": {
32
+ "id": "bCgZbeiavcDU"
33
+ },
34
+ @@ -352,7 +352,24 @@
35
+ "metadata": {
36
+ "id": "y_TeicMvyKHP"
37
+ },
38
+ - "outputs": [],
39
+ + "outputs": [
40
+ + {
41
+ + "name": "stderr",
42
+ + "output_type": "stream",
43
+ + "text": [
44
+ + "\u001b[36m[2025-08-24 18:32:35,257][22314] register_encoder_factory: <function make_vizdoom_encoder at 0x7fed145c0220>\u001b[0m\n",
45
+ + "\u001b[33m[2025-08-24 18:32:35,271][22314] Saved parameter configuration for experiment default_experiment not found!\u001b[0m\n",
46
+ + "\u001b[33m[2025-08-24 18:32:35,273][22314] Starting experiment from scratch!\u001b[0m\n",
47
+ + "\u001b[36m[2025-08-24 18:32:35,283][22314] Experiment dir /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment already exists!\u001b[0m\n",
48
+ + "\u001b[36m[2025-08-24 18:32:35,284][22314] Resuming existing experiment from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment...\u001b[0m\n",
49
+ + "\u001b[36m[2025-08-24 18:32:35,285][22314] Weights and Biases integration disabled\u001b[0m\n",
50
+ + "\u001b[36m[2025-08-24 18:32:38,518][22314] Queried available GPUs: 0\n",
51
+ + "\u001b[0m\n",
52
+ + "\u001b[37m\u001b[1m[2025-08-24 18:32:38,519][22314] Environment var CUDA_VISIBLE_DEVICES is 0\n",
53
+ + "\u001b[0m\n"
54
+ + ]
55
+ + }
56
+ + ],
57
+ "source": [
58
+ "## 开始训练，这大约需要 15 分钟\n",
59
+ "register_vizdoom_components()\n",
60
+ @@ -418,12 +435,12 @@
61
+ },
62
+ {
63
+ "cell_type": "markdown",
64
+ - "source": [
65
+ - "智能体学到了一些东西，但它的表现还可以更好。我们显然需要训练更长的时间。不过，让我们先把这个模型上传到 Hub。"
66
+ - ],
67
+ "metadata": {
68
+ "id": "2A4pf_1VwPqR"
69
+ - }
70
+ + },
71
+ + "source": [
72
+ + "智能体学到了一些东西，但它的表现还可以更好。我们显然需要训练更长的时间。不过，让我们先把这个模型上传到 Hub。"
73
+ + ]
74
+ },
75
+ {
76
+ "cell_type": "markdown",
77
+ @@ -495,14 +512,14 @@
78
+ },
79
+ {
80
+ "cell_type": "markdown",
81
+ + "metadata": {
82
+ + "id": "9PzeXx-qxVvw"
83
+ + },
84
+ "source": [
85
+ "## 让我们加载另一个模型\n",
86
+ "\n",
87
+ "\n"
88
+ - ],
89
+ - "metadata": {
90
+ - "id": "9PzeXx-qxVvw"
91
+ - }
92
+ + ]
93
+ },
94
+ {
95
+ "cell_type": "markdown",
96
+ @@ -568,16 +585,16 @@
97
+ },
98
+ {
99
+ "cell_type": "markdown",
100
+ + "metadata": {
101
+ + "id": "ie5YWC3NyKO8"
102
+ + },
103
+ "source": [
104
+ "## 一些额外挑战 🏆：Doom 死亡竞赛\n",
105
+ "\n",
106
+ "训练一个能玩 Doom 死亡竞赛的智能体**需要在一台比 Colab 中可用机器配置更强的机器上花费数小时**。\n",
107
+ "\n",
108
+ "幸运的是，我们**已经在这个场景中训练好了一个智能体，并且它可以在 🤗 Hub 上找到！** 让我们下载模型并可视化智能体的表现。"
109
+ - ],
110
+ - "metadata": {
111
+ - "id": "ie5YWC3NyKO8"
112
+ - }
113
+ + ]
114
+ },
115
+ {
116
+ "cell_type": "code",
117
+ @@ -593,12 +610,12 @@
118
+ },
119
+ {
120
+ "cell_type": "markdown",
121
+ - "source": [
122
+ - "由于智能体游戏时间很长，视频生成可能需要**10分钟**。"
123
+ - ],
124
+ "metadata": {
125
+ "id": "7AX_LwxR2FQ0"
126
+ - }
127
+ + },
128
+ + "source": [
129
+ + "由于智能体游戏时间很长，视频生成可能需要**10分钟**。"
130
+ + ]
131
+ },
132
+ {
133
+ "cell_type": "code",
134
+ @@ -625,17 +642,20 @@
135
+ },
136
+ {
137
+ "cell_type": "markdown",
138
+ + "metadata": {
139
+ + "id": "N6mEC-4zyihx"
140
+ + },
141
+ "source": [
142
+ "\n",
143
+ "您**可以尝试使用上面的代码在这个环境中训练您的智能体**，但不能在 Colab 上进行。\n",
144
+ "**祝您好运 🤞**"
145
+ - ],
146
+ - "metadata": {
147
+ - "id": "N6mEC-4zyihx"
148
+ - }
149
+ + ]
150
+ },
151
+ {
152
+ "cell_type": "markdown",
153
+ + "metadata": {
154
+ + "id": "YnDAngN6zeeI"
155
+ + },
156
+ "source": [
157
+ "如果您喜欢更简单的场景，**为什么不尝试在另一个 ViZDoom 场景中进行训练，例如 `doom_deadly_corridor` 或 `doom_defend_the_center`。**\n",
158
+ "\n",
159
+ @@ -647,33 +667,40 @@
160
+ "最后一个单元到此结束。但我们还没有完！ 🤗 接下来的**附加部分将包含深度强化学习中一些最有趣、最前沿和最尖端的工作**。\n",
161
+ "\n",
162
+ "## 保持学习，保持出色 🤗"
163
+ - ],
164
+ - "metadata": {
165
+ - "id": "YnDAngN6zeeI"
166
+ - }
167
+ + ]
168
+ }
169
+ ],
170
+ "metadata": {
171
+ "accelerator": "GPU",
172
+ "colab": {
173
+ - "provenance": [],
174
+ "collapsed_sections": [
175
+ "PU4FVzaoM6fC",
176
+ "nB68Eb9UgC94",
177
+ "ez5UhUtYcWXF",
178
+ "sgRy6wnrgnij"
179
+ ],
180
+ - "private_outputs": true
181
+ + "private_outputs": true,
182
+ + "provenance": []
183
+ },
184
+ "gpuClass": "standard",
185
+ "kernelspec": {
186
+ - "display_name": "Python 3",
187
+ + "display_name": "Python 3 (ipykernel)",
188
+ + "language": "python",
189
+ "name": "python3"
190
+ },
191
+ "language_info": {
192
+ - "name": "python"
193
+ + "codemirror_mode": {
194
+ + "name": "ipython",
195
+ + "version": 3
196
+ + },
197
+ + "file_extension": ".py",
198
+ + "mimetype": "text/x-python",
199
+ + "name": "python",
200
+ + "nbconvert_exporter": "python",
201
+ + "pygments_lexer": "ipython3",
202
+ + "version": "3.12.3"
203
+ }
204
+ },
205
+ "nbformat": 4,
206
+ "nbformat_minor": 0
207
+ -}
208
+
209
+ +}
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eab27ed76fe11833fa51d56932c7077374a10ee8f83bf07fbcc7e01a3d04d125
3
+ size 5304423
sf_log.txt ADDED
@@ -0,0 +1,971 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2025-08-24 18:32:40,982][22314] Saving configuration to /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json...
2
+ [2025-08-24 18:32:41,149][22314] Rollout worker 0 uses device cpu
3
+ [2025-08-24 18:32:41,150][22314] Rollout worker 1 uses device cpu
4
+ [2025-08-24 18:32:41,152][22314] Rollout worker 2 uses device cpu
5
+ [2025-08-24 18:32:41,152][22314] Rollout worker 3 uses device cpu
6
+ [2025-08-24 18:32:41,153][22314] Rollout worker 4 uses device cpu
7
+ [2025-08-24 18:32:41,154][22314] Rollout worker 5 uses device cpu
8
+ [2025-08-24 18:32:41,155][22314] Rollout worker 6 uses device cpu
9
+ [2025-08-24 18:32:41,155][22314] Rollout worker 7 uses device cpu
10
+ [2025-08-24 18:32:41,207][22314] Using GPUs [0] for process 0 (actually maps to GPUs [0])
11
+ [2025-08-24 18:32:41,208][22314] InferenceWorker_p0-w0: min num requests: 2
12
+ [2025-08-24 18:32:41,235][22314] Starting all processes...
13
+ [2025-08-24 18:32:41,236][22314] Starting process learner_proc0
14
+ [2025-08-24 18:32:41,280][22314] Starting all processes...
15
+ [2025-08-24 18:32:41,288][22314] Starting process inference_proc0-0
16
+ [2025-08-24 18:32:41,288][22314] Starting process rollout_proc0
17
+ [2025-08-24 18:32:41,289][22314] Starting process rollout_proc1
18
+ [2025-08-24 18:32:41,290][22314] Starting process rollout_proc2
19
+ [2025-08-24 18:32:41,290][22314] Starting process rollout_proc3
20
+ [2025-08-24 18:32:41,290][22314] Starting process rollout_proc4
21
+ [2025-08-24 18:32:41,291][22314] Starting process rollout_proc5
22
+ [2025-08-24 18:32:41,291][22314] Starting process rollout_proc6
23
+ [2025-08-24 18:32:41,291][22314] Starting process rollout_proc7
24
+ [2025-08-24 18:32:44,927][22801] Using GPUs [0] for process 0 (actually maps to GPUs [0])
25
+ [2025-08-24 18:32:44,928][22801] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
26
+ [2025-08-24 18:32:44,995][22814] Using GPUs [0] for process 0 (actually maps to GPUs [0])
27
+ [2025-08-24 18:32:44,996][22814] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
28
+ [2025-08-24 18:32:45,043][22801] Num visible devices: 1
29
+ [2025-08-24 18:32:45,043][22814] Num visible devices: 1
30
+ [2025-08-24 18:32:45,046][22801] Starting seed is not provided
31
+ [2025-08-24 18:32:45,047][22801] Using GPUs [0] for process 0 (actually maps to GPUs [0])
32
+ [2025-08-24 18:32:45,047][22801] Initializing actor-critic model on device cuda:0
33
+ [2025-08-24 18:32:45,047][22815] Worker 0 uses CPU cores [0, 1]
34
+ [2025-08-24 18:32:45,048][22801] RunningMeanStd input shape: (3, 72, 128)
35
+ [2025-08-24 18:32:45,050][22801] RunningMeanStd input shape: (1,)
36
+ [2025-08-24 18:32:45,060][22818] Worker 3 uses CPU cores [6, 7]
37
+ [2025-08-24 18:32:45,064][22801] ConvEncoder: input_channels=3
38
+ [2025-08-24 18:32:45,133][22835] Worker 5 uses CPU cores [10, 11]
39
+ [2025-08-24 18:32:45,133][22837] Worker 6 uses CPU cores [12, 13]
40
+ [2025-08-24 18:32:45,164][22816] Worker 1 uses CPU cores [2, 3]
41
+ [2025-08-24 18:32:45,219][22838] Worker 7 uses CPU cores [14, 15]
42
+ [2025-08-24 18:32:45,246][22834] Worker 4 uses CPU cores [8, 9]
43
+ [2025-08-24 18:32:45,247][22817] Worker 2 uses CPU cores [4, 5]
44
+ [2025-08-24 18:32:45,262][22801] Conv encoder output size: 512
45
+ [2025-08-24 18:32:45,262][22801] Policy head output size: 512
46
+ [2025-08-24 18:32:45,287][22801] Created Actor Critic model with architecture:
47
+ [2025-08-24 18:32:45,287][22801] ActorCriticSharedWeights(
48
+ (obs_normalizer): ObservationNormalizer(
49
+ (running_mean_std): RunningMeanStdDictInPlace(
50
+ (running_mean_std): ModuleDict(
51
+ (obs): RunningMeanStdInPlace()
52
+ )
53
+ )
54
+ )
55
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
56
+ (encoder): VizdoomEncoder(
57
+ (basic_encoder): ConvEncoder(
58
+ (enc): RecursiveScriptModule(
59
+ original_name=ConvEncoderImpl
60
+ (conv_head): RecursiveScriptModule(
61
+ original_name=Sequential
62
+ (0): RecursiveScriptModule(original_name=Conv2d)
63
+ (1): RecursiveScriptModule(original_name=ELU)
64
+ (2): RecursiveScriptModule(original_name=Conv2d)
65
+ (3): RecursiveScriptModule(original_name=ELU)
66
+ (4): RecursiveScriptModule(original_name=Conv2d)
67
+ (5): RecursiveScriptModule(original_name=ELU)
68
+ )
69
+ (mlp_layers): RecursiveScriptModule(
70
+ original_name=Sequential
71
+ (0): RecursiveScriptModule(original_name=Linear)
72
+ (1): RecursiveScriptModule(original_name=ELU)
73
+ )
74
+ )
75
+ )
76
+ )
77
+ (core): ModelCoreRNN(
78
+ (core): GRU(512, 512)
79
+ )
80
+ (decoder): MlpDecoder(
81
+ (mlp): Identity()
82
+ )
83
+ (critic_linear): Linear(in_features=512, out_features=1, bias=True)
84
+ (action_parameterization): ActionParameterizationDefault(
85
+ (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
86
+ )
87
+ )
88
+ [2025-08-24 18:32:46,126][22801] Using optimizer <class 'torch.optim.adam.Adam'>
89
+ [2025-08-24 18:32:47,722][22801] No checkpoints found
90
+ [2025-08-24 18:32:47,722][22801] Did not load from checkpoint, starting from scratch!
91
+ [2025-08-24 18:32:47,723][22801] Initialized policy 0 weights for model version 0
92
+ [2025-08-24 18:32:47,731][22801] LearnerWorker_p0 finished initialization!
93
+ [2025-08-24 18:32:47,731][22801] Using GPUs [0] for process 0 (actually maps to GPUs [0])
94
+ [2025-08-24 18:32:47,951][22814] RunningMeanStd input shape: (3, 72, 128)
95
+ [2025-08-24 18:32:47,952][22814] RunningMeanStd input shape: (1,)
96
+ [2025-08-24 18:32:47,963][22814] ConvEncoder: input_channels=3
97
+ [2025-08-24 18:32:48,057][22814] Conv encoder output size: 512
98
+ [2025-08-24 18:32:48,058][22814] Policy head output size: 512
99
+ [2025-08-24 18:32:48,112][22314] Inference worker 0-0 is ready!
100
+ [2025-08-24 18:32:48,114][22314] All inference workers are ready! Signal rollout workers to start!
101
+ [2025-08-24 18:32:48,171][22815] Doom resolution: 160x120, resize resolution: (128, 72)
102
+ [2025-08-24 18:32:48,174][22818] Doom resolution: 160x120, resize resolution: (128, 72)
103
+ [2025-08-24 18:32:48,174][22838] Doom resolution: 160x120, resize resolution: (128, 72)
104
+ [2025-08-24 18:32:48,174][22817] Doom resolution: 160x120, resize resolution: (128, 72)
105
+ [2025-08-24 18:32:48,175][22837] Doom resolution: 160x120, resize resolution: (128, 72)
106
+ [2025-08-24 18:32:48,176][22816] Doom resolution: 160x120, resize resolution: (128, 72)
107
+ [2025-08-24 18:32:48,176][22835] Doom resolution: 160x120, resize resolution: (128, 72)
108
+ [2025-08-24 18:32:48,177][22834] Doom resolution: 160x120, resize resolution: (128, 72)
109
+ [2025-08-24 18:32:48,623][22816] Decorrelating experience for 0 frames...
110
+ [2025-08-24 18:32:48,623][22817] Decorrelating experience for 0 frames...
111
+ [2025-08-24 18:32:48,623][22818] Decorrelating experience for 0 frames...
112
+ [2025-08-24 18:32:48,623][22815] Decorrelating experience for 0 frames...
113
+ [2025-08-24 18:32:48,627][22835] Decorrelating experience for 0 frames...
114
+ [2025-08-24 18:32:48,630][22837] Decorrelating experience for 0 frames...
115
+ [2025-08-24 18:32:48,811][22816] Decorrelating experience for 32 frames...
116
+ [2025-08-24 18:32:48,828][22818] Decorrelating experience for 32 frames...
117
+ [2025-08-24 18:32:48,846][22838] Decorrelating experience for 0 frames...
118
+ [2025-08-24 18:32:49,067][22834] Decorrelating experience for 0 frames...
119
+ [2025-08-24 18:32:49,076][22835] Decorrelating experience for 32 frames...
120
+ [2025-08-24 18:32:49,087][22838] Decorrelating experience for 32 frames...
121
+ [2025-08-24 18:32:49,145][22816] Decorrelating experience for 64 frames...
122
+ [2025-08-24 18:32:49,317][22834] Decorrelating experience for 32 frames...
123
+ [2025-08-24 18:32:49,329][22817] Decorrelating experience for 32 frames...
124
+ [2025-08-24 18:32:49,423][22818] Decorrelating experience for 64 frames...
125
+ [2025-08-24 18:32:49,469][22816] Decorrelating experience for 96 frames...
126
+ [2025-08-24 18:32:49,571][22815] Decorrelating experience for 32 frames...
127
+ [2025-08-24 18:32:49,598][22835] Decorrelating experience for 64 frames...
128
+ [2025-08-24 18:32:49,662][22834] Decorrelating experience for 64 frames...
129
+ [2025-08-24 18:32:49,743][22818] Decorrelating experience for 96 frames...
130
+ [2025-08-24 18:32:49,830][22837] Decorrelating experience for 32 frames...
131
+ [2025-08-24 18:32:49,930][22835] Decorrelating experience for 96 frames...
132
+ [2025-08-24 18:32:49,933][22815] Decorrelating experience for 64 frames...
133
+ [2025-08-24 18:32:50,097][22834] Decorrelating experience for 96 frames...
134
+ [2025-08-24 18:32:50,208][22837] Decorrelating experience for 64 frames...
135
+ [2025-08-24 18:32:50,251][22815] Decorrelating experience for 96 frames...
136
+ [2025-08-24 18:32:50,292][22314] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
137
+ [2025-08-24 18:32:50,331][22817] Decorrelating experience for 64 frames...
138
+ [2025-08-24 18:32:50,572][22838] Decorrelating experience for 64 frames...
139
+ [2025-08-24 18:32:50,592][22817] Decorrelating experience for 96 frames...
140
+ [2025-08-24 18:32:50,815][22837] Decorrelating experience for 96 frames...
141
+ [2025-08-24 18:32:50,882][22838] Decorrelating experience for 96 frames...
142
+ [2025-08-24 18:32:51,860][22801] Signal inference workers to stop experience collection...
143
+ [2025-08-24 18:32:51,868][22814] InferenceWorker_p0-w0: stopping experience collection
144
+ [2025-08-24 18:32:53,563][22801] Signal inference workers to resume experience collection...
145
+ [2025-08-24 18:32:53,564][22814] InferenceWorker_p0-w0: resuming experience collection
146
+ [2025-08-24 18:32:55,292][22314] Fps is (10 sec: 4915.1, 60 sec: 4915.1, 300 sec: 4915.1). Total num frames: 24576. Throughput: 0: 798.4. Samples: 3992. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
147
+ [2025-08-24 18:32:55,294][22314] Avg episode reward: [(0, '3.567')]
148
+ [2025-08-24 18:32:56,784][22814] Updated weights for policy 0, policy_version 10 (0.0017)
149
+ [2025-08-24 18:33:00,292][22314] Fps is (10 sec: 7782.4, 60 sec: 7782.4, 300 sec: 7782.4). Total num frames: 77824. Throughput: 0: 2010.2. Samples: 20102. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
150
+ [2025-08-24 18:33:00,293][22314] Avg episode reward: [(0, '4.637')]
151
+ [2025-08-24 18:33:00,503][22814] Updated weights for policy 0, policy_version 20 (0.0018)
152
+ [2025-08-24 18:33:01,201][22314] Heartbeat connected on Batcher_0
153
+ [2025-08-24 18:33:01,204][22314] Heartbeat connected on LearnerWorker_p0
154
+ [2025-08-24 18:33:01,213][22314] Heartbeat connected on RolloutWorker_w0
155
+ [2025-08-24 18:33:01,215][22314] Heartbeat connected on InferenceWorker_p0-w0
156
+ [2025-08-24 18:33:01,219][22314] Heartbeat connected on RolloutWorker_w2
157
+ [2025-08-24 18:33:01,221][22314] Heartbeat connected on RolloutWorker_w1
158
+ [2025-08-24 18:33:01,225][22314] Heartbeat connected on RolloutWorker_w3
159
+ [2025-08-24 18:33:01,226][22314] Heartbeat connected on RolloutWorker_w4
160
+ [2025-08-24 18:33:01,232][22314] Heartbeat connected on RolloutWorker_w6
161
+ [2025-08-24 18:33:01,234][22314] Heartbeat connected on RolloutWorker_w5
162
+ [2025-08-24 18:33:01,239][22314] Heartbeat connected on RolloutWorker_w7
163
+ [2025-08-24 18:33:05,123][22814] Updated weights for policy 0, policy_version 30 (0.0025)
164
+ [2025-08-24 18:33:05,292][22314] Fps is (10 sec: 9830.6, 60 sec: 8192.0, 300 sec: 8192.0). Total num frames: 122880. Throughput: 0: 1823.3. Samples: 27350. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
165
+ [2025-08-24 18:33:05,294][22314] Avg episode reward: [(0, '4.497')]
166
+ [2025-08-24 18:33:05,302][22801] Saving new best policy, reward=4.497!
167
+ [2025-08-24 18:33:10,291][22814] Updated weights for policy 0, policy_version 40 (0.0034)
168
+ [2025-08-24 18:33:10,292][22314] Fps is (10 sec: 8601.6, 60 sec: 8192.0, 300 sec: 8192.0). Total num frames: 163840. Throughput: 0: 1976.4. Samples: 39528. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
169
+ [2025-08-24 18:33:10,294][22314] Avg episode reward: [(0, '4.477')]
170
+ [2025-08-24 18:33:15,292][22314] Fps is (10 sec: 7372.7, 60 sec: 7864.3, 300 sec: 7864.3). Total num frames: 196608. Throughput: 0: 1985.0. Samples: 49626. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
171
+ [2025-08-24 18:33:15,294][22314] Avg episode reward: [(0, '4.510')]
172
+ [2025-08-24 18:33:15,304][22801] Saving new best policy, reward=4.510!
173
+ [2025-08-24 18:33:16,440][22814] Updated weights for policy 0, policy_version 50 (0.0041)
174
+ [2025-08-24 18:33:20,292][22314] Fps is (10 sec: 6553.6, 60 sec: 7645.9, 300 sec: 7645.9). Total num frames: 229376. Throughput: 0: 1825.2. Samples: 54756. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
175
+ [2025-08-24 18:33:20,294][22314] Avg episode reward: [(0, '4.691')]
176
+ [2025-08-24 18:33:20,297][22801] Saving new best policy, reward=4.691!
177
+ [2025-08-24 18:33:22,191][22814] Updated weights for policy 0, policy_version 60 (0.0026)
178
+ [2025-08-24 18:33:25,292][22314] Fps is (10 sec: 6963.2, 60 sec: 7606.8, 300 sec: 7606.8). Total num frames: 266240. Throughput: 0: 1864.3. Samples: 65252. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
179
+ [2025-08-24 18:33:25,294][22314] Avg episode reward: [(0, '4.740')]
180
+ [2025-08-24 18:33:25,302][22801] Saving new best policy, reward=4.740!
181
+ [2025-08-24 18:33:28,019][22814] Updated weights for policy 0, policy_version 70 (0.0023)
182
+ [2025-08-24 18:33:30,292][22314] Fps is (10 sec: 7372.8, 60 sec: 7577.6, 300 sec: 7577.6). Total num frames: 303104. Throughput: 0: 1920.9. Samples: 76836. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
183
+ [2025-08-24 18:33:30,294][22314] Avg episode reward: [(0, '4.530')]
184
+ [2025-08-24 18:33:33,059][22814] Updated weights for policy 0, policy_version 80 (0.0027)
185
+ [2025-08-24 18:33:35,292][22314] Fps is (10 sec: 7782.4, 60 sec: 7645.9, 300 sec: 7645.9). Total num frames: 344064. Throughput: 0: 1839.0. Samples: 82754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
186
+ [2025-08-24 18:33:35,294][22314] Avg episode reward: [(0, '4.332')]
187
+ [2025-08-24 18:33:38,049][22814] Updated weights for policy 0, policy_version 90 (0.0025)
188
+ [2025-08-24 18:33:40,292][22314] Fps is (10 sec: 8191.9, 60 sec: 7700.5, 300 sec: 7700.5). Total num frames: 385024. Throughput: 0: 2029.2. Samples: 95308. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
189
+ [2025-08-24 18:33:40,294][22314] Avg episode reward: [(0, '4.402')]
190
+ [2025-08-24 18:33:42,709][22814] Updated weights for policy 0, policy_version 100 (0.0028)
191
+ [2025-08-24 18:33:45,293][22314] Fps is (10 sec: 8600.4, 60 sec: 7819.4, 300 sec: 7819.4). Total num frames: 430080. Throughput: 0: 1964.7. Samples: 108518. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
192
+ [2025-08-24 18:33:45,296][22314] Avg episode reward: [(0, '4.524')]
193
+ [2025-08-24 18:33:47,055][22814] Updated weights for policy 0, policy_version 110 (0.0024)
194
+ [2025-08-24 18:33:50,292][22314] Fps is (10 sec: 9420.8, 60 sec: 7987.2, 300 sec: 7987.2). Total num frames: 479232. Throughput: 0: 1969.6. Samples: 115982. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
195
+ [2025-08-24 18:33:50,294][22314] Avg episode reward: [(0, '4.392')]
196
+ [2025-08-24 18:33:51,247][22814] Updated weights for policy 0, policy_version 120 (0.0022)
197
+ [2025-08-24 18:33:55,292][22314] Fps is (10 sec: 9422.2, 60 sec: 8328.6, 300 sec: 8066.0). Total num frames: 524288. Throughput: 0: 2008.2. Samples: 129898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
198
+ [2025-08-24 18:33:55,293][22314] Avg episode reward: [(0, '4.456')]
199
+ [2025-08-24 18:33:55,866][22814] Updated weights for policy 0, policy_version 130 (0.0027)
200
+ [2025-08-24 18:34:00,223][22814] Updated weights for policy 0, policy_version 140 (0.0027)
201
+ [2025-08-24 18:34:00,292][22314] Fps is (10 sec: 9420.9, 60 sec: 8260.3, 300 sec: 8192.0). Total num frames: 573440. Throughput: 0: 2094.8. Samples: 143890. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
202
+ [2025-08-24 18:34:00,293][22314] Avg episode reward: [(0, '4.418')]
203
+ [2025-08-24 18:34:05,075][22814] Updated weights for policy 0, policy_version 150 (0.0024)
204
+ [2025-08-24 18:34:05,292][22314] Fps is (10 sec: 9011.1, 60 sec: 8192.0, 300 sec: 8192.0). Total num frames: 614400. Throughput: 0: 2123.7. Samples: 150322. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
205
+ [2025-08-24 18:34:05,294][22314] Avg episode reward: [(0, '4.321')]
206
+ [2025-08-24 18:34:10,292][22314] Fps is (10 sec: 7782.5, 60 sec: 8123.7, 300 sec: 8140.8). Total num frames: 651264. Throughput: 0: 2136.0. Samples: 161372. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
207
+ [2025-08-24 18:34:10,293][22314] Avg episode reward: [(0, '4.659')]
208
+ [2025-08-24 18:34:10,596][22814] Updated weights for policy 0, policy_version 160 (0.0024)
209
+ [2025-08-24 18:34:15,292][22314] Fps is (10 sec: 7782.2, 60 sec: 8260.2, 300 sec: 8143.8). Total num frames: 692224. Throughput: 0: 2160.3. Samples: 174050. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
210
+ [2025-08-24 18:34:15,294][22314] Avg episode reward: [(0, '4.358')]
211
+ [2025-08-24 18:34:15,436][22814] Updated weights for policy 0, policy_version 170 (0.0029)
212
+ [2025-08-24 18:34:19,888][22814] Updated weights for policy 0, policy_version 180 (0.0023)
213
+ [2025-08-24 18:34:20,292][22314] Fps is (10 sec: 8601.5, 60 sec: 8465.1, 300 sec: 8192.0). Total num frames: 737280. Throughput: 0: 2180.6. Samples: 180882. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
214
+ [2025-08-24 18:34:20,294][22314] Avg episode reward: [(0, '4.411')]
215
+ [2025-08-24 18:34:25,292][22314] Fps is (10 sec: 8192.1, 60 sec: 8465.1, 300 sec: 8148.9). Total num frames: 774144. Throughput: 0: 2158.4. Samples: 192438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
216
+ [2025-08-24 18:34:25,294][22314] Avg episode reward: [(0, '4.714')]
217
+ [2025-08-24 18:34:25,827][22814] Updated weights for policy 0, policy_version 190 (0.0034)
218
+ [2025-08-24 18:34:30,292][22314] Fps is (10 sec: 6963.3, 60 sec: 8396.8, 300 sec: 8069.1). Total num frames: 806912. Throughput: 0: 2078.0. Samples: 202026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
219
+ [2025-08-24 18:34:30,294][22314] Avg episode reward: [(0, '4.570')]
220
+ [2025-08-24 18:34:31,916][22814] Updated weights for policy 0, policy_version 200 (0.0036)
221
+ [2025-08-24 18:34:35,292][22314] Fps is (10 sec: 6963.3, 60 sec: 8328.5, 300 sec: 8036.0). Total num frames: 843776. Throughput: 0: 2044.7. Samples: 207994. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
222
+ [2025-08-24 18:34:35,294][22314] Avg episode reward: [(0, '4.396')]
223
+ [2025-08-24 18:34:35,363][22801] Saving /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000207_847872.pth...
224
+ [2025-08-24 18:34:36,698][22814] Updated weights for policy 0, policy_version 210 (0.0033)
225
+ [2025-08-24 18:34:40,292][22314] Fps is (10 sec: 8192.0, 60 sec: 8396.8, 300 sec: 8080.3). Total num frames: 888832. Throughput: 0: 2020.2. Samples: 220806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
226
+ [2025-08-24 18:34:40,294][22314] Avg episode reward: [(0, '4.382')]
227
+ [2025-08-24 18:34:41,191][22814] Updated weights for policy 0, policy_version 220 (0.0023)
228
+ [2025-08-24 18:34:45,277][22814] Updated weights for policy 0, policy_version 230 (0.0020)
229
+ [2025-08-24 18:34:45,292][22314] Fps is (10 sec: 9830.3, 60 sec: 8533.5, 300 sec: 8192.0). Total num frames: 942080. Throughput: 0: 2044.0. Samples: 235872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
230
+ [2025-08-24 18:34:45,294][22314] Avg episode reward: [(0, '4.283')]
231
+ [2025-08-24 18:34:50,137][22814] Updated weights for policy 0, policy_version 240 (0.0031)
232
+ [2025-08-24 18:34:50,292][22314] Fps is (10 sec: 9420.9, 60 sec: 8396.8, 300 sec: 8192.0). Total num frames: 983040. Throughput: 0: 2045.3. Samples: 242362. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
233
+ [2025-08-24 18:34:50,293][22314] Avg episode reward: [(0, '4.406')]
234
+ [2025-08-24 18:34:54,607][22814] Updated weights for policy 0, policy_version 250 (0.0019)
235
+ [2025-08-24 18:34:55,292][22314] Fps is (10 sec: 8601.7, 60 sec: 8396.8, 300 sec: 8224.8). Total num frames: 1028096. Throughput: 0: 2094.0. Samples: 255602. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
236
+ [2025-08-24 18:34:55,294][22314] Avg episode reward: [(0, '4.361')]
237
+ [2025-08-24 18:34:58,725][22814] Updated weights for policy 0, policy_version 260 (0.0023)
238
+ [2025-08-24 18:35:00,292][22314] Fps is (10 sec: 9420.7, 60 sec: 8396.8, 300 sec: 8286.5). Total num frames: 1077248. Throughput: 0: 2142.6. Samples: 270468. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
239
+ [2025-08-24 18:35:00,294][22314] Avg episode reward: [(0, '4.408')]
240
+ [2025-08-24 18:35:02,709][22814] Updated weights for policy 0, policy_version 270 (0.0023)
241
+ [2025-08-24 18:35:05,291][22314] Fps is (10 sec: 10240.2, 60 sec: 8601.6, 300 sec: 8374.1). Total num frames: 1130496. Throughput: 0: 2166.1. Samples: 278354. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
242
+ [2025-08-24 18:35:05,293][22314] Avg episode reward: [(0, '4.478')]
243
+ [2025-08-24 18:35:06,461][22814] Updated weights for policy 0, policy_version 280 (0.0020)
244
+ [2025-08-24 18:35:10,292][22314] Fps is (10 sec: 10649.6, 60 sec: 8874.7, 300 sec: 8455.3). Total num frames: 1183744. Throughput: 0: 2266.7. Samples: 294438. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
245
+ [2025-08-24 18:35:10,293][22314] Avg episode reward: [(0, '4.752')]
246
+ [2025-08-24 18:35:10,312][22801] Saving new best policy, reward=4.752!
247
+ [2025-08-24 18:35:10,312][22814] Updated weights for policy 0, policy_version 290 (0.0020)
248
+ [2025-08-24 18:35:14,120][22814] Updated weights for policy 0, policy_version 300 (0.0020)
249
+ [2025-08-24 18:35:15,292][22314] Fps is (10 sec: 11059.0, 60 sec: 9147.8, 300 sec: 8559.2). Total num frames: 1241088. Throughput: 0: 2417.7. Samples: 310824. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
250
+ [2025-08-24 18:35:15,294][22314] Avg episode reward: [(0, '4.581')]
251
+ [2025-08-24 18:35:17,918][22814] Updated weights for policy 0, policy_version 310 (0.0022)
252
+ [2025-08-24 18:35:20,292][22314] Fps is (10 sec: 10649.5, 60 sec: 9216.0, 300 sec: 8601.6). Total num frames: 1290240. Throughput: 0: 2461.8. Samples: 318776. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
253
+ [2025-08-24 18:35:20,293][22314] Avg episode reward: [(0, '4.453')]
254
+ [2025-08-24 18:35:21,833][22814] Updated weights for policy 0, policy_version 320 (0.0024)
255
+ [2025-08-24 18:35:25,292][22314] Fps is (10 sec: 10240.0, 60 sec: 9489.1, 300 sec: 8667.7). Total num frames: 1343488. Throughput: 0: 2525.0. Samples: 334432. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
256
+ [2025-08-24 18:35:25,294][22314] Avg episode reward: [(0, '4.421')]
257
+ [2025-08-24 18:35:25,665][22814] Updated weights for policy 0, policy_version 330 (0.0019)
258
+ [2025-08-24 18:35:29,579][22814] Updated weights for policy 0, policy_version 340 (0.0021)
259
+ [2025-08-24 18:35:30,292][22314] Fps is (10 sec: 10649.7, 60 sec: 9830.4, 300 sec: 8729.6). Total num frames: 1396736. Throughput: 0: 2539.9. Samples: 350168. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
260
+ [2025-08-24 18:35:30,293][22314] Avg episode reward: [(0, '4.484')]
261
+ [2025-08-24 18:35:33,729][22814] Updated weights for policy 0, policy_version 350 (0.0026)
262
+ [2025-08-24 18:35:35,292][22314] Fps is (10 sec: 10240.1, 60 sec: 10035.2, 300 sec: 8763.0). Total num frames: 1445888. Throughput: 0: 2569.0. Samples: 357968. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
263
+ [2025-08-24 18:35:35,293][22314] Avg episode reward: [(0, '4.212')]
264
+ [2025-08-24 18:35:37,704][22814] Updated weights for policy 0, policy_version 360 (0.0021)
265
+ [2025-08-24 18:35:40,291][22314] Fps is (10 sec: 9830.5, 60 sec: 10103.5, 300 sec: 8794.4). Total num frames: 1495040. Throughput: 0: 2600.4. Samples: 372620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
266
+ [2025-08-24 18:35:40,293][22314] Avg episode reward: [(0, '4.548')]
267
+ [2025-08-24 18:35:41,922][22814] Updated weights for policy 0, policy_version 370 (0.0027)
268
+ [2025-08-24 18:35:45,292][22314] Fps is (10 sec: 10239.9, 60 sec: 10103.5, 300 sec: 8847.4). Total num frames: 1548288. Throughput: 0: 2603.7. Samples: 387636. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
269
+ [2025-08-24 18:35:45,294][22314] Avg episode reward: [(0, '4.488')]
270
+ [2025-08-24 18:35:45,969][22814] Updated weights for policy 0, policy_version 380 (0.0021)
271
+ [2025-08-24 18:35:49,918][22814] Updated weights for policy 0, policy_version 390 (0.0020)
272
+ [2025-08-24 18:35:50,292][22314] Fps is (10 sec: 10239.8, 60 sec: 10240.0, 300 sec: 8874.7). Total num frames: 1597440. Throughput: 0: 2610.1. Samples: 395808. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
273
+ [2025-08-24 18:35:50,294][22314] Avg episode reward: [(0, '4.378')]
274
+ [2025-08-24 18:35:53,877][22814] Updated weights for policy 0, policy_version 400 (0.0024)
275
+ [2025-08-24 18:35:55,292][22314] Fps is (10 sec: 10239.8, 60 sec: 10376.5, 300 sec: 8922.6). Total num frames: 1650688. Throughput: 0: 2587.0. Samples: 410852. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
276
+ [2025-08-24 18:35:55,294][22314] Avg episode reward: [(0, '4.416')]
277
+ [2025-08-24 18:35:57,891][22814] Updated weights for policy 0, policy_version 410 (0.0023)
278
+ [2025-08-24 18:36:00,292][22314] Fps is (10 sec: 10240.1, 60 sec: 10376.5, 300 sec: 8946.5). Total num frames: 1699840. Throughput: 0: 2563.5. Samples: 426182. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
279
+ [2025-08-24 18:36:00,293][22314] Avg episode reward: [(0, '4.575')]
280
+ [2025-08-24 18:36:01,835][22814] Updated weights for policy 0, policy_version 420 (0.0019)
281
+ [2025-08-24 18:36:05,292][22314] Fps is (10 sec: 9830.5, 60 sec: 10308.2, 300 sec: 8969.2). Total num frames: 1748992. Throughput: 0: 2555.2. Samples: 433760. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
282
+ [2025-08-24 18:36:05,293][22314] Avg episode reward: [(0, '4.477')]
283
+ [2025-08-24 18:36:06,581][22814] Updated weights for policy 0, policy_version 430 (0.0030)
284
+ [2025-08-24 18:36:10,292][22314] Fps is (10 sec: 9011.1, 60 sec: 10103.5, 300 sec: 8949.8). Total num frames: 1789952. Throughput: 0: 2493.1. Samples: 446620. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
285
+ [2025-08-24 18:36:10,294][22314] Avg episode reward: [(0, '4.409')]
286
+ [2025-08-24 18:36:11,351][22814] Updated weights for policy 0, policy_version 440 (0.0026)
287
+ [2025-08-24 18:36:15,292][22314] Fps is (10 sec: 8601.6, 60 sec: 9898.7, 300 sec: 8951.3). Total num frames: 1835008. Throughput: 0: 2430.4. Samples: 459536. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
288
+ [2025-08-24 18:36:15,294][22314] Avg episode reward: [(0, '4.300')]
289
+ [2025-08-24 18:36:16,021][22814] Updated weights for policy 0, policy_version 450 (0.0031)
290
+ [2025-08-24 18:36:20,292][22314] Fps is (10 sec: 8601.6, 60 sec: 9762.1, 300 sec: 8933.2). Total num frames: 1875968. Throughput: 0: 2394.0. Samples: 465700. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
291
+ [2025-08-24 18:36:20,294][22314] Avg episode reward: [(0, '4.548')]
292
+ [2025-08-24 18:36:21,182][22814] Updated weights for policy 0, policy_version 460 (0.0022)
293
+ [2025-08-24 18:36:25,292][22314] Fps is (10 sec: 8192.1, 60 sec: 9557.3, 300 sec: 8915.9). Total num frames: 1916928. Throughput: 0: 2340.7. Samples: 477950. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
294
+ [2025-08-24 18:36:25,293][22314] Avg episode reward: [(0, '4.455')]
295
+ [2025-08-24 18:36:25,806][22814] Updated weights for policy 0, policy_version 470 (0.0024)
296
+ [2025-08-24 18:36:29,984][22814] Updated weights for policy 0, policy_version 480 (0.0024)
297
+ [2025-08-24 18:36:30,291][22314] Fps is (10 sec: 9011.4, 60 sec: 9489.1, 300 sec: 8936.7). Total num frames: 1966080. Throughput: 0: 2332.8. Samples: 492610. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
298
+ [2025-08-24 18:36:30,293][22314] Avg episode reward: [(0, '4.589')]
299
+ [2025-08-24 18:36:33,901][22814] Updated weights for policy 0, policy_version 490 (0.0024)
300
+ [2025-08-24 18:36:35,292][22314] Fps is (10 sec: 10240.0, 60 sec: 9557.3, 300 sec: 8974.8). Total num frames: 2019328. Throughput: 0: 2327.8. Samples: 500558. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
301
+ [2025-08-24 18:36:35,293][22314] Avg episode reward: [(0, '4.355')]
302
+ [2025-08-24 18:36:35,303][22801] Saving /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000493_2019328.pth...
303
+ [2025-08-24 18:36:37,913][22814] Updated weights for policy 0, policy_version 500 (0.0023)
304
+ [2025-08-24 18:36:40,292][22314] Fps is (10 sec: 10239.8, 60 sec: 9557.3, 300 sec: 8993.4). Total num frames: 2068480. Throughput: 0: 2331.9. Samples: 515788. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
305
+ [2025-08-24 18:36:40,293][22314] Avg episode reward: [(0, '4.271')]
306
+ [2025-08-24 18:36:41,942][22814] Updated weights for policy 0, policy_version 510 (0.0023)
307
+ [2025-08-24 18:36:45,292][22314] Fps is (10 sec: 10239.9, 60 sec: 9557.3, 300 sec: 9028.6). Total num frames: 2121728. Throughput: 0: 2338.5. Samples: 531416. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
308
+ [2025-08-24 18:36:45,295][22314] Avg episode reward: [(0, '4.457')]
309
+ [2025-08-24 18:36:45,764][22814] Updated weights for policy 0, policy_version 520 (0.0018)
310
+ [2025-08-24 18:36:50,081][22814] Updated weights for policy 0, policy_version 530 (0.0022)
311
+ [2025-08-24 18:36:50,292][22314] Fps is (10 sec: 10240.0, 60 sec: 9557.3, 300 sec: 9045.3). Total num frames: 2170880. Throughput: 0: 2339.1. Samples: 539018. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
312
+ [2025-08-24 18:36:50,294][22314] Avg episode reward: [(0, '4.509')]
313
+ [2025-08-24 18:36:53,982][22814] Updated weights for policy 0, policy_version 540 (0.0021)
314
+ [2025-08-24 18:36:55,292][22314] Fps is (10 sec: 10240.1, 60 sec: 9557.4, 300 sec: 9078.1). Total num frames: 2224128. Throughput: 0: 2389.9. Samples: 554164. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
315
+ [2025-08-24 18:36:55,293][22314] Avg episode reward: [(0, '4.568')]
316
+ [2025-08-24 18:36:57,868][22814] Updated weights for policy 0, policy_version 550 (0.0020)
317
+ [2025-08-24 18:37:00,292][22314] Fps is (10 sec: 10649.7, 60 sec: 9625.6, 300 sec: 9109.5). Total num frames: 2277376. Throughput: 0: 2454.4. Samples: 569982. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
318
+ [2025-08-24 18:37:00,293][22314] Avg episode reward: [(0, '4.440')]
319
+ [2025-08-24 18:37:02,032][22814] Updated weights for policy 0, policy_version 560 (0.0023)
320
+ [2025-08-24 18:37:05,292][22314] Fps is (10 sec: 9011.0, 60 sec: 9420.8, 300 sec: 9075.4). Total num frames: 2314240. Throughput: 0: 2458.7. Samples: 576340. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
321
+ [2025-08-24 18:37:05,294][22314] Avg episode reward: [(0, '4.562')]
322
+ [2025-08-24 18:37:07,602][22814] Updated weights for policy 0, policy_version 570 (0.0031)
323
+ [2025-08-24 18:37:10,292][22314] Fps is (10 sec: 7372.7, 60 sec: 9352.5, 300 sec: 9042.7). Total num frames: 2351104. Throughput: 0: 2422.6. Samples: 586968. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
324
+ [2025-08-24 18:37:10,294][22314] Avg episode reward: [(0, '4.336')]
325
+ [2025-08-24 18:37:12,990][22814] Updated weights for policy 0, policy_version 580 (0.0041)
326
+ [2025-08-24 18:37:15,292][22314] Fps is (10 sec: 7782.6, 60 sec: 9284.3, 300 sec: 9026.7). Total num frames: 2392064. Throughput: 0: 2363.7. Samples: 598976. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
327
+ [2025-08-24 18:37:15,293][22314] Avg episode reward: [(0, '4.295')]
328
+ [2025-08-24 18:37:18,069][22814] Updated weights for policy 0, policy_version 590 (0.0021)
329
+ [2025-08-24 18:37:20,292][22314] Fps is (10 sec: 8192.0, 60 sec: 9284.3, 300 sec: 9011.2). Total num frames: 2433024. Throughput: 0: 2323.1. Samples: 605098. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
330
+ [2025-08-24 18:37:20,293][22314] Avg episode reward: [(0, '4.286')]
331
+ [2025-08-24 18:37:22,760][22814] Updated weights for policy 0, policy_version 600 (0.0021)
332
+ [2025-08-24 18:37:25,292][22314] Fps is (10 sec: 8601.5, 60 sec: 9352.5, 300 sec: 9011.2). Total num frames: 2478080. Throughput: 0: 2281.5. Samples: 618456. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
333
+ [2025-08-24 18:37:25,293][22314] Avg episode reward: [(0, '4.462')]
334
+ [2025-08-24 18:37:26,891][22814] Updated weights for policy 0, policy_version 610 (0.0023)
335
+ [2025-08-24 18:37:30,291][22314] Fps is (10 sec: 9830.5, 60 sec: 9420.8, 300 sec: 9040.5). Total num frames: 2531328. Throughput: 0: 2272.3. Samples: 633668. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
336
+ [2025-08-24 18:37:30,293][22314] Avg episode reward: [(0, '4.367')]
337
+ [2025-08-24 18:37:30,952][22814] Updated weights for policy 0, policy_version 620 (0.0019)
338
+ [2025-08-24 18:37:35,291][22314] Fps is (10 sec: 9830.6, 60 sec: 9284.3, 300 sec: 9039.9). Total num frames: 2576384. Throughput: 0: 2262.0. Samples: 640806. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
339
+ [2025-08-24 18:37:35,294][22314] Avg episode reward: [(0, '4.463')]
340
+ [2025-08-24 18:37:35,383][22814] Updated weights for policy 0, policy_version 630 (0.0023)
341
+ [2025-08-24 18:37:39,688][22814] Updated weights for policy 0, policy_version 640 (0.0023)
342
+ [2025-08-24 18:37:40,291][22314] Fps is (10 sec: 9420.8, 60 sec: 9284.3, 300 sec: 9053.6). Total num frames: 2625536. Throughput: 0: 2235.4. Samples: 654758. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
343
+ [2025-08-24 18:37:40,293][22314] Avg episode reward: [(0, '4.310')]
344
+ [2025-08-24 18:37:43,886][22814] Updated weights for policy 0, policy_version 650 (0.0020)
345
+ [2025-08-24 18:37:45,292][22314] Fps is (10 sec: 9830.3, 60 sec: 9216.0, 300 sec: 9066.7). Total num frames: 2674688. Throughput: 0: 2210.5. Samples: 669456. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
346
+ [2025-08-24 18:37:45,294][22314] Avg episode reward: [(0, '4.356')]
347
+ [2025-08-24 18:37:48,177][22814] Updated weights for policy 0, policy_version 660 (0.0026)
348
+ [2025-08-24 18:37:50,292][22314] Fps is (10 sec: 9420.6, 60 sec: 9147.7, 300 sec: 9136.2). Total num frames: 2719744. Throughput: 0: 2228.9. Samples: 676640. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
349
+ [2025-08-24 18:37:50,294][22314] Avg episode reward: [(0, '4.454')]
350
+ [2025-08-24 18:37:53,073][22814] Updated weights for policy 0, policy_version 670 (0.0030)
351
+ [2025-08-24 18:37:55,292][22314] Fps is (10 sec: 8601.6, 60 sec: 8942.9, 300 sec: 9094.5). Total num frames: 2760704. Throughput: 0: 2277.4. Samples: 689452. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
352
+ [2025-08-24 18:37:55,294][22314] Avg episode reward: [(0, '4.205')]
353
+ [2025-08-24 18:37:57,367][22814] Updated weights for policy 0, policy_version 680 (0.0024)
354
+ [2025-08-24 18:38:00,292][22314] Fps is (10 sec: 9011.3, 60 sec: 8874.7, 300 sec: 9108.4). Total num frames: 2809856. Throughput: 0: 2327.3. Samples: 703706. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
355
+ [2025-08-24 18:38:00,294][22314] Avg episode reward: [(0, '4.522')]
356
+ [2025-08-24 18:38:01,711][22814] Updated weights for policy 0, policy_version 690 (0.0021)
357
+ [2025-08-24 18:38:05,296][22314] Fps is (10 sec: 9007.3, 60 sec: 8942.3, 300 sec: 9108.3). Total num frames: 2850816. Throughput: 0: 2341.0. Samples: 710454. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
358
+ [2025-08-24 18:38:05,298][22314] Avg episode reward: [(0, '4.439')]
359
+ [2025-08-24 18:38:06,848][22814] Updated weights for policy 0, policy_version 700 (0.0024)
360
+ [2025-08-24 18:38:10,292][22314] Fps is (10 sec: 8601.6, 60 sec: 9079.5, 300 sec: 9150.1). Total num frames: 2895872. Throughput: 0: 2315.6. Samples: 722660. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
361
+ [2025-08-24 18:38:10,294][22314] Avg episode reward: [(0, '4.626')]
362
+ [2025-08-24 18:38:11,795][22814] Updated weights for policy 0, policy_version 710 (0.0025)
363
+ [2025-08-24 18:38:15,292][22314] Fps is (10 sec: 8605.4, 60 sec: 9079.5, 300 sec: 9177.8). Total num frames: 2936832. Throughput: 0: 2247.0. Samples: 734782. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
364
+ [2025-08-24 18:38:15,293][22314] Avg episode reward: [(0, '4.450')]
365
+ [2025-08-24 18:38:16,799][22814] Updated weights for policy 0, policy_version 720 (0.0023)
366
+ [2025-08-24 18:38:20,292][22314] Fps is (10 sec: 8191.9, 60 sec: 9079.4, 300 sec: 9191.7). Total num frames: 2977792. Throughput: 0: 2225.5. Samples: 740954. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
367
+ [2025-08-24 18:38:20,295][22314] Avg episode reward: [(0, '4.581')]
368
+ [2025-08-24 18:38:21,681][22814] Updated weights for policy 0, policy_version 730 (0.0031)
369
+ [2025-08-24 18:38:25,292][22314] Fps is (10 sec: 8192.1, 60 sec: 9011.2, 300 sec: 9205.6). Total num frames: 3018752. Throughput: 0: 2201.3. Samples: 753816. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
370
+ [2025-08-24 18:38:25,293][22314] Avg episode reward: [(0, '4.501')]
371
+ [2025-08-24 18:38:26,141][22814] Updated weights for policy 0, policy_version 740 (0.0024)
372
+ [2025-08-24 18:38:30,292][22314] Fps is (10 sec: 9011.2, 60 sec: 8942.9, 300 sec: 9233.4). Total num frames: 3067904. Throughput: 0: 2195.5. Samples: 768252. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
373
+ [2025-08-24 18:38:30,294][22314] Avg episode reward: [(0, '4.295')]
374
+ [2025-08-24 18:38:30,361][22814] Updated weights for policy 0, policy_version 750 (0.0023)
375
+ [2025-08-24 18:38:35,035][22814] Updated weights for policy 0, policy_version 760 (0.0031)
376
+ [2025-08-24 18:38:35,292][22314] Fps is (10 sec: 9420.7, 60 sec: 8942.9, 300 sec: 9247.2). Total num frames: 3112960. Throughput: 0: 2185.9. Samples: 775004. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
377
+ [2025-08-24 18:38:35,293][22314] Avg episode reward: [(0, '4.592')]
378
+ [2025-08-24 18:38:35,305][22801] Saving /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000760_3112960.pth...
379
+ [2025-08-24 18:38:35,391][22801] Removing /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000207_847872.pth
380
+ [2025-08-24 18:38:39,621][22814] Updated weights for policy 0, policy_version 770 (0.0028)
381
+ [2025-08-24 18:38:40,292][22314] Fps is (10 sec: 9011.3, 60 sec: 8874.6, 300 sec: 9247.3). Total num frames: 3158016. Throughput: 0: 2194.3. Samples: 788194. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
382
+ [2025-08-24 18:38:40,294][22314] Avg episode reward: [(0, '4.343')]
383
+ [2025-08-24 18:38:44,197][22814] Updated weights for policy 0, policy_version 780 (0.0027)
384
+ [2025-08-24 18:38:45,292][22314] Fps is (10 sec: 9011.2, 60 sec: 8806.4, 300 sec: 9233.4). Total num frames: 3203072. Throughput: 0: 2174.6. Samples: 801564. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
385
+ [2025-08-24 18:38:45,294][22314] Avg episode reward: [(0, '4.506')]
386
+ [2025-08-24 18:38:48,474][22814] Updated weights for policy 0, policy_version 790 (0.0024)
387
+ [2025-08-24 18:38:50,292][22314] Fps is (10 sec: 9420.9, 60 sec: 8874.7, 300 sec: 9247.2). Total num frames: 3252224. Throughput: 0: 2189.1. Samples: 808952. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
388
+ [2025-08-24 18:38:50,295][22314] Avg episode reward: [(0, '4.536')]
389
+ [2025-08-24 18:38:52,708][22814] Updated weights for policy 0, policy_version 800 (0.0018)
390
+ [2025-08-24 18:38:55,293][22314] Fps is (10 sec: 9830.2, 60 sec: 9011.2, 300 sec: 9247.2). Total num frames: 3301376. Throughput: 0: 2237.5. Samples: 823346. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
391
+ [2025-08-24 18:38:55,295][22314] Avg episode reward: [(0, '4.507')]
392
+ [2025-08-24 18:38:57,013][22814] Updated weights for policy 0, policy_version 810 (0.0022)
393
+ [2025-08-24 18:39:00,291][22314] Fps is (10 sec: 9420.9, 60 sec: 8943.0, 300 sec: 9261.1). Total num frames: 3346432. Throughput: 0: 2286.1. Samples: 837658. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
394
+ [2025-08-24 18:39:00,294][22314] Avg episode reward: [(0, '4.375')]
395
+ [2025-08-24 18:39:01,239][22814] Updated weights for policy 0, policy_version 820 (0.0019)
396
+ [2025-08-24 18:39:05,292][22314] Fps is (10 sec: 8601.8, 60 sec: 8943.6, 300 sec: 9275.0). Total num frames: 3387392. Throughput: 0: 2306.0. Samples: 844722. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
397
+ [2025-08-24 18:39:05,293][22314] Avg episode reward: [(0, '4.632')]
398
+ [2025-08-24 18:39:06,311][22814] Updated weights for policy 0, policy_version 830 (0.0031)
399
+ [2025-08-24 18:39:10,293][22314] Fps is (10 sec: 8190.6, 60 sec: 8874.4, 300 sec: 9275.0). Total num frames: 3428352. Throughput: 0: 2283.3. Samples: 856570. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
400
+ [2025-08-24 18:39:10,299][22314] Avg episode reward: [(0, '4.381')]
401
+ [2025-08-24 18:39:11,452][22814] Updated weights for policy 0, policy_version 840 (0.0024)
402
+ [2025-08-24 18:39:15,292][22314] Fps is (10 sec: 8192.0, 60 sec: 8874.7, 300 sec: 9261.1). Total num frames: 3469312. Throughput: 0: 2230.1. Samples: 868608. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
403
+ [2025-08-24 18:39:15,294][22314] Avg episode reward: [(0, '4.606')]
404
+ [2025-08-24 18:39:16,461][22814] Updated weights for policy 0, policy_version 850 (0.0031)
405
+ [2025-08-24 18:39:20,293][22314] Fps is (10 sec: 7782.2, 60 sec: 8806.1, 300 sec: 9261.1). Total num frames: 3506176. Throughput: 0: 2213.1. Samples: 874598. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
406
+ [2025-08-24 18:39:20,298][22314] Avg episode reward: [(0, '4.232')]
407
+ [2025-08-24 18:39:22,410][22814] Updated weights for policy 0, policy_version 860 (0.0044)
408
+ [2025-08-24 18:39:25,292][22314] Fps is (10 sec: 6963.2, 60 sec: 8669.8, 300 sec: 9261.1). Total num frames: 3538944. Throughput: 0: 2138.8. Samples: 884440. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
409
+ [2025-08-24 18:39:25,294][22314] Avg episode reward: [(0, '4.455')]
410
+ [2025-08-24 18:39:28,097][22814] Updated weights for policy 0, policy_version 870 (0.0027)
411
+ [2025-08-24 18:39:30,292][22314] Fps is (10 sec: 7374.2, 60 sec: 8533.4, 300 sec: 9275.0). Total num frames: 3579904. Throughput: 0: 2097.5. Samples: 895952. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
412
+ [2025-08-24 18:39:30,294][22314] Avg episode reward: [(0, '4.426')]
413
+ [2025-08-24 18:39:33,005][22814] Updated weights for policy 0, policy_version 880 (0.0030)
414
+ [2025-08-24 18:39:35,292][22314] Fps is (10 sec: 7782.2, 60 sec: 8396.8, 300 sec: 9247.2). Total num frames: 3616768. Throughput: 0: 2074.6. Samples: 902308. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
415
+ [2025-08-24 18:39:35,294][22314] Avg episode reward: [(0, '3.970')]
416
+ [2025-08-24 18:39:38,495][22814] Updated weights for policy 0, policy_version 890 (0.0032)
417
+ [2025-08-24 18:39:40,292][22314] Fps is (10 sec: 8192.1, 60 sec: 8396.8, 300 sec: 9219.5). Total num frames: 3661824. Throughput: 0: 2011.0. Samples: 913842. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
418
+ [2025-08-24 18:39:40,293][22314] Avg episode reward: [(0, '4.334')]
419
+ [2025-08-24 18:39:42,900][22814] Updated weights for policy 0, policy_version 900 (0.0022)
420
+ [2025-08-24 18:39:45,292][22314] Fps is (10 sec: 8601.7, 60 sec: 8328.5, 300 sec: 9219.5). Total num frames: 3702784. Throughput: 0: 1983.5. Samples: 926916. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
421
+ [2025-08-24 18:39:45,294][22314] Avg episode reward: [(0, '4.366')]
422
+ [2025-08-24 18:39:48,018][22814] Updated weights for policy 0, policy_version 910 (0.0029)
423
+ [2025-08-24 18:39:50,292][22314] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 9205.6). Total num frames: 3743744. Throughput: 0: 1959.8. Samples: 932914. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
424
+ [2025-08-24 18:39:50,294][22314] Avg episode reward: [(0, '4.174')]
425
+ [2025-08-24 18:39:52,776][22814] Updated weights for policy 0, policy_version 920 (0.0022)
426
+ [2025-08-24 18:39:55,292][22314] Fps is (10 sec: 8601.7, 60 sec: 8123.8, 300 sec: 9191.7). Total num frames: 3788800. Throughput: 0: 1988.1. Samples: 946030. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
427
+ [2025-08-24 18:39:55,294][22314] Avg episode reward: [(0, '4.510')]
428
+ [2025-08-24 18:39:57,446][22814] Updated weights for policy 0, policy_version 930 (0.0030)
429
+ [2025-08-24 18:40:00,292][22314] Fps is (10 sec: 8601.7, 60 sec: 8055.5, 300 sec: 9150.0). Total num frames: 3829760. Throughput: 0: 2003.0. Samples: 958742. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
430
+ [2025-08-24 18:40:00,294][22314] Avg episode reward: [(0, '4.434')]
431
+ [2025-08-24 18:40:02,163][22814] Updated weights for policy 0, policy_version 940 (0.0025)
432
+ [2025-08-24 18:40:05,292][22314] Fps is (10 sec: 7782.2, 60 sec: 7987.2, 300 sec: 9094.5). Total num frames: 3866624. Throughput: 0: 2006.7. Samples: 964896. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
433
+ [2025-08-24 18:40:05,295][22314] Avg episode reward: [(0, '4.515')]
434
+ [2025-08-24 18:40:08,372][22814] Updated weights for policy 0, policy_version 950 (0.0038)
435
+ [2025-08-24 18:40:10,292][22314] Fps is (10 sec: 7372.7, 60 sec: 7919.1, 300 sec: 9025.1). Total num frames: 3903488. Throughput: 0: 2007.6. Samples: 974784. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
436
+ [2025-08-24 18:40:10,294][22314] Avg episode reward: [(0, '4.295')]
437
+ [2025-08-24 18:40:14,493][22814] Updated weights for policy 0, policy_version 960 (0.0039)
438
+ [2025-08-24 18:40:15,292][22314] Fps is (10 sec: 6963.2, 60 sec: 7782.4, 300 sec: 8969.5). Total num frames: 3936256. Throughput: 0: 1976.8. Samples: 984908. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
439
+ [2025-08-24 18:40:15,294][22314] Avg episode reward: [(0, '4.216')]
440
+ [2025-08-24 18:40:20,292][22314] Fps is (10 sec: 6553.6, 60 sec: 7714.4, 300 sec: 8900.1). Total num frames: 3969024. Throughput: 0: 1950.5. Samples: 990080. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
441
+ [2025-08-24 18:40:20,294][22314] Avg episode reward: [(0, '4.393')]
442
+ [2025-08-24 18:40:20,375][22814] Updated weights for policy 0, policy_version 970 (0.0032)
443
+ [2025-08-24 18:40:24,890][22801] Saving /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
444
+ [2025-08-24 18:40:24,900][22314] Component Batcher_0 stopped!
445
+ [2025-08-24 18:40:24,900][22801] Stopping Batcher_0...
446
+ [2025-08-24 18:40:24,906][22801] Loop batcher_evt_loop terminating...
447
+ [2025-08-24 18:40:24,941][22814] Weights refcount: 2 0
448
+ [2025-08-24 18:40:24,948][22814] Stopping InferenceWorker_p0-w0...
449
+ [2025-08-24 18:40:24,949][22814] Loop inference_proc0-0_evt_loop terminating...
450
+ [2025-08-24 18:40:24,949][22314] Component InferenceWorker_p0-w0 stopped!
451
+ [2025-08-24 18:40:24,968][22837] Stopping RolloutWorker_w6...
452
+ [2025-08-24 18:40:24,968][22314] Component RolloutWorker_w6 stopped!
453
+ [2025-08-24 18:40:24,972][22837] Loop rollout_proc6_evt_loop terminating...
454
+ [2025-08-24 18:40:24,972][22838] Stopping RolloutWorker_w7...
455
+ [2025-08-24 18:40:24,973][22838] Loop rollout_proc7_evt_loop terminating...
456
+ [2025-08-24 18:40:24,974][22818] Stopping RolloutWorker_w3...
457
+ [2025-08-24 18:40:24,975][22818] Loop rollout_proc3_evt_loop terminating...
458
+ [2025-08-24 18:40:24,979][22834] Stopping RolloutWorker_w4...
459
+ [2025-08-24 18:40:24,979][22834] Loop rollout_proc4_evt_loop terminating...
460
+ [2025-08-24 18:40:24,980][22817] Stopping RolloutWorker_w2...
461
+ [2025-08-24 18:40:24,981][22817] Loop rollout_proc2_evt_loop terminating...
462
+ [2025-08-24 18:40:24,981][22816] Stopping RolloutWorker_w1...
463
+ [2025-08-24 18:40:24,982][22816] Loop rollout_proc1_evt_loop terminating...
464
+ [2025-08-24 18:40:24,984][22835] Stopping RolloutWorker_w5...
465
+ [2025-08-24 18:40:24,985][22835] Loop rollout_proc5_evt_loop terminating...
466
+ [2025-08-24 18:40:24,974][22314] Component RolloutWorker_w7 stopped!
467
+ [2025-08-24 18:40:24,991][22314] Component RolloutWorker_w3 stopped!
468
+ [2025-08-24 18:40:24,994][22314] Component RolloutWorker_w4 stopped!
469
+ [2025-08-24 18:40:24,996][22314] Component RolloutWorker_w2 stopped!
470
+ [2025-08-24 18:40:24,997][22314] Component RolloutWorker_w1 stopped!
471
+ [2025-08-24 18:40:24,999][22314] Component RolloutWorker_w5 stopped!
472
+ [2025-08-24 18:40:25,013][22815] Stopping RolloutWorker_w0...
473
+ [2025-08-24 18:40:25,014][22815] Loop rollout_proc0_evt_loop terminating...
474
+ [2025-08-24 18:40:25,013][22314] Component RolloutWorker_w0 stopped!
475
+ [2025-08-24 18:40:25,070][22801] Removing /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000493_2019328.pth
476
+ [2025-08-24 18:40:25,081][22801] Saving /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
477
+ [2025-08-24 18:40:25,256][22801] Stopping LearnerWorker_p0...
478
+ [2025-08-24 18:40:25,257][22801] Loop learner_proc0_evt_loop terminating...
479
+ [2025-08-24 18:40:25,257][22314] Component LearnerWorker_p0 stopped!
480
+ [2025-08-24 18:40:25,261][22314] Waiting for process learner_proc0 to stop...
481
+ [2025-08-24 18:40:27,251][22314] Waiting for process inference_proc0-0 to join...
482
+ [2025-08-24 18:40:27,253][22314] Waiting for process rollout_proc0 to join...
483
+ [2025-08-24 18:40:27,255][22314] Waiting for process rollout_proc1 to join...
484
+ [2025-08-24 18:40:27,257][22314] Waiting for process rollout_proc2 to join...
485
+ [2025-08-24 18:40:27,259][22314] Waiting for process rollout_proc3 to join...
486
+ [2025-08-24 18:40:27,260][22314] Waiting for process rollout_proc4 to join...
487
+ [2025-08-24 18:40:27,262][22314] Waiting for process rollout_proc5 to join...
488
+ [2025-08-24 18:40:27,264][22314] Waiting for process rollout_proc6 to join...
489
+ [2025-08-24 18:40:27,266][22314] Waiting for process rollout_proc7 to join...
490
+ [2025-08-24 18:40:27,268][22314] Batcher 0 profile tree view:
491
+ batching: 18.0026, releasing_batches: 0.0497
492
+ [2025-08-24 18:40:27,270][22314] InferenceWorker_p0-w0 profile tree view:
493
+ wait_policy: 0.0001
494
+ wait_policy_total: 6.6832
495
+ update_model: 8.2570
496
+ weight_update: 0.0038
497
+ one_step: 0.0132
498
+ handle_policy_step: 419.9654
499
+ deserialize: 14.5882, stack: 2.8454, obs_to_device_normalize: 104.2409, forward: 185.7047, send_messages: 39.8577
500
+ prepare_outputs: 56.7150
501
+ to_cpu: 41.0715
502
+ [2025-08-24 18:40:27,271][22314] Learner 0 profile tree view:
503
+ misc: 0.0058, prepare_batch: 14.6209
504
+ train: 49.6287
505
+ epoch_init: 0.0079, minibatch_init: 0.0117, losses_postprocess: 0.6793, kl_divergence: 0.8223, after_optimizer: 13.7673
506
+ calculate_losses: 20.2579
507
+ losses_init: 0.0043, forward_head: 1.7738, bptt_initial: 12.2704, tail: 1.1439, advantages_returns: 0.3356, losses: 2.1770
508
+ bptt: 2.2604
509
+ bptt_forward_core: 2.1293
510
+ update: 13.3471
511
+ clip: 1.3706
512
+ [2025-08-24 18:40:27,273][22314] RolloutWorker_w0 profile tree view:
513
+ wait_for_trajectories: 0.2174, enqueue_policy_requests: 13.2016, env_step: 190.0232, overhead: 11.3236, complete_rollouts: 0.4466
514
+ save_policy_outputs: 15.6623
515
+ split_output_tensors: 5.3806
516
+ [2025-08-24 18:40:27,274][22314] RolloutWorker_w7 profile tree view:
517
+ wait_for_trajectories: 0.2138, enqueue_policy_requests: 12.5841, env_step: 188.6272, overhead: 10.1826, complete_rollouts: 0.4320
518
+ save_policy_outputs: 15.1798
519
+ split_output_tensors: 5.2640
520
+ [2025-08-24 18:40:27,276][22314] Loop Runner_EvtLoop terminating...
521
+ [2025-08-24 18:40:27,278][22314] Runner profile tree view:
522
+ main_loop: 466.0443
523
+ [2025-08-24 18:40:27,280][22314] Collected {0: 4005888}, FPS: 8595.5
524
+ [2025-08-24 18:48:50,120][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
525
+ [2025-08-24 18:48:50,122][22314] Overriding arg 'num_workers' with value 1 passed from command line
526
+ [2025-08-24 18:48:50,123][22314] Adding new argument 'no_render'=True that is not in the saved config file!
527
+ [2025-08-24 18:48:50,124][22314] Adding new argument 'save_video'=True that is not in the saved config file!
528
+ [2025-08-24 18:48:50,125][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
529
+ [2025-08-24 18:48:50,126][22314] Adding new argument 'video_name'=None that is not in the saved config file!
530
+ [2025-08-24 18:48:50,127][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
531
+ [2025-08-24 18:48:50,127][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
532
+ [2025-08-24 18:48:50,128][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
533
+ [2025-08-24 18:48:50,129][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
534
+ [2025-08-24 18:48:50,129][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
535
+ [2025-08-24 18:48:50,130][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
536
+ [2025-08-24 18:48:50,131][22314] Adding new argument 'train_script'=None that is not in the saved config file!
537
+ [2025-08-24 18:48:50,131][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
538
+ [2025-08-24 18:48:50,132][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
539
+ [2025-08-24 18:48:50,165][22314] Doom resolution: 160x120, resize resolution: (128, 72)
540
+ [2025-08-24 18:48:50,169][22314] RunningMeanStd input shape: (3, 72, 128)
541
+ [2025-08-24 18:48:50,173][22314] RunningMeanStd input shape: (1,)
542
+ [2025-08-24 18:48:50,192][22314] ConvEncoder: input_channels=3
543
+ [2025-08-24 18:48:50,395][22314] Conv encoder output size: 512
544
+ [2025-08-24 18:48:50,397][22314] Policy head output size: 512
545
+ [2025-08-24 18:48:51,237][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
546
+ [2025-08-24 18:48:51,244][22314] Could not load from checkpoint, attempt 0
547
+ Traceback (most recent call last):
548
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
549
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
550
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
551
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1529, in load
552
+ raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
553
+ _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
554
+ (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
555
+ (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
556
+ WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([numpy.core.multiarray.scalar])` or the `torch.serialization.safe_globals([numpy.core.multiarray.scalar])` context manager to allowlist this global if you trust this class/function.
557
+
558
+ Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
559
+ [2025-08-24 18:48:51,247][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
560
+ [2025-08-24 18:48:51,249][22314] Could not load from checkpoint, attempt 1
561
+ Traceback (most recent call last):
562
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
563
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
564
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
565
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1529, in load
566
+ raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
567
+ _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
568
+ (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
569
+ (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
570
+ WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([numpy.core.multiarray.scalar])` or the `torch.serialization.safe_globals([numpy.core.multiarray.scalar])` context manager to allowlist this global if you trust this class/function.
571
+
572
+ Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
573
+ [2025-08-24 18:48:51,250][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
574
+ [2025-08-24 18:48:51,252][22314] Could not load from checkpoint, attempt 2
575
+ Traceback (most recent call last):
576
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
577
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
578
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
579
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1529, in load
580
+ raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
581
+ _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
582
+ (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
583
+ (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
584
+ WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([numpy.core.multiarray.scalar])` or the `torch.serialization.safe_globals([numpy.core.multiarray.scalar])` context manager to allowlist this global if you trust this class/function.
585
+
586
+ Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
587
+ [2025-08-24 18:51:13,652][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
588
+ [2025-08-24 18:51:13,653][22314] Overriding arg 'num_workers' with value 1 passed from command line
589
+ [2025-08-24 18:51:13,654][22314] Adding new argument 'no_render'=True that is not in the saved config file!
590
+ [2025-08-24 18:51:13,655][22314] Adding new argument 'save_video'=True that is not in the saved config file!
591
+ [2025-08-24 18:51:13,656][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
592
+ [2025-08-24 18:51:13,658][22314] Adding new argument 'video_name'=None that is not in the saved config file!
593
+ [2025-08-24 18:51:13,659][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
594
+ [2025-08-24 18:51:13,659][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
595
+ [2025-08-24 18:51:13,660][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
596
+ [2025-08-24 18:51:13,662][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
597
+ [2025-08-24 18:51:13,663][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
598
+ [2025-08-24 18:51:13,664][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
599
+ [2025-08-24 18:51:13,665][22314] Adding new argument 'train_script'=None that is not in the saved config file!
600
+ [2025-08-24 18:51:13,666][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
601
+ [2025-08-24 18:51:13,667][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
602
+ [2025-08-24 18:51:13,702][22314] RunningMeanStd input shape: (3, 72, 128)
603
+ [2025-08-24 18:51:13,704][22314] RunningMeanStd input shape: (1,)
604
+ [2025-08-24 18:51:13,721][22314] ConvEncoder: input_channels=3
605
+ [2025-08-24 18:51:13,791][22314] Conv encoder output size: 512
606
+ [2025-08-24 18:51:13,793][22314] Policy head output size: 512
607
+ [2025-08-24 18:51:13,831][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
608
+ [2025-08-24 18:51:13,834][22314] Could not load from checkpoint, attempt 0
609
+ Traceback (most recent call last):
610
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
611
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
612
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
613
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
614
+ return _load(
615
+ ^^^^^^
616
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
617
+ result = unpickler.load()
618
+ ^^^^^^^^^^^^^^^^
619
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
620
+ elif full_path in _get_user_allowed_globals():
621
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
622
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
623
+ module, name = f.__module__, f.__qualname__
624
+ ^^^^^^^^^^^^
625
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
626
+ [2025-08-24 18:51:13,839][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
627
+ [2025-08-24 18:51:13,841][22314] Could not load from checkpoint, attempt 1
628
+ Traceback (most recent call last):
629
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
630
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
631
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
632
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
633
+ return _load(
634
+ ^^^^^^
635
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
636
+ result = unpickler.load()
637
+ ^^^^^^^^^^^^^^^^
638
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
639
+ elif full_path in _get_user_allowed_globals():
640
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
641
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
642
+ module, name = f.__module__, f.__qualname__
643
+ ^^^^^^^^^^^^
644
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
645
+ [2025-08-24 18:51:13,843][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
646
+ [2025-08-24 18:51:13,845][22314] Could not load from checkpoint, attempt 2
647
+ Traceback (most recent call last):
648
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
649
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
650
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
651
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
652
+ return _load(
653
+ ^^^^^^
654
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
655
+ result = unpickler.load()
656
+ ^^^^^^^^^^^^^^^^
657
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
658
+ elif full_path in _get_user_allowed_globals():
659
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
660
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
661
+ module, name = f.__module__, f.__qualname__
662
+ ^^^^^^^^^^^^
663
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
664
+ [2025-08-24 18:54:07,775][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
665
+ [2025-08-24 18:54:07,778][22314] Overriding arg 'num_workers' with value 1 passed from command line
666
+ [2025-08-24 18:54:07,779][22314] Adding new argument 'no_render'=True that is not in the saved config file!
667
+ [2025-08-24 18:54:07,781][22314] Adding new argument 'save_video'=True that is not in the saved config file!
668
+ [2025-08-24 18:54:07,782][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
669
+ [2025-08-24 18:54:07,784][22314] Adding new argument 'video_name'=None that is not in the saved config file!
670
+ [2025-08-24 18:54:07,785][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
671
+ [2025-08-24 18:54:07,786][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
672
+ [2025-08-24 18:54:07,788][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
673
+ [2025-08-24 18:54:07,789][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
674
+ [2025-08-24 18:54:07,791][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
675
+ [2025-08-24 18:54:07,792][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
676
+ [2025-08-24 18:54:07,794][22314] Adding new argument 'train_script'=None that is not in the saved config file!
677
+ [2025-08-24 18:54:07,798][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
678
+ [2025-08-24 18:54:07,799][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
679
+ [2025-08-24 18:54:07,838][22314] RunningMeanStd input shape: (3, 72, 128)
680
+ [2025-08-24 18:54:07,840][22314] RunningMeanStd input shape: (1,)
681
+ [2025-08-24 18:54:07,856][22314] ConvEncoder: input_channels=3
682
+ [2025-08-24 18:54:07,908][22314] Conv encoder output size: 512
683
+ [2025-08-24 18:54:07,910][22314] Policy head output size: 512
684
+ [2025-08-24 18:54:07,940][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
685
+ [2025-08-24 18:54:07,945][22314] Could not load from checkpoint, attempt 0
686
+ Traceback (most recent call last):
687
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
688
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
689
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
690
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
691
+ return _load(
692
+ ^^^^^^
693
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
694
+ result = unpickler.load()
695
+ ^^^^^^^^^^^^^^^^
696
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
697
+ elif full_path in _get_user_allowed_globals():
698
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
699
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
700
+ module, name = f.__module__, f.__qualname__
701
+ ^^^^^^^^^^^^
702
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
703
+ [2025-08-24 18:54:07,950][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
704
+ [2025-08-24 18:54:07,952][22314] Could not load from checkpoint, attempt 1
705
+ Traceback (most recent call last):
706
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
707
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
708
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
709
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
710
+ return _load(
711
+ ^^^^^^
712
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
713
+ result = unpickler.load()
714
+ ^^^^^^^^^^^^^^^^
715
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
716
+ elif full_path in _get_user_allowed_globals():
717
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
718
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
719
+ module, name = f.__module__, f.__qualname__
720
+ ^^^^^^^^^^^^
721
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
722
+ [2025-08-24 18:54:07,955][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
723
+ [2025-08-24 18:54:07,957][22314] Could not load from checkpoint, attempt 2
724
+ Traceback (most recent call last):
725
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
726
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
727
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
728
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
729
+ return _load(
730
+ ^^^^^^
731
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
732
+ result = unpickler.load()
733
+ ^^^^^^^^^^^^^^^^
734
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
735
+ elif full_path in _get_user_allowed_globals():
736
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
737
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
738
+ module, name = f.__module__, f.__qualname__
739
+ ^^^^^^^^^^^^
740
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
741
+ [2025-08-24 18:58:07,932][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
742
+ [2025-08-24 18:58:07,934][22314] Overriding arg 'num_workers' with value 1 passed from command line
743
+ [2025-08-24 18:58:07,935][22314] Adding new argument 'no_render'=True that is not in the saved config file!
744
+ [2025-08-24 18:58:07,936][22314] Adding new argument 'save_video'=True that is not in the saved config file!
745
+ [2025-08-24 18:58:07,936][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
746
+ [2025-08-24 18:58:07,937][22314] Adding new argument 'video_name'=None that is not in the saved config file!
747
+ [2025-08-24 18:58:07,939][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
748
+ [2025-08-24 18:58:07,941][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
749
+ [2025-08-24 18:58:07,942][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
750
+ [2025-08-24 18:58:07,943][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
751
+ [2025-08-24 18:58:07,945][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
752
+ [2025-08-24 18:58:07,945][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
753
+ [2025-08-24 18:58:07,946][22314] Adding new argument 'train_script'=None that is not in the saved config file!
754
+ [2025-08-24 18:58:07,949][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
755
+ [2025-08-24 18:58:07,950][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
756
+ [2025-08-24 18:58:08,001][22314] RunningMeanStd input shape: (3, 72, 128)
757
+ [2025-08-24 18:58:08,004][22314] RunningMeanStd input shape: (1,)
758
+ [2025-08-24 18:58:08,017][22314] ConvEncoder: input_channels=3
759
+ [2025-08-24 18:58:08,073][22314] Conv encoder output size: 512
760
+ [2025-08-24 18:58:08,075][22314] Policy head output size: 512
761
+ [2025-08-24 18:58:08,113][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
762
+ [2025-08-24 18:58:08,117][22314] Could not load from checkpoint, attempt 0
763
+ Traceback (most recent call last):
764
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
765
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
766
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
767
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
768
+ return _load(
769
+ ^^^^^^
770
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
771
+ result = unpickler.load()
772
+ ^^^^^^^^^^^^^^^^
773
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
774
+ elif full_path in _get_user_allowed_globals():
775
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
776
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
777
+ module, name = f.__module__, f.__qualname__
778
+ ^^^^^^^^^^^^
779
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
780
+ [2025-08-24 18:58:08,120][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
781
+ [2025-08-24 18:58:08,122][22314] Could not load from checkpoint, attempt 1
782
+ Traceback (most recent call last):
783
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
784
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
785
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
786
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
787
+ return _load(
788
+ ^^^^^^
789
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
790
+ result = unpickler.load()
791
+ ^^^^^^^^^^^^^^^^
792
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
793
+ elif full_path in _get_user_allowed_globals():
794
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
795
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
796
+ module, name = f.__module__, f.__qualname__
797
+ ^^^^^^^^^^^^
798
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
799
+ [2025-08-24 18:58:08,125][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
800
+ [2025-08-24 18:58:08,128][22314] Could not load from checkpoint, attempt 2
801
+ Traceback (most recent call last):
802
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
803
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
804
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
805
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
806
+ return _load(
807
+ ^^^^^^
808
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
809
+ result = unpickler.load()
810
+ ^^^^^^^^^^^^^^^^
811
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
812
+ elif full_path in _get_user_allowed_globals():
813
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
814
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
815
+ module, name = f.__module__, f.__qualname__
816
+ ^^^^^^^^^^^^
817
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
818
+ [2025-08-24 18:58:17,446][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
819
+ [2025-08-24 18:58:17,447][22314] Overriding arg 'num_workers' with value 1 passed from command line
820
+ [2025-08-24 18:58:17,449][22314] Adding new argument 'no_render'=True that is not in the saved config file!
821
+ [2025-08-24 18:58:17,450][22314] Adding new argument 'save_video'=True that is not in the saved config file!
822
+ [2025-08-24 18:58:17,450][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
823
+ [2025-08-24 18:58:17,451][22314] Adding new argument 'video_name'=None that is not in the saved config file!
824
+ [2025-08-24 18:58:17,453][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
825
+ [2025-08-24 18:58:17,454][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
826
+ [2025-08-24 18:58:17,455][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
827
+ [2025-08-24 18:58:17,456][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
828
+ [2025-08-24 18:58:17,456][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
829
+ [2025-08-24 18:58:17,457][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
830
+ [2025-08-24 18:58:17,458][22314] Adding new argument 'train_script'=None that is not in the saved config file!
831
+ [2025-08-24 18:58:17,459][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
832
+ [2025-08-24 18:58:17,460][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
833
+ [2025-08-24 18:58:17,488][22314] RunningMeanStd input shape: (3, 72, 128)
834
+ [2025-08-24 18:58:17,489][22314] RunningMeanStd input shape: (1,)
835
+ [2025-08-24 18:58:17,499][22314] ConvEncoder: input_channels=3
836
+ [2025-08-24 18:58:17,538][22314] Conv encoder output size: 512
837
+ [2025-08-24 18:58:17,540][22314] Policy head output size: 512
838
+ [2025-08-24 18:58:17,571][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
839
+ [2025-08-24 18:58:17,573][22314] Could not load from checkpoint, attempt 0
840
+ Traceback (most recent call last):
841
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
842
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
843
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
844
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
845
+ return _load(
846
+ ^^^^^^
847
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
848
+ result = unpickler.load()
849
+ ^^^^^^^^^^^^^^^^
850
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
851
+ elif full_path in _get_user_allowed_globals():
852
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
853
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
854
+ module, name = f.__module__, f.__qualname__
855
+ ^^^^^^^^^^^^
856
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
857
+ [2025-08-24 18:58:17,575][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
858
+ [2025-08-24 18:58:17,577][22314] Could not load from checkpoint, attempt 1
859
+ Traceback (most recent call last):
860
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
861
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
862
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
863
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
864
+ return _load(
865
+ ^^^^^^
866
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
867
+ result = unpickler.load()
868
+ ^^^^^^^^^^^^^^^^
869
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
870
+ elif full_path in _get_user_allowed_globals():
871
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
872
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
873
+ module, name = f.__module__, f.__qualname__
874
+ ^^^^^^^^^^^^
875
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
876
+ [2025-08-24 18:58:17,579][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
877
+ [2025-08-24 18:58:17,581][22314] Could not load from checkpoint, attempt 2
878
+ Traceback (most recent call last):
879
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
880
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
881
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
882
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
883
+ return _load(
884
+ ^^^^^^
885
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
886
+ result = unpickler.load()
887
+ ^^^^^^^^^^^^^^^^
888
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
889
+ elif full_path in _get_user_allowed_globals():
890
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
891
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
892
+ module, name = f.__module__, f.__qualname__
893
+ ^^^^^^^^^^^^
894
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
895
+ [2025-08-24 19:27:37,397][22314] Loading existing experiment configuration from /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/config.json
896
+ [2025-08-24 19:27:37,399][22314] Overriding arg 'num_workers' with value 1 passed from command line
897
+ [2025-08-24 19:27:37,399][22314] Adding new argument 'no_render'=True that is not in the saved config file!
898
+ [2025-08-24 19:27:37,400][22314] Adding new argument 'save_video'=True that is not in the saved config file!
899
+ [2025-08-24 19:27:37,401][22314] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
900
+ [2025-08-24 19:27:37,401][22314] Adding new argument 'video_name'=None that is not in the saved config file!
901
+ [2025-08-24 19:27:37,402][22314] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
902
+ [2025-08-24 19:27:37,402][22314] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
903
+ [2025-08-24 19:27:37,403][22314] Adding new argument 'push_to_hub'=False that is not in the saved config file!
904
+ [2025-08-24 19:27:37,404][22314] Adding new argument 'hf_repository'=None that is not in the saved config file!
905
+ [2025-08-24 19:27:37,404][22314] Adding new argument 'policy_index'=0 that is not in the saved config file!
906
+ [2025-08-24 19:27:37,405][22314] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
907
+ [2025-08-24 19:27:37,406][22314] Adding new argument 'train_script'=None that is not in the saved config file!
908
+ [2025-08-24 19:27:37,408][22314] Adding new argument 'enjoy_script'=None that is not in the saved config file!
909
+ [2025-08-24 19:27:37,409][22314] Using frameskip 1 and render_action_repeat=4 for evaluation
910
+ [2025-08-24 19:27:37,438][22314] RunningMeanStd input shape: (3, 72, 128)
911
+ [2025-08-24 19:27:37,441][22314] RunningMeanStd input shape: (1,)
912
+ [2025-08-24 19:27:37,453][22314] ConvEncoder: input_channels=3
913
+ [2025-08-24 19:27:37,493][22314] Conv encoder output size: 512
914
+ [2025-08-24 19:27:37,495][22314] Policy head output size: 512
915
+ [2025-08-24 19:27:37,517][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
916
+ [2025-08-24 19:27:37,521][22314] Could not load from checkpoint, attempt 0
917
+ Traceback (most recent call last):
918
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
919
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
920
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
921
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
922
+ return _load(
923
+ ^^^^^^
924
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
925
+ result = unpickler.load()
926
+ ^^^^^^^^^^^^^^^^
927
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
928
+ elif full_path in _get_user_allowed_globals():
929
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
930
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
931
+ module, name = f.__module__, f.__qualname__
932
+ ^^^^^^^^^^^^
933
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
934
+ [2025-08-24 19:27:37,523][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
935
+ [2025-08-24 19:27:37,525][22314] Could not load from checkpoint, attempt 1
936
+ Traceback (most recent call last):
937
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
938
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
939
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
940
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
941
+ return _load(
942
+ ^^^^^^
943
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
944
+ result = unpickler.load()
945
+ ^^^^^^^^^^^^^^^^
946
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
947
+ elif full_path in _get_user_allowed_globals():
948
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
949
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
950
+ module, name = f.__module__, f.__qualname__
951
+ ^^^^^^^^^^^^
952
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?
953
+ [2025-08-24 19:27:37,527][22314] Loading state from checkpoint /home/luckye/_DevelopmentCode/RL_Boot/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
954
+ [2025-08-24 19:27:37,529][22314] Could not load from checkpoint, attempt 2
955
+ Traceback (most recent call last):
956
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
957
+ checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
958
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
959
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 1521, in load
960
+ return _load(
961
+ ^^^^^^
962
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/serialization.py", line 2119, in _load
963
+ result = unpickler.load()
964
+ ^^^^^^^^^^^^^^^^
965
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 337, in load
966
+ elif full_path in _get_user_allowed_globals():
967
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
968
+ File "/home/luckye/_DevelopmentCode/huggingface_doom/sf_env/lib/python3.12/site-packages/torch/_weights_only_unpickler.py", line 144, in _get_user_allowed_globals
969
+ module, name = f.__module__, f.__qualname__
970
+ ^^^^^^^^^^^^
971
+ AttributeError: 'str' object has no attribute '__module__'. Did you mean: '__mod__'?