[2025-02-22 19:09:49,670][02012] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-02-22 19:09:49,672][02012] Rollout worker 0 uses device cpu [2025-02-22 19:09:49,672][02012] Rollout worker 1 uses device cpu [2025-02-22 19:09:49,673][02012] Rollout worker 2 uses device cpu [2025-02-22 19:09:49,674][02012] Rollout worker 3 uses device cpu [2025-02-22 19:09:49,675][02012] Rollout worker 4 uses device cpu [2025-02-22 19:09:49,676][02012] Rollout worker 5 uses device cpu [2025-02-22 19:09:49,676][02012] Rollout worker 6 uses device cpu [2025-02-22 19:09:49,677][02012] Rollout worker 7 uses device cpu [2025-02-22 19:09:49,763][02012] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:09:49,764][02012] InferenceWorker_p0-w0: min num requests: 2 [2025-02-22 19:09:49,797][02012] Starting all processes... [2025-02-22 19:09:49,798][02012] Starting process learner_proc0 [2025-02-22 19:09:49,862][02012] Starting all processes... [2025-02-22 19:09:49,867][02012] Starting process inference_proc0-0 [2025-02-22 19:09:49,868][02012] Starting process rollout_proc0 [2025-02-22 19:09:49,869][02012] Starting process rollout_proc1 [2025-02-22 19:09:49,869][02012] Starting process rollout_proc2 [2025-02-22 19:09:49,870][02012] Starting process rollout_proc3 [2025-02-22 19:09:49,870][02012] Starting process rollout_proc4 [2025-02-22 19:09:49,873][02012] Starting process rollout_proc5 [2025-02-22 19:09:49,873][02012] Starting process rollout_proc6 [2025-02-22 19:09:49,874][02012] Starting process rollout_proc7 [2025-02-22 19:09:52,307][04694] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:09:52,307][04694] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2025-02-22 19:09:52,323][04694] Num visible devices: 1 [2025-02-22 19:09:52,607][04701] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:52,825][04698] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:52,883][04696] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:52,947][04702] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:53,015][04695] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:53,016][04681] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:09:53,016][04681] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2025-02-22 19:09:53,034][04681] Num visible devices: 1 [2025-02-22 19:09:53,052][04697] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:53,053][04681] Starting seed is not provided [2025-02-22 19:09:53,053][04681] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:09:53,053][04681] Initializing actor-critic model on device cuda:0 [2025-02-22 19:09:53,054][04681] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:09:53,056][04681] RunningMeanStd input shape: (1,) [2025-02-22 19:09:53,071][04681] ConvEncoder: input_channels=3 [2025-02-22 19:09:53,176][04700] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:53,177][04699] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:09:53,319][04681] Conv encoder output size: 512 [2025-02-22 19:09:53,319][04681] Policy head output size: 512 [2025-02-22 19:09:53,364][04681] Created Actor Critic model with architecture: [2025-02-22 19:09:53,364][04681] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-02-22 19:09:53,557][04681] Using optimizer [2025-02-22 19:09:58,087][04681] No checkpoints found [2025-02-22 19:09:58,087][04681] Did not load from checkpoint, starting from scratch! [2025-02-22 19:09:58,087][04681] Initialized policy 0 weights for model version 0 [2025-02-22 19:09:58,090][04681] LearnerWorker_p0 finished initialization! [2025-02-22 19:09:58,090][04681] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:09:58,176][04694] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:09:58,177][04694] RunningMeanStd input shape: (1,) [2025-02-22 19:09:58,190][04694] ConvEncoder: input_channels=3 [2025-02-22 19:09:58,301][04694] Conv encoder output size: 512 [2025-02-22 19:09:58,301][04694] Policy head output size: 512 [2025-02-22 19:09:58,339][02012] Inference worker 0-0 is ready! [2025-02-22 19:09:58,340][02012] All inference workers are ready! Signal rollout workers to start! [2025-02-22 19:09:58,374][04696] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,374][04700] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04702] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04699] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04697] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04695] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04698] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,396][04701] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:09:58,689][04696] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,689][04700] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,689][04701] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,700][04697] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,721][04702] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,941][04698] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,943][04700] Decorrelating experience for 32 frames... [2025-02-22 19:09:58,953][04699] Decorrelating experience for 0 frames... [2025-02-22 19:09:58,968][04702] Decorrelating experience for 32 frames... [2025-02-22 19:09:58,997][04695] Decorrelating experience for 0 frames... [2025-02-22 19:09:59,194][04696] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,208][04699] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,241][04701] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,258][04695] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,346][04702] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,349][04698] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,433][04700] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,519][04697] Decorrelating experience for 32 frames... [2025-02-22 19:09:59,543][04696] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,616][04695] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,627][04699] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,642][02012] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-02-22 19:09:59,685][04702] Decorrelating experience for 96 frames... [2025-02-22 19:09:59,698][04701] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,855][04696] Decorrelating experience for 96 frames... [2025-02-22 19:09:59,889][04697] Decorrelating experience for 64 frames... [2025-02-22 19:09:59,949][04699] Decorrelating experience for 96 frames... [2025-02-22 19:09:59,974][04700] Decorrelating experience for 96 frames... [2025-02-22 19:10:00,004][04698] Decorrelating experience for 64 frames... [2025-02-22 19:10:00,155][04695] Decorrelating experience for 96 frames... [2025-02-22 19:10:00,202][04697] Decorrelating experience for 96 frames... [2025-02-22 19:10:00,264][04701] Decorrelating experience for 96 frames... [2025-02-22 19:10:00,299][04698] Decorrelating experience for 96 frames... [2025-02-22 19:10:01,290][04681] Signal inference workers to stop experience collection... [2025-02-22 19:10:01,303][04694] InferenceWorker_p0-w0: stopping experience collection [2025-02-22 19:10:02,574][04681] Signal inference workers to resume experience collection... [2025-02-22 19:10:02,575][04694] InferenceWorker_p0-w0: resuming experience collection [2025-02-22 19:10:04,611][04694] Updated weights for policy 0, policy_version 10 (0.0012) [2025-02-22 19:10:04,642][02012] Fps is (10 sec: 8192.2, 60 sec: 8192.2, 300 sec: 8192.2). Total num frames: 40960. Throughput: 0: 1866.0. Samples: 9330. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2025-02-22 19:10:04,644][02012] Avg episode reward: [(0, '3.890')] [2025-02-22 19:10:06,993][04694] Updated weights for policy 0, policy_version 20 (0.0012) [2025-02-22 19:10:09,460][04694] Updated weights for policy 0, policy_version 30 (0.0013) [2025-02-22 19:10:09,642][02012] Fps is (10 sec: 12288.0, 60 sec: 12288.0, 300 sec: 12288.0). Total num frames: 122880. Throughput: 0: 2224.0. Samples: 22240. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2025-02-22 19:10:09,644][02012] Avg episode reward: [(0, '4.504')] [2025-02-22 19:10:09,646][04681] Saving new best policy, reward=4.504! [2025-02-22 19:10:09,755][02012] Heartbeat connected on Batcher_0 [2025-02-22 19:10:09,759][02012] Heartbeat connected on LearnerWorker_p0 [2025-02-22 19:10:09,770][02012] Heartbeat connected on RolloutWorker_w0 [2025-02-22 19:10:09,771][02012] Heartbeat connected on InferenceWorker_p0-w0 [2025-02-22 19:10:09,776][02012] Heartbeat connected on RolloutWorker_w1 [2025-02-22 19:10:09,783][02012] Heartbeat connected on RolloutWorker_w2 [2025-02-22 19:10:09,784][02012] Heartbeat connected on RolloutWorker_w3 [2025-02-22 19:10:09,787][02012] Heartbeat connected on RolloutWorker_w4 [2025-02-22 19:10:09,792][02012] Heartbeat connected on RolloutWorker_w5 [2025-02-22 19:10:09,795][02012] Heartbeat connected on RolloutWorker_w6 [2025-02-22 19:10:09,799][02012] Heartbeat connected on RolloutWorker_w7 [2025-02-22 19:10:11,857][04694] Updated weights for policy 0, policy_version 40 (0.0012) [2025-02-22 19:10:14,232][04694] Updated weights for policy 0, policy_version 50 (0.0012) [2025-02-22 19:10:14,642][02012] Fps is (10 sec: 16793.5, 60 sec: 13926.5, 300 sec: 13926.5). Total num frames: 208896. Throughput: 0: 3176.7. Samples: 47650. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2025-02-22 19:10:14,644][02012] Avg episode reward: [(0, '4.390')] [2025-02-22 19:10:16,613][04694] Updated weights for policy 0, policy_version 60 (0.0013) [2025-02-22 19:10:19,001][04694] Updated weights for policy 0, policy_version 70 (0.0013) [2025-02-22 19:10:19,642][02012] Fps is (10 sec: 17203.4, 60 sec: 14745.7, 300 sec: 14745.7). Total num frames: 294912. Throughput: 0: 3674.7. Samples: 73494. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2025-02-22 19:10:19,643][02012] Avg episode reward: [(0, '4.451')] [2025-02-22 19:10:21,485][04694] Updated weights for policy 0, policy_version 80 (0.0013) [2025-02-22 19:10:23,973][04694] Updated weights for policy 0, policy_version 90 (0.0012) [2025-02-22 19:10:24,642][02012] Fps is (10 sec: 16793.5, 60 sec: 15073.3, 300 sec: 15073.3). Total num frames: 376832. Throughput: 0: 3427.8. Samples: 85694. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:10:24,644][02012] Avg episode reward: [(0, '4.467')] [2025-02-22 19:10:26,382][04694] Updated weights for policy 0, policy_version 100 (0.0012) [2025-02-22 19:10:28,770][04694] Updated weights for policy 0, policy_version 110 (0.0012) [2025-02-22 19:10:29,642][02012] Fps is (10 sec: 16793.6, 60 sec: 15428.3, 300 sec: 15428.3). Total num frames: 462848. Throughput: 0: 3706.8. Samples: 111202. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:10:29,644][02012] Avg episode reward: [(0, '4.564')] [2025-02-22 19:10:29,645][04681] Saving new best policy, reward=4.564! [2025-02-22 19:10:31,127][04694] Updated weights for policy 0, policy_version 120 (0.0012) [2025-02-22 19:10:33,498][04694] Updated weights for policy 0, policy_version 130 (0.0013) [2025-02-22 19:10:34,642][02012] Fps is (10 sec: 17203.4, 60 sec: 15681.9, 300 sec: 15681.9). Total num frames: 548864. Throughput: 0: 3916.0. Samples: 137058. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:10:34,644][02012] Avg episode reward: [(0, '4.619')] [2025-02-22 19:10:34,651][04681] Saving new best policy, reward=4.619! [2025-02-22 19:10:35,969][04694] Updated weights for policy 0, policy_version 140 (0.0013) [2025-02-22 19:10:38,249][04694] Updated weights for policy 0, policy_version 150 (0.0012) [2025-02-22 19:10:39,642][02012] Fps is (10 sec: 17612.6, 60 sec: 15974.4, 300 sec: 15974.4). Total num frames: 638976. Throughput: 0: 3745.3. Samples: 149810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:10:39,644][02012] Avg episode reward: [(0, '4.553')] [2025-02-22 19:10:40,380][04694] Updated weights for policy 0, policy_version 160 (0.0012) [2025-02-22 19:10:42,638][04694] Updated weights for policy 0, policy_version 170 (0.0012) [2025-02-22 19:10:44,642][02012] Fps is (10 sec: 18431.7, 60 sec: 16293.0, 300 sec: 16293.0). Total num frames: 733184. Throughput: 0: 3951.7. Samples: 177828. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:10:44,644][02012] Avg episode reward: [(0, '4.537')] [2025-02-22 19:10:44,808][04694] Updated weights for policy 0, policy_version 180 (0.0012) [2025-02-22 19:10:46,978][04694] Updated weights for policy 0, policy_version 190 (0.0012) [2025-02-22 19:10:49,144][04694] Updated weights for policy 0, policy_version 200 (0.0012) [2025-02-22 19:10:49,642][02012] Fps is (10 sec: 18841.5, 60 sec: 16547.8, 300 sec: 16547.8). Total num frames: 827392. Throughput: 0: 4378.2. Samples: 206348. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:10:49,644][02012] Avg episode reward: [(0, '4.574')] [2025-02-22 19:10:51,368][04694] Updated weights for policy 0, policy_version 210 (0.0013) [2025-02-22 19:10:53,772][04694] Updated weights for policy 0, policy_version 220 (0.0012) [2025-02-22 19:10:54,642][02012] Fps is (10 sec: 18022.6, 60 sec: 16607.5, 300 sec: 16607.5). Total num frames: 913408. Throughput: 0: 4387.5. Samples: 219676. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:10:54,644][02012] Avg episode reward: [(0, '4.486')] [2025-02-22 19:10:56,152][04694] Updated weights for policy 0, policy_version 230 (0.0012) [2025-02-22 19:10:58,553][04694] Updated weights for policy 0, policy_version 240 (0.0012) [2025-02-22 19:10:59,642][02012] Fps is (10 sec: 17203.3, 60 sec: 16657.1, 300 sec: 16657.1). Total num frames: 999424. Throughput: 0: 4395.9. Samples: 245466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:10:59,644][02012] Avg episode reward: [(0, '4.702')] [2025-02-22 19:10:59,645][04681] Saving new best policy, reward=4.702! [2025-02-22 19:11:01,040][04694] Updated weights for policy 0, policy_version 250 (0.0013) [2025-02-22 19:11:03,431][04694] Updated weights for policy 0, policy_version 260 (0.0012) [2025-02-22 19:11:04,642][02012] Fps is (10 sec: 17202.9, 60 sec: 17408.0, 300 sec: 16699.1). Total num frames: 1085440. Throughput: 0: 4385.5. Samples: 270842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:04,644][02012] Avg episode reward: [(0, '5.309')] [2025-02-22 19:11:04,649][04681] Saving new best policy, reward=5.309! [2025-02-22 19:11:05,801][04694] Updated weights for policy 0, policy_version 270 (0.0012) [2025-02-22 19:11:08,166][04694] Updated weights for policy 0, policy_version 280 (0.0012) [2025-02-22 19:11:09,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17476.3, 300 sec: 16735.1). Total num frames: 1171456. Throughput: 0: 4400.8. Samples: 283732. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:11:09,644][02012] Avg episode reward: [(0, '6.151')] [2025-02-22 19:11:09,646][04681] Saving new best policy, reward=6.151! [2025-02-22 19:11:10,526][04694] Updated weights for policy 0, policy_version 290 (0.0012) [2025-02-22 19:11:12,958][04694] Updated weights for policy 0, policy_version 300 (0.0013) [2025-02-22 19:11:14,642][02012] Fps is (10 sec: 16793.7, 60 sec: 17408.0, 300 sec: 16711.7). Total num frames: 1253376. Throughput: 0: 4402.9. Samples: 309332. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:14,644][02012] Avg episode reward: [(0, '7.167')] [2025-02-22 19:11:14,655][04681] Saving new best policy, reward=7.167! [2025-02-22 19:11:15,372][04694] Updated weights for policy 0, policy_version 310 (0.0013) [2025-02-22 19:11:17,746][04694] Updated weights for policy 0, policy_version 320 (0.0011) [2025-02-22 19:11:19,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17476.2, 300 sec: 16793.6). Total num frames: 1343488. Throughput: 0: 4405.0. Samples: 335282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:19,645][02012] Avg episode reward: [(0, '8.096')] [2025-02-22 19:11:19,647][04681] Saving new best policy, reward=8.096! [2025-02-22 19:11:20,081][04694] Updated weights for policy 0, policy_version 330 (0.0012) [2025-02-22 19:11:22,446][04694] Updated weights for policy 0, policy_version 340 (0.0012) [2025-02-22 19:11:24,642][02012] Fps is (10 sec: 17613.0, 60 sec: 17544.6, 300 sec: 16817.7). Total num frames: 1429504. Throughput: 0: 4410.2. Samples: 348268. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:24,643][02012] Avg episode reward: [(0, '8.985')] [2025-02-22 19:11:24,649][04681] Saving new best policy, reward=8.985! [2025-02-22 19:11:24,869][04694] Updated weights for policy 0, policy_version 350 (0.0013) [2025-02-22 19:11:27,338][04694] Updated weights for policy 0, policy_version 360 (0.0012) [2025-02-22 19:11:29,642][02012] Fps is (10 sec: 16793.8, 60 sec: 17476.3, 300 sec: 16793.6). Total num frames: 1511424. Throughput: 0: 4351.5. Samples: 373646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:11:29,643][02012] Avg episode reward: [(0, '9.143')] [2025-02-22 19:11:29,645][04681] Saving new best policy, reward=9.143! [2025-02-22 19:11:29,758][04694] Updated weights for policy 0, policy_version 370 (0.0012) [2025-02-22 19:11:32,066][04694] Updated weights for policy 0, policy_version 380 (0.0012) [2025-02-22 19:11:34,428][04694] Updated weights for policy 0, policy_version 390 (0.0012) [2025-02-22 19:11:34,642][02012] Fps is (10 sec: 16793.4, 60 sec: 17476.2, 300 sec: 16815.2). Total num frames: 1597440. Throughput: 0: 4294.9. Samples: 399618. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:11:34,644][02012] Avg episode reward: [(0, '10.116')] [2025-02-22 19:11:34,660][04681] Saving new best policy, reward=10.116! [2025-02-22 19:11:36,818][04694] Updated weights for policy 0, policy_version 400 (0.0012) [2025-02-22 19:11:39,255][04694] Updated weights for policy 0, policy_version 410 (0.0013) [2025-02-22 19:11:39,642][02012] Fps is (10 sec: 17202.9, 60 sec: 17408.0, 300 sec: 16834.6). Total num frames: 1683456. Throughput: 0: 4283.6. Samples: 412440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:11:39,644][02012] Avg episode reward: [(0, '11.845')] [2025-02-22 19:11:39,645][04681] Saving new best policy, reward=11.845! [2025-02-22 19:11:41,637][04694] Updated weights for policy 0, policy_version 420 (0.0012) [2025-02-22 19:11:43,995][04694] Updated weights for policy 0, policy_version 430 (0.0012) [2025-02-22 19:11:44,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17271.5, 300 sec: 16852.1). Total num frames: 1769472. Throughput: 0: 4280.2. Samples: 438074. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:11:44,644][02012] Avg episode reward: [(0, '12.955')] [2025-02-22 19:11:44,650][04681] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000432_1769472.pth... [2025-02-22 19:11:44,729][04681] Saving new best policy, reward=12.955! [2025-02-22 19:11:46,364][04694] Updated weights for policy 0, policy_version 440 (0.0012) [2025-02-22 19:11:48,752][04694] Updated weights for policy 0, policy_version 450 (0.0012) [2025-02-22 19:11:49,642][02012] Fps is (10 sec: 17203.5, 60 sec: 17135.0, 300 sec: 16868.1). Total num frames: 1855488. Throughput: 0: 4290.1. Samples: 463894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:49,643][02012] Avg episode reward: [(0, '14.850')] [2025-02-22 19:11:49,645][04681] Saving new best policy, reward=14.850! [2025-02-22 19:11:51,219][04694] Updated weights for policy 0, policy_version 460 (0.0012) [2025-02-22 19:11:53,638][04694] Updated weights for policy 0, policy_version 470 (0.0012) [2025-02-22 19:11:54,642][02012] Fps is (10 sec: 17203.4, 60 sec: 17134.9, 300 sec: 16882.7). Total num frames: 1941504. Throughput: 0: 4278.9. Samples: 476284. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:54,644][02012] Avg episode reward: [(0, '14.658')] [2025-02-22 19:11:56,007][04694] Updated weights for policy 0, policy_version 480 (0.0012) [2025-02-22 19:11:58,355][04694] Updated weights for policy 0, policy_version 490 (0.0013) [2025-02-22 19:11:59,642][02012] Fps is (10 sec: 17203.0, 60 sec: 17134.9, 300 sec: 16896.0). Total num frames: 2027520. Throughput: 0: 4287.4. Samples: 502266. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:11:59,644][02012] Avg episode reward: [(0, '16.380')] [2025-02-22 19:11:59,645][04681] Saving new best policy, reward=16.380! [2025-02-22 19:12:00,680][04694] Updated weights for policy 0, policy_version 500 (0.0012) [2025-02-22 19:12:03,016][04694] Updated weights for policy 0, policy_version 510 (0.0012) [2025-02-22 19:12:04,642][02012] Fps is (10 sec: 17203.0, 60 sec: 17134.9, 300 sec: 16908.3). Total num frames: 2113536. Throughput: 0: 4290.4. Samples: 528348. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:04,643][02012] Avg episode reward: [(0, '16.847')] [2025-02-22 19:12:04,650][04681] Saving new best policy, reward=16.847! [2025-02-22 19:12:05,419][04694] Updated weights for policy 0, policy_version 520 (0.0012) [2025-02-22 19:12:07,741][04694] Updated weights for policy 0, policy_version 530 (0.0012) [2025-02-22 19:12:09,642][02012] Fps is (10 sec: 17612.8, 60 sec: 17203.2, 300 sec: 16951.1). Total num frames: 2203648. Throughput: 0: 4294.3. Samples: 541510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:09,644][02012] Avg episode reward: [(0, '16.929')] [2025-02-22 19:12:09,645][04681] Saving new best policy, reward=16.929! [2025-02-22 19:12:10,090][04694] Updated weights for policy 0, policy_version 540 (0.0012) [2025-02-22 19:12:12,436][04694] Updated weights for policy 0, policy_version 550 (0.0013) [2025-02-22 19:12:14,642][02012] Fps is (10 sec: 17612.8, 60 sec: 17271.5, 300 sec: 16960.5). Total num frames: 2289664. Throughput: 0: 4312.8. Samples: 567722. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:14,644][02012] Avg episode reward: [(0, '17.134')] [2025-02-22 19:12:14,649][04681] Saving new best policy, reward=17.134! [2025-02-22 19:12:14,796][04694] Updated weights for policy 0, policy_version 560 (0.0012) [2025-02-22 19:12:17,266][04694] Updated weights for policy 0, policy_version 570 (0.0013) [2025-02-22 19:12:19,628][04694] Updated weights for policy 0, policy_version 580 (0.0012) [2025-02-22 19:12:19,642][02012] Fps is (10 sec: 17203.4, 60 sec: 17203.2, 300 sec: 16969.2). Total num frames: 2375680. Throughput: 0: 4302.3. Samples: 593222. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:19,643][02012] Avg episode reward: [(0, '20.014')] [2025-02-22 19:12:19,645][04681] Saving new best policy, reward=20.014! [2025-02-22 19:12:21,958][04694] Updated weights for policy 0, policy_version 590 (0.0012) [2025-02-22 19:12:24,299][04694] Updated weights for policy 0, policy_version 600 (0.0012) [2025-02-22 19:12:24,642][02012] Fps is (10 sec: 17203.4, 60 sec: 17203.2, 300 sec: 16977.2). Total num frames: 2461696. Throughput: 0: 4309.5. Samples: 606366. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:24,644][02012] Avg episode reward: [(0, '21.659')] [2025-02-22 19:12:24,648][04681] Saving new best policy, reward=21.659! [2025-02-22 19:12:26,658][04694] Updated weights for policy 0, policy_version 610 (0.0012) [2025-02-22 19:12:29,061][04694] Updated weights for policy 0, policy_version 620 (0.0012) [2025-02-22 19:12:29,642][02012] Fps is (10 sec: 17203.0, 60 sec: 17271.4, 300 sec: 16984.8). Total num frames: 2547712. Throughput: 0: 4316.3. Samples: 632308. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:12:29,644][02012] Avg episode reward: [(0, '18.398')] [2025-02-22 19:12:31,466][04694] Updated weights for policy 0, policy_version 630 (0.0013) [2025-02-22 19:12:33,792][04694] Updated weights for policy 0, policy_version 640 (0.0012) [2025-02-22 19:12:34,642][02012] Fps is (10 sec: 17202.9, 60 sec: 17271.4, 300 sec: 16991.8). Total num frames: 2633728. Throughput: 0: 4317.1. Samples: 658166. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-22 19:12:34,644][02012] Avg episode reward: [(0, '20.156')] [2025-02-22 19:12:36,136][04694] Updated weights for policy 0, policy_version 650 (0.0012) [2025-02-22 19:12:38,504][04694] Updated weights for policy 0, policy_version 660 (0.0012) [2025-02-22 19:12:39,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17271.5, 300 sec: 16998.4). Total num frames: 2719744. Throughput: 0: 4332.0. Samples: 671226. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:39,643][02012] Avg episode reward: [(0, '19.847')] [2025-02-22 19:12:40,900][04694] Updated weights for policy 0, policy_version 670 (0.0013) [2025-02-22 19:12:43,366][04694] Updated weights for policy 0, policy_version 680 (0.0012) [2025-02-22 19:12:44,642][02012] Fps is (10 sec: 17203.5, 60 sec: 17271.5, 300 sec: 17004.6). Total num frames: 2805760. Throughput: 0: 4321.8. Samples: 696746. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:44,643][02012] Avg episode reward: [(0, '20.261')] [2025-02-22 19:12:45,718][04694] Updated weights for policy 0, policy_version 690 (0.0012) [2025-02-22 19:12:48,112][04694] Updated weights for policy 0, policy_version 700 (0.0012) [2025-02-22 19:12:49,642][02012] Fps is (10 sec: 17203.3, 60 sec: 17271.4, 300 sec: 17010.5). Total num frames: 2891776. Throughput: 0: 4316.3. Samples: 722580. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:49,644][02012] Avg episode reward: [(0, '19.804')] [2025-02-22 19:12:50,501][04694] Updated weights for policy 0, policy_version 710 (0.0013) [2025-02-22 19:12:52,834][04694] Updated weights for policy 0, policy_version 720 (0.0012) [2025-02-22 19:12:54,642][02012] Fps is (10 sec: 17203.1, 60 sec: 17271.4, 300 sec: 17016.0). Total num frames: 2977792. Throughput: 0: 4315.9. Samples: 735724. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:12:54,643][02012] Avg episode reward: [(0, '17.493')] [2025-02-22 19:12:55,288][04694] Updated weights for policy 0, policy_version 730 (0.0012) [2025-02-22 19:12:57,667][04694] Updated weights for policy 0, policy_version 740 (0.0012) [2025-02-22 19:12:59,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17271.5, 300 sec: 17021.2). Total num frames: 3063808. Throughput: 0: 4302.2. Samples: 761320. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-22 19:12:59,644][02012] Avg episode reward: [(0, '19.754')] [2025-02-22 19:13:00,024][04694] Updated weights for policy 0, policy_version 750 (0.0012) [2025-02-22 19:13:02,416][04694] Updated weights for policy 0, policy_version 760 (0.0012) [2025-02-22 19:13:04,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17271.5, 300 sec: 17026.1). Total num frames: 3149824. Throughput: 0: 4310.7. Samples: 787206. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-22 19:13:04,643][02012] Avg episode reward: [(0, '23.172')] [2025-02-22 19:13:04,649][04681] Saving new best policy, reward=23.172! [2025-02-22 19:13:04,781][04694] Updated weights for policy 0, policy_version 770 (0.0012) [2025-02-22 19:13:07,197][04694] Updated weights for policy 0, policy_version 780 (0.0013) [2025-02-22 19:13:09,589][04694] Updated weights for policy 0, policy_version 790 (0.0012) [2025-02-22 19:13:09,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17203.2, 300 sec: 17030.7). Total num frames: 3235840. Throughput: 0: 4300.5. Samples: 799888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-22 19:13:09,644][02012] Avg episode reward: [(0, '21.966')] [2025-02-22 19:13:11,920][04694] Updated weights for policy 0, policy_version 800 (0.0012) [2025-02-22 19:13:14,278][04694] Updated weights for policy 0, policy_version 810 (0.0012) [2025-02-22 19:13:14,642][02012] Fps is (10 sec: 17203.1, 60 sec: 17203.2, 300 sec: 17035.2). Total num frames: 3321856. Throughput: 0: 4301.2. Samples: 825862. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:13:14,644][02012] Avg episode reward: [(0, '21.500')] [2025-02-22 19:13:16,631][04694] Updated weights for policy 0, policy_version 820 (0.0012) [2025-02-22 19:13:18,939][04694] Updated weights for policy 0, policy_version 830 (0.0012) [2025-02-22 19:13:19,642][02012] Fps is (10 sec: 17203.3, 60 sec: 17203.2, 300 sec: 17039.4). Total num frames: 3407872. Throughput: 0: 4315.4. Samples: 852360. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-22 19:13:19,644][02012] Avg episode reward: [(0, '22.072')] [2025-02-22 19:13:21,325][04694] Updated weights for policy 0, policy_version 840 (0.0012) [2025-02-22 19:13:23,645][04694] Updated weights for policy 0, policy_version 850 (0.0013) [2025-02-22 19:13:24,642][02012] Fps is (10 sec: 17612.7, 60 sec: 17271.4, 300 sec: 17063.3). Total num frames: 3497984. Throughput: 0: 4314.7. Samples: 865388. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:13:24,644][02012] Avg episode reward: [(0, '21.698')] [2025-02-22 19:13:26,011][04694] Updated weights for policy 0, policy_version 860 (0.0011) [2025-02-22 19:13:28,315][04694] Updated weights for policy 0, policy_version 870 (0.0012) [2025-02-22 19:13:29,642][02012] Fps is (10 sec: 17612.9, 60 sec: 17271.5, 300 sec: 17066.7). Total num frames: 3584000. Throughput: 0: 4330.8. Samples: 891634. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-22 19:13:29,644][02012] Avg episode reward: [(0, '22.091')] [2025-02-22 19:13:30,663][04694] Updated weights for policy 0, policy_version 880 (0.0012) [2025-02-22 19:13:33,119][04694] Updated weights for policy 0, policy_version 890 (0.0012) [2025-02-22 19:13:34,642][02012] Fps is (10 sec: 17203.3, 60 sec: 17271.5, 300 sec: 17069.8). Total num frames: 3670016. Throughput: 0: 4328.4. Samples: 917358. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-22 19:13:34,644][02012] Avg episode reward: [(0, '22.206')] [2025-02-22 19:13:35,383][04694] Updated weights for policy 0, policy_version 900 (0.0012) [2025-02-22 19:13:37,525][04694] Updated weights for policy 0, policy_version 910 (0.0013) [2025-02-22 19:13:39,642][02012] Fps is (10 sec: 18022.4, 60 sec: 17408.0, 300 sec: 17110.1). Total num frames: 3764224. Throughput: 0: 4356.2. Samples: 931754. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:13:39,643][02012] Avg episode reward: [(0, '21.588')] [2025-02-22 19:13:39,774][04694] Updated weights for policy 0, policy_version 920 (0.0012) [2025-02-22 19:13:42,015][04694] Updated weights for policy 0, policy_version 930 (0.0012) [2025-02-22 19:13:44,161][04694] Updated weights for policy 0, policy_version 940 (0.0012) [2025-02-22 19:13:44,642][02012] Fps is (10 sec: 18841.6, 60 sec: 17544.5, 300 sec: 17148.6). Total num frames: 3858432. Throughput: 0: 4400.4. Samples: 959338. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:13:44,644][02012] Avg episode reward: [(0, '22.546')] [2025-02-22 19:13:44,651][04681] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000942_3858432.pth... [2025-02-22 19:13:46,395][04694] Updated weights for policy 0, policy_version 950 (0.0012) [2025-02-22 19:13:48,754][04694] Updated weights for policy 0, policy_version 960 (0.0012) [2025-02-22 19:13:49,642][02012] Fps is (10 sec: 18022.3, 60 sec: 17544.5, 300 sec: 17149.8). Total num frames: 3944448. Throughput: 0: 4427.1. Samples: 986426. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2025-02-22 19:13:49,644][02012] Avg episode reward: [(0, '21.779')] [2025-02-22 19:13:51,110][04694] Updated weights for policy 0, policy_version 970 (0.0012) [2025-02-22 19:13:53,454][04694] Updated weights for policy 0, policy_version 980 (0.0012) [2025-02-22 19:13:54,642][02012] Fps is (10 sec: 17612.8, 60 sec: 17612.8, 300 sec: 17168.3). Total num frames: 4034560. Throughput: 0: 4434.6. Samples: 999446. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2025-02-22 19:13:54,644][02012] Avg episode reward: [(0, '22.955')] [2025-02-22 19:13:55,799][04694] Updated weights for policy 0, policy_version 990 (0.0012) [2025-02-22 19:13:58,255][04694] Updated weights for policy 0, policy_version 1000 (0.0013) [2025-02-22 19:13:59,642][02012] Fps is (10 sec: 17203.2, 60 sec: 17544.5, 300 sec: 17152.0). Total num frames: 4116480. Throughput: 0: 4428.9. Samples: 1025160. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:13:59,644][02012] Avg episode reward: [(0, '21.506')] [2025-02-22 19:14:00,623][04694] Updated weights for policy 0, policy_version 1010 (0.0012) [2025-02-22 19:14:02,998][04694] Updated weights for policy 0, policy_version 1020 (0.0013) [2025-02-22 19:14:04,642][02012] Fps is (10 sec: 16793.6, 60 sec: 17544.5, 300 sec: 17153.0). Total num frames: 4202496. Throughput: 0: 4416.2. Samples: 1051088. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:14:04,644][02012] Avg episode reward: [(0, '22.015')] [2025-02-22 19:14:04,652][04681] Stopping Batcher_0... [2025-02-22 19:14:04,652][04681] Loop batcher_evt_loop terminating... [2025-02-22 19:14:04,652][02012] Component Batcher_0 stopped! [2025-02-22 19:14:04,653][04681] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:14:04,675][04694] Weights refcount: 2 0 [2025-02-22 19:14:04,677][04694] Stopping InferenceWorker_p0-w0... [2025-02-22 19:14:04,677][04694] Loop inference_proc0-0_evt_loop terminating... [2025-02-22 19:14:04,677][02012] Component InferenceWorker_p0-w0 stopped! [2025-02-22 19:14:04,697][04700] Stopping RolloutWorker_w7... [2025-02-22 19:14:04,697][04700] Loop rollout_proc7_evt_loop terminating... [2025-02-22 19:14:04,699][04699] Stopping RolloutWorker_w3... [2025-02-22 19:14:04,699][04699] Loop rollout_proc3_evt_loop terminating... [2025-02-22 19:14:04,697][02012] Component RolloutWorker_w7 stopped! [2025-02-22 19:14:04,700][04702] Stopping RolloutWorker_w6... [2025-02-22 19:14:04,700][04702] Loop rollout_proc6_evt_loop terminating... [2025-02-22 19:14:04,700][02012] Component RolloutWorker_w3 stopped! [2025-02-22 19:14:04,701][02012] Component RolloutWorker_w6 stopped! [2025-02-22 19:14:04,703][04698] Stopping RolloutWorker_w4... [2025-02-22 19:14:04,704][04698] Loop rollout_proc4_evt_loop terminating... [2025-02-22 19:14:04,704][04697] Stopping RolloutWorker_w2... [2025-02-22 19:14:04,703][02012] Component RolloutWorker_w4 stopped! [2025-02-22 19:14:04,704][04697] Loop rollout_proc2_evt_loop terminating... [2025-02-22 19:14:04,704][04696] Stopping RolloutWorker_w1... [2025-02-22 19:14:04,705][04696] Loop rollout_proc1_evt_loop terminating... [2025-02-22 19:14:04,705][04701] Stopping RolloutWorker_w5... [2025-02-22 19:14:04,705][02012] Component RolloutWorker_w2 stopped! [2025-02-22 19:14:04,706][04695] Stopping RolloutWorker_w0... [2025-02-22 19:14:04,706][02012] Component RolloutWorker_w1 stopped! [2025-02-22 19:14:04,707][04695] Loop rollout_proc0_evt_loop terminating... [2025-02-22 19:14:04,706][04701] Loop rollout_proc5_evt_loop terminating... [2025-02-22 19:14:04,707][02012] Component RolloutWorker_w5 stopped! [2025-02-22 19:14:04,708][02012] Component RolloutWorker_w0 stopped! [2025-02-22 19:14:04,731][04681] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000432_1769472.pth [2025-02-22 19:14:04,741][04681] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:14:04,859][04681] Stopping LearnerWorker_p0... [2025-02-22 19:14:04,860][04681] Loop learner_proc0_evt_loop terminating... [2025-02-22 19:14:04,860][02012] Component LearnerWorker_p0 stopped! [2025-02-22 19:14:04,862][02012] Waiting for process learner_proc0 to stop... [2025-02-22 19:14:05,913][02012] Waiting for process inference_proc0-0 to join... [2025-02-22 19:14:05,914][02012] Waiting for process rollout_proc0 to join... [2025-02-22 19:14:05,915][02012] Waiting for process rollout_proc1 to join... [2025-02-22 19:14:05,916][02012] Waiting for process rollout_proc2 to join... [2025-02-22 19:14:05,917][02012] Waiting for process rollout_proc3 to join... [2025-02-22 19:14:05,918][02012] Waiting for process rollout_proc4 to join... [2025-02-22 19:14:05,919][02012] Waiting for process rollout_proc5 to join... [2025-02-22 19:14:05,920][02012] Waiting for process rollout_proc6 to join... [2025-02-22 19:14:05,921][02012] Waiting for process rollout_proc7 to join... [2025-02-22 19:14:05,921][02012] Batcher 0 profile tree view: batching: 10.8347, releasing_batches: 0.0335 [2025-02-22 19:14:05,922][02012] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 4.2625 update_model: 4.1878 weight_update: 0.0013 one_step: 0.0035 handle_policy_step: 226.4649 deserialize: 9.8307, stack: 1.6521, obs_to_device_normalize: 52.9953, forward: 113.4508, send_messages: 15.3827 prepare_outputs: 24.8177 to_cpu: 14.3459 [2025-02-22 19:14:05,923][02012] Learner 0 profile tree view: misc: 0.0045, prepare_batch: 10.2700 train: 25.3999 epoch_init: 0.0049, minibatch_init: 0.0078, losses_postprocess: 0.2802, kl_divergence: 0.3631, after_optimizer: 5.3335 calculate_losses: 10.6395 losses_init: 0.0035, forward_head: 0.8904, bptt_initial: 6.2745, tail: 0.6728, advantages_returns: 0.1863, losses: 1.1900 bptt: 1.2299 bptt_forward_core: 1.1742 update: 8.3607 clip: 0.9202 [2025-02-22 19:14:05,924][02012] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.1401, enqueue_policy_requests: 11.0559, env_step: 147.8510, overhead: 6.7190, complete_rollouts: 0.2730 save_policy_outputs: 9.1621 split_output_tensors: 3.5299 [2025-02-22 19:14:05,924][02012] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.1416, enqueue_policy_requests: 11.0795, env_step: 147.6237, overhead: 6.8061, complete_rollouts: 0.2802 save_policy_outputs: 9.2647 split_output_tensors: 3.5787 [2025-02-22 19:14:05,925][02012] Loop Runner_EvtLoop terminating... [2025-02-22 19:14:05,927][02012] Runner profile tree view: main_loop: 256.1295 [2025-02-22 19:14:05,927][02012] Collected {0: 4206592}, FPS: 16423.7 [2025-02-22 19:14:45,013][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:14:45,014][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:14:45,014][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:14:45,015][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:14:45,016][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:14:45,016][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:14:45,017][02012] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:14:45,018][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:14:45,019][02012] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-02-22 19:14:45,019][02012] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-02-22 19:14:45,020][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:14:45,021][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:14:45,021][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:14:45,022][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:14:45,023][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:14:45,053][02012] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:14:45,055][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:14:45,057][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:14:45,072][02012] ConvEncoder: input_channels=3 [2025-02-22 19:14:45,185][02012] Conv encoder output size: 512 [2025-02-22 19:14:45,186][02012] Policy head output size: 512 [2025-02-22 19:14:45,344][02012] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:14:46,211][02012] Num frames 100... [2025-02-22 19:14:46,362][02012] Num frames 200... [2025-02-22 19:14:46,512][02012] Num frames 300... [2025-02-22 19:14:46,661][02012] Num frames 400... [2025-02-22 19:14:46,808][02012] Num frames 500... [2025-02-22 19:14:46,975][02012] Avg episode rewards: #0: 12.760, true rewards: #0: 5.760 [2025-02-22 19:14:46,976][02012] Avg episode reward: 12.760, avg true_objective: 5.760 [2025-02-22 19:14:47,012][02012] Num frames 600... [2025-02-22 19:14:47,160][02012] Num frames 700... [2025-02-22 19:14:47,306][02012] Num frames 800... [2025-02-22 19:14:47,454][02012] Num frames 900... [2025-02-22 19:14:47,598][02012] Num frames 1000... [2025-02-22 19:14:47,747][02012] Num frames 1100... [2025-02-22 19:14:47,894][02012] Num frames 1200... [2025-02-22 19:14:48,041][02012] Num frames 1300... [2025-02-22 19:14:48,190][02012] Num frames 1400... [2025-02-22 19:14:48,337][02012] Num frames 1500... [2025-02-22 19:14:48,481][02012] Num frames 1600... [2025-02-22 19:14:48,627][02012] Num frames 1700... [2025-02-22 19:14:48,753][02012] Avg episode rewards: #0: 20.240, true rewards: #0: 8.740 [2025-02-22 19:14:48,754][02012] Avg episode reward: 20.240, avg true_objective: 8.740 [2025-02-22 19:14:48,829][02012] Num frames 1800... [2025-02-22 19:14:48,972][02012] Num frames 1900... [2025-02-22 19:14:49,115][02012] Num frames 2000... [2025-02-22 19:14:49,266][02012] Num frames 2100... [2025-02-22 19:14:49,417][02012] Num frames 2200... [2025-02-22 19:14:49,513][02012] Avg episode rewards: #0: 16.093, true rewards: #0: 7.427 [2025-02-22 19:14:49,514][02012] Avg episode reward: 16.093, avg true_objective: 7.427 [2025-02-22 19:14:49,621][02012] Num frames 2300... [2025-02-22 19:14:49,768][02012] Num frames 2400... [2025-02-22 19:14:49,913][02012] Num frames 2500... [2025-02-22 19:14:50,064][02012] Num frames 2600... [2025-02-22 19:14:50,212][02012] Num frames 2700... [2025-02-22 19:14:50,361][02012] Num frames 2800... [2025-02-22 19:14:50,512][02012] Num frames 2900... [2025-02-22 19:14:50,663][02012] Num frames 3000... [2025-02-22 19:14:50,813][02012] Num frames 3100... [2025-02-22 19:14:50,962][02012] Num frames 3200... [2025-02-22 19:14:51,114][02012] Num frames 3300... [2025-02-22 19:14:51,263][02012] Num frames 3400... [2025-02-22 19:14:51,411][02012] Num frames 3500... [2025-02-22 19:14:51,562][02012] Num frames 3600... [2025-02-22 19:14:51,711][02012] Num frames 3700... [2025-02-22 19:14:51,855][02012] Num frames 3800... [2025-02-22 19:14:52,008][02012] Num frames 3900... [2025-02-22 19:14:52,157][02012] Num frames 4000... [2025-02-22 19:14:52,303][02012] Num frames 4100... [2025-02-22 19:14:52,452][02012] Num frames 4200... [2025-02-22 19:14:52,605][02012] Num frames 4300... [2025-02-22 19:14:52,701][02012] Avg episode rewards: #0: 26.820, true rewards: #0: 10.820 [2025-02-22 19:14:52,702][02012] Avg episode reward: 26.820, avg true_objective: 10.820 [2025-02-22 19:14:52,806][02012] Num frames 4400... [2025-02-22 19:14:52,951][02012] Num frames 4500... [2025-02-22 19:14:53,095][02012] Num frames 4600... [2025-02-22 19:14:53,236][02012] Num frames 4700... [2025-02-22 19:14:53,378][02012] Num frames 4800... [2025-02-22 19:14:53,523][02012] Num frames 4900... [2025-02-22 19:14:53,668][02012] Num frames 5000... [2025-02-22 19:14:53,812][02012] Num frames 5100... [2025-02-22 19:14:53,908][02012] Avg episode rewards: #0: 24.456, true rewards: #0: 10.256 [2025-02-22 19:14:53,909][02012] Avg episode reward: 24.456, avg true_objective: 10.256 [2025-02-22 19:14:54,017][02012] Num frames 5200... [2025-02-22 19:14:54,165][02012] Num frames 5300... [2025-02-22 19:14:54,309][02012] Num frames 5400... [2025-02-22 19:14:54,455][02012] Num frames 5500... [2025-02-22 19:14:54,600][02012] Num frames 5600... [2025-02-22 19:14:54,745][02012] Num frames 5700... [2025-02-22 19:14:54,901][02012] Num frames 5800... [2025-02-22 19:14:55,049][02012] Num frames 5900... [2025-02-22 19:14:55,197][02012] Num frames 6000... [2025-02-22 19:14:55,344][02012] Num frames 6100... [2025-02-22 19:14:55,499][02012] Num frames 6200... [2025-02-22 19:14:55,626][02012] Avg episode rewards: #0: 24.247, true rewards: #0: 10.413 [2025-02-22 19:14:55,627][02012] Avg episode reward: 24.247, avg true_objective: 10.413 [2025-02-22 19:14:55,707][02012] Num frames 6300... [2025-02-22 19:14:55,851][02012] Num frames 6400... [2025-02-22 19:14:56,000][02012] Num frames 6500... [2025-02-22 19:14:56,154][02012] Num frames 6600... [2025-02-22 19:14:56,305][02012] Num frames 6700... [2025-02-22 19:14:56,489][02012] Avg episode rewards: #0: 21.989, true rewards: #0: 9.703 [2025-02-22 19:14:56,490][02012] Avg episode reward: 21.989, avg true_objective: 9.703 [2025-02-22 19:14:56,504][02012] Num frames 6800... [2025-02-22 19:14:56,646][02012] Num frames 6900... [2025-02-22 19:14:56,790][02012] Num frames 7000... [2025-02-22 19:14:56,932][02012] Num frames 7100... [2025-02-22 19:14:57,077][02012] Num frames 7200... [2025-02-22 19:14:57,223][02012] Num frames 7300... [2025-02-22 19:14:57,367][02012] Num frames 7400... [2025-02-22 19:14:57,508][02012] Num frames 7500... [2025-02-22 19:14:57,649][02012] Num frames 7600... [2025-02-22 19:14:57,794][02012] Num frames 7700... [2025-02-22 19:14:57,941][02012] Num frames 7800... [2025-02-22 19:14:58,112][02012] Avg episode rewards: #0: 22.596, true rewards: #0: 9.846 [2025-02-22 19:14:58,113][02012] Avg episode reward: 22.596, avg true_objective: 9.846 [2025-02-22 19:14:58,149][02012] Num frames 7900... [2025-02-22 19:14:58,294][02012] Num frames 8000... [2025-02-22 19:14:58,437][02012] Num frames 8100... [2025-02-22 19:14:58,583][02012] Num frames 8200... [2025-02-22 19:14:58,730][02012] Num frames 8300... [2025-02-22 19:14:58,870][02012] Num frames 8400... [2025-02-22 19:14:59,012][02012] Num frames 8500... [2025-02-22 19:14:59,138][02012] Avg episode rewards: #0: 21.388, true rewards: #0: 9.499 [2025-02-22 19:14:59,139][02012] Avg episode reward: 21.388, avg true_objective: 9.499 [2025-02-22 19:14:59,214][02012] Num frames 8600... [2025-02-22 19:14:59,359][02012] Num frames 8700... [2025-02-22 19:14:59,500][02012] Num frames 8800... [2025-02-22 19:14:59,639][02012] Num frames 8900... [2025-02-22 19:14:59,782][02012] Num frames 9000... [2025-02-22 19:14:59,923][02012] Num frames 9100... [2025-02-22 19:15:00,069][02012] Num frames 9200... [2025-02-22 19:15:00,202][02012] Avg episode rewards: #0: 20.353, true rewards: #0: 9.253 [2025-02-22 19:15:00,203][02012] Avg episode reward: 20.353, avg true_objective: 9.253 [2025-02-22 19:15:22,810][02012] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-02-22 19:16:20,325][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:16:20,326][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:16:20,327][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:16:20,328][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:16:20,328][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:16:20,329][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:16:20,329][02012] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-02-22 19:16:20,330][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:16:20,331][02012] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-02-22 19:16:20,331][02012] Adding new argument 'hf_repository'='ThomasSimonini/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-02-22 19:16:20,332][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:16:20,333][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:16:20,334][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:16:20,334][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:16:20,335][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:16:20,359][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:16:20,361][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:16:20,373][02012] ConvEncoder: input_channels=3 [2025-02-22 19:16:20,411][02012] Conv encoder output size: 512 [2025-02-22 19:16:20,413][02012] Policy head output size: 512 [2025-02-22 19:16:20,432][02012] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:16:20,895][02012] Num frames 100... [2025-02-22 19:16:21,038][02012] Num frames 200... [2025-02-22 19:16:21,181][02012] Num frames 300... [2025-02-22 19:16:21,326][02012] Num frames 400... [2025-02-22 19:16:21,470][02012] Num frames 500... [2025-02-22 19:16:21,613][02012] Num frames 600... [2025-02-22 19:16:21,761][02012] Num frames 700... [2025-02-22 19:16:21,822][02012] Avg episode rewards: #0: 16.040, true rewards: #0: 7.040 [2025-02-22 19:16:21,823][02012] Avg episode reward: 16.040, avg true_objective: 7.040 [2025-02-22 19:16:21,959][02012] Num frames 800... [2025-02-22 19:16:22,102][02012] Num frames 900... [2025-02-22 19:16:22,244][02012] Num frames 1000... [2025-02-22 19:16:22,388][02012] Num frames 1100... [2025-02-22 19:16:22,533][02012] Num frames 1200... [2025-02-22 19:16:22,680][02012] Num frames 1300... [2025-02-22 19:16:22,845][02012] Avg episode rewards: #0: 14.880, true rewards: #0: 6.880 [2025-02-22 19:16:22,846][02012] Avg episode reward: 14.880, avg true_objective: 6.880 [2025-02-22 19:16:22,881][02012] Num frames 1400... [2025-02-22 19:16:23,024][02012] Num frames 1500... [2025-02-22 19:16:23,168][02012] Num frames 1600... [2025-02-22 19:16:23,310][02012] Num frames 1700... [2025-02-22 19:16:23,452][02012] Num frames 1800... [2025-02-22 19:16:23,592][02012] Num frames 1900... [2025-02-22 19:16:23,734][02012] Num frames 2000... [2025-02-22 19:16:23,878][02012] Num frames 2100... [2025-02-22 19:16:24,019][02012] Num frames 2200... [2025-02-22 19:16:24,163][02012] Num frames 2300... [2025-02-22 19:16:24,305][02012] Num frames 2400... [2025-02-22 19:16:24,446][02012] Num frames 2500... [2025-02-22 19:16:24,596][02012] Num frames 2600... [2025-02-22 19:16:24,741][02012] Num frames 2700... [2025-02-22 19:16:24,883][02012] Num frames 2800... [2025-02-22 19:16:25,026][02012] Num frames 2900... [2025-02-22 19:16:25,168][02012] Num frames 3000... [2025-02-22 19:16:39,671][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:16:39,672][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:16:39,673][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:16:39,674][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:16:39,675][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:16:39,675][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:16:39,676][02012] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-02-22 19:16:39,677][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:16:39,677][02012] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-02-22 19:16:39,678][02012] Adding new argument 'hf_repository'='iamandrewliao/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-02-22 19:16:39,678][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:16:39,679][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:16:39,680][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:16:39,680][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:16:39,681][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:16:39,710][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:16:39,711][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:16:39,722][02012] ConvEncoder: input_channels=3 [2025-02-22 19:16:39,758][02012] Conv encoder output size: 512 [2025-02-22 19:16:39,759][02012] Policy head output size: 512 [2025-02-22 19:16:39,779][02012] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:16:40,239][02012] Num frames 100... [2025-02-22 19:16:40,390][02012] Num frames 200... [2025-02-22 19:16:40,536][02012] Num frames 300... [2025-02-22 19:16:40,686][02012] Num frames 400... [2025-02-22 19:16:40,832][02012] Num frames 500... [2025-02-22 19:16:40,976][02012] Num frames 600... [2025-02-22 19:16:41,121][02012] Num frames 700... [2025-02-22 19:16:41,193][02012] Avg episode rewards: #0: 18.100, true rewards: #0: 7.100 [2025-02-22 19:16:41,194][02012] Avg episode reward: 18.100, avg true_objective: 7.100 [2025-02-22 19:16:41,321][02012] Num frames 800... [2025-02-22 19:16:41,466][02012] Num frames 900... [2025-02-22 19:16:41,615][02012] Num frames 1000... [2025-02-22 19:16:41,758][02012] Num frames 1100... [2025-02-22 19:16:41,901][02012] Num frames 1200... [2025-02-22 19:16:42,044][02012] Num frames 1300... [2025-02-22 19:16:42,190][02012] Num frames 1400... [2025-02-22 19:16:42,335][02012] Num frames 1500... [2025-02-22 19:16:42,480][02012] Num frames 1600... [2025-02-22 19:16:42,622][02012] Num frames 1700... [2025-02-22 19:16:42,770][02012] Num frames 1800... [2025-02-22 19:16:42,910][02012] Num frames 1900... [2025-02-22 19:16:43,023][02012] Avg episode rewards: #0: 22.200, true rewards: #0: 9.700 [2025-02-22 19:16:43,024][02012] Avg episode reward: 22.200, avg true_objective: 9.700 [2025-02-22 19:16:43,113][02012] Num frames 2000... [2025-02-22 19:16:43,257][02012] Num frames 2100... [2025-02-22 19:16:43,400][02012] Num frames 2200... [2025-02-22 19:16:43,542][02012] Num frames 2300... [2025-02-22 19:16:43,684][02012] Num frames 2400... [2025-02-22 19:16:43,825][02012] Num frames 2500... [2025-02-22 19:16:43,976][02012] Num frames 2600... [2025-02-22 19:16:44,125][02012] Num frames 2700... [2025-02-22 19:16:44,244][02012] Avg episode rewards: #0: 20.144, true rewards: #0: 9.143 [2025-02-22 19:16:44,246][02012] Avg episode reward: 20.144, avg true_objective: 9.143 [2025-02-22 19:16:44,330][02012] Num frames 2800... [2025-02-22 19:16:44,479][02012] Num frames 2900... [2025-02-22 19:16:44,625][02012] Num frames 3000... [2025-02-22 19:16:44,770][02012] Num frames 3100... [2025-02-22 19:16:44,914][02012] Num frames 3200... [2025-02-22 19:16:45,057][02012] Num frames 3300... [2025-02-22 19:16:45,201][02012] Num frames 3400... [2025-02-22 19:16:45,346][02012] Num frames 3500... [2025-02-22 19:16:45,489][02012] Num frames 3600... [2025-02-22 19:16:45,635][02012] Num frames 3700... [2025-02-22 19:16:45,781][02012] Num frames 3800... [2025-02-22 19:16:45,927][02012] Num frames 3900... [2025-02-22 19:16:46,073][02012] Num frames 4000... [2025-02-22 19:16:46,239][02012] Num frames 4100... [2025-02-22 19:16:46,386][02012] Num frames 4200... [2025-02-22 19:16:46,541][02012] Num frames 4300... [2025-02-22 19:16:46,703][02012] Num frames 4400... [2025-02-22 19:16:46,865][02012] Num frames 4500... [2025-02-22 19:16:47,029][02012] Num frames 4600... [2025-02-22 19:16:47,187][02012] Num frames 4700... [2025-02-22 19:16:47,348][02012] Num frames 4800... [2025-02-22 19:16:47,472][02012] Avg episode rewards: #0: 30.357, true rewards: #0: 12.108 [2025-02-22 19:16:47,473][02012] Avg episode reward: 30.357, avg true_objective: 12.108 [2025-02-22 19:16:47,559][02012] Num frames 4900... [2025-02-22 19:16:47,709][02012] Num frames 5000... [2025-02-22 19:16:47,858][02012] Num frames 5100... [2025-02-22 19:16:48,007][02012] Num frames 5200... [2025-02-22 19:16:48,159][02012] Num frames 5300... [2025-02-22 19:16:48,305][02012] Num frames 5400... [2025-02-22 19:16:48,451][02012] Num frames 5500... [2025-02-22 19:16:48,609][02012] Num frames 5600... [2025-02-22 19:16:48,770][02012] Num frames 5700... [2025-02-22 19:16:48,915][02012] Num frames 5800... [2025-02-22 19:16:49,064][02012] Num frames 5900... [2025-02-22 19:16:49,210][02012] Num frames 6000... [2025-02-22 19:16:49,360][02012] Num frames 6100... [2025-02-22 19:16:49,514][02012] Num frames 6200... [2025-02-22 19:16:49,664][02012] Num frames 6300... [2025-02-22 19:16:49,823][02012] Num frames 6400... [2025-02-22 19:16:49,976][02012] Num frames 6500... [2025-02-22 19:16:50,128][02012] Num frames 6600... [2025-02-22 19:16:50,276][02012] Num frames 6700... [2025-02-22 19:16:50,425][02012] Num frames 6800... [2025-02-22 19:16:50,613][02012] Avg episode rewards: #0: 35.982, true rewards: #0: 13.782 [2025-02-22 19:16:50,614][02012] Avg episode reward: 35.982, avg true_objective: 13.782 [2025-02-22 19:16:50,630][02012] Num frames 6900... [2025-02-22 19:16:50,795][02012] Num frames 7000... [2025-02-22 19:16:50,963][02012] Num frames 7100... [2025-02-22 19:16:51,123][02012] Num frames 7200... [2025-02-22 19:16:51,274][02012] Num frames 7300... [2025-02-22 19:16:51,420][02012] Num frames 7400... [2025-02-22 19:16:51,574][02012] Avg episode rewards: #0: 31.611, true rewards: #0: 12.445 [2025-02-22 19:16:51,575][02012] Avg episode reward: 31.611, avg true_objective: 12.445 [2025-02-22 19:16:51,626][02012] Num frames 7500... [2025-02-22 19:16:51,773][02012] Num frames 7600... [2025-02-22 19:16:51,917][02012] Num frames 7700... [2025-02-22 19:16:52,063][02012] Num frames 7800... [2025-02-22 19:16:52,209][02012] Num frames 7900... [2025-02-22 19:16:52,354][02012] Num frames 8000... [2025-02-22 19:16:52,519][02012] Avg episode rewards: #0: 28.393, true rewards: #0: 11.536 [2025-02-22 19:16:52,520][02012] Avg episode reward: 28.393, avg true_objective: 11.536 [2025-02-22 19:16:52,559][02012] Num frames 8100... [2025-02-22 19:16:52,714][02012] Num frames 8200... [2025-02-22 19:16:52,865][02012] Num frames 8300... [2025-02-22 19:16:53,011][02012] Num frames 8400... [2025-02-22 19:16:53,157][02012] Num frames 8500... [2025-02-22 19:16:53,302][02012] Num frames 8600... [2025-02-22 19:16:53,450][02012] Num frames 8700... [2025-02-22 19:16:53,601][02012] Num frames 8800... [2025-02-22 19:16:53,751][02012] Num frames 8900... [2025-02-22 19:16:53,817][02012] Avg episode rewards: #0: 26.759, true rewards: #0: 11.134 [2025-02-22 19:16:53,818][02012] Avg episode reward: 26.759, avg true_objective: 11.134 [2025-02-22 19:16:53,956][02012] Num frames 9000... [2025-02-22 19:16:54,103][02012] Num frames 9100... [2025-02-22 19:16:54,249][02012] Num frames 9200... [2025-02-22 19:16:54,394][02012] Num frames 9300... [2025-02-22 19:16:54,541][02012] Num frames 9400... [2025-02-22 19:16:54,692][02012] Num frames 9500... [2025-02-22 19:16:54,765][02012] Avg episode rewards: #0: 25.456, true rewards: #0: 10.568 [2025-02-22 19:16:54,766][02012] Avg episode reward: 25.456, avg true_objective: 10.568 [2025-02-22 19:16:54,899][02012] Num frames 9600... [2025-02-22 19:16:55,045][02012] Num frames 9700... [2025-02-22 19:16:55,194][02012] Num frames 9800... [2025-02-22 19:16:55,341][02012] Num frames 9900... [2025-02-22 19:16:55,485][02012] Num frames 10000... [2025-02-22 19:16:55,630][02012] Avg episode rewards: #0: 23.964, true rewards: #0: 10.064 [2025-02-22 19:16:55,631][02012] Avg episode reward: 23.964, avg true_objective: 10.064 [2025-02-22 19:17:19,906][02012] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-02-22 19:17:29,652][02012] The model has been pushed to https://huggingface.co/iamandrewliao/rl_course_vizdoom_health_gathering_supreme [2025-02-22 19:22:35,216][02012] Loading legacy config file train_dir/doom_health_gathering_supreme_2222/cfg.json instead of train_dir/doom_health_gathering_supreme_2222/config.json [2025-02-22 19:22:35,217][02012] Loading existing experiment configuration from train_dir/doom_health_gathering_supreme_2222/config.json [2025-02-22 19:22:35,218][02012] Overriding arg 'experiment' with value 'doom_health_gathering_supreme_2222' passed from command line [2025-02-22 19:22:35,219][02012] Overriding arg 'train_dir' with value 'train_dir' passed from command line [2025-02-22 19:22:35,220][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:22:35,221][02012] Adding new argument 'lr_adaptive_min'=1e-06 that is not in the saved config file! [2025-02-22 19:22:35,221][02012] Adding new argument 'lr_adaptive_max'=0.01 that is not in the saved config file! [2025-02-22 19:22:35,222][02012] Adding new argument 'env_gpu_observations'=True that is not in the saved config file! [2025-02-22 19:22:35,223][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:22:35,223][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:22:35,224][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:22:35,225][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:22:35,225][02012] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:22:35,226][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:22:35,227][02012] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-02-22 19:22:35,227][02012] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-02-22 19:22:35,228][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:22:35,229][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:22:35,229][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:22:35,230][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:22:35,231][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:22:35,255][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:22:35,257][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:22:35,269][02012] ConvEncoder: input_channels=3 [2025-02-22 19:22:35,318][02012] Conv encoder output size: 512 [2025-02-22 19:22:35,319][02012] Policy head output size: 512 [2025-02-22 19:22:35,345][02012] Loading state from checkpoint train_dir/doom_health_gathering_supreme_2222/checkpoint_p0/checkpoint_000539850_4422451200.pth... [2025-02-22 19:22:35,836][02012] Num frames 100... [2025-02-22 19:22:35,983][02012] Num frames 200... [2025-02-22 19:22:36,130][02012] Num frames 300... [2025-02-22 19:22:36,278][02012] Num frames 400... [2025-02-22 19:22:36,425][02012] Num frames 500... [2025-02-22 19:22:36,568][02012] Num frames 600... [2025-02-22 19:22:36,723][02012] Num frames 700... [2025-02-22 19:22:36,870][02012] Num frames 800... [2025-02-22 19:22:37,017][02012] Num frames 900... [2025-02-22 19:22:37,163][02012] Num frames 1000... [2025-02-22 19:22:37,309][02012] Num frames 1100... [2025-02-22 19:22:37,453][02012] Num frames 1200... [2025-02-22 19:22:37,603][02012] Num frames 1300... [2025-02-22 19:22:37,753][02012] Num frames 1400... [2025-02-22 19:22:37,902][02012] Num frames 1500... [2025-02-22 19:22:38,048][02012] Num frames 1600... [2025-02-22 19:22:38,195][02012] Num frames 1700... [2025-02-22 19:22:38,342][02012] Num frames 1800... [2025-02-22 19:22:38,488][02012] Num frames 1900... [2025-02-22 19:22:38,634][02012] Num frames 2000... [2025-02-22 19:22:38,787][02012] Num frames 2100... [2025-02-22 19:22:38,839][02012] Avg episode rewards: #0: 66.998, true rewards: #0: 21.000 [2025-02-22 19:22:38,840][02012] Avg episode reward: 66.998, avg true_objective: 21.000 [2025-02-22 19:22:38,985][02012] Num frames 2200... [2025-02-22 19:22:39,134][02012] Num frames 2300... [2025-02-22 19:22:39,281][02012] Num frames 2400... [2025-02-22 19:22:39,428][02012] Num frames 2500... [2025-02-22 19:22:39,576][02012] Num frames 2600... [2025-02-22 19:22:39,721][02012] Num frames 2700... [2025-02-22 19:22:39,865][02012] Num frames 2800... [2025-02-22 19:22:40,011][02012] Num frames 2900... [2025-02-22 19:22:40,154][02012] Num frames 3000... [2025-02-22 19:22:40,298][02012] Num frames 3100... [2025-02-22 19:22:40,450][02012] Num frames 3200... [2025-02-22 19:22:40,594][02012] Num frames 3300... [2025-02-22 19:22:40,742][02012] Num frames 3400... [2025-02-22 19:22:40,890][02012] Num frames 3500... [2025-02-22 19:22:41,039][02012] Num frames 3600... [2025-02-22 19:22:41,185][02012] Num frames 3700... [2025-02-22 19:22:41,335][02012] Num frames 3800... [2025-02-22 19:22:41,485][02012] Num frames 3900... [2025-02-22 19:22:41,631][02012] Num frames 4000... [2025-02-22 19:22:41,782][02012] Num frames 4100... [2025-02-22 19:22:41,930][02012] Num frames 4200... [2025-02-22 19:22:41,982][02012] Avg episode rewards: #0: 62.499, true rewards: #0: 21.000 [2025-02-22 19:22:41,983][02012] Avg episode reward: 62.499, avg true_objective: 21.000 [2025-02-22 19:22:42,130][02012] Num frames 4300... [2025-02-22 19:22:42,276][02012] Num frames 4400... [2025-02-22 19:22:42,422][02012] Num frames 4500... [2025-02-22 19:22:42,575][02012] Num frames 4600... [2025-02-22 19:22:42,727][02012] Num frames 4700... [2025-02-22 19:22:42,875][02012] Num frames 4800... [2025-02-22 19:22:43,029][02012] Num frames 4900... [2025-02-22 19:22:43,181][02012] Num frames 5000... [2025-02-22 19:22:43,327][02012] Num frames 5100... [2025-02-22 19:22:43,478][02012] Num frames 5200... [2025-02-22 19:22:43,628][02012] Num frames 5300... [2025-02-22 19:22:43,778][02012] Num frames 5400... [2025-02-22 19:22:43,928][02012] Num frames 5500... [2025-02-22 19:22:44,078][02012] Num frames 5600... [2025-02-22 19:22:44,227][02012] Num frames 5700... [2025-02-22 19:22:44,374][02012] Num frames 5800... [2025-02-22 19:22:44,526][02012] Num frames 5900... [2025-02-22 19:22:44,679][02012] Num frames 6000... [2025-02-22 19:22:44,834][02012] Num frames 6100... [2025-02-22 19:22:44,990][02012] Num frames 6200... [2025-02-22 19:22:45,152][02012] Num frames 6300... [2025-02-22 19:22:45,204][02012] Avg episode rewards: #0: 61.999, true rewards: #0: 21.000 [2025-02-22 19:22:45,205][02012] Avg episode reward: 61.999, avg true_objective: 21.000 [2025-02-22 19:22:45,357][02012] Num frames 6400... [2025-02-22 19:22:45,502][02012] Num frames 6500... [2025-02-22 19:22:45,656][02012] Num frames 6600... [2025-02-22 19:22:45,817][02012] Num frames 6700... [2025-02-22 19:22:45,984][02012] Num frames 6800... [2025-02-22 19:22:46,140][02012] Num frames 6900... [2025-02-22 19:22:46,300][02012] Num frames 7000... [2025-02-22 19:22:46,460][02012] Num frames 7100... [2025-02-22 19:22:46,629][02012] Num frames 7200... [2025-02-22 19:22:46,789][02012] Num frames 7300... [2025-02-22 19:22:46,949][02012] Num frames 7400... [2025-02-22 19:22:47,100][02012] Num frames 7500... [2025-02-22 19:22:47,246][02012] Num frames 7600... [2025-02-22 19:22:47,403][02012] Num frames 7700... [2025-02-22 19:22:47,561][02012] Num frames 7800... [2025-02-22 19:22:47,708][02012] Num frames 7900... [2025-02-22 19:22:47,857][02012] Num frames 8000... [2025-02-22 19:22:48,005][02012] Num frames 8100... [2025-02-22 19:22:48,153][02012] Num frames 8200... [2025-02-22 19:22:48,302][02012] Num frames 8300... [2025-02-22 19:22:48,456][02012] Num frames 8400... [2025-02-22 19:22:48,507][02012] Avg episode rewards: #0: 62.999, true rewards: #0: 21.000 [2025-02-22 19:22:48,509][02012] Avg episode reward: 62.999, avg true_objective: 21.000 [2025-02-22 19:22:48,668][02012] Num frames 8500... [2025-02-22 19:22:48,813][02012] Num frames 8600... [2025-02-22 19:22:48,960][02012] Num frames 8700... [2025-02-22 19:22:49,110][02012] Num frames 8800... [2025-02-22 19:22:49,258][02012] Num frames 8900... [2025-02-22 19:22:49,405][02012] Num frames 9000... [2025-02-22 19:22:49,549][02012] Num frames 9100... [2025-02-22 19:22:49,700][02012] Num frames 9200... [2025-02-22 19:22:49,849][02012] Num frames 9300... [2025-02-22 19:22:49,998][02012] Num frames 9400... [2025-02-22 19:22:50,146][02012] Num frames 9500... [2025-02-22 19:22:50,295][02012] Num frames 9600... [2025-02-22 19:22:50,443][02012] Num frames 9700... [2025-02-22 19:22:50,592][02012] Num frames 9800... [2025-02-22 19:22:50,743][02012] Num frames 9900... [2025-02-22 19:22:50,893][02012] Num frames 10000... [2025-02-22 19:22:51,043][02012] Num frames 10100... [2025-02-22 19:22:51,193][02012] Num frames 10200... [2025-02-22 19:22:51,341][02012] Num frames 10300... [2025-02-22 19:22:51,493][02012] Num frames 10400... [2025-02-22 19:22:51,650][02012] Num frames 10500... [2025-02-22 19:22:51,702][02012] Avg episode rewards: #0: 63.599, true rewards: #0: 21.000 [2025-02-22 19:22:51,703][02012] Avg episode reward: 63.599, avg true_objective: 21.000 [2025-02-22 19:22:51,852][02012] Num frames 10600... [2025-02-22 19:22:52,003][02012] Num frames 10700... [2025-02-22 19:22:52,148][02012] Num frames 10800... [2025-02-22 19:22:52,295][02012] Num frames 10900... [2025-02-22 19:22:52,443][02012] Num frames 11000... [2025-02-22 19:22:52,592][02012] Num frames 11100... [2025-02-22 19:22:52,743][02012] Num frames 11200... [2025-02-22 19:22:52,892][02012] Num frames 11300... [2025-02-22 19:22:53,045][02012] Num frames 11400... [2025-02-22 19:22:53,195][02012] Num frames 11500... [2025-02-22 19:22:53,353][02012] Num frames 11600... [2025-02-22 19:22:53,516][02012] Num frames 11700... [2025-02-22 19:22:53,673][02012] Num frames 11800... [2025-02-22 19:22:53,832][02012] Num frames 11900... [2025-02-22 19:22:53,988][02012] Num frames 12000... [2025-02-22 19:22:54,146][02012] Num frames 12100... [2025-02-22 19:22:54,307][02012] Num frames 12200... [2025-02-22 19:22:54,465][02012] Num frames 12300... [2025-02-22 19:22:54,626][02012] Num frames 12400... [2025-02-22 19:22:54,765][02012] Avg episode rewards: #0: 61.419, true rewards: #0: 20.753 [2025-02-22 19:22:54,766][02012] Avg episode reward: 61.419, avg true_objective: 20.753 [2025-02-22 19:22:54,842][02012] Num frames 12500... [2025-02-22 19:22:55,002][02012] Num frames 12600... [2025-02-22 19:22:55,166][02012] Num frames 12700... [2025-02-22 19:22:55,328][02012] Num frames 12800... [2025-02-22 19:22:55,486][02012] Num frames 12900... [2025-02-22 19:22:55,644][02012] Num frames 13000... [2025-02-22 19:22:55,796][02012] Num frames 13100... [2025-02-22 19:22:55,947][02012] Num frames 13200... [2025-02-22 19:22:56,100][02012] Num frames 13300... [2025-02-22 19:22:56,246][02012] Num frames 13400... [2025-02-22 19:22:56,402][02012] Num frames 13500... [2025-02-22 19:22:56,555][02012] Num frames 13600... [2025-02-22 19:22:56,717][02012] Num frames 13700... [2025-02-22 19:22:56,882][02012] Num frames 13800... [2025-02-22 19:22:57,043][02012] Num frames 13900... [2025-02-22 19:22:57,199][02012] Num frames 14000... [2025-02-22 19:22:57,346][02012] Num frames 14100... [2025-02-22 19:22:57,495][02012] Num frames 14200... [2025-02-22 19:22:57,649][02012] Num frames 14300... [2025-02-22 19:22:57,807][02012] Num frames 14400... [2025-02-22 19:22:57,962][02012] Num frames 14500... [2025-02-22 19:22:58,099][02012] Avg episode rewards: #0: 62.644, true rewards: #0: 20.789 [2025-02-22 19:22:58,100][02012] Avg episode reward: 62.644, avg true_objective: 20.789 [2025-02-22 19:22:58,174][02012] Num frames 14600... [2025-02-22 19:22:58,323][02012] Num frames 14700... [2025-02-22 19:22:58,472][02012] Num frames 14800... [2025-02-22 19:22:58,623][02012] Num frames 14900... [2025-02-22 19:22:58,774][02012] Num frames 15000... [2025-02-22 19:22:58,925][02012] Num frames 15100... [2025-02-22 19:22:59,076][02012] Num frames 15200... [2025-02-22 19:22:59,224][02012] Num frames 15300... [2025-02-22 19:22:59,374][02012] Num frames 15400... [2025-02-22 19:22:59,524][02012] Num frames 15500... [2025-02-22 19:22:59,675][02012] Num frames 15600... [2025-02-22 19:22:59,827][02012] Num frames 15700... [2025-02-22 19:22:59,977][02012] Num frames 15800... [2025-02-22 19:23:00,125][02012] Num frames 15900... [2025-02-22 19:23:00,273][02012] Num frames 16000... [2025-02-22 19:23:00,424][02012] Num frames 16100... [2025-02-22 19:23:00,574][02012] Num frames 16200... [2025-02-22 19:23:00,729][02012] Num frames 16300... [2025-02-22 19:23:00,880][02012] Num frames 16400... [2025-02-22 19:23:01,037][02012] Num frames 16500... [2025-02-22 19:23:01,191][02012] Num frames 16600... [2025-02-22 19:23:01,293][02012] Avg episode rewards: #0: 63.039, true rewards: #0: 20.790 [2025-02-22 19:23:01,294][02012] Avg episode reward: 63.039, avg true_objective: 20.790 [2025-02-22 19:23:01,397][02012] Num frames 16700... [2025-02-22 19:23:01,545][02012] Num frames 16800... [2025-02-22 19:23:01,694][02012] Num frames 16900... [2025-02-22 19:23:01,841][02012] Num frames 17000... [2025-02-22 19:23:01,990][02012] Num frames 17100... [2025-02-22 19:23:02,139][02012] Num frames 17200... [2025-02-22 19:23:02,288][02012] Num frames 17300... [2025-02-22 19:23:02,441][02012] Num frames 17400... [2025-02-22 19:23:02,591][02012] Num frames 17500... [2025-02-22 19:23:02,741][02012] Num frames 17600... [2025-02-22 19:23:02,891][02012] Num frames 17700... [2025-02-22 19:23:03,043][02012] Num frames 17800... [2025-02-22 19:23:03,191][02012] Num frames 17900... [2025-02-22 19:23:03,340][02012] Num frames 18000... [2025-02-22 19:23:03,487][02012] Num frames 18100... [2025-02-22 19:23:03,633][02012] Num frames 18200... [2025-02-22 19:23:03,782][02012] Num frames 18300... [2025-02-22 19:23:03,933][02012] Num frames 18400... [2025-02-22 19:23:04,090][02012] Num frames 18500... [2025-02-22 19:23:04,244][02012] Num frames 18600... [2025-02-22 19:23:04,400][02012] Num frames 18700... [2025-02-22 19:23:04,502][02012] Avg episode rewards: #0: 63.701, true rewards: #0: 20.813 [2025-02-22 19:23:04,503][02012] Avg episode reward: 63.701, avg true_objective: 20.813 [2025-02-22 19:23:04,608][02012] Num frames 18800... [2025-02-22 19:23:04,757][02012] Num frames 18900... [2025-02-22 19:23:04,906][02012] Num frames 19000... [2025-02-22 19:23:05,055][02012] Num frames 19100... [2025-02-22 19:23:05,202][02012] Num frames 19200... [2025-02-22 19:23:05,351][02012] Num frames 19300... [2025-02-22 19:23:05,501][02012] Num frames 19400... [2025-02-22 19:23:05,654][02012] Num frames 19500... [2025-02-22 19:23:05,806][02012] Num frames 19600... [2025-02-22 19:23:05,958][02012] Num frames 19700... [2025-02-22 19:23:06,109][02012] Num frames 19800... [2025-02-22 19:23:06,256][02012] Num frames 19900... [2025-02-22 19:23:06,404][02012] Num frames 20000... [2025-02-22 19:23:06,557][02012] Num frames 20100... [2025-02-22 19:23:06,625][02012] Avg episode rewards: #0: 61.507, true rewards: #0: 20.108 [2025-02-22 19:23:06,626][02012] Avg episode reward: 61.507, avg true_objective: 20.108 [2025-02-22 19:23:55,110][02012] Replay video saved to train_dir/doom_health_gathering_supreme_2222/replay.mp4! [2025-02-22 19:25:20,646][02012] Environment doom_basic already registered, overwriting... [2025-02-22 19:25:20,647][02012] Environment doom_two_colors_easy already registered, overwriting... [2025-02-22 19:25:20,648][02012] Environment doom_two_colors_hard already registered, overwriting... [2025-02-22 19:25:20,648][02012] Environment doom_dm already registered, overwriting... [2025-02-22 19:25:20,650][02012] Environment doom_dwango5 already registered, overwriting... [2025-02-22 19:25:20,650][02012] Environment doom_my_way_home_flat_actions already registered, overwriting... [2025-02-22 19:25:20,651][02012] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2025-02-22 19:25:20,652][02012] Environment doom_my_way_home already registered, overwriting... [2025-02-22 19:25:20,652][02012] Environment doom_deadly_corridor already registered, overwriting... [2025-02-22 19:25:20,653][02012] Environment doom_defend_the_center already registered, overwriting... [2025-02-22 19:25:20,654][02012] Environment doom_defend_the_line already registered, overwriting... [2025-02-22 19:25:20,655][02012] Environment doom_health_gathering already registered, overwriting... [2025-02-22 19:25:20,656][02012] Environment doom_health_gathering_supreme already registered, overwriting... [2025-02-22 19:25:20,656][02012] Environment doom_battle already registered, overwriting... [2025-02-22 19:25:20,657][02012] Environment doom_battle2 already registered, overwriting... [2025-02-22 19:25:20,658][02012] Environment doom_duel_bots already registered, overwriting... [2025-02-22 19:25:20,659][02012] Environment doom_deathmatch_bots already registered, overwriting... [2025-02-22 19:25:20,659][02012] Environment doom_duel already registered, overwriting... [2025-02-22 19:25:20,661][02012] Environment doom_deathmatch_full already registered, overwriting... [2025-02-22 19:25:20,661][02012] Environment doom_benchmark already registered, overwriting... [2025-02-22 19:25:20,662][02012] register_encoder_factory: [2025-02-22 19:25:20,671][02012] Loading legacy config file train_dir/doom_deathmatch_bots_2222/cfg.json instead of train_dir/doom_deathmatch_bots_2222/config.json [2025-02-22 19:25:20,672][02012] Loading existing experiment configuration from train_dir/doom_deathmatch_bots_2222/config.json [2025-02-22 19:25:20,672][02012] Overriding arg 'experiment' with value 'doom_deathmatch_bots_2222' passed from command line [2025-02-22 19:25:20,673][02012] Overriding arg 'train_dir' with value 'train_dir' passed from command line [2025-02-22 19:25:20,674][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:25:20,674][02012] Adding new argument 'lr_adaptive_min'=1e-06 that is not in the saved config file! [2025-02-22 19:25:20,675][02012] Adding new argument 'lr_adaptive_max'=0.01 that is not in the saved config file! [2025-02-22 19:25:20,676][02012] Adding new argument 'env_gpu_observations'=True that is not in the saved config file! [2025-02-22 19:25:20,677][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:25:20,678][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:25:20,678][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:25:20,679][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:25:20,680][02012] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:25:20,681][02012] Adding new argument 'max_num_episodes'=1 that is not in the saved config file! [2025-02-22 19:25:20,681][02012] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-02-22 19:25:20,682][02012] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-02-22 19:25:20,683][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:25:20,684][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:25:20,684][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:25:20,685][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:25:20,685][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:25:20,718][02012] Port 40300 is available [2025-02-22 19:25:20,719][02012] Using port 40300 [2025-02-22 19:25:20,720][02012] RunningMeanStd input shape: (23,) [2025-02-22 19:25:20,722][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:25:20,723][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:25:20,736][02012] ConvEncoder: input_channels=3 [2025-02-22 19:25:20,776][02012] Conv encoder output size: 512 [2025-02-22 19:25:20,778][02012] Policy head output size: 640 [2025-02-22 19:25:20,845][02012] Loading state from checkpoint train_dir/doom_deathmatch_bots_2222/checkpoint_p0/checkpoint_000282220_2311946240.pth... [2025-02-22 19:26:13,749][02012] Environment doom_basic already registered, overwriting... [2025-02-22 19:26:13,750][02012] Environment doom_two_colors_easy already registered, overwriting... [2025-02-22 19:26:13,751][02012] Environment doom_two_colors_hard already registered, overwriting... [2025-02-22 19:26:13,752][02012] Environment doom_dm already registered, overwriting... [2025-02-22 19:26:13,753][02012] Environment doom_dwango5 already registered, overwriting... [2025-02-22 19:26:13,753][02012] Environment doom_my_way_home_flat_actions already registered, overwriting... [2025-02-22 19:26:13,754][02012] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2025-02-22 19:26:13,755][02012] Environment doom_my_way_home already registered, overwriting... [2025-02-22 19:26:13,755][02012] Environment doom_deadly_corridor already registered, overwriting... [2025-02-22 19:26:13,756][02012] Environment doom_defend_the_center already registered, overwriting... [2025-02-22 19:26:13,757][02012] Environment doom_defend_the_line already registered, overwriting... [2025-02-22 19:26:13,757][02012] Environment doom_health_gathering already registered, overwriting... [2025-02-22 19:26:13,758][02012] Environment doom_health_gathering_supreme already registered, overwriting... [2025-02-22 19:26:13,759][02012] Environment doom_battle already registered, overwriting... [2025-02-22 19:26:13,759][02012] Environment doom_battle2 already registered, overwriting... [2025-02-22 19:26:13,760][02012] Environment doom_duel_bots already registered, overwriting... [2025-02-22 19:26:13,761][02012] Environment doom_deathmatch_bots already registered, overwriting... [2025-02-22 19:26:13,761][02012] Environment doom_duel already registered, overwriting... [2025-02-22 19:26:13,762][02012] Environment doom_deathmatch_full already registered, overwriting... [2025-02-22 19:26:13,763][02012] Environment doom_benchmark already registered, overwriting... [2025-02-22 19:26:13,764][02012] register_encoder_factory: [2025-02-22 19:26:13,772][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:26:13,773][02012] Overriding arg 'train_for_env_steps' with value 4500000 passed from command line [2025-02-22 19:26:13,778][02012] Experiment dir /content/train_dir/default_experiment already exists! [2025-02-22 19:26:13,779][02012] Resuming existing experiment from /content/train_dir/default_experiment... [2025-02-22 19:26:13,780][02012] Weights and Biases integration disabled [2025-02-22 19:26:13,782][02012] Environment var CUDA_VISIBLE_DEVICES is 0 [2025-02-22 19:26:16,214][02012] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=4500000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4200000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4200000} git_hash=unknown git_repo_name=not a git repository [2025-02-22 19:26:16,215][02012] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-02-22 19:26:16,217][02012] Rollout worker 0 uses device cpu [2025-02-22 19:26:16,218][02012] Rollout worker 1 uses device cpu [2025-02-22 19:26:16,218][02012] Rollout worker 2 uses device cpu [2025-02-22 19:26:16,219][02012] Rollout worker 3 uses device cpu [2025-02-22 19:26:16,220][02012] Rollout worker 4 uses device cpu [2025-02-22 19:26:16,220][02012] Rollout worker 5 uses device cpu [2025-02-22 19:26:16,221][02012] Rollout worker 6 uses device cpu [2025-02-22 19:26:16,222][02012] Rollout worker 7 uses device cpu [2025-02-22 19:26:16,257][02012] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:26:16,258][02012] InferenceWorker_p0-w0: min num requests: 2 [2025-02-22 19:26:16,290][02012] Starting all processes... [2025-02-22 19:26:16,291][02012] Starting process learner_proc0 [2025-02-22 19:26:16,345][02012] Starting all processes... [2025-02-22 19:26:16,349][02012] Starting process inference_proc0-0 [2025-02-22 19:26:16,350][02012] Starting process rollout_proc0 [2025-02-22 19:26:16,351][02012] Starting process rollout_proc1 [2025-02-22 19:26:16,351][02012] Starting process rollout_proc2 [2025-02-22 19:26:16,352][02012] Starting process rollout_proc3 [2025-02-22 19:26:16,356][02012] Starting process rollout_proc4 [2025-02-22 19:26:16,357][02012] Starting process rollout_proc5 [2025-02-22 19:26:16,357][02012] Starting process rollout_proc6 [2025-02-22 19:26:16,358][02012] Starting process rollout_proc7 [2025-02-22 19:26:18,821][10332] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,083][10335] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,210][10331] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:26:19,211][10331] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2025-02-22 19:26:19,227][10331] Num visible devices: 1 [2025-02-22 19:26:19,312][10318] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:26:19,312][10318] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2025-02-22 19:26:19,328][10318] Num visible devices: 1 [2025-02-22 19:26:19,353][10318] Starting seed is not provided [2025-02-22 19:26:19,354][10318] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:26:19,354][10318] Initializing actor-critic model on device cuda:0 [2025-02-22 19:26:19,354][10318] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:26:19,355][10318] RunningMeanStd input shape: (1,) [2025-02-22 19:26:19,368][10318] ConvEncoder: input_channels=3 [2025-02-22 19:26:19,480][10318] Conv encoder output size: 512 [2025-02-22 19:26:19,481][10318] Policy head output size: 512 [2025-02-22 19:26:19,485][10339] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,487][10333] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,500][10318] Created Actor Critic model with architecture: [2025-02-22 19:26:19,500][10318] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-02-22 19:26:19,501][10334] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,547][10336] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,567][10338] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:19,628][10318] Using optimizer [2025-02-22 19:26:19,680][10337] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] [2025-02-22 19:26:20,632][10318] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001027_4206592.pth... [2025-02-22 19:26:20,671][10318] Loading model from checkpoint [2025-02-22 19:26:20,673][10318] Loaded experiment state at self.train_step=1027, self.env_steps=4206592 [2025-02-22 19:26:20,673][10318] Initialized policy 0 weights for model version 1027 [2025-02-22 19:26:20,675][10318] LearnerWorker_p0 finished initialization! [2025-02-22 19:26:20,675][10318] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-22 19:26:20,760][10331] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:26:20,761][10331] RunningMeanStd input shape: (1,) [2025-02-22 19:26:20,773][10331] ConvEncoder: input_channels=3 [2025-02-22 19:26:20,883][10331] Conv encoder output size: 512 [2025-02-22 19:26:20,883][10331] Policy head output size: 512 [2025-02-22 19:26:20,921][02012] Inference worker 0-0 is ready! [2025-02-22 19:26:20,922][02012] All inference workers are ready! Signal rollout workers to start! [2025-02-22 19:26:20,956][10333] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,956][10339] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10335] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10332] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10337] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10338] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10336] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:20,977][10334] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-22 19:26:21,308][10333] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,308][10339] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,318][10334] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,318][10336] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,319][10338] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,319][10337] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,566][10339] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,582][10338] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,584][10337] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,588][10334] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,595][10335] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,713][10332] Decorrelating experience for 0 frames... [2025-02-22 19:26:21,740][10333] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,875][10336] Decorrelating experience for 32 frames... [2025-02-22 19:26:21,905][10339] Decorrelating experience for 64 frames... [2025-02-22 19:26:21,959][10338] Decorrelating experience for 64 frames... [2025-02-22 19:26:21,966][10334] Decorrelating experience for 64 frames... [2025-02-22 19:26:21,989][10335] Decorrelating experience for 32 frames... [2025-02-22 19:26:22,066][10337] Decorrelating experience for 64 frames... [2025-02-22 19:26:22,197][10339] Decorrelating experience for 96 frames... [2025-02-22 19:26:22,211][10332] Decorrelating experience for 32 frames... [2025-02-22 19:26:22,269][10338] Decorrelating experience for 96 frames... [2025-02-22 19:26:22,276][10336] Decorrelating experience for 64 frames... [2025-02-22 19:26:22,475][10334] Decorrelating experience for 96 frames... [2025-02-22 19:26:22,476][10337] Decorrelating experience for 96 frames... [2025-02-22 19:26:22,562][10336] Decorrelating experience for 96 frames... [2025-02-22 19:26:22,574][10332] Decorrelating experience for 64 frames... [2025-02-22 19:26:22,725][10333] Decorrelating experience for 64 frames... [2025-02-22 19:26:22,877][10332] Decorrelating experience for 96 frames... [2025-02-22 19:26:23,005][10335] Decorrelating experience for 64 frames... [2025-02-22 19:26:23,058][10333] Decorrelating experience for 96 frames... [2025-02-22 19:26:23,373][10335] Decorrelating experience for 96 frames... [2025-02-22 19:26:23,615][10318] Signal inference workers to stop experience collection... [2025-02-22 19:26:23,622][10331] InferenceWorker_p0-w0: stopping experience collection [2025-02-22 19:26:23,783][02012] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4206592. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-02-22 19:26:23,784][02012] Avg episode reward: [(0, '3.287')] [2025-02-22 19:26:24,648][10318] Signal inference workers to resume experience collection... [2025-02-22 19:26:24,649][10331] InferenceWorker_p0-w0: resuming experience collection [2025-02-22 19:26:26,793][10331] Updated weights for policy 0, policy_version 1037 (0.0092) [2025-02-22 19:26:28,783][02012] Fps is (10 sec: 14745.3, 60 sec: 14745.3, 300 sec: 14745.3). Total num frames: 4280320. Throughput: 0: 2306.8. Samples: 11534. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:26:28,785][02012] Avg episode reward: [(0, '17.377')] [2025-02-22 19:26:29,113][10331] Updated weights for policy 0, policy_version 1047 (0.0012) [2025-02-22 19:26:31,384][10331] Updated weights for policy 0, policy_version 1057 (0.0012) [2025-02-22 19:26:33,702][10331] Updated weights for policy 0, policy_version 1067 (0.0012) [2025-02-22 19:26:33,782][02012] Fps is (10 sec: 16384.0, 60 sec: 16384.0, 300 sec: 16384.0). Total num frames: 4370432. Throughput: 0: 3830.2. Samples: 38302. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:26:33,784][02012] Avg episode reward: [(0, '22.064')] [2025-02-22 19:26:36,044][10331] Updated weights for policy 0, policy_version 1077 (0.0012) [2025-02-22 19:26:36,249][02012] Heartbeat connected on Batcher_0 [2025-02-22 19:26:36,262][02012] Heartbeat connected on InferenceWorker_p0-w0 [2025-02-22 19:26:36,265][02012] Heartbeat connected on RolloutWorker_w0 [2025-02-22 19:26:36,269][02012] Heartbeat connected on RolloutWorker_w1 [2025-02-22 19:26:36,271][02012] Heartbeat connected on RolloutWorker_w2 [2025-02-22 19:26:36,276][02012] Heartbeat connected on RolloutWorker_w3 [2025-02-22 19:26:36,276][02012] Heartbeat connected on LearnerWorker_p0 [2025-02-22 19:26:36,280][02012] Heartbeat connected on RolloutWorker_w4 [2025-02-22 19:26:36,284][02012] Heartbeat connected on RolloutWorker_w5 [2025-02-22 19:26:36,288][02012] Heartbeat connected on RolloutWorker_w6 [2025-02-22 19:26:36,292][02012] Heartbeat connected on RolloutWorker_w7 [2025-02-22 19:26:38,460][10331] Updated weights for policy 0, policy_version 1087 (0.0013) [2025-02-22 19:26:38,782][02012] Fps is (10 sec: 17613.1, 60 sec: 16657.1, 300 sec: 16657.1). Total num frames: 4456448. Throughput: 0: 3428.9. Samples: 51434. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-22 19:26:38,784][02012] Avg episode reward: [(0, '23.483')] [2025-02-22 19:26:38,785][10318] Saving new best policy, reward=23.483! [2025-02-22 19:26:40,823][10331] Updated weights for policy 0, policy_version 1097 (0.0012) [2025-02-22 19:26:41,519][10318] Stopping Batcher_0... [2025-02-22 19:26:41,519][10318] Loop batcher_evt_loop terminating... [2025-02-22 19:26:41,519][02012] Component Batcher_0 stopped! [2025-02-22 19:26:41,520][10318] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001100_4505600.pth... [2025-02-22 19:26:41,542][10331] Weights refcount: 2 0 [2025-02-22 19:26:41,544][10331] Stopping InferenceWorker_p0-w0... [2025-02-22 19:26:41,545][10331] Loop inference_proc0-0_evt_loop terminating... [2025-02-22 19:26:41,545][02012] Component InferenceWorker_p0-w0 stopped! [2025-02-22 19:26:41,566][10339] Stopping RolloutWorker_w6... [2025-02-22 19:26:41,567][10339] Loop rollout_proc6_evt_loop terminating... [2025-02-22 19:26:41,566][02012] Component RolloutWorker_w6 stopped! [2025-02-22 19:26:41,568][10338] Stopping RolloutWorker_w7... [2025-02-22 19:26:41,568][10338] Loop rollout_proc7_evt_loop terminating... [2025-02-22 19:26:41,569][10332] Stopping RolloutWorker_w0... [2025-02-22 19:26:41,569][10332] Loop rollout_proc0_evt_loop terminating... [2025-02-22 19:26:41,569][10334] Stopping RolloutWorker_w3... [2025-02-22 19:26:41,569][10337] Stopping RolloutWorker_w2... [2025-02-22 19:26:41,570][10337] Loop rollout_proc2_evt_loop terminating... [2025-02-22 19:26:41,570][10334] Loop rollout_proc3_evt_loop terminating... [2025-02-22 19:26:41,570][10335] Stopping RolloutWorker_w4... [2025-02-22 19:26:41,570][10333] Stopping RolloutWorker_w1... [2025-02-22 19:26:41,571][10335] Loop rollout_proc4_evt_loop terminating... [2025-02-22 19:26:41,571][10333] Loop rollout_proc1_evt_loop terminating... [2025-02-22 19:26:41,569][02012] Component RolloutWorker_w7 stopped! [2025-02-22 19:26:41,571][02012] Component RolloutWorker_w0 stopped! [2025-02-22 19:26:41,572][02012] Component RolloutWorker_w3 stopped! [2025-02-22 19:26:41,573][02012] Component RolloutWorker_w2 stopped! [2025-02-22 19:26:41,574][10336] Stopping RolloutWorker_w5... [2025-02-22 19:26:41,574][10336] Loop rollout_proc5_evt_loop terminating... [2025-02-22 19:26:41,574][02012] Component RolloutWorker_w4 stopped! [2025-02-22 19:26:41,575][02012] Component RolloutWorker_w1 stopped! [2025-02-22 19:26:41,576][02012] Component RolloutWorker_w5 stopped! [2025-02-22 19:26:41,603][10318] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000942_3858432.pth [2025-02-22 19:26:41,613][10318] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001100_4505600.pth... [2025-02-22 19:26:41,741][10318] Stopping LearnerWorker_p0... [2025-02-22 19:26:41,741][10318] Loop learner_proc0_evt_loop terminating... [2025-02-22 19:26:41,741][02012] Component LearnerWorker_p0 stopped! [2025-02-22 19:26:41,742][02012] Waiting for process learner_proc0 to stop... [2025-02-22 19:26:42,739][02012] Waiting for process inference_proc0-0 to join... [2025-02-22 19:26:42,740][02012] Waiting for process rollout_proc0 to join... [2025-02-22 19:26:42,741][02012] Waiting for process rollout_proc1 to join... [2025-02-22 19:26:42,742][02012] Waiting for process rollout_proc2 to join... [2025-02-22 19:26:42,743][02012] Waiting for process rollout_proc3 to join... [2025-02-22 19:26:42,743][02012] Waiting for process rollout_proc4 to join... [2025-02-22 19:26:42,744][02012] Waiting for process rollout_proc5 to join... [2025-02-22 19:26:42,745][02012] Waiting for process rollout_proc6 to join... [2025-02-22 19:26:42,746][02012] Waiting for process rollout_proc7 to join... [2025-02-22 19:26:42,747][02012] Batcher 0 profile tree view: batching: 0.9334, releasing_batches: 0.0023 [2025-02-22 19:26:42,748][02012] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 1.5267 update_model: 0.3773 weight_update: 0.0011 one_step: 0.0037 handle_policy_step: 16.9405 deserialize: 0.6933, stack: 0.1171, obs_to_device_normalize: 3.8782, forward: 8.7838, send_messages: 1.0990 prepare_outputs: 1.7696 to_cpu: 1.0117 [2025-02-22 19:26:42,749][02012] Learner 0 profile tree view: misc: 0.0003, prepare_batch: 1.5026 train: 2.4575 epoch_init: 0.0003, minibatch_init: 0.0005, losses_postprocess: 0.0189, kl_divergence: 0.0349, after_optimizer: 0.0961 calculate_losses: 0.9306 losses_init: 0.0002, forward_head: 0.1507, bptt_initial: 0.4940, tail: 0.0670, advantages_returns: 0.0140, losses: 0.1032 bptt: 0.0889 bptt_forward_core: 0.0847 update: 1.3479 clip: 0.1070 [2025-02-22 19:26:42,749][02012] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.0103, enqueue_policy_requests: 0.8050, env_step: 10.5465, overhead: 0.4729, complete_rollouts: 0.0197 save_policy_outputs: 0.6665 split_output_tensors: 0.2580 [2025-02-22 19:26:42,750][02012] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.0104, enqueue_policy_requests: 0.8040, env_step: 10.6529, overhead: 0.4772, complete_rollouts: 0.0194 save_policy_outputs: 0.6606 split_output_tensors: 0.2551 [2025-02-22 19:26:42,751][02012] Loop Runner_EvtLoop terminating... [2025-02-22 19:26:42,752][02012] Runner profile tree view: main_loop: 26.4626 [2025-02-22 19:26:42,753][02012] Collected {0: 4505600}, FPS: 11299.3 [2025-02-22 19:27:54,802][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:27:54,803][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:27:54,804][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:27:54,804][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:27:54,805][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:27:54,805][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:27:54,806][02012] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:27:54,807][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:27:54,808][02012] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-02-22 19:27:54,808][02012] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-02-22 19:27:54,809][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:27:54,810][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:27:54,810][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:27:54,811][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:27:54,812][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:27:54,835][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:27:54,838][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:27:54,848][02012] ConvEncoder: input_channels=3 [2025-02-22 19:27:54,885][02012] Conv encoder output size: 512 [2025-02-22 19:27:54,886][02012] Policy head output size: 512 [2025-02-22 19:27:54,906][02012] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001100_4505600.pth... [2025-02-22 19:27:55,383][02012] Num frames 100... [2025-02-22 19:27:55,518][02012] Num frames 200... [2025-02-22 19:27:55,656][02012] Num frames 300... [2025-02-22 19:27:55,798][02012] Num frames 400... [2025-02-22 19:27:55,932][02012] Num frames 500... [2025-02-22 19:27:56,067][02012] Num frames 600... [2025-02-22 19:27:56,205][02012] Num frames 700... [2025-02-22 19:27:56,265][02012] Avg episode rewards: #0: 15.040, true rewards: #0: 7.040 [2025-02-22 19:27:56,267][02012] Avg episode reward: 15.040, avg true_objective: 7.040 [2025-02-22 19:27:56,399][02012] Num frames 800... [2025-02-22 19:27:56,537][02012] Num frames 900... [2025-02-22 19:27:56,672][02012] Num frames 1000... [2025-02-22 19:27:56,807][02012] Num frames 1100... [2025-02-22 19:27:56,944][02012] Num frames 1200... [2025-02-22 19:27:57,082][02012] Num frames 1300... [2025-02-22 19:27:57,218][02012] Num frames 1400... [2025-02-22 19:27:57,358][02012] Num frames 1500... [2025-02-22 19:27:57,496][02012] Num frames 1600... [2025-02-22 19:27:57,633][02012] Num frames 1700... [2025-02-22 19:27:57,728][02012] Avg episode rewards: #0: 19.140, true rewards: #0: 8.640 [2025-02-22 19:27:57,729][02012] Avg episode reward: 19.140, avg true_objective: 8.640 [2025-02-22 19:27:57,830][02012] Num frames 1800... [2025-02-22 19:27:57,966][02012] Num frames 1900... [2025-02-22 19:27:58,101][02012] Num frames 2000... [2025-02-22 19:27:58,242][02012] Num frames 2100... [2025-02-22 19:27:58,381][02012] Num frames 2200... [2025-02-22 19:27:58,513][02012] Num frames 2300... [2025-02-22 19:27:58,641][02012] Num frames 2400... [2025-02-22 19:27:58,769][02012] Num frames 2500... [2025-02-22 19:27:58,897][02012] Num frames 2600... [2025-02-22 19:27:59,028][02012] Num frames 2700... [2025-02-22 19:27:59,168][02012] Num frames 2800... [2025-02-22 19:27:59,310][02012] Num frames 2900... [2025-02-22 19:27:59,453][02012] Num frames 3000... [2025-02-22 19:27:59,519][02012] Avg episode rewards: #0: 23.693, true rewards: #0: 10.027 [2025-02-22 19:27:59,520][02012] Avg episode reward: 23.693, avg true_objective: 10.027 [2025-02-22 19:27:59,647][02012] Num frames 3100... [2025-02-22 19:27:59,781][02012] Num frames 3200... [2025-02-22 19:27:59,914][02012] Num frames 3300... [2025-02-22 19:28:00,046][02012] Num frames 3400... [2025-02-22 19:28:00,176][02012] Num frames 3500... [2025-02-22 19:28:00,304][02012] Num frames 3600... [2025-02-22 19:28:00,433][02012] Num frames 3700... [2025-02-22 19:28:00,560][02012] Num frames 3800... [2025-02-22 19:28:00,696][02012] Num frames 3900... [2025-02-22 19:28:00,830][02012] Num frames 4000... [2025-02-22 19:28:00,963][02012] Num frames 4100... [2025-02-22 19:28:01,095][02012] Num frames 4200... [2025-02-22 19:28:01,225][02012] Num frames 4300... [2025-02-22 19:28:01,366][02012] Num frames 4400... [2025-02-22 19:28:01,475][02012] Avg episode rewards: #0: 26.103, true rewards: #0: 11.102 [2025-02-22 19:28:01,476][02012] Avg episode reward: 26.103, avg true_objective: 11.102 [2025-02-22 19:28:01,554][02012] Num frames 4500... [2025-02-22 19:28:01,686][02012] Num frames 4600... [2025-02-22 19:28:01,815][02012] Num frames 4700... [2025-02-22 19:28:01,944][02012] Num frames 4800... [2025-02-22 19:28:02,071][02012] Num frames 4900... [2025-02-22 19:28:02,201][02012] Num frames 5000... [2025-02-22 19:28:02,328][02012] Num frames 5100... [2025-02-22 19:28:02,460][02012] Num frames 5200... [2025-02-22 19:28:02,594][02012] Num frames 5300... [2025-02-22 19:28:02,732][02012] Num frames 5400... [2025-02-22 19:28:02,865][02012] Num frames 5500... [2025-02-22 19:28:02,926][02012] Avg episode rewards: #0: 25.008, true rewards: #0: 11.008 [2025-02-22 19:28:02,927][02012] Avg episode reward: 25.008, avg true_objective: 11.008 [2025-02-22 19:28:03,052][02012] Num frames 5600... [2025-02-22 19:28:03,182][02012] Num frames 5700... [2025-02-22 19:28:03,314][02012] Num frames 5800... [2025-02-22 19:28:03,443][02012] Num frames 5900... [2025-02-22 19:28:03,579][02012] Num frames 6000... [2025-02-22 19:28:03,726][02012] Num frames 6100... [2025-02-22 19:28:03,870][02012] Num frames 6200... [2025-02-22 19:28:04,016][02012] Num frames 6300... [2025-02-22 19:28:04,165][02012] Num frames 6400... [2025-02-22 19:28:04,309][02012] Num frames 6500... [2025-02-22 19:28:04,456][02012] Num frames 6600... [2025-02-22 19:28:04,607][02012] Avg episode rewards: #0: 25.277, true rewards: #0: 11.110 [2025-02-22 19:28:04,609][02012] Avg episode reward: 25.277, avg true_objective: 11.110 [2025-02-22 19:28:04,659][02012] Num frames 6700... [2025-02-22 19:28:04,801][02012] Num frames 6800... [2025-02-22 19:28:04,944][02012] Num frames 6900... [2025-02-22 19:28:05,086][02012] Num frames 7000... [2025-02-22 19:28:05,230][02012] Num frames 7100... [2025-02-22 19:28:05,373][02012] Num frames 7200... [2025-02-22 19:28:05,517][02012] Num frames 7300... [2025-02-22 19:28:05,661][02012] Num frames 7400... [2025-02-22 19:28:05,806][02012] Num frames 7500... [2025-02-22 19:28:05,950][02012] Num frames 7600... [2025-02-22 19:28:06,093][02012] Num frames 7700... [2025-02-22 19:28:06,236][02012] Num frames 7800... [2025-02-22 19:28:06,380][02012] Num frames 7900... [2025-02-22 19:28:06,457][02012] Avg episode rewards: #0: 25.877, true rewards: #0: 11.306 [2025-02-22 19:28:06,458][02012] Avg episode reward: 25.877, avg true_objective: 11.306 [2025-02-22 19:28:06,586][02012] Num frames 8000... [2025-02-22 19:28:06,731][02012] Num frames 8100... [2025-02-22 19:28:06,875][02012] Num frames 8200... [2025-02-22 19:28:07,018][02012] Num frames 8300... [2025-02-22 19:28:07,166][02012] Num frames 8400... [2025-02-22 19:28:07,313][02012] Num frames 8500... [2025-02-22 19:28:07,468][02012] Num frames 8600... [2025-02-22 19:28:07,618][02012] Num frames 8700... [2025-02-22 19:28:07,766][02012] Num frames 8800... [2025-02-22 19:28:07,911][02012] Num frames 8900... [2025-02-22 19:28:08,058][02012] Num frames 9000... [2025-02-22 19:28:08,200][02012] Num frames 9100... [2025-02-22 19:28:08,345][02012] Num frames 9200... [2025-02-22 19:28:08,488][02012] Num frames 9300... [2025-02-22 19:28:08,622][02012] Avg episode rewards: #0: 26.943, true rewards: #0: 11.692 [2025-02-22 19:28:08,623][02012] Avg episode reward: 26.943, avg true_objective: 11.692 [2025-02-22 19:28:08,690][02012] Num frames 9400... [2025-02-22 19:28:08,831][02012] Num frames 9500... [2025-02-22 19:28:08,974][02012] Num frames 9600... [2025-02-22 19:28:09,117][02012] Num frames 9700... [2025-02-22 19:28:09,260][02012] Num frames 9800... [2025-02-22 19:28:09,401][02012] Num frames 9900... [2025-02-22 19:28:09,543][02012] Num frames 10000... [2025-02-22 19:28:09,688][02012] Num frames 10100... [2025-02-22 19:28:09,850][02012] Num frames 10200... [2025-02-22 19:28:10,004][02012] Num frames 10300... [2025-02-22 19:28:10,156][02012] Num frames 10400... [2025-02-22 19:28:10,285][02012] Avg episode rewards: #0: 26.609, true rewards: #0: 11.609 [2025-02-22 19:28:10,286][02012] Avg episode reward: 26.609, avg true_objective: 11.609 [2025-02-22 19:28:10,367][02012] Num frames 10500... [2025-02-22 19:28:10,521][02012] Num frames 10600... [2025-02-22 19:28:10,684][02012] Num frames 10700... [2025-02-22 19:28:10,833][02012] Num frames 10800... [2025-02-22 19:28:10,992][02012] Num frames 10900... [2025-02-22 19:28:11,151][02012] Num frames 11000... [2025-02-22 19:28:11,293][02012] Avg episode rewards: #0: 25.056, true rewards: #0: 11.056 [2025-02-22 19:28:11,294][02012] Avg episode reward: 25.056, avg true_objective: 11.056 [2025-02-22 19:28:38,184][02012] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-02-22 19:29:16,636][02012] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-22 19:29:16,637][02012] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-22 19:29:16,637][02012] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-22 19:29:16,638][02012] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-22 19:29:16,639][02012] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-22 19:29:16,639][02012] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-22 19:29:16,640][02012] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-02-22 19:29:16,640][02012] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-22 19:29:16,641][02012] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-02-22 19:29:16,642][02012] Adding new argument 'hf_repository'='iamandrewliao/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-02-22 19:29:16,643][02012] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-22 19:29:16,644][02012] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-22 19:29:16,644][02012] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-22 19:29:16,645][02012] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-22 19:29:16,646][02012] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-22 19:29:16,671][02012] RunningMeanStd input shape: (3, 72, 128) [2025-02-22 19:29:16,673][02012] RunningMeanStd input shape: (1,) [2025-02-22 19:29:16,684][02012] ConvEncoder: input_channels=3 [2025-02-22 19:29:16,722][02012] Conv encoder output size: 512 [2025-02-22 19:29:16,723][02012] Policy head output size: 512 [2025-02-22 19:29:16,743][02012] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001100_4505600.pth... [2025-02-22 19:29:17,221][02012] Num frames 100... [2025-02-22 19:29:17,364][02012] Num frames 200... [2025-02-22 19:29:17,504][02012] Num frames 300... [2025-02-22 19:29:17,649][02012] Num frames 400... [2025-02-22 19:29:17,790][02012] Num frames 500... [2025-02-22 19:29:17,931][02012] Num frames 600... [2025-02-22 19:29:18,074][02012] Num frames 700... [2025-02-22 19:29:18,135][02012] Avg episode rewards: #0: 14.040, true rewards: #0: 7.040 [2025-02-22 19:29:18,136][02012] Avg episode reward: 14.040, avg true_objective: 7.040 [2025-02-22 19:29:18,273][02012] Num frames 800... [2025-02-22 19:29:18,417][02012] Num frames 900... [2025-02-22 19:29:18,558][02012] Num frames 1000... [2025-02-22 19:29:18,705][02012] Num frames 1100... [2025-02-22 19:29:18,844][02012] Num frames 1200... [2025-02-22 19:29:18,986][02012] Num frames 1300... [2025-02-22 19:29:19,128][02012] Num frames 1400... [2025-02-22 19:29:19,267][02012] Num frames 1500... [2025-02-22 19:29:19,421][02012] Num frames 1600... [2025-02-22 19:29:19,622][02012] Avg episode rewards: #0: 16.480, true rewards: #0: 8.480 [2025-02-22 19:29:19,623][02012] Avg episode reward: 16.480, avg true_objective: 8.480 [2025-02-22 19:29:19,631][02012] Num frames 1700... [2025-02-22 19:29:19,775][02012] Num frames 1800... [2025-02-22 19:29:19,916][02012] Num frames 1900... [2025-02-22 19:29:20,063][02012] Num frames 2000... [2025-02-22 19:29:20,211][02012] Num frames 2100... [2025-02-22 19:29:20,364][02012] Num frames 2200... [2025-02-22 19:29:20,518][02012] Num frames 2300... [2025-02-22 19:29:20,676][02012] Num frames 2400... [2025-02-22 19:29:20,831][02012] Num frames 2500... [2025-02-22 19:29:21,024][02012] Avg episode rewards: #0: 17.307, true rewards: #0: 8.640 [2025-02-22 19:29:21,025][02012] Avg episode reward: 17.307, avg true_objective: 8.640 [2025-02-22 19:29:21,039][02012] Num frames 2600... [2025-02-22 19:29:21,185][02012] Num frames 2700... [2025-02-22 19:29:21,326][02012] Num frames 2800... [2025-02-22 19:29:21,464][02012] Num frames 2900... [2025-02-22 19:29:21,606][02012] Num frames 3000... [2025-02-22 19:29:21,751][02012] Num frames 3100... [2025-02-22 19:29:21,893][02012] Num frames 3200... [2025-02-22 19:29:22,036][02012] Num frames 3300... [2025-02-22 19:29:22,179][02012] Num frames 3400... [2025-02-22 19:29:22,370][02012] Avg episode rewards: #0: 17.740, true rewards: #0: 8.740 [2025-02-22 19:29:22,371][02012] Avg episode reward: 17.740, avg true_objective: 8.740 [2025-02-22 19:29:22,379][02012] Num frames 3500... [2025-02-22 19:29:22,521][02012] Num frames 3600... [2025-02-22 19:29:22,659][02012] Num frames 3700... [2025-02-22 19:29:22,800][02012] Num frames 3800... [2025-02-22 19:29:22,939][02012] Num frames 3900... [2025-02-22 19:29:23,079][02012] Num frames 4000... [2025-02-22 19:29:23,219][02012] Num frames 4100... [2025-02-22 19:29:23,359][02012] Num frames 4200... [2025-02-22 19:29:23,502][02012] Num frames 4300... [2025-02-22 19:29:23,639][02012] Avg episode rewards: #0: 17.720, true rewards: #0: 8.720 [2025-02-22 19:29:23,640][02012] Avg episode reward: 17.720, avg true_objective: 8.720 [2025-02-22 19:29:23,698][02012] Num frames 4400... [2025-02-22 19:29:23,839][02012] Num frames 4500... [2025-02-22 19:29:23,983][02012] Num frames 4600... [2025-02-22 19:29:24,126][02012] Num frames 4700... [2025-02-22 19:29:24,267][02012] Num frames 4800... [2025-02-22 19:29:24,409][02012] Num frames 4900... [2025-02-22 19:29:24,469][02012] Avg episode rewards: #0: 16.007, true rewards: #0: 8.173 [2025-02-22 19:29:24,470][02012] Avg episode reward: 16.007, avg true_objective: 8.173 [2025-02-22 19:29:24,608][02012] Num frames 5000... [2025-02-22 19:29:24,755][02012] Num frames 5100... [2025-02-22 19:29:24,900][02012] Num frames 5200... [2025-02-22 19:29:25,044][02012] Num frames 5300... [2025-02-22 19:29:25,188][02012] Num frames 5400... [2025-02-22 19:29:25,333][02012] Num frames 5500... [2025-02-22 19:29:25,479][02012] Num frames 5600... [2025-02-22 19:29:25,623][02012] Num frames 5700... [2025-02-22 19:29:25,770][02012] Num frames 5800... [2025-02-22 19:29:25,915][02012] Num frames 5900... [2025-02-22 19:29:26,058][02012] Num frames 6000... [2025-02-22 19:29:26,202][02012] Num frames 6100... [2025-02-22 19:29:26,345][02012] Num frames 6200... [2025-02-22 19:29:26,487][02012] Num frames 6300... [2025-02-22 19:29:26,633][02012] Num frames 6400... [2025-02-22 19:29:26,700][02012] Avg episode rewards: #0: 19.012, true rewards: #0: 9.154 [2025-02-22 19:29:26,701][02012] Avg episode reward: 19.012, avg true_objective: 9.154 [2025-02-22 19:29:26,834][02012] Num frames 6500... [2025-02-22 19:29:26,972][02012] Num frames 6600... [2025-02-22 19:29:27,107][02012] Num frames 6700... [2025-02-22 19:29:27,242][02012] Num frames 6800... [2025-02-22 19:29:27,375][02012] Num frames 6900... [2025-02-22 19:29:27,508][02012] Num frames 7000... [2025-02-22 19:29:27,642][02012] Num frames 7100... [2025-02-22 19:29:27,780][02012] Num frames 7200... [2025-02-22 19:29:27,846][02012] Avg episode rewards: #0: 18.510, true rewards: #0: 9.010 [2025-02-22 19:29:27,847][02012] Avg episode reward: 18.510, avg true_objective: 9.010 [2025-02-22 19:29:27,974][02012] Num frames 7300... [2025-02-22 19:29:28,109][02012] Num frames 7400... [2025-02-22 19:29:28,247][02012] Num frames 7500... [2025-02-22 19:29:28,384][02012] Num frames 7600... [2025-02-22 19:29:28,524][02012] Num frames 7700... [2025-02-22 19:29:28,664][02012] Num frames 7800... [2025-02-22 19:29:28,803][02012] Num frames 7900... [2025-02-22 19:29:28,936][02012] Num frames 8000... [2025-02-22 19:29:29,110][02012] Avg episode rewards: #0: 18.431, true rewards: #0: 8.987 [2025-02-22 19:29:29,111][02012] Avg episode reward: 18.431, avg true_objective: 8.987 [2025-02-22 19:29:29,129][02012] Num frames 8100... [2025-02-22 19:29:29,264][02012] Num frames 8200... [2025-02-22 19:29:29,412][02012] Num frames 8300... [2025-02-22 19:29:29,552][02012] Num frames 8400... [2025-02-22 19:29:29,695][02012] Num frames 8500... [2025-02-22 19:29:29,834][02012] Num frames 8600... [2025-02-22 19:29:29,975][02012] Num frames 8700... [2025-02-22 19:29:30,112][02012] Num frames 8800... [2025-02-22 19:29:30,256][02012] Num frames 8900... [2025-02-22 19:29:30,369][02012] Avg episode rewards: #0: 18.640, true rewards: #0: 8.940 [2025-02-22 19:29:30,370][02012] Avg episode reward: 18.640, avg true_objective: 8.940 [2025-02-22 19:29:52,000][02012] Replay video saved to /content/train_dir/default_experiment/replay.mp4!