| [2025-04-06 21:21:03,390][29458] Saving configuration to /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/config.json... | |
| [2025-04-06 21:21:03,429][29458] Rollout worker 0 uses device cpu | |
| [2025-04-06 21:21:03,429][29458] Rollout worker 1 uses device cpu | |
| [2025-04-06 21:21:03,429][29458] Rollout worker 2 uses device cpu | |
| [2025-04-06 21:21:03,430][29458] Rollout worker 3 uses device cpu | |
| [2025-04-06 21:21:03,430][29458] Rollout worker 4 uses device cpu | |
| [2025-04-06 21:21:03,430][29458] Rollout worker 5 uses device cpu | |
| [2025-04-06 21:21:03,430][29458] Rollout worker 6 uses device cpu | |
| [2025-04-06 21:21:03,431][29458] Rollout worker 7 uses device cpu | |
| [2025-04-06 21:21:03,542][29458] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-04-06 21:21:03,542][29458] InferenceWorker_p0-w0: min num requests: 2 | |
| [2025-04-06 21:21:03,595][29458] Starting all processes... | |
| [2025-04-06 21:21:03,596][29458] Starting process learner_proc0 | |
| [2025-04-06 21:21:12,041][29458] Starting all processes... | |
| [2025-04-06 21:21:12,055][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-04-06 21:21:12,056][29697] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2025-04-06 21:21:12,068][29458] Starting process inference_proc0-0 | |
| [2025-04-06 21:21:12,068][29458] Starting process rollout_proc0 | |
| [2025-04-06 21:21:12,068][29458] Starting process rollout_proc1 | |
| [2025-04-06 21:21:12,071][29458] Starting process rollout_proc6 | |
| [2025-04-06 21:21:12,078][29697] Num visible devices: 1 | |
| [2025-04-06 21:21:12,086][29697] Starting seed is not provided | |
| [2025-04-06 21:21:12,087][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-04-06 21:21:12,088][29697] Initializing actor-critic model on device cuda:0 | |
| [2025-04-06 21:21:12,090][29697] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-04-06 21:21:12,095][29697] RunningMeanStd input shape: (1,) | |
| [2025-04-06 21:21:12,069][29458] Starting process rollout_proc3 | |
| [2025-04-06 21:21:12,069][29458] Starting process rollout_proc4 | |
| [2025-04-06 21:21:12,070][29458] Starting process rollout_proc5 | |
| [2025-04-06 21:21:12,068][29458] Starting process rollout_proc2 | |
| [2025-04-06 21:21:12,071][29458] Starting process rollout_proc7 | |
| [2025-04-06 21:21:12,144][29697] ConvEncoder: input_channels=3 | |
| [2025-04-06 21:21:12,576][29697] Conv encoder output size: 512 | |
| [2025-04-06 21:21:12,577][29697] Policy head output size: 512 | |
| [2025-04-06 21:21:12,610][29697] Created Actor Critic model with architecture: | |
| [2025-04-06 21:21:12,611][29697] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2025-04-06 21:21:12,813][29697] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2025-04-06 21:21:18,468][29697] No checkpoints found | |
| [2025-04-06 21:21:18,471][29697] Did not load from checkpoint, starting from scratch! | |
| [2025-04-06 21:21:18,474][29697] Initialized policy 0 weights for model version 0 | |
| [2025-04-06 21:21:18,488][29697] LearnerWorker_p0 finished initialization! | |
| [2025-04-06 21:21:18,488][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-04-06 21:21:27,570][29817] Worker 6 uses CPU cores [6] | |
| [2025-04-06 21:21:32,697][29818] Worker 0 uses CPU cores [0] | |
| [2025-04-06 21:21:34,112][29822] Worker 5 uses CPU cores [5] | |
| [2025-04-06 21:21:40,610][29819] Worker 3 uses CPU cores [3] | |
| [2025-04-06 21:21:50,398][29816] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-04-06 21:21:50,399][29816] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2025-04-06 21:21:50,418][29816] Num visible devices: 1 | |
| [2025-04-06 21:21:50,551][29816] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-04-06 21:21:50,553][29816] RunningMeanStd input shape: (1,) | |
| [2025-04-06 21:21:50,583][29816] ConvEncoder: input_channels=3 | |
| [2025-04-06 21:21:50,754][29816] Conv encoder output size: 512 | |
| [2025-04-06 21:21:50,755][29816] Policy head output size: 512 | |
| [2025-04-06 21:21:56,879][29823] Worker 7 uses CPU cores [7] | |
| [2025-04-06 21:22:03,459][29815] Worker 1 uses CPU cores [1] | |
| [2025-04-06 21:22:25,110][29820] Worker 4 uses CPU cores [4] | |
| [2025-04-06 21:22:27,189][29458] Heartbeat connected on Batcher_0 | |
| [2025-04-06 21:22:27,191][29458] Heartbeat connected on LearnerWorker_p0 | |
| [2025-04-06 21:22:27,191][29458] Heartbeat connected on RolloutWorker_w6 | |
| [2025-04-06 21:22:27,192][29458] Heartbeat connected on RolloutWorker_w0 | |
| [2025-04-06 21:22:27,192][29458] Heartbeat connected on RolloutWorker_w5 | |
| [2025-04-06 21:22:27,193][29458] Heartbeat connected on RolloutWorker_w3 | |
| [2025-04-06 21:22:27,193][29458] Inference worker 0-0 is ready! | |
| [2025-04-06 21:22:27,194][29458] All inference workers are ready! Signal rollout workers to start! | |
| [2025-04-06 21:22:27,195][29458] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2025-04-06 21:22:27,195][29458] Heartbeat connected on RolloutWorker_w7 | |
| [2025-04-06 21:22:27,196][29821] Worker 2 uses CPU cores [2] | |
| [2025-04-06 21:22:27,198][29458] Heartbeat connected on RolloutWorker_w1 | |
| [2025-04-06 21:22:27,200][29458] Heartbeat connected on RolloutWorker_w4 | |
| [2025-04-06 21:22:27,202][29458] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,224][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,231][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,236][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,239][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,248][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,252][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,255][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,257][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,260][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,261][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,263][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,264][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,265][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,267][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,270][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,271][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:27,276][29820] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,276][29818] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,295][29823] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,317][29819] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,319][29815] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,343][29821] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,380][29817] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,394][29822] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:22:27,868][29815] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:27,868][29820] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:27,869][29823] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:27,871][29819] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:27,882][29818] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:27,902][29817] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:28,222][29818] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,231][29819] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,240][29817] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,256][29821] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:28,260][29820] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,275][29823] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,582][29822] Decorrelating experience for 0 frames... | |
| [2025-04-06 21:22:28,598][29821] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,707][29819] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:28,748][29817] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:28,760][29823] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:28,783][29818] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:28,980][29822] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:28,985][29820] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:29,096][29819] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,163][29817] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,237][29818] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,391][29815] Decorrelating experience for 32 frames... | |
| [2025-04-06 21:22:29,393][29820] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,460][29822] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:29,589][29823] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,790][29821] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:29,836][29822] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:29,856][29815] Decorrelating experience for 64 frames... | |
| [2025-04-06 21:22:30,133][29821] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:30,204][29815] Decorrelating experience for 96 frames... | |
| [2025-04-06 21:22:30,234][29458] Heartbeat connected on RolloutWorker_w2 | |
| [2025-04-06 21:22:30,818][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 9.0. Samples: 32. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-04-06 21:22:30,819][29458] Avg episode reward: [(0, '0.480')] | |
| [2025-04-06 21:22:31,550][29697] Signal inference workers to stop experience collection... | |
| [2025-04-06 21:22:31,561][29816] InferenceWorker_p0-w0: stopping experience collection | |
| [2025-04-06 21:22:34,456][29697] Signal inference workers to resume experience collection... | |
| [2025-04-06 21:22:34,458][29816] InferenceWorker_p0-w0: resuming experience collection | |
| [2025-04-06 21:22:35,818][29458] Fps is (10 sec: 1917.0, 60 sec: 1912.7, 300 sec: 1901.5). Total num frames: 16384. Throughput: 0: 571.1. Samples: 4888. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2025-04-06 21:22:35,819][29458] Avg episode reward: [(0, '3.035')] | |
| [2025-04-06 21:22:38,690][29816] Updated weights for policy 0, policy_version 10 (0.0118) | |
| [2025-04-06 21:22:40,819][29458] Fps is (10 sec: 5734.4, 60 sec: 4227.9, 300 sec: 4211.3). Total num frames: 57344. Throughput: 0: 800.0. Samples: 10846. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:22:40,820][29458] Avg episode reward: [(0, '4.419')] | |
| [2025-04-06 21:22:43,570][29816] Updated weights for policy 0, policy_version 20 (0.0035) | |
| [2025-04-06 21:22:45,818][29458] Fps is (10 sec: 8191.9, 60 sec: 5296.3, 300 sec: 5280.4). Total num frames: 98304. Throughput: 0: 1273.0. Samples: 23622. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:22:45,819][29458] Avg episode reward: [(0, '4.432')] | |
| [2025-04-06 21:22:48,379][29816] Updated weights for policy 0, policy_version 30 (0.0034) | |
| [2025-04-06 21:22:50,818][29458] Fps is (10 sec: 8601.9, 60 sec: 6085.3, 300 sec: 6070.3). Total num frames: 143360. Throughput: 0: 1556.9. Samples: 36672. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:22:50,819][29458] Avg episode reward: [(0, '4.405')] | |
| [2025-04-06 21:22:50,836][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000035_143360.pth... | |
| [2025-04-06 21:22:50,967][29697] Saving new best policy, reward=4.405! | |
| [2025-04-06 21:22:53,235][29816] Updated weights for policy 0, policy_version 40 (0.0038) | |
| [2025-04-06 21:22:55,818][29458] Fps is (10 sec: 8601.6, 60 sec: 6454.5, 300 sec: 6441.0). Total num frames: 184320. Throughput: 0: 1497.7. Samples: 42764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:22:55,819][29458] Avg episode reward: [(0, '4.547')] | |
| [2025-04-06 21:22:55,820][29697] Saving new best policy, reward=4.547! | |
| [2025-04-06 21:22:57,999][29816] Updated weights for policy 0, policy_version 50 (0.0035) | |
| [2025-04-06 21:23:00,818][29458] Fps is (10 sec: 8191.9, 60 sec: 6713.7, 300 sec: 6701.5). Total num frames: 225280. Throughput: 0: 1658.8. Samples: 55656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:23:00,819][29458] Avg episode reward: [(0, '4.311')] | |
| [2025-04-06 21:23:02,747][29816] Updated weights for policy 0, policy_version 60 (0.0040) | |
| [2025-04-06 21:23:05,818][29458] Fps is (10 sec: 8601.7, 60 sec: 7011.9, 300 sec: 7000.5). Total num frames: 270336. Throughput: 0: 1783.5. Samples: 68752. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:23:05,819][29458] Avg episode reward: [(0, '4.297')] | |
| [2025-04-06 21:23:07,502][29816] Updated weights for policy 0, policy_version 70 (0.0031) | |
| [2025-04-06 21:23:10,819][29458] Fps is (10 sec: 8601.2, 60 sec: 7147.5, 300 sec: 7137.0). Total num frames: 311296. Throughput: 0: 1724.7. Samples: 75108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:23:10,819][29458] Avg episode reward: [(0, '4.408')] | |
| [2025-04-06 21:23:12,285][29816] Updated weights for policy 0, policy_version 80 (0.0035) | |
| [2025-04-06 21:23:15,819][29458] Fps is (10 sec: 8600.7, 60 sec: 7339.6, 300 sec: 7329.7). Total num frames: 356352. Throughput: 0: 1957.3. Samples: 88112. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:23:15,820][29458] Avg episode reward: [(0, '4.271')] | |
| [2025-04-06 21:23:17,010][29816] Updated weights for policy 0, policy_version 90 (0.0031) | |
| [2025-04-06 21:23:20,818][29458] Fps is (10 sec: 9011.5, 60 sec: 7496.2, 300 sec: 7486.6). Total num frames: 401408. Throughput: 0: 1990.8. Samples: 94474. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:23:20,819][29458] Avg episode reward: [(0, '4.516')] | |
| [2025-04-06 21:23:21,817][29816] Updated weights for policy 0, policy_version 100 (0.0038) | |
| [2025-04-06 21:23:25,818][29458] Fps is (10 sec: 8602.4, 60 sec: 7555.8, 300 sec: 7546.8). Total num frames: 442368. Throughput: 0: 2147.5. Samples: 107482. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:23:25,819][29458] Avg episode reward: [(0, '4.406')] | |
| [2025-04-06 21:23:26,458][29816] Updated weights for policy 0, policy_version 110 (0.0034) | |
| [2025-04-06 21:23:30,818][29458] Fps is (10 sec: 8601.7, 60 sec: 8123.8, 300 sec: 7661.9). Total num frames: 487424. Throughput: 0: 2158.1. Samples: 120734. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2025-04-06 21:23:30,819][29458] Avg episode reward: [(0, '4.321')] | |
| [2025-04-06 21:23:30,997][29816] Updated weights for policy 0, policy_version 120 (0.0036) | |
| [2025-04-06 21:23:35,819][29458] Fps is (10 sec: 8601.2, 60 sec: 8533.3, 300 sec: 7700.5). Total num frames: 528384. Throughput: 0: 2155.0. Samples: 133648. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:23:35,819][29458] Avg episode reward: [(0, '4.625')] | |
| [2025-04-06 21:23:35,863][29697] Saving new best policy, reward=4.625! | |
| [2025-04-06 21:23:35,869][29816] Updated weights for policy 0, policy_version 130 (0.0033) | |
| [2025-04-06 21:23:40,637][29816] Updated weights for policy 0, policy_version 140 (0.0035) | |
| [2025-04-06 21:23:40,819][29458] Fps is (10 sec: 8601.1, 60 sec: 8601.6, 300 sec: 7789.5). Total num frames: 573440. Throughput: 0: 2162.5. Samples: 140078. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:23:40,820][29458] Avg episode reward: [(0, '4.268')] | |
| [2025-04-06 21:23:45,417][29816] Updated weights for policy 0, policy_version 150 (0.0037) | |
| [2025-04-06 21:23:45,818][29458] Fps is (10 sec: 8602.0, 60 sec: 8601.6, 300 sec: 7815.1). Total num frames: 614400. Throughput: 0: 2163.8. Samples: 153026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:23:45,819][29458] Avg episode reward: [(0, '4.387')] | |
| [2025-04-06 21:23:50,154][29816] Updated weights for policy 0, policy_version 160 (0.0039) | |
| [2025-04-06 21:23:50,819][29458] Fps is (10 sec: 8601.6, 60 sec: 8601.5, 300 sec: 7886.6). Total num frames: 659456. Throughput: 0: 2159.5. Samples: 165932. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:23:50,819][29458] Avg episode reward: [(0, '4.397')] | |
| [2025-04-06 21:23:54,952][29816] Updated weights for policy 0, policy_version 170 (0.0036) | |
| [2025-04-06 21:23:55,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8601.6, 300 sec: 7903.9). Total num frames: 700416. Throughput: 0: 2161.7. Samples: 172382. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:23:55,819][29458] Avg episode reward: [(0, '4.640')] | |
| [2025-04-06 21:23:55,866][29697] Saving new best policy, reward=4.640! | |
| [2025-04-06 21:23:59,759][29816] Updated weights for policy 0, policy_version 180 (0.0037) | |
| [2025-04-06 21:24:00,818][29458] Fps is (10 sec: 8602.0, 60 sec: 8669.8, 300 sec: 7963.0). Total num frames: 745472. Throughput: 0: 2154.8. Samples: 185076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:24:00,819][29458] Avg episode reward: [(0, '4.844')] | |
| [2025-04-06 21:24:00,830][29697] Saving new best policy, reward=4.844! | |
| [2025-04-06 21:24:04,949][29816] Updated weights for policy 0, policy_version 190 (0.0039) | |
| [2025-04-06 21:24:05,819][29458] Fps is (10 sec: 8191.9, 60 sec: 8533.3, 300 sec: 7933.1). Total num frames: 782336. Throughput: 0: 2273.2. Samples: 196770. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:24:05,820][29458] Avg episode reward: [(0, '4.955')] | |
| [2025-04-06 21:24:05,821][29697] Saving new best policy, reward=4.955! | |
| [2025-04-06 21:24:10,673][29816] Updated weights for policy 0, policy_version 200 (0.0037) | |
| [2025-04-06 21:24:10,819][29458] Fps is (10 sec: 7372.7, 60 sec: 8465.1, 300 sec: 7906.1). Total num frames: 819200. Throughput: 0: 2103.5. Samples: 202142. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:24:10,819][29458] Avg episode reward: [(0, '4.772')] | |
| [2025-04-06 21:24:15,818][29458] Fps is (10 sec: 7373.1, 60 sec: 8328.7, 300 sec: 7881.5). Total num frames: 856064. Throughput: 0: 2042.1. Samples: 212630. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:24:15,819][29458] Avg episode reward: [(0, '4.667')] | |
| [2025-04-06 21:24:16,335][29816] Updated weights for policy 0, policy_version 210 (0.0039) | |
| [2025-04-06 21:24:20,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8260.3, 300 sec: 7895.2). Total num frames: 897024. Throughput: 0: 2038.9. Samples: 225400. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2025-04-06 21:24:20,819][29458] Avg episode reward: [(0, '4.723')] | |
| [2025-04-06 21:24:21,104][29816] Updated weights for policy 0, policy_version 220 (0.0041) | |
| [2025-04-06 21:24:25,818][29458] Fps is (10 sec: 8191.8, 60 sec: 8260.3, 300 sec: 7907.7). Total num frames: 937984. Throughput: 0: 2025.0. Samples: 231204. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2025-04-06 21:24:25,819][29458] Avg episode reward: [(0, '4.701')] | |
| [2025-04-06 21:24:26,084][29816] Updated weights for policy 0, policy_version 230 (0.0039) | |
| [2025-04-06 21:24:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7919.2). Total num frames: 978944. Throughput: 0: 2021.2. Samples: 243982. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2025-04-06 21:24:30,819][29458] Avg episode reward: [(0, '5.029')] | |
| [2025-04-06 21:24:30,877][29697] Saving new best policy, reward=5.029! | |
| [2025-04-06 21:24:30,885][29816] Updated weights for policy 0, policy_version 240 (0.0038) | |
| [2025-04-06 21:24:35,684][29816] Updated weights for policy 0, policy_version 250 (0.0036) | |
| [2025-04-06 21:24:35,819][29458] Fps is (10 sec: 8601.5, 60 sec: 8260.3, 300 sec: 7961.6). Total num frames: 1024000. Throughput: 0: 1876.3. Samples: 250366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:24:35,819][29458] Avg episode reward: [(0, '5.175')] | |
| [2025-04-06 21:24:35,821][29697] Saving new best policy, reward=5.175! | |
| [2025-04-06 21:24:40,475][29816] Updated weights for policy 0, policy_version 260 (0.0035) | |
| [2025-04-06 21:24:40,818][29458] Fps is (10 sec: 8601.7, 60 sec: 8192.1, 300 sec: 7970.3). Total num frames: 1064960. Throughput: 0: 2019.1. Samples: 263240. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:24:40,819][29458] Avg episode reward: [(0, '5.215')] | |
| [2025-04-06 21:24:40,831][29697] Saving new best policy, reward=5.215! | |
| [2025-04-06 21:24:45,776][29816] Updated weights for policy 0, policy_version 270 (0.0034) | |
| [2025-04-06 21:24:45,819][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7978.3). Total num frames: 1105920. Throughput: 0: 2003.9. Samples: 275254. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:24:45,819][29458] Avg episode reward: [(0, '5.764')] | |
| [2025-04-06 21:24:45,821][29697] Saving new best policy, reward=5.764! | |
| [2025-04-06 21:24:50,818][29458] Fps is (10 sec: 7782.3, 60 sec: 8055.5, 300 sec: 7957.2). Total num frames: 1142784. Throughput: 0: 2011.9. Samples: 287306. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:24:50,819][29458] Avg episode reward: [(0, '5.685')] | |
| [2025-04-06 21:24:50,827][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth... | |
| [2025-04-06 21:24:51,023][29816] Updated weights for policy 0, policy_version 280 (0.0037) | |
| [2025-04-06 21:24:55,726][29816] Updated weights for policy 0, policy_version 290 (0.0038) | |
| [2025-04-06 21:24:55,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8123.7, 300 sec: 7992.6). Total num frames: 1187840. Throughput: 0: 2025.6. Samples: 293296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:24:55,819][29458] Avg episode reward: [(0, '5.926')] | |
| [2025-04-06 21:24:55,820][29697] Saving new best policy, reward=5.926! | |
| [2025-04-06 21:25:00,818][29458] Fps is (10 sec: 8601.6, 60 sec: 8055.5, 300 sec: 7999.1). Total num frames: 1228800. Throughput: 0: 2065.0. Samples: 305554. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:25:00,818][29816] Updated weights for policy 0, policy_version 300 (0.0041) | |
| [2025-04-06 21:25:00,819][29458] Avg episode reward: [(0, '6.334')] | |
| [2025-04-06 21:25:00,829][29697] Saving new best policy, reward=6.334! | |
| [2025-04-06 21:25:05,818][29816] Updated weights for policy 0, policy_version 310 (0.0033) | |
| [2025-04-06 21:25:05,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8123.8, 300 sec: 8005.2). Total num frames: 1269760. Throughput: 0: 2058.7. Samples: 318040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:05,819][29458] Avg episode reward: [(0, '7.399')] | |
| [2025-04-06 21:25:05,820][29697] Saving new best policy, reward=7.399! | |
| [2025-04-06 21:25:10,767][29816] Updated weights for policy 0, policy_version 320 (0.0035) | |
| [2025-04-06 21:25:10,818][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 8010.9). Total num frames: 1310720. Throughput: 0: 2065.3. Samples: 324144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:10,819][29458] Avg episode reward: [(0, '7.160')] | |
| [2025-04-06 21:25:15,614][29816] Updated weights for policy 0, policy_version 330 (0.0037) | |
| [2025-04-06 21:25:15,819][29458] Fps is (10 sec: 8191.6, 60 sec: 8260.2, 300 sec: 8016.3). Total num frames: 1351680. Throughput: 0: 2061.4. Samples: 336744. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:25:15,820][29458] Avg episode reward: [(0, '7.225')] | |
| [2025-04-06 21:25:20,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8192.0, 300 sec: 7997.8). Total num frames: 1388544. Throughput: 0: 2182.8. Samples: 348592. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:20,819][29458] Avg episode reward: [(0, '7.720')] | |
| [2025-04-06 21:25:20,833][29697] Saving new best policy, reward=7.720! | |
| [2025-04-06 21:25:21,072][29816] Updated weights for policy 0, policy_version 340 (0.0041) | |
| [2025-04-06 21:25:25,818][29458] Fps is (10 sec: 7782.6, 60 sec: 8192.0, 300 sec: 8003.2). Total num frames: 1429504. Throughput: 0: 2022.7. Samples: 354260. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:25:25,819][29458] Avg episode reward: [(0, '8.824')] | |
| [2025-04-06 21:25:25,821][29697] Saving new best policy, reward=8.824! | |
| [2025-04-06 21:25:26,135][29816] Updated weights for policy 0, policy_version 350 (0.0035) | |
| [2025-04-06 21:25:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 8008.3). Total num frames: 1470464. Throughput: 0: 2030.5. Samples: 366626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:30,819][29458] Avg episode reward: [(0, '9.869')] | |
| [2025-04-06 21:25:30,833][29697] Saving new best policy, reward=9.869! | |
| [2025-04-06 21:25:31,162][29816] Updated weights for policy 0, policy_version 360 (0.0034) | |
| [2025-04-06 21:25:35,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8123.8, 300 sec: 8013.2). Total num frames: 1511424. Throughput: 0: 2039.2. Samples: 379070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:35,819][29458] Avg episode reward: [(0, '9.580')] | |
| [2025-04-06 21:25:36,085][29816] Updated weights for policy 0, policy_version 370 (0.0032) | |
| [2025-04-06 21:25:40,820][29458] Fps is (10 sec: 8191.0, 60 sec: 8123.6, 300 sec: 8017.8). Total num frames: 1552384. Throughput: 0: 2039.1. Samples: 385060. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:25:40,821][29458] Avg episode reward: [(0, '9.251')] | |
| [2025-04-06 21:25:41,112][29816] Updated weights for policy 0, policy_version 380 (0.0035) | |
| [2025-04-06 21:25:45,819][29458] Fps is (10 sec: 7782.3, 60 sec: 8055.5, 300 sec: 8001.6). Total num frames: 1589248. Throughput: 0: 2029.6. Samples: 396886. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:25:45,819][29458] Avg episode reward: [(0, '10.675')] | |
| [2025-04-06 21:25:45,822][29697] Saving new best policy, reward=10.675! | |
| [2025-04-06 21:25:46,567][29816] Updated weights for policy 0, policy_version 390 (0.0037) | |
| [2025-04-06 21:25:50,819][29458] Fps is (10 sec: 7373.1, 60 sec: 8055.4, 300 sec: 7986.1). Total num frames: 1626112. Throughput: 0: 1989.9. Samples: 407588. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2025-04-06 21:25:50,825][29458] Avg episode reward: [(0, '12.079')] | |
| [2025-04-06 21:25:50,840][29697] Saving new best policy, reward=12.079! | |
| [2025-04-06 21:25:52,204][29816] Updated weights for policy 0, policy_version 400 (0.0036) | |
| [2025-04-06 21:25:55,819][29458] Fps is (10 sec: 7372.8, 60 sec: 7918.9, 300 sec: 7971.4). Total num frames: 1662976. Throughput: 0: 1977.1. Samples: 413114. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:25:55,819][29458] Avg episode reward: [(0, '11.502')] | |
| [2025-04-06 21:25:57,689][29816] Updated weights for policy 0, policy_version 410 (0.0042) | |
| [2025-04-06 21:26:00,819][29458] Fps is (10 sec: 7373.3, 60 sec: 7850.6, 300 sec: 7957.4). Total num frames: 1699840. Throughput: 0: 1938.5. Samples: 423974. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:26:00,820][29458] Avg episode reward: [(0, '11.994')] | |
| [2025-04-06 21:26:03,596][29816] Updated weights for policy 0, policy_version 420 (0.0038) | |
| [2025-04-06 21:26:05,818][29458] Fps is (10 sec: 7373.0, 60 sec: 7782.4, 300 sec: 7944.9). Total num frames: 1736704. Throughput: 0: 1792.6. Samples: 429258. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:26:05,819][29458] Avg episode reward: [(0, '13.856')] | |
| [2025-04-06 21:26:05,820][29697] Saving new best policy, reward=13.856! | |
| [2025-04-06 21:26:09,764][29816] Updated weights for policy 0, policy_version 430 (0.0048) | |
| [2025-04-06 21:26:10,818][29458] Fps is (10 sec: 6553.7, 60 sec: 7577.6, 300 sec: 7895.7). Total num frames: 1765376. Throughput: 0: 1894.1. Samples: 439496. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:26:10,819][29458] Avg episode reward: [(0, '14.682')] | |
| [2025-04-06 21:26:10,837][29697] Saving new best policy, reward=14.682! | |
| [2025-04-06 21:26:15,819][29458] Fps is (10 sec: 6143.8, 60 sec: 7441.1, 300 sec: 7866.5). Total num frames: 1798144. Throughput: 0: 1833.9. Samples: 449150. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:26:15,819][29458] Avg episode reward: [(0, '13.383')] | |
| [2025-04-06 21:26:15,829][29816] Updated weights for policy 0, policy_version 440 (0.0045) | |
| [2025-04-06 21:26:20,818][29458] Fps is (10 sec: 7372.9, 60 sec: 7509.3, 300 sec: 7873.6). Total num frames: 1839104. Throughput: 0: 1821.6. Samples: 461042. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:26:20,819][29458] Avg episode reward: [(0, '12.544')] | |
| [2025-04-06 21:26:21,138][29816] Updated weights for policy 0, policy_version 450 (0.0035) | |
| [2025-04-06 21:26:25,818][29458] Fps is (10 sec: 8192.4, 60 sec: 7509.4, 300 sec: 7880.6). Total num frames: 1880064. Throughput: 0: 1814.0. Samples: 466688. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:26:25,820][29458] Avg episode reward: [(0, '14.011')] | |
| [2025-04-06 21:26:26,363][29816] Updated weights for policy 0, policy_version 460 (0.0037) | |
| [2025-04-06 21:26:30,820][29458] Fps is (10 sec: 7781.1, 60 sec: 7440.9, 300 sec: 7870.2). Total num frames: 1916928. Throughput: 0: 1812.5. Samples: 478450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:26:30,821][29458] Avg episode reward: [(0, '14.741')] | |
| [2025-04-06 21:26:30,836][29697] Saving new best policy, reward=14.741! | |
| [2025-04-06 21:26:31,636][29816] Updated weights for policy 0, policy_version 470 (0.0037) | |
| [2025-04-06 21:26:35,818][29458] Fps is (10 sec: 7782.4, 60 sec: 7441.1, 300 sec: 7876.8). Total num frames: 1957888. Throughput: 0: 1701.0. Samples: 484130. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2025-04-06 21:26:35,819][29458] Avg episode reward: [(0, '14.143')] | |
| [2025-04-06 21:26:36,932][29816] Updated weights for policy 0, policy_version 480 (0.0038) | |
| [2025-04-06 21:26:40,819][29458] Fps is (10 sec: 7783.3, 60 sec: 7372.9, 300 sec: 7866.9). Total num frames: 1994752. Throughput: 0: 1839.4. Samples: 495888. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:26:40,819][29458] Avg episode reward: [(0, '16.219')] | |
| [2025-04-06 21:26:40,831][29697] Saving new best policy, reward=16.219! | |
| [2025-04-06 21:26:42,188][29816] Updated weights for policy 0, policy_version 490 (0.0044) | |
| [2025-04-06 21:26:45,819][29458] Fps is (10 sec: 7372.5, 60 sec: 7372.8, 300 sec: 7857.5). Total num frames: 2031616. Throughput: 0: 1852.7. Samples: 507346. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:26:45,820][29458] Avg episode reward: [(0, '16.662')] | |
| [2025-04-06 21:26:45,822][29697] Saving new best policy, reward=16.662! | |
| [2025-04-06 21:26:47,546][29816] Updated weights for policy 0, policy_version 500 (0.0040) | |
| [2025-04-06 21:26:50,818][29458] Fps is (10 sec: 7782.7, 60 sec: 7441.2, 300 sec: 7863.9). Total num frames: 2072576. Throughput: 0: 1861.4. Samples: 513022. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:26:50,819][29458] Avg episode reward: [(0, '16.306')] | |
| [2025-04-06 21:26:50,834][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000506_2072576.pth... | |
| [2025-04-06 21:26:50,935][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000035_143360.pth | |
| [2025-04-06 21:26:52,874][29816] Updated weights for policy 0, policy_version 510 (0.0048) | |
| [2025-04-06 21:26:55,819][29458] Fps is (10 sec: 7782.1, 60 sec: 7441.0, 300 sec: 7854.7). Total num frames: 2109440. Throughput: 0: 1893.9. Samples: 524724. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:26:55,820][29458] Avg episode reward: [(0, '16.752')] | |
| [2025-04-06 21:26:55,822][29697] Saving new best policy, reward=16.752! | |
| [2025-04-06 21:26:58,215][29816] Updated weights for policy 0, policy_version 520 (0.0033) | |
| [2025-04-06 21:27:00,819][29458] Fps is (10 sec: 7372.7, 60 sec: 7441.1, 300 sec: 7846.0). Total num frames: 2146304. Throughput: 0: 1937.2. Samples: 536322. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:27:00,820][29458] Avg episode reward: [(0, '16.356')] | |
| [2025-04-06 21:27:03,417][29816] Updated weights for policy 0, policy_version 530 (0.0040) | |
| [2025-04-06 21:27:05,818][29458] Fps is (10 sec: 7782.9, 60 sec: 7509.3, 300 sec: 7852.2). Total num frames: 2187264. Throughput: 0: 1932.6. Samples: 548010. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:27:05,820][29458] Avg episode reward: [(0, '16.246')] | |
| [2025-04-06 21:27:08,808][29816] Updated weights for policy 0, policy_version 540 (0.0049) | |
| [2025-04-06 21:27:10,819][29458] Fps is (10 sec: 8191.7, 60 sec: 7714.1, 300 sec: 7858.3). Total num frames: 2228224. Throughput: 0: 1934.2. Samples: 553728. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:27:10,819][29458] Avg episode reward: [(0, '16.217')] | |
| [2025-04-06 21:27:14,157][29816] Updated weights for policy 0, policy_version 550 (0.0039) | |
| [2025-04-06 21:27:15,818][29458] Fps is (10 sec: 7782.4, 60 sec: 7782.4, 300 sec: 7849.9). Total num frames: 2265088. Throughput: 0: 1924.3. Samples: 565040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:27:15,819][29458] Avg episode reward: [(0, '15.185')] | |
| [2025-04-06 21:27:19,607][29816] Updated weights for policy 0, policy_version 560 (0.0038) | |
| [2025-04-06 21:27:20,819][29458] Fps is (10 sec: 7373.0, 60 sec: 7714.1, 300 sec: 7841.8). Total num frames: 2301952. Throughput: 0: 2051.4. Samples: 576444. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-04-06 21:27:20,820][29458] Avg episode reward: [(0, '18.130')] | |
| [2025-04-06 21:27:20,837][29697] Saving new best policy, reward=18.130! | |
| [2025-04-06 21:27:25,069][29816] Updated weights for policy 0, policy_version 570 (0.0041) | |
| [2025-04-06 21:27:25,819][29458] Fps is (10 sec: 7372.7, 60 sec: 7645.8, 300 sec: 7928.2). Total num frames: 2338816. Throughput: 0: 1910.8. Samples: 581872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:27:25,819][29458] Avg episode reward: [(0, '17.818')] | |
| [2025-04-06 21:27:30,819][29458] Fps is (10 sec: 6963.2, 60 sec: 7577.8, 300 sec: 7983.7). Total num frames: 2371584. Throughput: 0: 1894.0. Samples: 592576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:27:30,820][29458] Avg episode reward: [(0, '19.286')] | |
| [2025-04-06 21:27:30,854][29697] Saving new best policy, reward=19.286! | |
| [2025-04-06 21:27:30,862][29816] Updated weights for policy 0, policy_version 580 (0.0041) | |
| [2025-04-06 21:27:35,818][29458] Fps is (10 sec: 7372.8, 60 sec: 7577.6, 300 sec: 7983.7). Total num frames: 2412544. Throughput: 0: 2027.3. Samples: 604252. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2025-04-06 21:27:35,819][29458] Avg episode reward: [(0, '18.906')] | |
| [2025-04-06 21:27:36,156][29816] Updated weights for policy 0, policy_version 590 (0.0034) | |
| [2025-04-06 21:27:40,818][29458] Fps is (10 sec: 8192.3, 60 sec: 7645.9, 300 sec: 7983.7). Total num frames: 2453504. Throughput: 0: 1903.0. Samples: 610358. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:27:40,819][29458] Avg episode reward: [(0, '19.766')] | |
| [2025-04-06 21:27:40,831][29697] Saving new best policy, reward=19.766! | |
| [2025-04-06 21:27:41,189][29816] Updated weights for policy 0, policy_version 600 (0.0036) | |
| [2025-04-06 21:27:45,818][29458] Fps is (10 sec: 8192.1, 60 sec: 7714.2, 300 sec: 7969.8). Total num frames: 2494464. Throughput: 0: 1922.3. Samples: 622826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2025-04-06 21:27:45,819][29458] Avg episode reward: [(0, '19.846')] | |
| [2025-04-06 21:27:45,821][29697] Saving new best policy, reward=19.846! | |
| [2025-04-06 21:27:46,042][29816] Updated weights for policy 0, policy_version 610 (0.0037) | |
| [2025-04-06 21:27:50,818][29458] Fps is (10 sec: 8192.0, 60 sec: 7714.1, 300 sec: 7969.8). Total num frames: 2535424. Throughput: 0: 1945.3. Samples: 635548. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:27:50,819][29458] Avg episode reward: [(0, '20.144')] | |
| [2025-04-06 21:27:50,847][29697] Saving new best policy, reward=20.144! | |
| [2025-04-06 21:27:50,856][29816] Updated weights for policy 0, policy_version 620 (0.0037) | |
| [2025-04-06 21:27:55,689][29816] Updated weights for policy 0, policy_version 630 (0.0032) | |
| [2025-04-06 21:27:55,819][29458] Fps is (10 sec: 8601.3, 60 sec: 7850.7, 300 sec: 7983.7). Total num frames: 2580480. Throughput: 0: 1955.3. Samples: 641716. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:27:55,820][29458] Avg episode reward: [(0, '21.052')] | |
| [2025-04-06 21:27:55,822][29697] Saving new best policy, reward=21.052! | |
| [2025-04-06 21:28:00,625][29816] Updated weights for policy 0, policy_version 640 (0.0037) | |
| [2025-04-06 21:28:00,819][29458] Fps is (10 sec: 8601.4, 60 sec: 7918.9, 300 sec: 7969.8). Total num frames: 2621440. Throughput: 0: 1981.0. Samples: 654186. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:28:00,819][29458] Avg episode reward: [(0, '19.409')] | |
| [2025-04-06 21:28:05,536][29816] Updated weights for policy 0, policy_version 650 (0.0039) | |
| [2025-04-06 21:28:05,818][29458] Fps is (10 sec: 8192.3, 60 sec: 7918.9, 300 sec: 7969.9). Total num frames: 2662400. Throughput: 0: 2005.8. Samples: 666706. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:28:05,819][29458] Avg episode reward: [(0, '19.812')] | |
| [2025-04-06 21:28:10,511][29816] Updated weights for policy 0, policy_version 660 (0.0040) | |
| [2025-04-06 21:28:10,819][29458] Fps is (10 sec: 8192.0, 60 sec: 7919.0, 300 sec: 7956.0). Total num frames: 2703360. Throughput: 0: 2023.9. Samples: 672948. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:28:10,820][29458] Avg episode reward: [(0, '21.072')] | |
| [2025-04-06 21:28:10,835][29697] Saving new best policy, reward=21.072! | |
| [2025-04-06 21:28:15,671][29816] Updated weights for policy 0, policy_version 670 (0.0036) | |
| [2025-04-06 21:28:15,819][29458] Fps is (10 sec: 8191.9, 60 sec: 7987.2, 300 sec: 7942.1). Total num frames: 2744320. Throughput: 0: 2050.1. Samples: 684832. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:28:15,819][29458] Avg episode reward: [(0, '21.161')] | |
| [2025-04-06 21:28:15,821][29697] Saving new best policy, reward=21.161! | |
| [2025-04-06 21:28:20,590][29816] Updated weights for policy 0, policy_version 680 (0.0038) | |
| [2025-04-06 21:28:20,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8055.5, 300 sec: 7942.1). Total num frames: 2785280. Throughput: 0: 1928.6. Samples: 691040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:28:20,819][29458] Avg episode reward: [(0, '20.833')] | |
| [2025-04-06 21:28:25,516][29816] Updated weights for policy 0, policy_version 690 (0.0037) | |
| [2025-04-06 21:28:25,819][29458] Fps is (10 sec: 8192.1, 60 sec: 8123.7, 300 sec: 7928.2). Total num frames: 2826240. Throughput: 0: 2073.1. Samples: 703646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:28:25,820][29458] Avg episode reward: [(0, '21.685')] | |
| [2025-04-06 21:28:25,821][29697] Saving new best policy, reward=21.685! | |
| [2025-04-06 21:28:30,510][29816] Updated weights for policy 0, policy_version 700 (0.0037) | |
| [2025-04-06 21:28:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 2867200. Throughput: 0: 2069.4. Samples: 715948. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:28:30,819][29458] Avg episode reward: [(0, '21.848')] | |
| [2025-04-06 21:28:30,831][29697] Saving new best policy, reward=21.848! | |
| [2025-04-06 21:28:35,417][29816] Updated weights for policy 0, policy_version 710 (0.0031) | |
| [2025-04-06 21:28:35,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7914.3). Total num frames: 2908160. Throughput: 0: 2064.6. Samples: 728454. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:28:35,819][29458] Avg episode reward: [(0, '21.461')] | |
| [2025-04-06 21:28:40,339][29816] Updated weights for policy 0, policy_version 720 (0.0037) | |
| [2025-04-06 21:28:40,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8328.5, 300 sec: 7928.2). Total num frames: 2953216. Throughput: 0: 2066.6. Samples: 734712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:28:40,819][29458] Avg episode reward: [(0, '21.480')] | |
| [2025-04-06 21:28:45,138][29816] Updated weights for policy 0, policy_version 730 (0.0035) | |
| [2025-04-06 21:28:45,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8328.5, 300 sec: 7914.3). Total num frames: 2994176. Throughput: 0: 2069.0. Samples: 747290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:28:45,819][29458] Avg episode reward: [(0, '18.329')] | |
| [2025-04-06 21:28:50,138][29816] Updated weights for policy 0, policy_version 740 (0.0042) | |
| [2025-04-06 21:28:50,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8328.5, 300 sec: 7914.3). Total num frames: 3035136. Throughput: 0: 2067.3. Samples: 759734. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:28:50,819][29458] Avg episode reward: [(0, '18.835')] | |
| [2025-04-06 21:28:50,834][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000741_3035136.pth... | |
| [2025-04-06 21:28:50,964][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth | |
| [2025-04-06 21:28:55,225][29816] Updated weights for policy 0, policy_version 750 (0.0041) | |
| [2025-04-06 21:28:55,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7900.4). Total num frames: 3076096. Throughput: 0: 2063.3. Samples: 765794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:28:55,820][29458] Avg episode reward: [(0, '19.178')] | |
| [2025-04-06 21:29:00,235][29816] Updated weights for policy 0, policy_version 760 (0.0037) | |
| [2025-04-06 21:29:00,820][29458] Fps is (10 sec: 8191.0, 60 sec: 8260.1, 300 sec: 7914.3). Total num frames: 3117056. Throughput: 0: 2072.2. Samples: 778084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:29:00,820][29458] Avg episode reward: [(0, '19.598')] | |
| [2025-04-06 21:29:05,337][29816] Updated weights for policy 0, policy_version 770 (0.0039) | |
| [2025-04-06 21:29:05,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8192.0, 300 sec: 7914.3). Total num frames: 3153920. Throughput: 0: 2203.1. Samples: 790178. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:29:05,819][29458] Avg episode reward: [(0, '21.833')] | |
| [2025-04-06 21:29:10,278][29816] Updated weights for policy 0, policy_version 780 (0.0039) | |
| [2025-04-06 21:29:10,819][29458] Fps is (10 sec: 8192.6, 60 sec: 8260.2, 300 sec: 7942.1). Total num frames: 3198976. Throughput: 0: 2061.4. Samples: 796412. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:29:10,820][29458] Avg episode reward: [(0, '22.750')] | |
| [2025-04-06 21:29:10,838][29697] Saving new best policy, reward=22.750! | |
| [2025-04-06 21:29:15,200][29816] Updated weights for policy 0, policy_version 790 (0.0034) | |
| [2025-04-06 21:29:15,818][29458] Fps is (10 sec: 8601.4, 60 sec: 8260.3, 300 sec: 7942.1). Total num frames: 3239936. Throughput: 0: 2060.7. Samples: 808680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:29:15,819][29458] Avg episode reward: [(0, '24.524')] | |
| [2025-04-06 21:29:15,821][29697] Saving new best policy, reward=24.524! | |
| [2025-04-06 21:29:20,195][29816] Updated weights for policy 0, policy_version 800 (0.0039) | |
| [2025-04-06 21:29:20,819][29458] Fps is (10 sec: 8192.2, 60 sec: 8260.2, 300 sec: 7942.1). Total num frames: 3280896. Throughput: 0: 2057.5. Samples: 821042. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:29:20,820][29458] Avg episode reward: [(0, '25.083')] | |
| [2025-04-06 21:29:20,830][29697] Saving new best policy, reward=25.083! | |
| [2025-04-06 21:29:25,181][29816] Updated weights for policy 0, policy_version 810 (0.0037) | |
| [2025-04-06 21:29:25,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7942.1). Total num frames: 3321856. Throughput: 0: 2053.8. Samples: 827132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:29:25,819][29458] Avg episode reward: [(0, '23.715')] | |
| [2025-04-06 21:29:30,162][29816] Updated weights for policy 0, policy_version 820 (0.0041) | |
| [2025-04-06 21:29:30,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3362816. Throughput: 0: 2050.0. Samples: 839542. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:29:30,820][29458] Avg episode reward: [(0, '22.336')] | |
| [2025-04-06 21:29:35,117][29816] Updated weights for policy 0, policy_version 830 (0.0038) | |
| [2025-04-06 21:29:35,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3403776. Throughput: 0: 2050.0. Samples: 851982. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:29:35,819][29458] Avg episode reward: [(0, '23.183')] | |
| [2025-04-06 21:29:40,031][29816] Updated weights for policy 0, policy_version 840 (0.0037) | |
| [2025-04-06 21:29:40,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3444736. Throughput: 0: 2052.9. Samples: 858176. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:29:40,820][29458] Avg episode reward: [(0, '21.936')] | |
| [2025-04-06 21:29:44,985][29816] Updated weights for policy 0, policy_version 850 (0.0035) | |
| [2025-04-06 21:29:45,819][29458] Fps is (10 sec: 8191.7, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3485696. Throughput: 0: 2055.8. Samples: 870594. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2025-04-06 21:29:45,821][29458] Avg episode reward: [(0, '22.511')] | |
| [2025-04-06 21:29:50,137][29816] Updated weights for policy 0, policy_version 860 (0.0032) | |
| [2025-04-06 21:29:50,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3526656. Throughput: 0: 2053.7. Samples: 882596. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:29:50,819][29458] Avg episode reward: [(0, '22.732')] | |
| [2025-04-06 21:29:55,161][29816] Updated weights for policy 0, policy_version 870 (0.0035) | |
| [2025-04-06 21:29:55,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3567616. Throughput: 0: 2051.9. Samples: 888746. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:29:55,819][29458] Avg episode reward: [(0, '23.068')] | |
| [2025-04-06 21:30:00,144][29816] Updated weights for policy 0, policy_version 880 (0.0037) | |
| [2025-04-06 21:30:00,819][29458] Fps is (10 sec: 8191.6, 60 sec: 8192.1, 300 sec: 7928.2). Total num frames: 3608576. Throughput: 0: 2055.1. Samples: 901160. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:30:00,820][29458] Avg episode reward: [(0, '23.628')] | |
| [2025-04-06 21:30:05,140][29816] Updated weights for policy 0, policy_version 890 (0.0037) | |
| [2025-04-06 21:30:05,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3649536. Throughput: 0: 2053.1. Samples: 913430. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:30:05,819][29458] Avg episode reward: [(0, '24.405')] | |
| [2025-04-06 21:30:09,984][29816] Updated weights for policy 0, policy_version 900 (0.0036) | |
| [2025-04-06 21:30:10,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8192.1, 300 sec: 7928.2). Total num frames: 3690496. Throughput: 0: 2055.4. Samples: 919626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2025-04-06 21:30:10,820][29458] Avg episode reward: [(0, '22.835')] | |
| [2025-04-06 21:30:15,093][29816] Updated weights for policy 0, policy_version 910 (0.0035) | |
| [2025-04-06 21:30:15,818][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3731456. Throughput: 0: 2050.0. Samples: 931792. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2025-04-06 21:30:15,819][29458] Avg episode reward: [(0, '21.146')] | |
| [2025-04-06 21:30:20,125][29816] Updated weights for policy 0, policy_version 920 (0.0033) | |
| [2025-04-06 21:30:20,819][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3772416. Throughput: 0: 2047.9. Samples: 944140. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:30:20,819][29458] Avg episode reward: [(0, '22.432')] | |
| [2025-04-06 21:30:25,108][29816] Updated weights for policy 0, policy_version 930 (0.0038) | |
| [2025-04-06 21:30:25,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3813376. Throughput: 0: 2047.9. Samples: 950334. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:30:25,819][29458] Avg episode reward: [(0, '25.029')] | |
| [2025-04-06 21:30:30,143][29816] Updated weights for policy 0, policy_version 940 (0.0034) | |
| [2025-04-06 21:30:30,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3854336. Throughput: 0: 2041.6. Samples: 962464. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2025-04-06 21:30:30,819][29458] Avg episode reward: [(0, '24.218')] | |
| [2025-04-06 21:30:35,169][29816] Updated weights for policy 0, policy_version 950 (0.0038) | |
| [2025-04-06 21:30:35,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8191.9, 300 sec: 7942.1). Total num frames: 3895296. Throughput: 0: 2046.6. Samples: 974694. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-04-06 21:30:35,820][29458] Avg episode reward: [(0, '26.537')] | |
| [2025-04-06 21:30:35,821][29697] Saving new best policy, reward=26.537! | |
| [2025-04-06 21:30:40,154][29816] Updated weights for policy 0, policy_version 960 (0.0037) | |
| [2025-04-06 21:30:40,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8192.0, 300 sec: 7956.0). Total num frames: 3936256. Throughput: 0: 2046.8. Samples: 980854. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:30:40,819][29458] Avg episode reward: [(0, '24.753')] | |
| [2025-04-06 21:30:45,077][29816] Updated weights for policy 0, policy_version 970 (0.0034) | |
| [2025-04-06 21:30:45,818][29458] Fps is (10 sec: 8192.4, 60 sec: 8192.0, 300 sec: 7969.9). Total num frames: 3977216. Throughput: 0: 2048.8. Samples: 993354. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-04-06 21:30:45,819][29458] Avg episode reward: [(0, '23.041')] | |
| [2025-04-06 21:30:49,011][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-04-06 21:30:49,019][29458] Component Batcher_0 stopped! | |
| [2025-04-06 21:30:49,025][29697] Stopping Batcher_0... | |
| [2025-04-06 21:30:49,035][29697] Loop batcher_evt_loop terminating... | |
| [2025-04-06 21:30:49,050][29816] Weights refcount: 2 0 | |
| [2025-04-06 21:30:49,055][29458] Component InferenceWorker_p0-w0 stopped! | |
| [2025-04-06 21:30:49,054][29816] Stopping InferenceWorker_p0-w0... | |
| [2025-04-06 21:30:49,058][29816] Loop inference_proc0-0_evt_loop terminating... | |
| [2025-04-06 21:30:49,088][29458] Component RolloutWorker_w6 stopped! | |
| [2025-04-06 21:30:49,088][29817] Stopping RolloutWorker_w6... | |
| [2025-04-06 21:30:49,093][29817] Loop rollout_proc6_evt_loop terminating... | |
| [2025-04-06 21:30:49,128][29458] Component RolloutWorker_w3 stopped! | |
| [2025-04-06 21:30:49,129][29458] Component RolloutWorker_w7 stopped! | |
| [2025-04-06 21:30:49,128][29823] Stopping RolloutWorker_w7... | |
| [2025-04-06 21:30:49,131][29458] Component RolloutWorker_w1 stopped! | |
| [2025-04-06 21:30:49,128][29819] Stopping RolloutWorker_w3... | |
| [2025-04-06 21:30:49,131][29823] Loop rollout_proc7_evt_loop terminating... | |
| [2025-04-06 21:30:49,133][29815] Stopping RolloutWorker_w1... | |
| [2025-04-06 21:30:49,138][29815] Loop rollout_proc1_evt_loop terminating... | |
| [2025-04-06 21:30:49,134][29819] Loop rollout_proc3_evt_loop terminating... | |
| [2025-04-06 21:30:49,140][29458] Component RolloutWorker_w2 stopped! | |
| [2025-04-06 21:30:49,142][29821] Stopping RolloutWorker_w2... | |
| [2025-04-06 21:30:49,145][29458] Component RolloutWorker_w4 stopped! | |
| [2025-04-06 21:30:49,147][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000506_2072576.pth | |
| [2025-04-06 21:30:49,148][29458] Component RolloutWorker_w0 stopped! | |
| [2025-04-06 21:30:49,144][29821] Loop rollout_proc2_evt_loop terminating... | |
| [2025-04-06 21:30:49,147][29818] Stopping RolloutWorker_w0... | |
| [2025-04-06 21:30:49,150][29818] Loop rollout_proc0_evt_loop terminating... | |
| [2025-04-06 21:30:49,145][29820] Stopping RolloutWorker_w4... | |
| [2025-04-06 21:30:49,154][29820] Loop rollout_proc4_evt_loop terminating... | |
| [2025-04-06 21:30:49,164][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-04-06 21:30:49,175][29458] Component RolloutWorker_w5 stopped! | |
| [2025-04-06 21:30:49,175][29822] Stopping RolloutWorker_w5... | |
| [2025-04-06 21:30:49,177][29822] Loop rollout_proc5_evt_loop terminating... | |
| [2025-04-06 21:30:49,340][29697] Stopping LearnerWorker_p0... | |
| [2025-04-06 21:30:49,341][29458] Component LearnerWorker_p0 stopped! | |
| [2025-04-06 21:30:49,342][29458] Waiting for process learner_proc0 to stop... | |
| [2025-04-06 21:30:49,343][29697] Loop learner_proc0_evt_loop terminating... | |
| [2025-04-06 21:30:51,627][29458] Waiting for process inference_proc0-0 to join... | |
| [2025-04-06 21:30:51,628][29458] Waiting for process rollout_proc0 to join... | |
| [2025-04-06 21:30:51,629][29458] Waiting for process rollout_proc1 to join... | |
| [2025-04-06 21:30:51,630][29458] Waiting for process rollout_proc2 to join... | |
| [2025-04-06 21:30:51,631][29458] Waiting for process rollout_proc3 to join... | |
| [2025-04-06 21:30:51,632][29458] Waiting for process rollout_proc4 to join... | |
| [2025-04-06 21:30:51,633][29458] Waiting for process rollout_proc5 to join... | |
| [2025-04-06 21:30:51,634][29458] Waiting for process rollout_proc6 to join... | |
| [2025-04-06 21:30:51,634][29458] Waiting for process rollout_proc7 to join... | |
| [2025-04-06 21:30:51,635][29458] Batcher 0 profile tree view: | |
| batching: 17.8770, releasing_batches: 0.0695 | |
| [2025-04-06 21:30:51,636][29458] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0001 | |
| wait_policy_total: 42.1090 | |
| update_model: 10.7099 | |
| weight_update: 0.0035 | |
| one_step: 0.0087 | |
| handle_policy_step: 451.9330 | |
| deserialize: 16.9719, stack: 3.9921, obs_to_device_normalize: 136.9680, forward: 196.5648, send_messages: 25.9704 | |
| prepare_outputs: 43.6267 | |
| to_cpu: 23.3038 | |
| [2025-04-06 21:30:51,637][29458] Learner 0 profile tree view: | |
| misc: 0.0148, prepare_batch: 11.7053 | |
| train: 50.0566 | |
| epoch_init: 0.0177, minibatch_init: 0.0135, losses_postprocess: 0.3649, kl_divergence: 0.5450, after_optimizer: 16.5465 | |
| calculate_losses: 17.8696 | |
| losses_init: 0.0104, forward_head: 1.2988, bptt_initial: 11.1267, tail: 1.2122, advantages_returns: 0.2812, losses: 1.7146 | |
| bptt: 1.7721 | |
| bptt_forward_core: 1.6535 | |
| update: 13.9069 | |
| clip: 1.7712 | |
| [2025-04-06 21:30:51,637][29458] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.3932, enqueue_policy_requests: 14.9665, env_step: 208.0064, overhead: 24.6133, complete_rollouts: 0.6180 | |
| save_policy_outputs: 26.4317 | |
| split_output_tensors: 13.2113 | |
| [2025-04-06 21:30:51,638][29458] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.4155, enqueue_policy_requests: 16.2129, env_step: 211.3192, overhead: 26.4352, complete_rollouts: 0.6806 | |
| save_policy_outputs: 28.5057 | |
| split_output_tensors: 14.0651 | |
| [2025-04-06 21:30:51,639][29458] Loop Runner_EvtLoop terminating... | |
| [2025-04-06 21:30:51,640][29458] Runner profile tree view: | |
| main_loop: 588.0457 | |
| [2025-04-06 21:30:51,641][29458] Collected {0: 4005888}, FPS: 6812.2 | |
| [2025-04-06 21:45:24,892][29458] Loading existing experiment configuration from /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/config.json | |
| [2025-04-06 21:45:24,892][29458] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2025-04-06 21:45:24,892][29458] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2025-04-06 21:45:24,893][29458] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2025-04-06 21:45:24,894][29458] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2025-04-06 21:45:24,894][29458] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2025-04-06 21:45:24,895][29458] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2025-04-06 21:45:24,895][29458] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2025-04-06 21:45:24,896][29458] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2025-04-06 21:45:24,896][29458] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2025-04-06 21:45:24,897][29458] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2025-04-06 21:45:24,898][29458] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2025-04-06 21:45:24,899][29458] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2025-04-06 21:45:24,899][29458] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2025-04-06 21:45:24,900][29458] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2025-04-06 21:45:24,939][29458] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-04-06 21:45:24,942][29458] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-04-06 21:45:24,944][29458] RunningMeanStd input shape: (1,) | |
| [2025-04-06 21:45:24,978][29458] ConvEncoder: input_channels=3 | |
| [2025-04-06 21:45:25,206][29458] Conv encoder output size: 512 | |
| [2025-04-06 21:45:25,206][29458] Policy head output size: 512 | |
| [2025-04-06 21:45:25,370][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-04-06 21:45:25,373][29458] Could not load from checkpoint, attempt 0 | |
| Traceback (most recent call last): | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint | |
| checkpoint_dict = torch.load(latest_checkpoint, map_location=device) | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load | |
| raise pickle.UnpicklingError(_get_wo_message(str(e))) from None | |
| _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, [1mdo those steps only if you trust the source of the checkpoint[0m. | |
| (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. | |
| (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message. | |
| WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function. | |
| Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html. | |
| [2025-04-06 21:45:25,379][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-04-06 21:45:25,380][29458] Could not load from checkpoint, attempt 1 | |
| Traceback (most recent call last): | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint | |
| checkpoint_dict = torch.load(latest_checkpoint, map_location=device) | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load | |
| raise pickle.UnpicklingError(_get_wo_message(str(e))) from None | |
| _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, [1mdo those steps only if you trust the source of the checkpoint[0m. | |
| (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. | |
| (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message. | |
| WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function. | |
| Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html. | |
| [2025-04-06 21:45:25,380][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-04-06 21:45:25,381][29458] Could not load from checkpoint, attempt 2 | |
| Traceback (most recent call last): | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint | |
| checkpoint_dict = torch.load(latest_checkpoint, map_location=device) | |
| File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load | |
| raise pickle.UnpicklingError(_get_wo_message(str(e))) from None | |
| _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, [1mdo those steps only if you trust the source of the checkpoint[0m. | |
| (1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. | |
| (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message. | |
| WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function. | |
| Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html. | |