Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+replay.mp4 filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1743885927.tguz-ASUS

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65ee1c34cc0c05587f7fcb1f39bf15ee8c9f58d529e17914973f2a9ca5db57fb
+size 40

.summary/0/events.out.tfevents.1743886088.tguz-ASUS

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ee1fda7daa65ca7b0e8273c8150b7dde7d39e8218fd1f1288c4c4f0e40cd0bc
+size 40

.summary/0/events.out.tfevents.1743966725.tguz-ASUS

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2f6b6868c238950ee32dc71f37a9d9006ec49a76f40e08b713c4e161c994fae
+size 40

.summary/0/events.out.tfevents.1743967250.tguz-ASUS

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:649a664593a7681e3007efbed7ba706689d0b9d01473f5645b6a92f9ab1e9083
+size 466054

README.md

@@ -0,0 +1,56 @@
+---
+library_name: sample-factory
+tags:
+- deep-reinforcement-learning
+- reinforcement-learning
+- sample-factory
+model-index:
+- name: APPO
+  results:
+  - task:
+      type: reinforcement-learning
+      name: reinforcement-learning
+    dataset:
+      name: doom_health_gathering_supreme
+      type: doom_health_gathering_supreme
+    metrics:
+    - type: mean_reward
+      value: 12.38 +/- 4.75
+      name: mean_reward
+      verified: false
+---
+
+A(n) **APPO** model trained on the **doom_health_gathering_supreme** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r togu6669/rl_course_vizdoom_health_gathering_supreme
+```
+
+    
+## Using the model
+
+To run the model after download, use the `enjoy` script corresponding to this environment:
+```
+python -m <path.to.enjoy.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme
+```
+
+
+You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
+See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
+    
+## Training with this model
+
+To continue training with this model, use the `train` script corresponding to this environment:
+```
+python -m <path.to.train.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme --restart_behavior=resume --train_for_env_steps=10000000000
+```
+
+Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
+    

checkpoint_p0/best_000000951_3895296_reward_26.537.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaec76dea9c52ebc495d5371c2fa9445ee9acd768eabeed59a6a83fe6ded4a6b
+size 34929051

checkpoint_p0/checkpoint_000000741_3035136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:302c35205f5097fc1a4c70b5dad5335570768a60d626dcc762db3cbf804b3dfc
+size 34929541

checkpoint_p0/checkpoint_000000978_4005888.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e969c26b45be6027caa26141c5d7189cc7ce8ba8a637577e76ff95757b08afd
+size 34929541

config.json

@@ -0,0 +1,142 @@
+{
+  "help": false,
+  "algo": "APPO",
+  "env": "doom_health_gathering_supreme",
+  "experiment": "default_experiment",
+  "train_dir": "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir",
+  "restart_behavior": "resume",
+  "device": "gpu",
+  "seed": null,
+  "num_policies": 1,
+  "async_rl": true,
+  "serial_mode": false,
+  "batched_sampling": false,
+  "num_batches_to_accumulate": 2,
+  "worker_num_splits": 2,
+  "policy_workers_per_policy": 1,
+  "max_policy_lag": 1000,
+  "num_workers": 8,
+  "num_envs_per_worker": 4,
+  "batch_size": 1024,
+  "num_batches_per_epoch": 1,
+  "num_epochs": 1,
+  "rollout": 32,
+  "recurrence": 32,
+  "shuffle_minibatches": false,
+  "gamma": 0.99,
+  "reward_scale": 1.0,
+  "reward_clip": 1000.0,
+  "value_bootstrap": false,
+  "normalize_returns": true,
+  "exploration_loss_coeff": 0.001,
+  "value_loss_coeff": 0.5,
+  "kl_loss_coeff": 0.0,
+  "exploration_loss": "symmetric_kl",
+  "gae_lambda": 0.95,
+  "ppo_clip_ratio": 0.1,
+  "ppo_clip_value": 0.2,
+  "with_vtrace": false,
+  "vtrace_rho": 1.0,
+  "vtrace_c": 1.0,
+  "optimizer": "adam",
+  "adam_eps": 1e-06,
+  "adam_beta1": 0.9,
+  "adam_beta2": 0.999,
+  "max_grad_norm": 4.0,
+  "learning_rate": 0.0001,
+  "lr_schedule": "constant",
+  "lr_schedule_kl_threshold": 0.008,
+  "lr_adaptive_min": 1e-06,
+  "lr_adaptive_max": 0.01,
+  "obs_subtract_mean": 0.0,
+  "obs_scale": 255.0,
+  "normalize_input": true,
+  "normalize_input_keys": null,
+  "decorrelate_experience_max_seconds": 0,
+  "decorrelate_envs_on_one_worker": true,
+  "actor_worker_gpus": [],
+  "set_workers_cpu_affinity": true,
+  "force_envs_single_thread": false,
+  "default_niceness": 0,
+  "log_to_file": true,
+  "experiment_summaries_interval": 10,
+  "flush_summaries_interval": 30,
+  "stats_avg": 100,
+  "summaries_use_frameskip": true,
+  "heartbeat_interval": 20,
+  "heartbeat_reporting_interval": 600,
+  "train_for_env_steps": 4000000,
+  "train_for_seconds": 10000000000,
+  "save_every_sec": 120,
+  "keep_checkpoints": 2,
+  "load_checkpoint_kind": "latest",
+  "save_milestones_sec": -1,
+  "save_best_every_sec": 5,
+  "save_best_metric": "reward",
+  "save_best_after": 100000,
+  "benchmark": false,
+  "encoder_mlp_layers": [
+    512,
+    512
+  ],
+  "encoder_conv_architecture": "convnet_simple",
+  "encoder_conv_mlp_layers": [
+    512
+  ],
+  "use_rnn": true,
+  "rnn_size": 512,
+  "rnn_type": "gru",
+  "rnn_num_layers": 1,
+  "decoder_mlp_layers": [],
+  "nonlinearity": "elu",
+  "policy_initialization": "orthogonal",
+  "policy_init_gain": 1.0,
+  "actor_critic_share_weights": true,
+  "adaptive_stddev": true,
+  "continuous_tanh_scale": 0.0,
+  "initial_stddev": 1.0,
+  "use_env_info_cache": false,
+  "env_gpu_actions": false,
+  "env_gpu_observations": true,
+  "env_frameskip": 4,
+  "env_framestack": 1,
+  "pixel_format": "CHW",
+  "use_record_episode_statistics": false,
+  "with_wandb": false,
+  "wandb_user": null,
+  "wandb_project": "sample_factory",
+  "wandb_group": null,
+  "wandb_job_type": "SF",
+  "wandb_tags": [],
+  "with_pbt": false,
+  "pbt_mix_policies_in_one_env": true,
+  "pbt_period_env_steps": 5000000,
+  "pbt_start_mutation": 20000000,
+  "pbt_replace_fraction": 0.3,
+  "pbt_mutation_rate": 0.15,
+  "pbt_replace_reward_gap": 0.1,
+  "pbt_replace_reward_gap_absolute": 1e-06,
+  "pbt_optimize_gamma": false,
+  "pbt_target_objective": "true_objective",
+  "pbt_perturb_min": 1.1,
+  "pbt_perturb_max": 1.5,
+  "num_agents": -1,
+  "num_humans": 0,
+  "num_bots": -1,
+  "start_bot_difficulty": null,
+  "timelimit": null,
+  "res_w": 128,
+  "res_h": 72,
+  "wide_aspect_ratio": false,
+  "eval_env_frameskip": 1,
+  "fps": 35,
+  "command_line": "--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000",
+  "cli_args": {
+    "env": "doom_health_gathering_supreme",
+    "num_workers": 8,
+    "num_envs_per_worker": 4,
+    "train_for_env_steps": 4000000
+  },
+  "git_hash": "9b997b38e3980d2faacd862b544705c166fa246f",
+  "git_repo_name": "https://github.com/togu6669/Hugging-Face-RL.git"
+}

git.diff

@@ -0,0 +1,125 @@
+diff --git a/Python/PPOVizDoom.py b/Python/PPOVizDoom.py
+index 29087d6..6a61571 100644
+--- a/Python/PPOVizDoom.py
++++ b/Python/PPOVizDoom.py
+@@ -40,62 +40,62 @@ def parse_vizdoom_cfg(argv=None, evaluation=False):
+     return final_cfg
+ 
+ 
+-## Start the training, this should take around 15 minutes
+-register_vizdoom_components()
+-
+-# The scenario we train on today is health gathering
+-# other scenarios include "doom_basic", "doom_two_colors_easy", "doom_dm", "doom_dwango5", "doom_my_way_home", "doom_deadly_corridor", "doom_defend_the_center", "doom_defend_the_line"
+-env = "doom_health_gathering_supreme"
+-cfg = parse_vizdoom_cfg(
+-    argv=[f"--env={env}", "--num_workers=8", "--num_envs_per_worker=4", "--train_for_env_steps=4000000"]
+-)
+-
+-status = run_rl(cfg)
+-
+-
+-from sample_factory.enjoy import enjoy
+-
+-cfg = parse_vizdoom_cfg(
+-    argv=[f"--env={env}", "--num_workers=1", "--save_video", "--no_render", "--max_num_episodes=10"], evaluation=True
+-)
+-status = enjoy(cfg)
+-
+-
+-# from base64 import b64encode
+-# from IPython.display import HTML
+-
+-# mp4 = open("/content/train_dir/default_experiment/replay.mp4", "rb").read()
+-# data_url = "data:video/mp4;base64," + b64encode(mp4).decode()
+-# HTML(
+-#     """
+-# <video width=640 controls>
+-#       <source src="%s" type="video/mp4">
+-# </video>
+-# """
+-#     % data_url
+-# )
+-
+-from huggingface_hub import notebook_login
+-notebook_login()
+-
+-# !git config --global credential.helper store
+-
+-from sample_factory.enjoy import enjoy
+-
+-hf_username = "ThomasSimonini"  # insert your HuggingFace username here
+-
+-cfg = parse_vizdoom_cfg(
+-    argv=[
+-        f"--env={env}",
+-        "--num_workers=1",
+-        "--save_video",
+-        "--no_render",
+-        "--max_num_episodes=10",
+-        "--max_num_frames=100000",
+-        "--push_to_hub",
+-        f"--hf_repository={hf_username}/rl_course_vizdoom_health_gathering_supreme",
+-    ],
+-    evaluation=True,
+-)
+-status = enjoy(cfg)
++if __name__ == '__main__':
++    ## Start the training, this should take around 15 minutes
++    register_vizdoom_components()
++
++    # The scenario we train on today is health gathering
++    # other scenarios include "doom_basic", "doom_two_colors_easy", "doom_dm", "doom_dwango5", "doom_my_way_home", "doom_deadly_corridor", "doom_defend_the_center", "doom_defend_the_line"
++    env = "doom_health_gathering_supreme"
++    cfg = parse_vizdoom_cfg(
++        argv=[f"--env={env}", "--num_workers=8", "--num_envs_per_worker=4", "--train_for_env_steps=4000000"]
++    )
++    status = run_rl(cfg)
++
++
++    from sample_factory.enjoy import enjoy
++
++    cfg = parse_vizdoom_cfg(
++        argv=[f"--env={env}", "--num_workers=1", "--save_video", "--no_render", "--max_num_episodes=10"], evaluation=True
++    )
++    status = enjoy(cfg)
++
++
++    # from base64 import b64encode
++    # from IPython.display import HTML
++
++    # mp4 = open("/content/train_dir/default_experiment/replay.mp4", "rb").read()
++    # data_url = "data:video/mp4;base64," + b64encode(mp4).decode()
++    # HTML(
++    #     """
++    # <video width=640 controls>
++    #       <source src="%s" type="video/mp4">
++    # </video>
++    # """
++    #     % data_url
++    # )
++
++    from huggingface_hub import notebook_login
++    notebook_login()
++
++    # !git config --global credential.helper store
++
++    from sample_factory.enjoy import enjoy
++
++    hf_username = "togu6669"  # insert your HuggingFace username here
++
++    cfg = parse_vizdoom_cfg(
++        argv=[
++            f"--env={env}",
++            "--num_workers=1",
++            "--save_video",
++            "--no_render",
++            "--max_num_episodes=10",
++            "--max_num_frames=100000",
++            "--push_to_hub",
++            f"--hf_repository={hf_username}/rl_course_vizdoom_health_gathering_supreme",
++        ],
++        evaluation=True,
++    )
++    status = enjoy(cfg)
+ 

replay.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6654a5248c11972a00859c9e17904a952899c319b0049cd593909923780071c
+size 23803452

sf_log.txt

@@ -0,0 +1,654 @@
+[2025-04-06 21:21:03,390][29458] Saving configuration to /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/config.json...
+[2025-04-06 21:21:03,429][29458] Rollout worker 0 uses device cpu
+[2025-04-06 21:21:03,429][29458] Rollout worker 1 uses device cpu
+[2025-04-06 21:21:03,429][29458] Rollout worker 2 uses device cpu
+[2025-04-06 21:21:03,430][29458] Rollout worker 3 uses device cpu
+[2025-04-06 21:21:03,430][29458] Rollout worker 4 uses device cpu
+[2025-04-06 21:21:03,430][29458] Rollout worker 5 uses device cpu
+[2025-04-06 21:21:03,430][29458] Rollout worker 6 uses device cpu
+[2025-04-06 21:21:03,431][29458] Rollout worker 7 uses device cpu
+[2025-04-06 21:21:03,542][29458] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2025-04-06 21:21:03,542][29458] InferenceWorker_p0-w0: min num requests: 2
+[2025-04-06 21:21:03,595][29458] Starting all processes...
+[2025-04-06 21:21:03,596][29458] Starting process learner_proc0
+[2025-04-06 21:21:12,041][29458] Starting all processes...
+[2025-04-06 21:21:12,055][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2025-04-06 21:21:12,056][29697] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
+[2025-04-06 21:21:12,068][29458] Starting process inference_proc0-0
+[2025-04-06 21:21:12,068][29458] Starting process rollout_proc0
+[2025-04-06 21:21:12,068][29458] Starting process rollout_proc1
+[2025-04-06 21:21:12,071][29458] Starting process rollout_proc6
+[2025-04-06 21:21:12,078][29697] Num visible devices: 1
+[2025-04-06 21:21:12,086][29697] Starting seed is not provided
+[2025-04-06 21:21:12,087][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2025-04-06 21:21:12,088][29697] Initializing actor-critic model on device cuda:0
+[2025-04-06 21:21:12,090][29697] RunningMeanStd input shape: (3, 72, 128)
+[2025-04-06 21:21:12,095][29697] RunningMeanStd input shape: (1,)
+[2025-04-06 21:21:12,069][29458] Starting process rollout_proc3
+[2025-04-06 21:21:12,069][29458] Starting process rollout_proc4
+[2025-04-06 21:21:12,070][29458] Starting process rollout_proc5
+[2025-04-06 21:21:12,068][29458] Starting process rollout_proc2
+[2025-04-06 21:21:12,071][29458] Starting process rollout_proc7
+[2025-04-06 21:21:12,144][29697] ConvEncoder: input_channels=3
+[2025-04-06 21:21:12,576][29697] Conv encoder output size: 512
+[2025-04-06 21:21:12,577][29697] Policy head output size: 512
+[2025-04-06 21:21:12,610][29697] Created Actor Critic model with architecture:
+[2025-04-06 21:21:12,611][29697] ActorCriticSharedWeights(
+  (obs_normalizer): ObservationNormalizer(
+    (running_mean_std): RunningMeanStdDictInPlace(
+      (running_mean_std): ModuleDict(
+        (obs): RunningMeanStdInPlace()
+      )
+    )
+  )
+  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
+  (encoder): VizdoomEncoder(
+    (basic_encoder): ConvEncoder(
+      (enc): RecursiveScriptModule(
+        original_name=ConvEncoderImpl
+        (conv_head): RecursiveScriptModule(
+          original_name=Sequential
+          (0): RecursiveScriptModule(original_name=Conv2d)
+          (1): RecursiveScriptModule(original_name=ELU)
+          (2): RecursiveScriptModule(original_name=Conv2d)
+          (3): RecursiveScriptModule(original_name=ELU)
+          (4): RecursiveScriptModule(original_name=Conv2d)
+          (5): RecursiveScriptModule(original_name=ELU)
+        )
+        (mlp_layers): RecursiveScriptModule(
+          original_name=Sequential
+          (0): RecursiveScriptModule(original_name=Linear)
+          (1): RecursiveScriptModule(original_name=ELU)
+        )
+      )
+    )
+  )
+  (core): ModelCoreRNN(
+    (core): GRU(512, 512)
+  )
+  (decoder): MlpDecoder(
+    (mlp): Identity()
+  )
+  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
+  (action_parameterization): ActionParameterizationDefault(
+    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
+  )
+)
+[2025-04-06 21:21:12,813][29697] Using optimizer <class 'torch.optim.adam.Adam'>
+[2025-04-06 21:21:18,468][29697] No checkpoints found
+[2025-04-06 21:21:18,471][29697] Did not load from checkpoint, starting from scratch!
+[2025-04-06 21:21:18,474][29697] Initialized policy 0 weights for model version 0
+[2025-04-06 21:21:18,488][29697] LearnerWorker_p0 finished initialization!
+[2025-04-06 21:21:18,488][29697] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2025-04-06 21:21:27,570][29817] Worker 6 uses CPU cores [6]
+[2025-04-06 21:21:32,697][29818] Worker 0 uses CPU cores [0]
+[2025-04-06 21:21:34,112][29822] Worker 5 uses CPU cores [5]
+[2025-04-06 21:21:40,610][29819] Worker 3 uses CPU cores [3]
+[2025-04-06 21:21:50,398][29816] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2025-04-06 21:21:50,399][29816] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
+[2025-04-06 21:21:50,418][29816] Num visible devices: 1
+[2025-04-06 21:21:50,551][29816] RunningMeanStd input shape: (3, 72, 128)
+[2025-04-06 21:21:50,553][29816] RunningMeanStd input shape: (1,)
+[2025-04-06 21:21:50,583][29816] ConvEncoder: input_channels=3
+[2025-04-06 21:21:50,754][29816] Conv encoder output size: 512
+[2025-04-06 21:21:50,755][29816] Policy head output size: 512
+[2025-04-06 21:21:56,879][29823] Worker 7 uses CPU cores [7]
+[2025-04-06 21:22:03,459][29815] Worker 1 uses CPU cores [1]
+[2025-04-06 21:22:25,110][29820] Worker 4 uses CPU cores [4]
+[2025-04-06 21:22:27,189][29458] Heartbeat connected on Batcher_0
+[2025-04-06 21:22:27,191][29458] Heartbeat connected on LearnerWorker_p0
+[2025-04-06 21:22:27,191][29458] Heartbeat connected on RolloutWorker_w6
+[2025-04-06 21:22:27,192][29458] Heartbeat connected on RolloutWorker_w0
+[2025-04-06 21:22:27,192][29458] Heartbeat connected on RolloutWorker_w5
+[2025-04-06 21:22:27,193][29458] Heartbeat connected on RolloutWorker_w3
+[2025-04-06 21:22:27,193][29458] Inference worker 0-0 is ready!
+[2025-04-06 21:22:27,194][29458] All inference workers are ready! Signal rollout workers to start!
+[2025-04-06 21:22:27,195][29458] Heartbeat connected on InferenceWorker_p0-w0
+[2025-04-06 21:22:27,195][29458] Heartbeat connected on RolloutWorker_w7
+[2025-04-06 21:22:27,196][29821] Worker 2 uses CPU cores [2]
+[2025-04-06 21:22:27,198][29458] Heartbeat connected on RolloutWorker_w1
+[2025-04-06 21:22:27,200][29458] Heartbeat connected on RolloutWorker_w4
+[2025-04-06 21:22:27,202][29458] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,224][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,231][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,236][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,239][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,248][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,252][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,255][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,257][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,260][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,261][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,263][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,264][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,265][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,267][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,270][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,271][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:27,276][29820] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,276][29818] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,295][29823] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,317][29819] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,319][29815] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,343][29821] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,380][29817] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,394][29822] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:22:27,868][29815] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:27,868][29820] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:27,869][29823] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:27,871][29819] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:27,882][29818] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:27,902][29817] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:28,222][29818] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,231][29819] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,240][29817] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,256][29821] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:28,260][29820] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,275][29823] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,582][29822] Decorrelating experience for 0 frames...
+[2025-04-06 21:22:28,598][29821] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,707][29819] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:28,748][29817] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:28,760][29823] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:28,783][29818] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:28,980][29822] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:28,985][29820] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:29,096][29819] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,163][29817] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,237][29818] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,391][29815] Decorrelating experience for 32 frames...
+[2025-04-06 21:22:29,393][29820] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,460][29822] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:29,589][29823] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,790][29821] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:29,836][29822] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:29,856][29815] Decorrelating experience for 64 frames...
+[2025-04-06 21:22:30,133][29821] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:30,204][29815] Decorrelating experience for 96 frames...
+[2025-04-06 21:22:30,234][29458] Heartbeat connected on RolloutWorker_w2
+[2025-04-06 21:22:30,818][29458] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 9.0. Samples: 32. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2025-04-06 21:22:30,819][29458] Avg episode reward: [(0, '0.480')]
+[2025-04-06 21:22:31,550][29697] Signal inference workers to stop experience collection...
+[2025-04-06 21:22:31,561][29816] InferenceWorker_p0-w0: stopping experience collection
+[2025-04-06 21:22:34,456][29697] Signal inference workers to resume experience collection...
+[2025-04-06 21:22:34,458][29816] InferenceWorker_p0-w0: resuming experience collection
+[2025-04-06 21:22:35,818][29458] Fps is (10 sec: 1917.0, 60 sec: 1912.7, 300 sec: 1901.5). Total num frames: 16384. Throughput: 0: 571.1. Samples: 4888. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
+[2025-04-06 21:22:35,819][29458] Avg episode reward: [(0, '3.035')]
+[2025-04-06 21:22:38,690][29816] Updated weights for policy 0, policy_version 10 (0.0118)
+[2025-04-06 21:22:40,819][29458] Fps is (10 sec: 5734.4, 60 sec: 4227.9, 300 sec: 4211.3). Total num frames: 57344. Throughput: 0: 800.0. Samples: 10846. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:22:40,820][29458] Avg episode reward: [(0, '4.419')]
+[2025-04-06 21:22:43,570][29816] Updated weights for policy 0, policy_version 20 (0.0035)
+[2025-04-06 21:22:45,818][29458] Fps is (10 sec: 8191.9, 60 sec: 5296.3, 300 sec: 5280.4). Total num frames: 98304. Throughput: 0: 1273.0. Samples: 23622. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:22:45,819][29458] Avg episode reward: [(0, '4.432')]
+[2025-04-06 21:22:48,379][29816] Updated weights for policy 0, policy_version 30 (0.0034)
+[2025-04-06 21:22:50,818][29458] Fps is (10 sec: 8601.9, 60 sec: 6085.3, 300 sec: 6070.3). Total num frames: 143360. Throughput: 0: 1556.9. Samples: 36672. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:22:50,819][29458] Avg episode reward: [(0, '4.405')]
+[2025-04-06 21:22:50,836][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000035_143360.pth...
+[2025-04-06 21:22:50,967][29697] Saving new best policy, reward=4.405!
+[2025-04-06 21:22:53,235][29816] Updated weights for policy 0, policy_version 40 (0.0038)
+[2025-04-06 21:22:55,818][29458] Fps is (10 sec: 8601.6, 60 sec: 6454.5, 300 sec: 6441.0). Total num frames: 184320. Throughput: 0: 1497.7. Samples: 42764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:22:55,819][29458] Avg episode reward: [(0, '4.547')]
+[2025-04-06 21:22:55,820][29697] Saving new best policy, reward=4.547!
+[2025-04-06 21:22:57,999][29816] Updated weights for policy 0, policy_version 50 (0.0035)
+[2025-04-06 21:23:00,818][29458] Fps is (10 sec: 8191.9, 60 sec: 6713.7, 300 sec: 6701.5). Total num frames: 225280. Throughput: 0: 1658.8. Samples: 55656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:23:00,819][29458] Avg episode reward: [(0, '4.311')]
+[2025-04-06 21:23:02,747][29816] Updated weights for policy 0, policy_version 60 (0.0040)
+[2025-04-06 21:23:05,818][29458] Fps is (10 sec: 8601.7, 60 sec: 7011.9, 300 sec: 7000.5). Total num frames: 270336. Throughput: 0: 1783.5. Samples: 68752. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:23:05,819][29458] Avg episode reward: [(0, '4.297')]
+[2025-04-06 21:23:07,502][29816] Updated weights for policy 0, policy_version 70 (0.0031)
+[2025-04-06 21:23:10,819][29458] Fps is (10 sec: 8601.2, 60 sec: 7147.5, 300 sec: 7137.0). Total num frames: 311296. Throughput: 0: 1724.7. Samples: 75108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:23:10,819][29458] Avg episode reward: [(0, '4.408')]
+[2025-04-06 21:23:12,285][29816] Updated weights for policy 0, policy_version 80 (0.0035)
+[2025-04-06 21:23:15,819][29458] Fps is (10 sec: 8600.7, 60 sec: 7339.6, 300 sec: 7329.7). Total num frames: 356352. Throughput: 0: 1957.3. Samples: 88112. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:23:15,820][29458] Avg episode reward: [(0, '4.271')]
+[2025-04-06 21:23:17,010][29816] Updated weights for policy 0, policy_version 90 (0.0031)
+[2025-04-06 21:23:20,818][29458] Fps is (10 sec: 9011.5, 60 sec: 7496.2, 300 sec: 7486.6). Total num frames: 401408. Throughput: 0: 1990.8. Samples: 94474. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:23:20,819][29458] Avg episode reward: [(0, '4.516')]
+[2025-04-06 21:23:21,817][29816] Updated weights for policy 0, policy_version 100 (0.0038)
+[2025-04-06 21:23:25,818][29458] Fps is (10 sec: 8602.4, 60 sec: 7555.8, 300 sec: 7546.8). Total num frames: 442368. Throughput: 0: 2147.5. Samples: 107482. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:23:25,819][29458] Avg episode reward: [(0, '4.406')]
+[2025-04-06 21:23:26,458][29816] Updated weights for policy 0, policy_version 110 (0.0034)
+[2025-04-06 21:23:30,818][29458] Fps is (10 sec: 8601.7, 60 sec: 8123.8, 300 sec: 7661.9). Total num frames: 487424. Throughput: 0: 2158.1. Samples: 120734. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
+[2025-04-06 21:23:30,819][29458] Avg episode reward: [(0, '4.321')]
+[2025-04-06 21:23:30,997][29816] Updated weights for policy 0, policy_version 120 (0.0036)
+[2025-04-06 21:23:35,819][29458] Fps is (10 sec: 8601.2, 60 sec: 8533.3, 300 sec: 7700.5). Total num frames: 528384. Throughput: 0: 2155.0. Samples: 133648. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:23:35,819][29458] Avg episode reward: [(0, '4.625')]
+[2025-04-06 21:23:35,863][29697] Saving new best policy, reward=4.625!
+[2025-04-06 21:23:35,869][29816] Updated weights for policy 0, policy_version 130 (0.0033)
+[2025-04-06 21:23:40,637][29816] Updated weights for policy 0, policy_version 140 (0.0035)
+[2025-04-06 21:23:40,819][29458] Fps is (10 sec: 8601.1, 60 sec: 8601.6, 300 sec: 7789.5). Total num frames: 573440. Throughput: 0: 2162.5. Samples: 140078. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:23:40,820][29458] Avg episode reward: [(0, '4.268')]
+[2025-04-06 21:23:45,417][29816] Updated weights for policy 0, policy_version 150 (0.0037)
+[2025-04-06 21:23:45,818][29458] Fps is (10 sec: 8602.0, 60 sec: 8601.6, 300 sec: 7815.1). Total num frames: 614400. Throughput: 0: 2163.8. Samples: 153026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:23:45,819][29458] Avg episode reward: [(0, '4.387')]
+[2025-04-06 21:23:50,154][29816] Updated weights for policy 0, policy_version 160 (0.0039)
+[2025-04-06 21:23:50,819][29458] Fps is (10 sec: 8601.6, 60 sec: 8601.5, 300 sec: 7886.6). Total num frames: 659456. Throughput: 0: 2159.5. Samples: 165932. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:23:50,819][29458] Avg episode reward: [(0, '4.397')]
+[2025-04-06 21:23:54,952][29816] Updated weights for policy 0, policy_version 170 (0.0036)
+[2025-04-06 21:23:55,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8601.6, 300 sec: 7903.9). Total num frames: 700416. Throughput: 0: 2161.7. Samples: 172382. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:23:55,819][29458] Avg episode reward: [(0, '4.640')]
+[2025-04-06 21:23:55,866][29697] Saving new best policy, reward=4.640!
+[2025-04-06 21:23:59,759][29816] Updated weights for policy 0, policy_version 180 (0.0037)
+[2025-04-06 21:24:00,818][29458] Fps is (10 sec: 8602.0, 60 sec: 8669.8, 300 sec: 7963.0). Total num frames: 745472. Throughput: 0: 2154.8. Samples: 185076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:24:00,819][29458] Avg episode reward: [(0, '4.844')]
+[2025-04-06 21:24:00,830][29697] Saving new best policy, reward=4.844!
+[2025-04-06 21:24:04,949][29816] Updated weights for policy 0, policy_version 190 (0.0039)
+[2025-04-06 21:24:05,819][29458] Fps is (10 sec: 8191.9, 60 sec: 8533.3, 300 sec: 7933.1). Total num frames: 782336. Throughput: 0: 2273.2. Samples: 196770. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:24:05,820][29458] Avg episode reward: [(0, '4.955')]
+[2025-04-06 21:24:05,821][29697] Saving new best policy, reward=4.955!
+[2025-04-06 21:24:10,673][29816] Updated weights for policy 0, policy_version 200 (0.0037)
+[2025-04-06 21:24:10,819][29458] Fps is (10 sec: 7372.7, 60 sec: 8465.1, 300 sec: 7906.1). Total num frames: 819200. Throughput: 0: 2103.5. Samples: 202142. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:24:10,819][29458] Avg episode reward: [(0, '4.772')]
+[2025-04-06 21:24:15,818][29458] Fps is (10 sec: 7373.1, 60 sec: 8328.7, 300 sec: 7881.5). Total num frames: 856064. Throughput: 0: 2042.1. Samples: 212630. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:24:15,819][29458] Avg episode reward: [(0, '4.667')]
+[2025-04-06 21:24:16,335][29816] Updated weights for policy 0, policy_version 210 (0.0039)
+[2025-04-06 21:24:20,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8260.3, 300 sec: 7895.2). Total num frames: 897024. Throughput: 0: 2038.9. Samples: 225400. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2025-04-06 21:24:20,819][29458] Avg episode reward: [(0, '4.723')]
+[2025-04-06 21:24:21,104][29816] Updated weights for policy 0, policy_version 220 (0.0041)
+[2025-04-06 21:24:25,818][29458] Fps is (10 sec: 8191.8, 60 sec: 8260.3, 300 sec: 7907.7). Total num frames: 937984. Throughput: 0: 2025.0. Samples: 231204. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2025-04-06 21:24:25,819][29458] Avg episode reward: [(0, '4.701')]
+[2025-04-06 21:24:26,084][29816] Updated weights for policy 0, policy_version 230 (0.0039)
+[2025-04-06 21:24:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7919.2). Total num frames: 978944. Throughput: 0: 2021.2. Samples: 243982. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2025-04-06 21:24:30,819][29458] Avg episode reward: [(0, '5.029')]
+[2025-04-06 21:24:30,877][29697] Saving new best policy, reward=5.029!
+[2025-04-06 21:24:30,885][29816] Updated weights for policy 0, policy_version 240 (0.0038)
+[2025-04-06 21:24:35,684][29816] Updated weights for policy 0, policy_version 250 (0.0036)
+[2025-04-06 21:24:35,819][29458] Fps is (10 sec: 8601.5, 60 sec: 8260.3, 300 sec: 7961.6). Total num frames: 1024000. Throughput: 0: 1876.3. Samples: 250366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:24:35,819][29458] Avg episode reward: [(0, '5.175')]
+[2025-04-06 21:24:35,821][29697] Saving new best policy, reward=5.175!
+[2025-04-06 21:24:40,475][29816] Updated weights for policy 0, policy_version 260 (0.0035)
+[2025-04-06 21:24:40,818][29458] Fps is (10 sec: 8601.7, 60 sec: 8192.1, 300 sec: 7970.3). Total num frames: 1064960. Throughput: 0: 2019.1. Samples: 263240. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:24:40,819][29458] Avg episode reward: [(0, '5.215')]
+[2025-04-06 21:24:40,831][29697] Saving new best policy, reward=5.215!
+[2025-04-06 21:24:45,776][29816] Updated weights for policy 0, policy_version 270 (0.0034)
+[2025-04-06 21:24:45,819][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7978.3). Total num frames: 1105920. Throughput: 0: 2003.9. Samples: 275254. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:24:45,819][29458] Avg episode reward: [(0, '5.764')]
+[2025-04-06 21:24:45,821][29697] Saving new best policy, reward=5.764!
+[2025-04-06 21:24:50,818][29458] Fps is (10 sec: 7782.3, 60 sec: 8055.5, 300 sec: 7957.2). Total num frames: 1142784. Throughput: 0: 2011.9. Samples: 287306. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:24:50,819][29458] Avg episode reward: [(0, '5.685')]
+[2025-04-06 21:24:50,827][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth...
+[2025-04-06 21:24:51,023][29816] Updated weights for policy 0, policy_version 280 (0.0037)
+[2025-04-06 21:24:55,726][29816] Updated weights for policy 0, policy_version 290 (0.0038)
+[2025-04-06 21:24:55,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8123.7, 300 sec: 7992.6). Total num frames: 1187840. Throughput: 0: 2025.6. Samples: 293296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:24:55,819][29458] Avg episode reward: [(0, '5.926')]
+[2025-04-06 21:24:55,820][29697] Saving new best policy, reward=5.926!
+[2025-04-06 21:25:00,818][29458] Fps is (10 sec: 8601.6, 60 sec: 8055.5, 300 sec: 7999.1). Total num frames: 1228800. Throughput: 0: 2065.0. Samples: 305554. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:25:00,818][29816] Updated weights for policy 0, policy_version 300 (0.0041)
+[2025-04-06 21:25:00,819][29458] Avg episode reward: [(0, '6.334')]
+[2025-04-06 21:25:00,829][29697] Saving new best policy, reward=6.334!
+[2025-04-06 21:25:05,818][29816] Updated weights for policy 0, policy_version 310 (0.0033)
+[2025-04-06 21:25:05,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8123.8, 300 sec: 8005.2). Total num frames: 1269760. Throughput: 0: 2058.7. Samples: 318040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:05,819][29458] Avg episode reward: [(0, '7.399')]
+[2025-04-06 21:25:05,820][29697] Saving new best policy, reward=7.399!
+[2025-04-06 21:25:10,767][29816] Updated weights for policy 0, policy_version 320 (0.0035)
+[2025-04-06 21:25:10,818][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 8010.9). Total num frames: 1310720. Throughput: 0: 2065.3. Samples: 324144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:10,819][29458] Avg episode reward: [(0, '7.160')]
+[2025-04-06 21:25:15,614][29816] Updated weights for policy 0, policy_version 330 (0.0037)
+[2025-04-06 21:25:15,819][29458] Fps is (10 sec: 8191.6, 60 sec: 8260.2, 300 sec: 8016.3). Total num frames: 1351680. Throughput: 0: 2061.4. Samples: 336744. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:25:15,820][29458] Avg episode reward: [(0, '7.225')]
+[2025-04-06 21:25:20,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8192.0, 300 sec: 7997.8). Total num frames: 1388544. Throughput: 0: 2182.8. Samples: 348592. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:20,819][29458] Avg episode reward: [(0, '7.720')]
+[2025-04-06 21:25:20,833][29697] Saving new best policy, reward=7.720!
+[2025-04-06 21:25:21,072][29816] Updated weights for policy 0, policy_version 340 (0.0041)
+[2025-04-06 21:25:25,818][29458] Fps is (10 sec: 7782.6, 60 sec: 8192.0, 300 sec: 8003.2). Total num frames: 1429504. Throughput: 0: 2022.7. Samples: 354260. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:25:25,819][29458] Avg episode reward: [(0, '8.824')]
+[2025-04-06 21:25:25,821][29697] Saving new best policy, reward=8.824!
+[2025-04-06 21:25:26,135][29816] Updated weights for policy 0, policy_version 350 (0.0035)
+[2025-04-06 21:25:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 8008.3). Total num frames: 1470464. Throughput: 0: 2030.5. Samples: 366626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:30,819][29458] Avg episode reward: [(0, '9.869')]
+[2025-04-06 21:25:30,833][29697] Saving new best policy, reward=9.869!
+[2025-04-06 21:25:31,162][29816] Updated weights for policy 0, policy_version 360 (0.0034)
+[2025-04-06 21:25:35,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8123.8, 300 sec: 8013.2). Total num frames: 1511424. Throughput: 0: 2039.2. Samples: 379070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:35,819][29458] Avg episode reward: [(0, '9.580')]
+[2025-04-06 21:25:36,085][29816] Updated weights for policy 0, policy_version 370 (0.0032)
+[2025-04-06 21:25:40,820][29458] Fps is (10 sec: 8191.0, 60 sec: 8123.6, 300 sec: 8017.8). Total num frames: 1552384. Throughput: 0: 2039.1. Samples: 385060. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:25:40,821][29458] Avg episode reward: [(0, '9.251')]
+[2025-04-06 21:25:41,112][29816] Updated weights for policy 0, policy_version 380 (0.0035)
+[2025-04-06 21:25:45,819][29458] Fps is (10 sec: 7782.3, 60 sec: 8055.5, 300 sec: 8001.6). Total num frames: 1589248. Throughput: 0: 2029.6. Samples: 396886. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:25:45,819][29458] Avg episode reward: [(0, '10.675')]
+[2025-04-06 21:25:45,822][29697] Saving new best policy, reward=10.675!
+[2025-04-06 21:25:46,567][29816] Updated weights for policy 0, policy_version 390 (0.0037)
+[2025-04-06 21:25:50,819][29458] Fps is (10 sec: 7373.1, 60 sec: 8055.4, 300 sec: 7986.1). Total num frames: 1626112. Throughput: 0: 1989.9. Samples: 407588. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2025-04-06 21:25:50,825][29458] Avg episode reward: [(0, '12.079')]
+[2025-04-06 21:25:50,840][29697] Saving new best policy, reward=12.079!
+[2025-04-06 21:25:52,204][29816] Updated weights for policy 0, policy_version 400 (0.0036)
+[2025-04-06 21:25:55,819][29458] Fps is (10 sec: 7372.8, 60 sec: 7918.9, 300 sec: 7971.4). Total num frames: 1662976. Throughput: 0: 1977.1. Samples: 413114. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:25:55,819][29458] Avg episode reward: [(0, '11.502')]
+[2025-04-06 21:25:57,689][29816] Updated weights for policy 0, policy_version 410 (0.0042)
+[2025-04-06 21:26:00,819][29458] Fps is (10 sec: 7373.3, 60 sec: 7850.6, 300 sec: 7957.4). Total num frames: 1699840. Throughput: 0: 1938.5. Samples: 423974. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:26:00,820][29458] Avg episode reward: [(0, '11.994')]
+[2025-04-06 21:26:03,596][29816] Updated weights for policy 0, policy_version 420 (0.0038)
+[2025-04-06 21:26:05,818][29458] Fps is (10 sec: 7373.0, 60 sec: 7782.4, 300 sec: 7944.9). Total num frames: 1736704. Throughput: 0: 1792.6. Samples: 429258. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:26:05,819][29458] Avg episode reward: [(0, '13.856')]
+[2025-04-06 21:26:05,820][29697] Saving new best policy, reward=13.856!
+[2025-04-06 21:26:09,764][29816] Updated weights for policy 0, policy_version 430 (0.0048)
+[2025-04-06 21:26:10,818][29458] Fps is (10 sec: 6553.7, 60 sec: 7577.6, 300 sec: 7895.7). Total num frames: 1765376. Throughput: 0: 1894.1. Samples: 439496. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:26:10,819][29458] Avg episode reward: [(0, '14.682')]
+[2025-04-06 21:26:10,837][29697] Saving new best policy, reward=14.682!
+[2025-04-06 21:26:15,819][29458] Fps is (10 sec: 6143.8, 60 sec: 7441.1, 300 sec: 7866.5). Total num frames: 1798144. Throughput: 0: 1833.9. Samples: 449150. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:26:15,819][29458] Avg episode reward: [(0, '13.383')]
+[2025-04-06 21:26:15,829][29816] Updated weights for policy 0, policy_version 440 (0.0045)
+[2025-04-06 21:26:20,818][29458] Fps is (10 sec: 7372.9, 60 sec: 7509.3, 300 sec: 7873.6). Total num frames: 1839104. Throughput: 0: 1821.6. Samples: 461042. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:26:20,819][29458] Avg episode reward: [(0, '12.544')]
+[2025-04-06 21:26:21,138][29816] Updated weights for policy 0, policy_version 450 (0.0035)
+[2025-04-06 21:26:25,818][29458] Fps is (10 sec: 8192.4, 60 sec: 7509.4, 300 sec: 7880.6). Total num frames: 1880064. Throughput: 0: 1814.0. Samples: 466688. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:26:25,820][29458] Avg episode reward: [(0, '14.011')]
+[2025-04-06 21:26:26,363][29816] Updated weights for policy 0, policy_version 460 (0.0037)
+[2025-04-06 21:26:30,820][29458] Fps is (10 sec: 7781.1, 60 sec: 7440.9, 300 sec: 7870.2). Total num frames: 1916928. Throughput: 0: 1812.5. Samples: 478450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:26:30,821][29458] Avg episode reward: [(0, '14.741')]
+[2025-04-06 21:26:30,836][29697] Saving new best policy, reward=14.741!
+[2025-04-06 21:26:31,636][29816] Updated weights for policy 0, policy_version 470 (0.0037)
+[2025-04-06 21:26:35,818][29458] Fps is (10 sec: 7782.4, 60 sec: 7441.1, 300 sec: 7876.8). Total num frames: 1957888. Throughput: 0: 1701.0. Samples: 484130. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2025-04-06 21:26:35,819][29458] Avg episode reward: [(0, '14.143')]
+[2025-04-06 21:26:36,932][29816] Updated weights for policy 0, policy_version 480 (0.0038)
+[2025-04-06 21:26:40,819][29458] Fps is (10 sec: 7783.3, 60 sec: 7372.9, 300 sec: 7866.9). Total num frames: 1994752. Throughput: 0: 1839.4. Samples: 495888. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:26:40,819][29458] Avg episode reward: [(0, '16.219')]
+[2025-04-06 21:26:40,831][29697] Saving new best policy, reward=16.219!
+[2025-04-06 21:26:42,188][29816] Updated weights for policy 0, policy_version 490 (0.0044)
+[2025-04-06 21:26:45,819][29458] Fps is (10 sec: 7372.5, 60 sec: 7372.8, 300 sec: 7857.5). Total num frames: 2031616. Throughput: 0: 1852.7. Samples: 507346. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:26:45,820][29458] Avg episode reward: [(0, '16.662')]
+[2025-04-06 21:26:45,822][29697] Saving new best policy, reward=16.662!
+[2025-04-06 21:26:47,546][29816] Updated weights for policy 0, policy_version 500 (0.0040)
+[2025-04-06 21:26:50,818][29458] Fps is (10 sec: 7782.7, 60 sec: 7441.2, 300 sec: 7863.9). Total num frames: 2072576. Throughput: 0: 1861.4. Samples: 513022. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:26:50,819][29458] Avg episode reward: [(0, '16.306')]
+[2025-04-06 21:26:50,834][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000506_2072576.pth...
+[2025-04-06 21:26:50,935][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000035_143360.pth
+[2025-04-06 21:26:52,874][29816] Updated weights for policy 0, policy_version 510 (0.0048)
+[2025-04-06 21:26:55,819][29458] Fps is (10 sec: 7782.1, 60 sec: 7441.0, 300 sec: 7854.7). Total num frames: 2109440. Throughput: 0: 1893.9. Samples: 524724. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:26:55,820][29458] Avg episode reward: [(0, '16.752')]
+[2025-04-06 21:26:55,822][29697] Saving new best policy, reward=16.752!
+[2025-04-06 21:26:58,215][29816] Updated weights for policy 0, policy_version 520 (0.0033)
+[2025-04-06 21:27:00,819][29458] Fps is (10 sec: 7372.7, 60 sec: 7441.1, 300 sec: 7846.0). Total num frames: 2146304. Throughput: 0: 1937.2. Samples: 536322. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:27:00,820][29458] Avg episode reward: [(0, '16.356')]
+[2025-04-06 21:27:03,417][29816] Updated weights for policy 0, policy_version 530 (0.0040)
+[2025-04-06 21:27:05,818][29458] Fps is (10 sec: 7782.9, 60 sec: 7509.3, 300 sec: 7852.2). Total num frames: 2187264. Throughput: 0: 1932.6. Samples: 548010. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:27:05,820][29458] Avg episode reward: [(0, '16.246')]
+[2025-04-06 21:27:08,808][29816] Updated weights for policy 0, policy_version 540 (0.0049)
+[2025-04-06 21:27:10,819][29458] Fps is (10 sec: 8191.7, 60 sec: 7714.1, 300 sec: 7858.3). Total num frames: 2228224. Throughput: 0: 1934.2. Samples: 553728. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:27:10,819][29458] Avg episode reward: [(0, '16.217')]
+[2025-04-06 21:27:14,157][29816] Updated weights for policy 0, policy_version 550 (0.0039)
+[2025-04-06 21:27:15,818][29458] Fps is (10 sec: 7782.4, 60 sec: 7782.4, 300 sec: 7849.9). Total num frames: 2265088. Throughput: 0: 1924.3. Samples: 565040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:27:15,819][29458] Avg episode reward: [(0, '15.185')]
+[2025-04-06 21:27:19,607][29816] Updated weights for policy 0, policy_version 560 (0.0038)
+[2025-04-06 21:27:20,819][29458] Fps is (10 sec: 7373.0, 60 sec: 7714.1, 300 sec: 7841.8). Total num frames: 2301952. Throughput: 0: 2051.4. Samples: 576444. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
+[2025-04-06 21:27:20,820][29458] Avg episode reward: [(0, '18.130')]
+[2025-04-06 21:27:20,837][29697] Saving new best policy, reward=18.130!
+[2025-04-06 21:27:25,069][29816] Updated weights for policy 0, policy_version 570 (0.0041)
+[2025-04-06 21:27:25,819][29458] Fps is (10 sec: 7372.7, 60 sec: 7645.8, 300 sec: 7928.2). Total num frames: 2338816. Throughput: 0: 1910.8. Samples: 581872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:27:25,819][29458] Avg episode reward: [(0, '17.818')]
+[2025-04-06 21:27:30,819][29458] Fps is (10 sec: 6963.2, 60 sec: 7577.8, 300 sec: 7983.7). Total num frames: 2371584. Throughput: 0: 1894.0. Samples: 592576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:27:30,820][29458] Avg episode reward: [(0, '19.286')]
+[2025-04-06 21:27:30,854][29697] Saving new best policy, reward=19.286!
+[2025-04-06 21:27:30,862][29816] Updated weights for policy 0, policy_version 580 (0.0041)
+[2025-04-06 21:27:35,818][29458] Fps is (10 sec: 7372.8, 60 sec: 7577.6, 300 sec: 7983.7). Total num frames: 2412544. Throughput: 0: 2027.3. Samples: 604252. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2025-04-06 21:27:35,819][29458] Avg episode reward: [(0, '18.906')]
+[2025-04-06 21:27:36,156][29816] Updated weights for policy 0, policy_version 590 (0.0034)
+[2025-04-06 21:27:40,818][29458] Fps is (10 sec: 8192.3, 60 sec: 7645.9, 300 sec: 7983.7). Total num frames: 2453504. Throughput: 0: 1903.0. Samples: 610358. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:27:40,819][29458] Avg episode reward: [(0, '19.766')]
+[2025-04-06 21:27:40,831][29697] Saving new best policy, reward=19.766!
+[2025-04-06 21:27:41,189][29816] Updated weights for policy 0, policy_version 600 (0.0036)
+[2025-04-06 21:27:45,818][29458] Fps is (10 sec: 8192.1, 60 sec: 7714.2, 300 sec: 7969.8). Total num frames: 2494464. Throughput: 0: 1922.3. Samples: 622826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
+[2025-04-06 21:27:45,819][29458] Avg episode reward: [(0, '19.846')]
+[2025-04-06 21:27:45,821][29697] Saving new best policy, reward=19.846!
+[2025-04-06 21:27:46,042][29816] Updated weights for policy 0, policy_version 610 (0.0037)
+[2025-04-06 21:27:50,818][29458] Fps is (10 sec: 8192.0, 60 sec: 7714.1, 300 sec: 7969.8). Total num frames: 2535424. Throughput: 0: 1945.3. Samples: 635548. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:27:50,819][29458] Avg episode reward: [(0, '20.144')]
+[2025-04-06 21:27:50,847][29697] Saving new best policy, reward=20.144!
+[2025-04-06 21:27:50,856][29816] Updated weights for policy 0, policy_version 620 (0.0037)
+[2025-04-06 21:27:55,689][29816] Updated weights for policy 0, policy_version 630 (0.0032)
+[2025-04-06 21:27:55,819][29458] Fps is (10 sec: 8601.3, 60 sec: 7850.7, 300 sec: 7983.7). Total num frames: 2580480. Throughput: 0: 1955.3. Samples: 641716. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:27:55,820][29458] Avg episode reward: [(0, '21.052')]
+[2025-04-06 21:27:55,822][29697] Saving new best policy, reward=21.052!
+[2025-04-06 21:28:00,625][29816] Updated weights for policy 0, policy_version 640 (0.0037)
+[2025-04-06 21:28:00,819][29458] Fps is (10 sec: 8601.4, 60 sec: 7918.9, 300 sec: 7969.8). Total num frames: 2621440. Throughput: 0: 1981.0. Samples: 654186. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:28:00,819][29458] Avg episode reward: [(0, '19.409')]
+[2025-04-06 21:28:05,536][29816] Updated weights for policy 0, policy_version 650 (0.0039)
+[2025-04-06 21:28:05,818][29458] Fps is (10 sec: 8192.3, 60 sec: 7918.9, 300 sec: 7969.9). Total num frames: 2662400. Throughput: 0: 2005.8. Samples: 666706. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:28:05,819][29458] Avg episode reward: [(0, '19.812')]
+[2025-04-06 21:28:10,511][29816] Updated weights for policy 0, policy_version 660 (0.0040)
+[2025-04-06 21:28:10,819][29458] Fps is (10 sec: 8192.0, 60 sec: 7919.0, 300 sec: 7956.0). Total num frames: 2703360. Throughput: 0: 2023.9. Samples: 672948. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:28:10,820][29458] Avg episode reward: [(0, '21.072')]
+[2025-04-06 21:28:10,835][29697] Saving new best policy, reward=21.072!
+[2025-04-06 21:28:15,671][29816] Updated weights for policy 0, policy_version 670 (0.0036)
+[2025-04-06 21:28:15,819][29458] Fps is (10 sec: 8191.9, 60 sec: 7987.2, 300 sec: 7942.1). Total num frames: 2744320. Throughput: 0: 2050.1. Samples: 684832. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:28:15,819][29458] Avg episode reward: [(0, '21.161')]
+[2025-04-06 21:28:15,821][29697] Saving new best policy, reward=21.161!
+[2025-04-06 21:28:20,590][29816] Updated weights for policy 0, policy_version 680 (0.0038)
+[2025-04-06 21:28:20,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8055.5, 300 sec: 7942.1). Total num frames: 2785280. Throughput: 0: 1928.6. Samples: 691040. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:28:20,819][29458] Avg episode reward: [(0, '20.833')]
+[2025-04-06 21:28:25,516][29816] Updated weights for policy 0, policy_version 690 (0.0037)
+[2025-04-06 21:28:25,819][29458] Fps is (10 sec: 8192.1, 60 sec: 8123.7, 300 sec: 7928.2). Total num frames: 2826240. Throughput: 0: 2073.1. Samples: 703646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:28:25,820][29458] Avg episode reward: [(0, '21.685')]
+[2025-04-06 21:28:25,821][29697] Saving new best policy, reward=21.685!
+[2025-04-06 21:28:30,510][29816] Updated weights for policy 0, policy_version 700 (0.0037)
+[2025-04-06 21:28:30,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 2867200. Throughput: 0: 2069.4. Samples: 715948. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:28:30,819][29458] Avg episode reward: [(0, '21.848')]
+[2025-04-06 21:28:30,831][29697] Saving new best policy, reward=21.848!
+[2025-04-06 21:28:35,417][29816] Updated weights for policy 0, policy_version 710 (0.0031)
+[2025-04-06 21:28:35,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7914.3). Total num frames: 2908160. Throughput: 0: 2064.6. Samples: 728454. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:28:35,819][29458] Avg episode reward: [(0, '21.461')]
+[2025-04-06 21:28:40,339][29816] Updated weights for policy 0, policy_version 720 (0.0037)
+[2025-04-06 21:28:40,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8328.5, 300 sec: 7928.2). Total num frames: 2953216. Throughput: 0: 2066.6. Samples: 734712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:28:40,819][29458] Avg episode reward: [(0, '21.480')]
+[2025-04-06 21:28:45,138][29816] Updated weights for policy 0, policy_version 730 (0.0035)
+[2025-04-06 21:28:45,818][29458] Fps is (10 sec: 8601.5, 60 sec: 8328.5, 300 sec: 7914.3). Total num frames: 2994176. Throughput: 0: 2069.0. Samples: 747290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:28:45,819][29458] Avg episode reward: [(0, '18.329')]
+[2025-04-06 21:28:50,138][29816] Updated weights for policy 0, policy_version 740 (0.0042)
+[2025-04-06 21:28:50,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8328.5, 300 sec: 7914.3). Total num frames: 3035136. Throughput: 0: 2067.3. Samples: 759734. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:28:50,819][29458] Avg episode reward: [(0, '18.835')]
+[2025-04-06 21:28:50,834][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000741_3035136.pth...
+[2025-04-06 21:28:50,964][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth
+[2025-04-06 21:28:55,225][29816] Updated weights for policy 0, policy_version 750 (0.0041)
+[2025-04-06 21:28:55,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7900.4). Total num frames: 3076096. Throughput: 0: 2063.3. Samples: 765794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:28:55,820][29458] Avg episode reward: [(0, '19.178')]
+[2025-04-06 21:29:00,235][29816] Updated weights for policy 0, policy_version 760 (0.0037)
+[2025-04-06 21:29:00,820][29458] Fps is (10 sec: 8191.0, 60 sec: 8260.1, 300 sec: 7914.3). Total num frames: 3117056. Throughput: 0: 2072.2. Samples: 778084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:29:00,820][29458] Avg episode reward: [(0, '19.598')]
+[2025-04-06 21:29:05,337][29816] Updated weights for policy 0, policy_version 770 (0.0039)
+[2025-04-06 21:29:05,818][29458] Fps is (10 sec: 7782.5, 60 sec: 8192.0, 300 sec: 7914.3). Total num frames: 3153920. Throughput: 0: 2203.1. Samples: 790178. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:29:05,819][29458] Avg episode reward: [(0, '21.833')]
+[2025-04-06 21:29:10,278][29816] Updated weights for policy 0, policy_version 780 (0.0039)
+[2025-04-06 21:29:10,819][29458] Fps is (10 sec: 8192.6, 60 sec: 8260.2, 300 sec: 7942.1). Total num frames: 3198976. Throughput: 0: 2061.4. Samples: 796412. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:29:10,820][29458] Avg episode reward: [(0, '22.750')]
+[2025-04-06 21:29:10,838][29697] Saving new best policy, reward=22.750!
+[2025-04-06 21:29:15,200][29816] Updated weights for policy 0, policy_version 790 (0.0034)
+[2025-04-06 21:29:15,818][29458] Fps is (10 sec: 8601.4, 60 sec: 8260.3, 300 sec: 7942.1). Total num frames: 3239936. Throughput: 0: 2060.7. Samples: 808680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:29:15,819][29458] Avg episode reward: [(0, '24.524')]
+[2025-04-06 21:29:15,821][29697] Saving new best policy, reward=24.524!
+[2025-04-06 21:29:20,195][29816] Updated weights for policy 0, policy_version 800 (0.0039)
+[2025-04-06 21:29:20,819][29458] Fps is (10 sec: 8192.2, 60 sec: 8260.2, 300 sec: 7942.1). Total num frames: 3280896. Throughput: 0: 2057.5. Samples: 821042. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:29:20,820][29458] Avg episode reward: [(0, '25.083')]
+[2025-04-06 21:29:20,830][29697] Saving new best policy, reward=25.083!
+[2025-04-06 21:29:25,181][29816] Updated weights for policy 0, policy_version 810 (0.0037)
+[2025-04-06 21:29:25,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7942.1). Total num frames: 3321856. Throughput: 0: 2053.8. Samples: 827132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:29:25,819][29458] Avg episode reward: [(0, '23.715')]
+[2025-04-06 21:29:30,162][29816] Updated weights for policy 0, policy_version 820 (0.0041)
+[2025-04-06 21:29:30,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3362816. Throughput: 0: 2050.0. Samples: 839542. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:29:30,820][29458] Avg episode reward: [(0, '22.336')]
+[2025-04-06 21:29:35,117][29816] Updated weights for policy 0, policy_version 830 (0.0038)
+[2025-04-06 21:29:35,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3403776. Throughput: 0: 2050.0. Samples: 851982. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:29:35,819][29458] Avg episode reward: [(0, '23.183')]
+[2025-04-06 21:29:40,031][29816] Updated weights for policy 0, policy_version 840 (0.0037)
+[2025-04-06 21:29:40,818][29458] Fps is (10 sec: 8192.1, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3444736. Throughput: 0: 2052.9. Samples: 858176. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:29:40,820][29458] Avg episode reward: [(0, '21.936')]
+[2025-04-06 21:29:44,985][29816] Updated weights for policy 0, policy_version 850 (0.0035)
+[2025-04-06 21:29:45,819][29458] Fps is (10 sec: 8191.7, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3485696. Throughput: 0: 2055.8. Samples: 870594. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
+[2025-04-06 21:29:45,821][29458] Avg episode reward: [(0, '22.511')]
+[2025-04-06 21:29:50,137][29816] Updated weights for policy 0, policy_version 860 (0.0032)
+[2025-04-06 21:29:50,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3526656. Throughput: 0: 2053.7. Samples: 882596. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:29:50,819][29458] Avg episode reward: [(0, '22.732')]
+[2025-04-06 21:29:55,161][29816] Updated weights for policy 0, policy_version 870 (0.0035)
+[2025-04-06 21:29:55,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8192.0, 300 sec: 7928.2). Total num frames: 3567616. Throughput: 0: 2051.9. Samples: 888746. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:29:55,819][29458] Avg episode reward: [(0, '23.068')]
+[2025-04-06 21:30:00,144][29816] Updated weights for policy 0, policy_version 880 (0.0037)
+[2025-04-06 21:30:00,819][29458] Fps is (10 sec: 8191.6, 60 sec: 8192.1, 300 sec: 7928.2). Total num frames: 3608576. Throughput: 0: 2055.1. Samples: 901160. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:30:00,820][29458] Avg episode reward: [(0, '23.628')]
+[2025-04-06 21:30:05,140][29816] Updated weights for policy 0, policy_version 890 (0.0037)
+[2025-04-06 21:30:05,818][29458] Fps is (10 sec: 8192.0, 60 sec: 8260.3, 300 sec: 7928.2). Total num frames: 3649536. Throughput: 0: 2053.1. Samples: 913430. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:30:05,819][29458] Avg episode reward: [(0, '24.405')]
+[2025-04-06 21:30:09,984][29816] Updated weights for policy 0, policy_version 900 (0.0036)
+[2025-04-06 21:30:10,818][29458] Fps is (10 sec: 8192.3, 60 sec: 8192.1, 300 sec: 7928.2). Total num frames: 3690496. Throughput: 0: 2055.4. Samples: 919626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
+[2025-04-06 21:30:10,820][29458] Avg episode reward: [(0, '22.835')]
+[2025-04-06 21:30:15,093][29816] Updated weights for policy 0, policy_version 910 (0.0035)
+[2025-04-06 21:30:15,818][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3731456. Throughput: 0: 2050.0. Samples: 931792. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
+[2025-04-06 21:30:15,819][29458] Avg episode reward: [(0, '21.146')]
+[2025-04-06 21:30:20,125][29816] Updated weights for policy 0, policy_version 920 (0.0033)
+[2025-04-06 21:30:20,819][29458] Fps is (10 sec: 8191.9, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3772416. Throughput: 0: 2047.9. Samples: 944140. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:30:20,819][29458] Avg episode reward: [(0, '22.432')]
+[2025-04-06 21:30:25,108][29816] Updated weights for policy 0, policy_version 930 (0.0038)
+[2025-04-06 21:30:25,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3813376. Throughput: 0: 2047.9. Samples: 950334. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:30:25,819][29458] Avg episode reward: [(0, '25.029')]
+[2025-04-06 21:30:30,143][29816] Updated weights for policy 0, policy_version 940 (0.0034)
+[2025-04-06 21:30:30,818][29458] Fps is (10 sec: 8192.2, 60 sec: 8192.0, 300 sec: 7942.1). Total num frames: 3854336. Throughput: 0: 2041.6. Samples: 962464. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
+[2025-04-06 21:30:30,819][29458] Avg episode reward: [(0, '24.218')]
+[2025-04-06 21:30:35,169][29816] Updated weights for policy 0, policy_version 950 (0.0038)
+[2025-04-06 21:30:35,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8191.9, 300 sec: 7942.1). Total num frames: 3895296. Throughput: 0: 2046.6. Samples: 974694. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
+[2025-04-06 21:30:35,820][29458] Avg episode reward: [(0, '26.537')]
+[2025-04-06 21:30:35,821][29697] Saving new best policy, reward=26.537!
+[2025-04-06 21:30:40,154][29816] Updated weights for policy 0, policy_version 960 (0.0037)
+[2025-04-06 21:30:40,819][29458] Fps is (10 sec: 8191.8, 60 sec: 8192.0, 300 sec: 7956.0). Total num frames: 3936256. Throughput: 0: 2046.8. Samples: 980854. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:30:40,819][29458] Avg episode reward: [(0, '24.753')]
+[2025-04-06 21:30:45,077][29816] Updated weights for policy 0, policy_version 970 (0.0034)
+[2025-04-06 21:30:45,818][29458] Fps is (10 sec: 8192.4, 60 sec: 8192.0, 300 sec: 7969.9). Total num frames: 3977216. Throughput: 0: 2048.8. Samples: 993354. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
+[2025-04-06 21:30:45,819][29458] Avg episode reward: [(0, '23.041')]
+[2025-04-06 21:30:49,011][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2025-04-06 21:30:49,019][29458] Component Batcher_0 stopped!
+[2025-04-06 21:30:49,025][29697] Stopping Batcher_0...
+[2025-04-06 21:30:49,035][29697] Loop batcher_evt_loop terminating...
+[2025-04-06 21:30:49,050][29816] Weights refcount: 2 0
+[2025-04-06 21:30:49,055][29458] Component InferenceWorker_p0-w0 stopped!
+[2025-04-06 21:30:49,054][29816] Stopping InferenceWorker_p0-w0...
+[2025-04-06 21:30:49,058][29816] Loop inference_proc0-0_evt_loop terminating...
+[2025-04-06 21:30:49,088][29458] Component RolloutWorker_w6 stopped!
+[2025-04-06 21:30:49,088][29817] Stopping RolloutWorker_w6...
+[2025-04-06 21:30:49,093][29817] Loop rollout_proc6_evt_loop terminating...
+[2025-04-06 21:30:49,128][29458] Component RolloutWorker_w3 stopped!
+[2025-04-06 21:30:49,129][29458] Component RolloutWorker_w7 stopped!
+[2025-04-06 21:30:49,128][29823] Stopping RolloutWorker_w7...
+[2025-04-06 21:30:49,131][29458] Component RolloutWorker_w1 stopped!
+[2025-04-06 21:30:49,128][29819] Stopping RolloutWorker_w3...
+[2025-04-06 21:30:49,131][29823] Loop rollout_proc7_evt_loop terminating...
+[2025-04-06 21:30:49,133][29815] Stopping RolloutWorker_w1...
+[2025-04-06 21:30:49,138][29815] Loop rollout_proc1_evt_loop terminating...
+[2025-04-06 21:30:49,134][29819] Loop rollout_proc3_evt_loop terminating...
+[2025-04-06 21:30:49,140][29458] Component RolloutWorker_w2 stopped!
+[2025-04-06 21:30:49,142][29821] Stopping RolloutWorker_w2...
+[2025-04-06 21:30:49,145][29458] Component RolloutWorker_w4 stopped!
+[2025-04-06 21:30:49,147][29697] Removing /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000506_2072576.pth
+[2025-04-06 21:30:49,148][29458] Component RolloutWorker_w0 stopped!
+[2025-04-06 21:30:49,144][29821] Loop rollout_proc2_evt_loop terminating...
+[2025-04-06 21:30:49,147][29818] Stopping RolloutWorker_w0...
+[2025-04-06 21:30:49,150][29818] Loop rollout_proc0_evt_loop terminating...
+[2025-04-06 21:30:49,145][29820] Stopping RolloutWorker_w4...
+[2025-04-06 21:30:49,154][29820] Loop rollout_proc4_evt_loop terminating...
+[2025-04-06 21:30:49,164][29697] Saving /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2025-04-06 21:30:49,175][29458] Component RolloutWorker_w5 stopped!
+[2025-04-06 21:30:49,175][29822] Stopping RolloutWorker_w5...
+[2025-04-06 21:30:49,177][29822] Loop rollout_proc5_evt_loop terminating...
+[2025-04-06 21:30:49,340][29697] Stopping LearnerWorker_p0...
+[2025-04-06 21:30:49,341][29458] Component LearnerWorker_p0 stopped!
+[2025-04-06 21:30:49,342][29458] Waiting for process learner_proc0 to stop...
+[2025-04-06 21:30:49,343][29697] Loop learner_proc0_evt_loop terminating...
+[2025-04-06 21:30:51,627][29458] Waiting for process inference_proc0-0 to join...
+[2025-04-06 21:30:51,628][29458] Waiting for process rollout_proc0 to join...
+[2025-04-06 21:30:51,629][29458] Waiting for process rollout_proc1 to join...
+[2025-04-06 21:30:51,630][29458] Waiting for process rollout_proc2 to join...
+[2025-04-06 21:30:51,631][29458] Waiting for process rollout_proc3 to join...
+[2025-04-06 21:30:51,632][29458] Waiting for process rollout_proc4 to join...
+[2025-04-06 21:30:51,633][29458] Waiting for process rollout_proc5 to join...
+[2025-04-06 21:30:51,634][29458] Waiting for process rollout_proc6 to join...
+[2025-04-06 21:30:51,634][29458] Waiting for process rollout_proc7 to join...
+[2025-04-06 21:30:51,635][29458] Batcher 0 profile tree view:
+batching: 17.8770, releasing_batches: 0.0695
+[2025-04-06 21:30:51,636][29458] InferenceWorker_p0-w0 profile tree view:
+wait_policy: 0.0001
+  wait_policy_total: 42.1090
+update_model: 10.7099
+  weight_update: 0.0035
+one_step: 0.0087
+  handle_policy_step: 451.9330
+    deserialize: 16.9719, stack: 3.9921, obs_to_device_normalize: 136.9680, forward: 196.5648, send_messages: 25.9704
+    prepare_outputs: 43.6267
+      to_cpu: 23.3038
+[2025-04-06 21:30:51,637][29458] Learner 0 profile tree view:
+misc: 0.0148, prepare_batch: 11.7053
+train: 50.0566
+  epoch_init: 0.0177, minibatch_init: 0.0135, losses_postprocess: 0.3649, kl_divergence: 0.5450, after_optimizer: 16.5465
+  calculate_losses: 17.8696
+    losses_init: 0.0104, forward_head: 1.2988, bptt_initial: 11.1267, tail: 1.2122, advantages_returns: 0.2812, losses: 1.7146
+    bptt: 1.7721
+      bptt_forward_core: 1.6535
+  update: 13.9069
+    clip: 1.7712
+[2025-04-06 21:30:51,637][29458] RolloutWorker_w0 profile tree view:
+wait_for_trajectories: 0.3932, enqueue_policy_requests: 14.9665, env_step: 208.0064, overhead: 24.6133, complete_rollouts: 0.6180
+save_policy_outputs: 26.4317
+  split_output_tensors: 13.2113
+[2025-04-06 21:30:51,638][29458] RolloutWorker_w7 profile tree view:
+wait_for_trajectories: 0.4155, enqueue_policy_requests: 16.2129, env_step: 211.3192, overhead: 26.4352, complete_rollouts: 0.6806
+save_policy_outputs: 28.5057
+  split_output_tensors: 14.0651
+[2025-04-06 21:30:51,639][29458] Loop Runner_EvtLoop terminating...
+[2025-04-06 21:30:51,640][29458] Runner profile tree view:
+main_loop: 588.0457
+[2025-04-06 21:30:51,641][29458] Collected {0: 4005888}, FPS: 6812.2
+[2025-04-06 21:45:24,892][29458] Loading existing experiment configuration from /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/config.json
+[2025-04-06 21:45:24,892][29458] Overriding arg 'num_workers' with value 1 passed from command line
+[2025-04-06 21:45:24,892][29458] Adding new argument 'no_render'=True that is not in the saved config file!
+[2025-04-06 21:45:24,893][29458] Adding new argument 'save_video'=True that is not in the saved config file!
+[2025-04-06 21:45:24,894][29458] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
+[2025-04-06 21:45:24,894][29458] Adding new argument 'video_name'=None that is not in the saved config file!
+[2025-04-06 21:45:24,895][29458] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
+[2025-04-06 21:45:24,895][29458] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
+[2025-04-06 21:45:24,896][29458] Adding new argument 'push_to_hub'=False that is not in the saved config file!
+[2025-04-06 21:45:24,896][29458] Adding new argument 'hf_repository'=None that is not in the saved config file!
+[2025-04-06 21:45:24,897][29458] Adding new argument 'policy_index'=0 that is not in the saved config file!
+[2025-04-06 21:45:24,898][29458] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
+[2025-04-06 21:45:24,899][29458] Adding new argument 'train_script'=None that is not in the saved config file!
+[2025-04-06 21:45:24,899][29458] Adding new argument 'enjoy_script'=None that is not in the saved config file!
+[2025-04-06 21:45:24,900][29458] Using frameskip 1 and render_action_repeat=4 for evaluation
+[2025-04-06 21:45:24,939][29458] Doom resolution: 160x120, resize resolution: (128, 72)
+[2025-04-06 21:45:24,942][29458] RunningMeanStd input shape: (3, 72, 128)
+[2025-04-06 21:45:24,944][29458] RunningMeanStd input shape: (1,)
+[2025-04-06 21:45:24,978][29458] ConvEncoder: input_channels=3
+[2025-04-06 21:45:25,206][29458] Conv encoder output size: 512
+[2025-04-06 21:45:25,206][29458] Policy head output size: 512
+[2025-04-06 21:45:25,370][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2025-04-06 21:45:25,373][29458] Could not load from checkpoint, attempt 0
+Traceback (most recent call last):
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
+    checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load
+    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
+_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. 
+	(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
+	(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
+	WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function.
+
+Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
+[2025-04-06 21:45:25,379][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2025-04-06 21:45:25,380][29458] Could not load from checkpoint, attempt 1
+Traceback (most recent call last):
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
+    checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load
+    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
+_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. 
+	(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
+	(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
+	WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function.
+
+Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
+[2025-04-06 21:45:25,380][29458] Loading state from checkpoint /home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
+[2025-04-06 21:45:25,381][29458] Could not load from checkpoint, attempt 2
+Traceback (most recent call last):
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/sample_factory/algo/learning/learner.py", line 281, in load_checkpoint
+    checkpoint_dict = torch.load(latest_checkpoint, map_location=device)
+  File "/home/tguz/Proj/PhD/RL/RL_courses/Hugging-Face-RL/venv/lib/python3.10/site-packages/torch/serialization.py", line 1470, in load
+    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
+_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. 
+	(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
+	(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
+	WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray.scalar was not an allowed global by default. Please use `torch.serialization.add_safe_globals([scalar])` or the `torch.serialization.safe_globals([scalar])` context manager to allowlist this global if you trust this class/function.
+
+Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.