Hugging Face's logo

ezrab
/
rl_course_vizdoom_health_gathering_supreme

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files
xet
Metrics Community
ezrab commited on
Commit
75e3292
·
verified ·
1 Parent(s): 479543e

Upload folder using huggingface_hub

Browse files
Files changed (9) hide show
  1. .gitattributes +1 -0
  2. .summary/0/events.out.tfevents.1745493270.eb5ae90ee160 +3 -0
  3. README.md +56 -0
  4. checkpoint_p0/best_000000913_3739648_reward_26.397.pth +3 -0
  5. checkpoint_p0/checkpoint_000000783_3207168.pth +3 -0
  6. checkpoint_p0/checkpoint_000000978_4005888.pth +3 -0
  7. config.json +142 -0
  8. replay.mp4 +3 -0
  9. sf_log.txt +835 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1745493270.eb5ae90ee160 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:438affdcfa81bb791ca8b53de5220e11ff83ed1843c58349d6057f0494d5b77a
3
+ size 398932
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: doom_health_gathering_supreme
15
+ type: doom_health_gathering_supreme
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 9.09 +/- 4.45
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ A(n) **APPO** model trained on the **doom_health_gathering_supreme** environment.
24
+
25
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
26
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
27
+
28
+
29
+ ## Downloading the model
30
+
31
+ After installing Sample-Factory, download the model with:
32
+ ```
33
+ python -m sample_factory.huggingface.load_from_hub -r ezrab/rl_course_vizdoom_health_gathering_supreme
34
+ ```
35
+
36
+
37
+ ## Using the model
38
+
39
+ To run the model after download, use the `enjoy` script corresponding to this environment:
40
+ ```
41
+ python -m <path.to.enjoy.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme
42
+ ```
43
+
44
+
45
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
46
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
47
+
48
+ ## Training with this model
49
+
50
+ To continue training with this model, use the `train` script corresponding to this environment:
51
+ ```
52
+ python -m <path.to.train.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme --restart_behavior=resume --train_for_env_steps=10000000000
53
+ ```
54
+
55
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
56
+
checkpoint_p0/best_000000913_3739648_reward_26.397.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:140756452eb55b8c4aa6fa7fba9ec77db9a4800fff5f6b3e2e48a4640795cd9f
3
+ size 34929051
checkpoint_p0/checkpoint_000000783_3207168.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1e3c46e2ada18c7a4d07470f6d6dbd09e329a9dc24483042f04842b06ff513e
3
+ size 34929541
checkpoint_p0/checkpoint_000000978_4005888.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcfa30c2af43062fb1dbb47910ed05319636f6fbf20cc9784174b4a30f7316c2
3
+ size 34929477
config.json ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "help": false,
3
+ "algo": "APPO",
4
+ "env": "doom_health_gathering_supreme",
5
+ "experiment": "default_experiment",
6
+ "train_dir": "/kaggle/working/train_dir",
7
+ "restart_behavior": "resume",
8
+ "device": "gpu",
9
+ "seed": null,
10
+ "num_policies": 1,
11
+ "async_rl": true,
12
+ "serial_mode": false,
13
+ "batched_sampling": false,
14
+ "num_batches_to_accumulate": 2,
15
+ "worker_num_splits": 2,
16
+ "policy_workers_per_policy": 1,
17
+ "max_policy_lag": 1000,
18
+ "num_workers": 8,
19
+ "num_envs_per_worker": 4,
20
+ "batch_size": 1024,
21
+ "num_batches_per_epoch": 1,
22
+ "num_epochs": 1,
23
+ "rollout": 32,
24
+ "recurrence": 32,
25
+ "shuffle_minibatches": false,
26
+ "gamma": 0.99,
27
+ "reward_scale": 1.0,
28
+ "reward_clip": 1000.0,
29
+ "value_bootstrap": false,
30
+ "normalize_returns": true,
31
+ "exploration_loss_coeff": 0.001,
32
+ "value_loss_coeff": 0.5,
33
+ "kl_loss_coeff": 0.0,
34
+ "exploration_loss": "symmetric_kl",
35
+ "gae_lambda": 0.95,
36
+ "ppo_clip_ratio": 0.1,
37
+ "ppo_clip_value": 0.2,
38
+ "with_vtrace": false,
39
+ "vtrace_rho": 1.0,
40
+ "vtrace_c": 1.0,
41
+ "optimizer": "adam",
42
+ "adam_eps": 1e-06,
43
+ "adam_beta1": 0.9,
44
+ "adam_beta2": 0.999,
45
+ "max_grad_norm": 4.0,
46
+ "learning_rate": 0.0001,
47
+ "lr_schedule": "constant",
48
+ "lr_schedule_kl_threshold": 0.008,
49
+ "lr_adaptive_min": 1e-06,
50
+ "lr_adaptive_max": 0.01,
51
+ "obs_subtract_mean": 0.0,
52
+ "obs_scale": 255.0,
53
+ "normalize_input": true,
54
+ "normalize_input_keys": null,
55
+ "decorrelate_experience_max_seconds": 0,
56
+ "decorrelate_envs_on_one_worker": true,
57
+ "actor_worker_gpus": [],
58
+ "set_workers_cpu_affinity": true,
59
+ "force_envs_single_thread": false,
60
+ "default_niceness": 0,
61
+ "log_to_file": true,
62
+ "experiment_summaries_interval": 10,
63
+ "flush_summaries_interval": 30,
64
+ "stats_avg": 100,
65
+ "summaries_use_frameskip": true,
66
+ "heartbeat_interval": 20,
67
+ "heartbeat_reporting_interval": 600,
68
+ "train_for_env_steps": 4000000,
69
+ "train_for_seconds": 10000000000,
70
+ "save_every_sec": 120,
71
+ "keep_checkpoints": 2,
72
+ "load_checkpoint_kind": "latest",
73
+ "save_milestones_sec": -1,
74
+ "save_best_every_sec": 5,
75
+ "save_best_metric": "reward",
76
+ "save_best_after": 100000,
77
+ "benchmark": false,
78
+ "encoder_mlp_layers": [
79
+ 512,
80
+ 512
81
+ ],
82
+ "encoder_conv_architecture": "convnet_simple",
83
+ "encoder_conv_mlp_layers": [
84
+ 512
85
+ ],
86
+ "use_rnn": true,
87
+ "rnn_size": 512,
88
+ "rnn_type": "gru",
89
+ "rnn_num_layers": 1,
90
+ "decoder_mlp_layers": [],
91
+ "nonlinearity": "elu",
92
+ "policy_initialization": "orthogonal",
93
+ "policy_init_gain": 1.0,
94
+ "actor_critic_share_weights": true,
95
+ "adaptive_stddev": true,
96
+ "continuous_tanh_scale": 0.0,
97
+ "initial_stddev": 1.0,
98
+ "use_env_info_cache": false,
99
+ "env_gpu_actions": false,
100
+ "env_gpu_observations": true,
101
+ "env_frameskip": 4,
102
+ "env_framestack": 1,
103
+ "pixel_format": "CHW",
104
+ "use_record_episode_statistics": false,
105
+ "with_wandb": false,
106
+ "wandb_user": null,
107
+ "wandb_project": "sample_factory",
108
+ "wandb_group": null,
109
+ "wandb_job_type": "SF",
110
+ "wandb_tags": [],
111
+ "with_pbt": false,
112
+ "pbt_mix_policies_in_one_env": true,
113
+ "pbt_period_env_steps": 5000000,
114
+ "pbt_start_mutation": 20000000,
115
+ "pbt_replace_fraction": 0.3,
116
+ "pbt_mutation_rate": 0.15,
117
+ "pbt_replace_reward_gap": 0.1,
118
+ "pbt_replace_reward_gap_absolute": 1e-06,
119
+ "pbt_optimize_gamma": false,
120
+ "pbt_target_objective": "true_objective",
121
+ "pbt_perturb_min": 1.1,
122
+ "pbt_perturb_max": 1.5,
123
+ "num_agents": -1,
124
+ "num_humans": 0,
125
+ "num_bots": -1,
126
+ "start_bot_difficulty": null,
127
+ "timelimit": null,
128
+ "res_w": 128,
129
+ "res_h": 72,
130
+ "wide_aspect_ratio": false,
131
+ "eval_env_frameskip": 1,
132
+ "fps": 35,
133
+ "command_line": "--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000",
134
+ "cli_args": {
135
+ "env": "doom_health_gathering_supreme",
136
+ "num_workers": 8,
137
+ "num_envs_per_worker": 4,
138
+ "train_for_env_steps": 4000000
139
+ },
140
+ "git_hash": "unknown",
141
+ "git_repo_name": "not a git repository"
142
+ }
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b871711f352dc9265ebba5485955992fc53cbf54e5aa24122f59c2191e40618e
3
+ size 17369696
sf_log.txt ADDED
@@ -0,0 +1,835 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2025-04-24 11:14:34,762][00031] Saving configuration to /kaggle/working/train_dir/default_experiment/config.json...
2
+ [2025-04-24 11:14:34,764][00031] Rollout worker 0 uses device cpu
3
+ [2025-04-24 11:14:34,764][00031] Rollout worker 1 uses device cpu
4
+ [2025-04-24 11:14:34,765][00031] Rollout worker 2 uses device cpu
5
+ [2025-04-24 11:14:34,766][00031] Rollout worker 3 uses device cpu
6
+ [2025-04-24 11:14:34,767][00031] Rollout worker 4 uses device cpu
7
+ [2025-04-24 11:14:34,767][00031] Rollout worker 5 uses device cpu
8
+ [2025-04-24 11:14:34,768][00031] Rollout worker 6 uses device cpu
9
+ [2025-04-24 11:14:34,769][00031] Rollout worker 7 uses device cpu
10
+ [2025-04-24 11:14:34,892][00031] Using GPUs [0] for process 0 (actually maps to GPUs [0])
11
+ [2025-04-24 11:14:34,893][00031] InferenceWorker_p0-w0: min num requests: 2
12
+ [2025-04-24 11:14:34,935][00031] Starting all processes...
13
+ [2025-04-24 11:14:34,936][00031] Starting process learner_proc0
14
+ [2025-04-24 11:14:35,031][00031] Starting all processes...
15
+ [2025-04-24 11:14:35,039][00031] Starting process inference_proc0-0
16
+ [2025-04-24 11:14:35,039][00031] Starting process rollout_proc0
17
+ [2025-04-24 11:14:35,040][00031] Starting process rollout_proc1
18
+ [2025-04-24 11:14:35,041][00031] Starting process rollout_proc2
19
+ [2025-04-24 11:14:35,041][00031] Starting process rollout_proc3
20
+ [2025-04-24 11:14:35,042][00031] Starting process rollout_proc4
21
+ [2025-04-24 11:14:35,042][00031] Starting process rollout_proc5
22
+ [2025-04-24 11:14:35,048][00031] Starting process rollout_proc6
23
+ [2025-04-24 11:14:35,049][00031] Starting process rollout_proc7
24
+ [2025-04-24 11:14:41,781][01244] Worker 7 uses CPU cores [3]
25
+ [2025-04-24 11:14:43,142][01240] Worker 3 uses CPU cores [3]
26
+ [2025-04-24 11:14:43,216][01238] Worker 2 uses CPU cores [2]
27
+ [2025-04-24 11:14:43,250][01223] Using GPUs [0] for process 0 (actually maps to GPUs [0])
28
+ [2025-04-24 11:14:43,251][01223] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
29
+ [2025-04-24 11:14:43,296][01223] Num visible devices: 1
30
+ [2025-04-24 11:14:43,307][01223] Starting seed is not provided
31
+ [2025-04-24 11:14:43,308][01223] Using GPUs [0] for process 0 (actually maps to GPUs [0])
32
+ [2025-04-24 11:14:43,308][01223] Initializing actor-critic model on device cuda:0
33
+ [2025-04-24 11:14:43,308][01223] RunningMeanStd input shape: (3, 72, 128)
34
+ [2025-04-24 11:14:43,312][01223] RunningMeanStd input shape: (1,)
35
+ [2025-04-24 11:14:43,361][01223] ConvEncoder: input_channels=3
36
+ [2025-04-24 11:14:43,494][01243] Worker 6 uses CPU cores [2]
37
+ [2025-04-24 11:14:43,607][01236] Using GPUs [0] for process 0 (actually maps to GPUs [0])
38
+ [2025-04-24 11:14:43,607][01236] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
39
+ [2025-04-24 11:14:43,658][01236] Num visible devices: 1
40
+ [2025-04-24 11:14:43,682][01237] Worker 0 uses CPU cores [0]
41
+ [2025-04-24 11:14:43,712][01241] Worker 4 uses CPU cores [0]
42
+ [2025-04-24 11:14:43,743][01223] Conv encoder output size: 512
43
+ [2025-04-24 11:14:43,743][01223] Policy head output size: 512
44
+ [2025-04-24 11:14:43,764][01242] Worker 5 uses CPU cores [1]
45
+ [2025-04-24 11:14:43,788][01239] Worker 1 uses CPU cores [1]
46
+ [2025-04-24 11:14:43,800][01223] Created Actor Critic model with architecture:
47
+ [2025-04-24 11:14:43,801][01223] ActorCriticSharedWeights(
48
+ (obs_normalizer): ObservationNormalizer(
49
+ (running_mean_std): RunningMeanStdDictInPlace(
50
+ (running_mean_std): ModuleDict(
51
+ (obs): RunningMeanStdInPlace()
52
+ )
53
+ )
54
+ )
55
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
56
+ (encoder): VizdoomEncoder(
57
+ (basic_encoder): ConvEncoder(
58
+ (enc): RecursiveScriptModule(
59
+ original_name=ConvEncoderImpl
60
+ (conv_head): RecursiveScriptModule(
61
+ original_name=Sequential
62
+ (0): RecursiveScriptModule(original_name=Conv2d)
63
+ (1): RecursiveScriptModule(original_name=ELU)
64
+ (2): RecursiveScriptModule(original_name=Conv2d)
65
+ (3): RecursiveScriptModule(original_name=ELU)
66
+ (4): RecursiveScriptModule(original_name=Conv2d)
67
+ (5): RecursiveScriptModule(original_name=ELU)
68
+ )
69
+ (mlp_layers): RecursiveScriptModule(
70
+ original_name=Sequential
71
+ (0): RecursiveScriptModule(original_name=Linear)
72
+ (1): RecursiveScriptModule(original_name=ELU)
73
+ )
74
+ )
75
+ )
76
+ )
77
+ (core): ModelCoreRNN(
78
+ (core): GRU(512, 512)
79
+ )
80
+ (decoder): MlpDecoder(
81
+ (mlp): Identity()
82
+ )
83
+ (critic_linear): Linear(in_features=512, out_features=1, bias=True)
84
+ (action_parameterization): ActionParameterizationDefault(
85
+ (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
86
+ )
87
+ )
88
+ [2025-04-24 11:14:44,053][01223] Using optimizer <class 'torch.optim.adam.Adam'>
89
+ [2025-04-24 11:14:48,057][01223] No checkpoints found
90
+ [2025-04-24 11:14:48,057][01223] Did not load from checkpoint, starting from scratch!
91
+ [2025-04-24 11:14:48,057][01223] Initialized policy 0 weights for model version 0
92
+ [2025-04-24 11:14:48,060][01223] LearnerWorker_p0 finished initialization!
93
+ [2025-04-24 11:14:48,060][01223] Using GPUs [0] for process 0 (actually maps to GPUs [0])
94
+ [2025-04-24 11:14:48,175][01236] RunningMeanStd input shape: (3, 72, 128)
95
+ [2025-04-24 11:14:48,176][01236] RunningMeanStd input shape: (1,)
96
+ [2025-04-24 11:14:48,187][01236] ConvEncoder: input_channels=3
97
+ [2025-04-24 11:14:48,296][01236] Conv encoder output size: 512
98
+ [2025-04-24 11:14:48,296][01236] Policy head output size: 512
99
+ [2025-04-24 11:14:48,352][00031] Inference worker 0-0 is ready!
100
+ [2025-04-24 11:14:48,353][00031] All inference workers are ready! Signal rollout workers to start!
101
+ [2025-04-24 11:14:48,480][01238] Doom resolution: 160x120, resize resolution: (128, 72)
102
+ [2025-04-24 11:14:48,480][01239] Doom resolution: 160x120, resize resolution: (128, 72)
103
+ [2025-04-24 11:14:48,479][01240] Doom resolution: 160x120, resize resolution: (128, 72)
104
+ [2025-04-24 11:14:48,480][01242] Doom resolution: 160x120, resize resolution: (128, 72)
105
+ [2025-04-24 11:14:48,480][01241] Doom resolution: 160x120, resize resolution: (128, 72)
106
+ [2025-04-24 11:14:48,479][01237] Doom resolution: 160x120, resize resolution: (128, 72)
107
+ [2025-04-24 11:14:48,478][01243] Doom resolution: 160x120, resize resolution: (128, 72)
108
+ [2025-04-24 11:14:48,483][01244] Doom resolution: 160x120, resize resolution: (128, 72)
109
+ [2025-04-24 11:14:48,962][01244] Decorrelating experience for 0 frames...
110
+ [2025-04-24 11:14:49,024][01237] Decorrelating experience for 0 frames...
111
+ [2025-04-24 11:14:49,297][01243] Decorrelating experience for 0 frames...
112
+ [2025-04-24 11:14:49,300][01238] Decorrelating experience for 0 frames...
113
+ [2025-04-24 11:14:49,484][01240] Decorrelating experience for 0 frames...
114
+ [2025-04-24 11:14:49,486][01244] Decorrelating experience for 32 frames...
115
+ [2025-04-24 11:14:49,836][01240] Decorrelating experience for 32 frames...
116
+ [2025-04-24 11:14:49,957][01237] Decorrelating experience for 32 frames...
117
+ [2025-04-24 11:14:49,998][01243] Decorrelating experience for 32 frames...
118
+ [2025-04-24 11:14:50,007][01238] Decorrelating experience for 32 frames...
119
+ [2025-04-24 11:14:50,036][00031] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
120
+ [2025-04-24 11:14:50,325][01240] Decorrelating experience for 64 frames...
121
+ [2025-04-24 11:14:50,590][01244] Decorrelating experience for 64 frames...
122
+ [2025-04-24 11:14:50,891][01241] Decorrelating experience for 0 frames...
123
+ [2025-04-24 11:14:50,949][01240] Decorrelating experience for 96 frames...
124
+ [2025-04-24 11:14:51,026][01237] Decorrelating experience for 64 frames...
125
+ [2025-04-24 11:14:51,140][01243] Decorrelating experience for 64 frames...
126
+ [2025-04-24 11:14:51,143][01238] Decorrelating experience for 64 frames...
127
+ [2025-04-24 11:14:51,351][01242] Decorrelating experience for 0 frames...
128
+ [2025-04-24 11:14:51,355][01244] Decorrelating experience for 96 frames...
129
+ [2025-04-24 11:14:51,612][01241] Decorrelating experience for 32 frames...
130
+ [2025-04-24 11:14:51,944][01237] Decorrelating experience for 96 frames...
131
+ [2025-04-24 11:14:52,049][01243] Decorrelating experience for 96 frames...
132
+ [2025-04-24 11:14:52,416][01242] Decorrelating experience for 32 frames...
133
+ [2025-04-24 11:14:52,624][01241] Decorrelating experience for 64 frames...
134
+ [2025-04-24 11:14:53,036][01242] Decorrelating experience for 64 frames...
135
+ [2025-04-24 11:14:53,517][01238] Decorrelating experience for 96 frames...
136
+ [2025-04-24 11:14:53,571][01242] Decorrelating experience for 96 frames...
137
+ [2025-04-24 11:14:53,883][01241] Decorrelating experience for 96 frames...
138
+ [2025-04-24 11:14:54,406][01223] Signal inference workers to stop experience collection...
139
+ [2025-04-24 11:14:54,413][01236] InferenceWorker_p0-w0: stopping experience collection
140
+ [2025-04-24 11:14:54,881][00031] Heartbeat connected on Batcher_0
141
+ [2025-04-24 11:14:54,892][00031] Heartbeat connected on InferenceWorker_p0-w0
142
+ [2025-04-24 11:14:54,901][00031] Heartbeat connected on RolloutWorker_w0
143
+ [2025-04-24 11:14:54,910][00031] Heartbeat connected on RolloutWorker_w2
144
+ [2025-04-24 11:14:54,915][00031] Heartbeat connected on RolloutWorker_w3
145
+ [2025-04-24 11:14:54,920][00031] Heartbeat connected on RolloutWorker_w4
146
+ [2025-04-24 11:14:54,925][00031] Heartbeat connected on RolloutWorker_w5
147
+ [2025-04-24 11:14:54,930][00031] Heartbeat connected on RolloutWorker_w6
148
+ [2025-04-24 11:14:54,935][00031] Heartbeat connected on RolloutWorker_w7
149
+ [2025-04-24 11:14:55,036][00031] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 520.8. Samples: 2604. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
150
+ [2025-04-24 11:14:55,037][00031] Avg episode reward: [(0, '3.606')]
151
+ [2025-04-24 11:14:56,414][01223] Signal inference workers to resume experience collection...
152
+ [2025-04-24 11:14:56,415][01236] InferenceWorker_p0-w0: resuming experience collection
153
+ [2025-04-24 11:14:56,698][00031] Heartbeat connected on LearnerWorker_p0
154
+ [2025-04-24 11:14:59,977][01236] Updated weights for policy 0, policy_version 10 (0.0096)
155
+ [2025-04-24 11:15:00,036][00031] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 4096.0). Total num frames: 40960. Throughput: 0: 458.6. Samples: 4586. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
156
+ [2025-04-24 11:15:00,037][00031] Avg episode reward: [(0, '4.281')]
157
+ [2025-04-24 11:15:04,358][01236] Updated weights for policy 0, policy_version 20 (0.0016)
158
+ [2025-04-24 11:15:05,036][00031] Fps is (10 sec: 8601.5, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 86016. Throughput: 0: 1245.9. Samples: 18688. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
159
+ [2025-04-24 11:15:05,037][00031] Avg episode reward: [(0, '4.305')]
160
+ [2025-04-24 11:15:08,433][01236] Updated weights for policy 0, policy_version 30 (0.0015)
161
+ [2025-04-24 11:15:10,036][00031] Fps is (10 sec: 9420.8, 60 sec: 6758.4, 300 sec: 6758.4). Total num frames: 135168. Throughput: 0: 1674.2. Samples: 33484. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
162
+ [2025-04-24 11:15:10,037][00031] Avg episode reward: [(0, '4.341')]
163
+ [2025-04-24 11:15:10,048][01223] Saving new best policy, reward=4.341!
164
+ [2025-04-24 11:15:12,925][01236] Updated weights for policy 0, policy_version 40 (0.0014)
165
+ [2025-04-24 11:15:15,036][00031] Fps is (10 sec: 9420.9, 60 sec: 7209.0, 300 sec: 7209.0). Total num frames: 180224. Throughput: 0: 1617.0. Samples: 40426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
166
+ [2025-04-24 11:15:15,037][00031] Avg episode reward: [(0, '4.340')]
167
+ [2025-04-24 11:15:17,596][01236] Updated weights for policy 0, policy_version 50 (0.0015)
168
+ [2025-04-24 11:15:20,036][00031] Fps is (10 sec: 9011.0, 60 sec: 7509.3, 300 sec: 7509.3). Total num frames: 225280. Throughput: 0: 1792.9. Samples: 53786. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
169
+ [2025-04-24 11:15:20,037][00031] Avg episode reward: [(0, '4.642')]
170
+ [2025-04-24 11:15:20,047][01223] Saving new best policy, reward=4.642!
171
+ [2025-04-24 11:15:21,863][01236] Updated weights for policy 0, policy_version 60 (0.0015)
172
+ [2025-04-24 11:15:25,036][00031] Fps is (10 sec: 9420.8, 60 sec: 7840.9, 300 sec: 7840.9). Total num frames: 274432. Throughput: 0: 1950.2. Samples: 68256. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
173
+ [2025-04-24 11:15:25,037][00031] Avg episode reward: [(0, '4.390')]
174
+ [2025-04-24 11:15:25,994][01236] Updated weights for policy 0, policy_version 70 (0.0014)
175
+ [2025-04-24 11:15:30,036][00031] Fps is (10 sec: 9830.6, 60 sec: 8089.6, 300 sec: 8089.6). Total num frames: 323584. Throughput: 0: 1892.8. Samples: 75710. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
176
+ [2025-04-24 11:15:30,037][00031] Avg episode reward: [(0, '4.583')]
177
+ [2025-04-24 11:15:30,044][01236] Updated weights for policy 0, policy_version 80 (0.0017)
178
+ [2025-04-24 11:15:34,305][01236] Updated weights for policy 0, policy_version 90 (0.0014)
179
+ [2025-04-24 11:15:35,036][00031] Fps is (10 sec: 9830.4, 60 sec: 8283.0, 300 sec: 8283.0). Total num frames: 372736. Throughput: 0: 2009.9. Samples: 90446. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
180
+ [2025-04-24 11:15:35,038][00031] Avg episode reward: [(0, '4.367')]
181
+ [2025-04-24 11:15:38,477][01236] Updated weights for policy 0, policy_version 100 (0.0015)
182
+ [2025-04-24 11:15:40,036][00031] Fps is (10 sec: 9830.3, 60 sec: 8437.8, 300 sec: 8437.8). Total num frames: 421888. Throughput: 0: 2281.6. Samples: 105276. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
183
+ [2025-04-24 11:15:40,038][00031] Avg episode reward: [(0, '4.761')]
184
+ [2025-04-24 11:15:40,057][01223] Saving new best policy, reward=4.761!
185
+ [2025-04-24 11:15:42,713][01236] Updated weights for policy 0, policy_version 110 (0.0013)
186
+ [2025-04-24 11:15:45,036][00031] Fps is (10 sec: 9830.3, 60 sec: 8564.4, 300 sec: 8564.4). Total num frames: 471040. Throughput: 0: 2395.7. Samples: 112392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
187
+ [2025-04-24 11:15:45,038][00031] Avg episode reward: [(0, '4.426')]
188
+ [2025-04-24 11:15:47,483][01236] Updated weights for policy 0, policy_version 120 (0.0013)
189
+ [2025-04-24 11:15:50,036][00031] Fps is (10 sec: 9420.9, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 516096. Throughput: 0: 2375.8. Samples: 125598. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
190
+ [2025-04-24 11:15:50,037][00031] Avg episode reward: [(0, '4.289')]
191
+ [2025-04-24 11:15:51,700][01236] Updated weights for policy 0, policy_version 130 (0.0016)
192
+ [2025-04-24 11:15:55,036][00031] Fps is (10 sec: 9420.9, 60 sec: 9420.8, 300 sec: 8696.1). Total num frames: 565248. Throughput: 0: 2372.0. Samples: 140226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
193
+ [2025-04-24 11:15:55,038][00031] Avg episode reward: [(0, '4.247')]
194
+ [2025-04-24 11:15:55,898][01236] Updated weights for policy 0, policy_version 140 (0.0014)
195
+ [2025-04-24 11:16:00,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9489.1, 300 sec: 8718.6). Total num frames: 610304. Throughput: 0: 2382.0. Samples: 147618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
196
+ [2025-04-24 11:16:00,039][00031] Avg episode reward: [(0, '4.777')]
197
+ [2025-04-24 11:16:00,076][01223] Saving new best policy, reward=4.777!
198
+ [2025-04-24 11:16:00,078][01236] Updated weights for policy 0, policy_version 150 (0.0016)
199
+ [2025-04-24 11:16:04,302][01236] Updated weights for policy 0, policy_version 160 (0.0015)
200
+ [2025-04-24 11:16:05,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 8792.7). Total num frames: 659456. Throughput: 0: 2406.9. Samples: 162098. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
201
+ [2025-04-24 11:16:05,037][00031] Avg episode reward: [(0, '4.729')]
202
+ [2025-04-24 11:16:08,466][01236] Updated weights for policy 0, policy_version 170 (0.0016)
203
+ [2025-04-24 11:16:10,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9557.3, 300 sec: 8857.6). Total num frames: 708608. Throughput: 0: 2411.7. Samples: 176782. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
204
+ [2025-04-24 11:16:10,037][00031] Avg episode reward: [(0, '4.455')]
205
+ [2025-04-24 11:16:12,695][01236] Updated weights for policy 0, policy_version 180 (0.0013)
206
+ [2025-04-24 11:16:15,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 8914.8). Total num frames: 757760. Throughput: 0: 2408.8. Samples: 184104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
207
+ [2025-04-24 11:16:15,037][00031] Avg episode reward: [(0, '4.411')]
208
+ [2025-04-24 11:16:16,835][01236] Updated weights for policy 0, policy_version 190 (0.0013)
209
+ [2025-04-24 11:16:20,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 8920.2). Total num frames: 802816. Throughput: 0: 2392.0. Samples: 198084. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
210
+ [2025-04-24 11:16:20,040][00031] Avg episode reward: [(0, '4.550')]
211
+ [2025-04-24 11:16:21,489][01236] Updated weights for policy 0, policy_version 200 (0.0014)
212
+ [2025-04-24 11:16:25,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 8968.1). Total num frames: 851968. Throughput: 0: 2374.2. Samples: 212116. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
213
+ [2025-04-24 11:16:25,037][00031] Avg episode reward: [(0, '4.593')]
214
+ [2025-04-24 11:16:25,855][01236] Updated weights for policy 0, policy_version 210 (0.0014)
215
+ [2025-04-24 11:16:29,970][01236] Updated weights for policy 0, policy_version 220 (0.0016)
216
+ [2025-04-24 11:16:30,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9011.2). Total num frames: 901120. Throughput: 0: 2381.2. Samples: 219544. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
217
+ [2025-04-24 11:16:30,037][00031] Avg episode reward: [(0, '4.448')]
218
+ [2025-04-24 11:16:30,044][01223] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000220_901120.pth...
219
+ [2025-04-24 11:16:34,224][01236] Updated weights for policy 0, policy_version 230 (0.0015)
220
+ [2025-04-24 11:16:35,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9011.2). Total num frames: 946176. Throughput: 0: 2408.4. Samples: 233976. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
221
+ [2025-04-24 11:16:35,038][00031] Avg episode reward: [(0, '4.831')]
222
+ [2025-04-24 11:16:35,049][01223] Saving new best policy, reward=4.831!
223
+ [2025-04-24 11:16:38,482][01236] Updated weights for policy 0, policy_version 240 (0.0016)
224
+ [2025-04-24 11:16:40,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9048.4). Total num frames: 995328. Throughput: 0: 2405.8. Samples: 248488. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
225
+ [2025-04-24 11:16:40,038][00031] Avg episode reward: [(0, '4.574')]
226
+ [2025-04-24 11:16:42,697][01236] Updated weights for policy 0, policy_version 250 (0.0015)
227
+ [2025-04-24 11:16:45,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9557.3, 300 sec: 9082.4). Total num frames: 1044480. Throughput: 0: 2402.0. Samples: 255708. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
228
+ [2025-04-24 11:16:45,037][00031] Avg episode reward: [(0, '4.846')]
229
+ [2025-04-24 11:16:45,039][01223] Saving new best policy, reward=4.846!
230
+ [2025-04-24 11:16:46,856][01236] Updated weights for policy 0, policy_version 260 (0.0013)
231
+ [2025-04-24 11:16:50,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9113.6). Total num frames: 1093632. Throughput: 0: 2409.2. Samples: 270514. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
232
+ [2025-04-24 11:16:50,037][00031] Avg episode reward: [(0, '4.838')]
233
+ [2025-04-24 11:16:51,453][01236] Updated weights for policy 0, policy_version 270 (0.0014)
234
+ [2025-04-24 11:16:55,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9109.5). Total num frames: 1138688. Throughput: 0: 2377.1. Samples: 283750. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
235
+ [2025-04-24 11:16:55,037][00031] Avg episode reward: [(0, '5.109')]
236
+ [2025-04-24 11:16:55,039][01223] Saving new best policy, reward=5.109!
237
+ [2025-04-24 11:16:55,786][01236] Updated weights for policy 0, policy_version 280 (0.0015)
238
+ [2025-04-24 11:16:59,968][01236] Updated weights for policy 0, policy_version 290 (0.0016)
239
+ [2025-04-24 11:17:00,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 9137.2). Total num frames: 1187840. Throughput: 0: 2379.0. Samples: 291158. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
240
+ [2025-04-24 11:17:00,037][00031] Avg episode reward: [(0, '5.459')]
241
+ [2025-04-24 11:17:00,046][01223] Saving new best policy, reward=5.459!
242
+ [2025-04-24 11:17:04,249][01236] Updated weights for policy 0, policy_version 300 (0.0015)
243
+ [2025-04-24 11:17:05,036][00031] Fps is (10 sec: 9829.9, 60 sec: 9625.5, 300 sec: 9162.9). Total num frames: 1236992. Throughput: 0: 2391.1. Samples: 305684. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
244
+ [2025-04-24 11:17:05,037][00031] Avg episode reward: [(0, '4.949')]
245
+ [2025-04-24 11:17:08,363][01236] Updated weights for policy 0, policy_version 310 (0.0016)
246
+ [2025-04-24 11:17:10,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9186.7). Total num frames: 1286144. Throughput: 0: 2410.0. Samples: 320564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
247
+ [2025-04-24 11:17:10,037][00031] Avg episode reward: [(0, '5.748')]
248
+ [2025-04-24 11:17:10,046][01223] Saving new best policy, reward=5.748!
249
+ [2025-04-24 11:17:12,515][01236] Updated weights for policy 0, policy_version 320 (0.0015)
250
+ [2025-04-24 11:17:15,036][00031] Fps is (10 sec: 9421.3, 60 sec: 9557.3, 300 sec: 9180.7). Total num frames: 1331200. Throughput: 0: 2404.3. Samples: 327736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
251
+ [2025-04-24 11:17:15,037][00031] Avg episode reward: [(0, '5.493')]
252
+ [2025-04-24 11:17:16,654][01236] Updated weights for policy 0, policy_version 330 (0.0015)
253
+ [2025-04-24 11:17:20,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 9202.3). Total num frames: 1380352. Throughput: 0: 2412.3. Samples: 342530. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
254
+ [2025-04-24 11:17:20,037][00031] Avg episode reward: [(0, '5.467')]
255
+ [2025-04-24 11:17:20,898][01236] Updated weights for policy 0, policy_version 340 (0.0015)
256
+ [2025-04-24 11:17:25,036][00031] Fps is (10 sec: 9420.4, 60 sec: 9557.3, 300 sec: 9196.2). Total num frames: 1425408. Throughput: 0: 2388.3. Samples: 355964. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
257
+ [2025-04-24 11:17:25,039][00031] Avg episode reward: [(0, '6.098')]
258
+ [2025-04-24 11:17:25,040][01223] Saving new best policy, reward=6.098!
259
+ [2025-04-24 11:17:25,745][01236] Updated weights for policy 0, policy_version 350 (0.0012)
260
+ [2025-04-24 11:17:29,847][01236] Updated weights for policy 0, policy_version 360 (0.0014)
261
+ [2025-04-24 11:17:30,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9216.0). Total num frames: 1474560. Throughput: 0: 2383.1. Samples: 362948. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
262
+ [2025-04-24 11:17:30,037][00031] Avg episode reward: [(0, '6.238')]
263
+ [2025-04-24 11:17:30,047][01223] Saving new best policy, reward=6.238!
264
+ [2025-04-24 11:17:34,043][01236] Updated weights for policy 0, policy_version 370 (0.0014)
265
+ [2025-04-24 11:17:35,036][00031] Fps is (10 sec: 9830.9, 60 sec: 9625.6, 300 sec: 9234.6). Total num frames: 1523712. Throughput: 0: 2380.4. Samples: 377634. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
266
+ [2025-04-24 11:17:35,038][00031] Avg episode reward: [(0, '6.331')]
267
+ [2025-04-24 11:17:35,039][01223] Saving new best policy, reward=6.331!
268
+ [2025-04-24 11:17:38,233][01236] Updated weights for policy 0, policy_version 380 (0.0017)
269
+ [2025-04-24 11:17:40,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9252.1). Total num frames: 1572864. Throughput: 0: 2414.0. Samples: 392380. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
270
+ [2025-04-24 11:17:40,037][00031] Avg episode reward: [(0, '7.161')]
271
+ [2025-04-24 11:17:40,048][01223] Saving new best policy, reward=7.161!
272
+ [2025-04-24 11:17:42,489][01236] Updated weights for policy 0, policy_version 390 (0.0013)
273
+ [2025-04-24 11:17:45,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9268.7). Total num frames: 1622016. Throughput: 0: 2408.5. Samples: 399542. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
274
+ [2025-04-24 11:17:45,037][00031] Avg episode reward: [(0, '7.800')]
275
+ [2025-04-24 11:17:45,039][01223] Saving new best policy, reward=7.800!
276
+ [2025-04-24 11:17:46,697][01236] Updated weights for policy 0, policy_version 400 (0.0014)
277
+ [2025-04-24 11:17:50,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9261.5). Total num frames: 1667072. Throughput: 0: 2411.9. Samples: 414220. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
278
+ [2025-04-24 11:17:50,037][00031] Avg episode reward: [(0, '7.745')]
279
+ [2025-04-24 11:17:50,826][01236] Updated weights for policy 0, policy_version 410 (0.0015)
280
+ [2025-04-24 11:17:54,951][01236] Updated weights for policy 0, policy_version 420 (0.0013)
281
+ [2025-04-24 11:17:55,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9299.0). Total num frames: 1720320. Throughput: 0: 2411.9. Samples: 429098. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
282
+ [2025-04-24 11:17:55,037][00031] Avg episode reward: [(0, '7.834')]
283
+ [2025-04-24 11:17:55,039][01223] Saving new best policy, reward=7.834!
284
+ [2025-04-24 11:17:59,777][01236] Updated weights for policy 0, policy_version 430 (0.0015)
285
+ [2025-04-24 11:18:00,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9269.9). Total num frames: 1761280. Throughput: 0: 2381.0. Samples: 434882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
286
+ [2025-04-24 11:18:00,037][00031] Avg episode reward: [(0, '8.294')]
287
+ [2025-04-24 11:18:00,047][01223] Saving new best policy, reward=8.294!
288
+ [2025-04-24 11:18:04,100][01236] Updated weights for policy 0, policy_version 440 (0.0015)
289
+ [2025-04-24 11:18:05,036][00031] Fps is (10 sec: 9011.2, 60 sec: 9557.4, 300 sec: 9284.3). Total num frames: 1810432. Throughput: 0: 2374.3. Samples: 449374. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
290
+ [2025-04-24 11:18:05,037][00031] Avg episode reward: [(0, '8.266')]
291
+ [2025-04-24 11:18:08,228][01236] Updated weights for policy 0, policy_version 450 (0.0016)
292
+ [2025-04-24 11:18:10,036][00031] Fps is (10 sec: 9830.3, 60 sec: 9557.3, 300 sec: 9297.9). Total num frames: 1859584. Throughput: 0: 2405.9. Samples: 464230. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
293
+ [2025-04-24 11:18:10,038][00031] Avg episode reward: [(0, '8.283')]
294
+ [2025-04-24 11:18:12,461][01236] Updated weights for policy 0, policy_version 460 (0.0014)
295
+ [2025-04-24 11:18:15,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9310.9). Total num frames: 1908736. Throughput: 0: 2411.0. Samples: 471442. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
296
+ [2025-04-24 11:18:15,037][00031] Avg episode reward: [(0, '8.972')]
297
+ [2025-04-24 11:18:15,039][01223] Saving new best policy, reward=8.972!
298
+ [2025-04-24 11:18:16,676][01236] Updated weights for policy 0, policy_version 470 (0.0016)
299
+ [2025-04-24 11:18:20,036][00031] Fps is (10 sec: 9830.5, 60 sec: 9625.6, 300 sec: 9323.3). Total num frames: 1957888. Throughput: 0: 2409.0. Samples: 486040. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
300
+ [2025-04-24 11:18:20,037][00031] Avg episode reward: [(0, '9.329')]
301
+ [2025-04-24 11:18:20,048][01223] Saving new best policy, reward=9.329!
302
+ [2025-04-24 11:18:20,906][01236] Updated weights for policy 0, policy_version 480 (0.0015)
303
+ [2025-04-24 11:18:25,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.7, 300 sec: 9316.0). Total num frames: 2002944. Throughput: 0: 2404.4. Samples: 500576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
304
+ [2025-04-24 11:18:25,038][00031] Avg episode reward: [(0, '9.135')]
305
+ [2025-04-24 11:18:25,072][01236] Updated weights for policy 0, policy_version 490 (0.0014)
306
+ [2025-04-24 11:18:29,782][01236] Updated weights for policy 0, policy_version 500 (0.0014)
307
+ [2025-04-24 11:18:30,038][00031] Fps is (10 sec: 9009.6, 60 sec: 9557.0, 300 sec: 9309.0). Total num frames: 2048000. Throughput: 0: 2409.6. Samples: 507980. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
308
+ [2025-04-24 11:18:30,039][00031] Avg episode reward: [(0, '10.339')]
309
+ [2025-04-24 11:18:30,048][01223] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000500_2048000.pth...
310
+ [2025-04-24 11:18:30,136][01223] Saving new best policy, reward=10.339!
311
+ [2025-04-24 11:18:34,129][01236] Updated weights for policy 0, policy_version 510 (0.0012)
312
+ [2025-04-24 11:18:35,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9557.3, 300 sec: 9320.7). Total num frames: 2097152. Throughput: 0: 2371.7. Samples: 520948. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
313
+ [2025-04-24 11:18:35,037][00031] Avg episode reward: [(0, '10.323')]
314
+ [2025-04-24 11:18:38,241][01236] Updated weights for policy 0, policy_version 520 (0.0016)
315
+ [2025-04-24 11:18:40,036][00031] Fps is (10 sec: 9832.2, 60 sec: 9557.3, 300 sec: 9331.8). Total num frames: 2146304. Throughput: 0: 2372.6. Samples: 535864. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
316
+ [2025-04-24 11:18:40,037][00031] Avg episode reward: [(0, '11.560')]
317
+ [2025-04-24 11:18:40,048][01223] Saving new best policy, reward=11.560!
318
+ [2025-04-24 11:18:42,462][01236] Updated weights for policy 0, policy_version 530 (0.0015)
319
+ [2025-04-24 11:18:45,036][00031] Fps is (10 sec: 9830.2, 60 sec: 9557.3, 300 sec: 9342.4). Total num frames: 2195456. Throughput: 0: 2403.9. Samples: 543058. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
320
+ [2025-04-24 11:18:45,038][00031] Avg episode reward: [(0, '11.406')]
321
+ [2025-04-24 11:18:46,577][01236] Updated weights for policy 0, policy_version 540 (0.0014)
322
+ [2025-04-24 11:18:50,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9352.5). Total num frames: 2244608. Throughput: 0: 2412.4. Samples: 557934. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
323
+ [2025-04-24 11:18:50,037][00031] Avg episode reward: [(0, '13.607')]
324
+ [2025-04-24 11:18:50,046][01223] Saving new best policy, reward=13.607!
325
+ [2025-04-24 11:18:50,738][01236] Updated weights for policy 0, policy_version 550 (0.0014)
326
+ [2025-04-24 11:18:54,950][01236] Updated weights for policy 0, policy_version 560 (0.0016)
327
+ [2025-04-24 11:18:55,036][00031] Fps is (10 sec: 9830.2, 60 sec: 9557.3, 300 sec: 9362.3). Total num frames: 2293760. Throughput: 0: 2409.2. Samples: 572644. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
328
+ [2025-04-24 11:18:55,037][00031] Avg episode reward: [(0, '13.353')]
329
+ [2025-04-24 11:18:59,000][01236] Updated weights for policy 0, policy_version 570 (0.0012)
330
+ [2025-04-24 11:19:00,036][00031] Fps is (10 sec: 9830.1, 60 sec: 9693.8, 300 sec: 9371.6). Total num frames: 2342912. Throughput: 0: 2413.8. Samples: 580066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
331
+ [2025-04-24 11:19:00,037][00031] Avg episode reward: [(0, '14.495')]
332
+ [2025-04-24 11:19:00,049][01223] Saving new best policy, reward=14.495!
333
+ [2025-04-24 11:19:03,858][01236] Updated weights for policy 0, policy_version 580 (0.0013)
334
+ [2025-04-24 11:19:05,036][00031] Fps is (10 sec: 9011.6, 60 sec: 9557.3, 300 sec: 9348.5). Total num frames: 2383872. Throughput: 0: 2382.8. Samples: 593264. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
335
+ [2025-04-24 11:19:05,037][00031] Avg episode reward: [(0, '15.183')]
336
+ [2025-04-24 11:19:05,039][01223] Saving new best policy, reward=15.183!
337
+ [2025-04-24 11:19:08,089][01236] Updated weights for policy 0, policy_version 590 (0.0015)
338
+ [2025-04-24 11:19:10,036][00031] Fps is (10 sec: 9011.3, 60 sec: 9557.3, 300 sec: 9357.8). Total num frames: 2433024. Throughput: 0: 2389.1. Samples: 608084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
339
+ [2025-04-24 11:19:10,038][00031] Avg episode reward: [(0, '16.189')]
340
+ [2025-04-24 11:19:10,049][01223] Saving new best policy, reward=16.189!
341
+ [2025-04-24 11:19:12,261][01236] Updated weights for policy 0, policy_version 600 (0.0016)
342
+ [2025-04-24 11:19:15,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9557.3, 300 sec: 9366.7). Total num frames: 2482176. Throughput: 0: 2385.9. Samples: 615340. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
343
+ [2025-04-24 11:19:15,038][00031] Avg episode reward: [(0, '16.561')]
344
+ [2025-04-24 11:19:15,078][01223] Saving new best policy, reward=16.561!
345
+ [2025-04-24 11:19:16,318][01236] Updated weights for policy 0, policy_version 610 (0.0014)
346
+ [2025-04-24 11:19:20,036][00031] Fps is (10 sec: 9830.5, 60 sec: 9557.3, 300 sec: 9375.3). Total num frames: 2531328. Throughput: 0: 2427.4. Samples: 630182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
347
+ [2025-04-24 11:19:20,039][00031] Avg episode reward: [(0, '17.174')]
348
+ [2025-04-24 11:19:20,052][01223] Saving new best policy, reward=17.174!
349
+ [2025-04-24 11:19:20,501][01236] Updated weights for policy 0, policy_version 620 (0.0016)
350
+ [2025-04-24 11:19:24,772][01236] Updated weights for policy 0, policy_version 630 (0.0017)
351
+ [2025-04-24 11:19:25,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9383.6). Total num frames: 2580480. Throughput: 0: 2420.2. Samples: 644774. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
352
+ [2025-04-24 11:19:25,037][00031] Avg episode reward: [(0, '15.530')]
353
+ [2025-04-24 11:19:28,769][01236] Updated weights for policy 0, policy_version 640 (0.0014)
354
+ [2025-04-24 11:19:30,036][00031] Fps is (10 sec: 9830.3, 60 sec: 9694.1, 300 sec: 9391.5). Total num frames: 2629632. Throughput: 0: 2425.2. Samples: 652194. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
355
+ [2025-04-24 11:19:30,039][00031] Avg episode reward: [(0, '16.167')]
356
+ [2025-04-24 11:19:33,010][01236] Updated weights for policy 0, policy_version 650 (0.0017)
357
+ [2025-04-24 11:19:35,038][00031] Fps is (10 sec: 9419.1, 60 sec: 9625.3, 300 sec: 9384.8). Total num frames: 2674688. Throughput: 0: 2420.1. Samples: 666842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
358
+ [2025-04-24 11:19:35,039][00031] Avg episode reward: [(0, '17.109')]
359
+ [2025-04-24 11:19:37,731][01236] Updated weights for policy 0, policy_version 660 (0.0016)
360
+ [2025-04-24 11:19:40,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 9392.5). Total num frames: 2723840. Throughput: 0: 2395.0. Samples: 680418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
361
+ [2025-04-24 11:19:40,037][00031] Avg episode reward: [(0, '18.018')]
362
+ [2025-04-24 11:19:40,049][01223] Saving new best policy, reward=18.018!
363
+ [2025-04-24 11:19:41,966][01236] Updated weights for policy 0, policy_version 670 (0.0014)
364
+ [2025-04-24 11:19:45,036][00031] Fps is (10 sec: 9832.1, 60 sec: 9625.6, 300 sec: 9400.0). Total num frames: 2772992. Throughput: 0: 2388.9. Samples: 687564. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
365
+ [2025-04-24 11:19:45,037][00031] Avg episode reward: [(0, '20.164')]
366
+ [2025-04-24 11:19:45,039][01223] Saving new best policy, reward=20.164!
367
+ [2025-04-24 11:19:46,146][01236] Updated weights for policy 0, policy_version 680 (0.0013)
368
+ [2025-04-24 11:19:50,036][00031] Fps is (10 sec: 9830.6, 60 sec: 9625.6, 300 sec: 9566.6). Total num frames: 2822144. Throughput: 0: 2419.2. Samples: 702128. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
369
+ [2025-04-24 11:19:50,037][00031] Avg episode reward: [(0, '21.040')]
370
+ [2025-04-24 11:19:50,049][01223] Saving new best policy, reward=21.040!
371
+ [2025-04-24 11:19:50,364][01236] Updated weights for policy 0, policy_version 690 (0.0016)
372
+ [2025-04-24 11:19:54,497][01236] Updated weights for policy 0, policy_version 700 (0.0013)
373
+ [2025-04-24 11:19:55,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.7, 300 sec: 9594.4). Total num frames: 2871296. Throughput: 0: 2418.1. Samples: 716898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
374
+ [2025-04-24 11:19:55,037][00031] Avg episode reward: [(0, '19.314')]
375
+ [2025-04-24 11:19:58,638][01236] Updated weights for policy 0, policy_version 710 (0.0013)
376
+ [2025-04-24 11:20:00,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9608.2). Total num frames: 2920448. Throughput: 0: 2421.2. Samples: 724294. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
377
+ [2025-04-24 11:20:00,038][00031] Avg episode reward: [(0, '17.601')]
378
+ [2025-04-24 11:20:02,902][01236] Updated weights for policy 0, policy_version 720 (0.0018)
379
+ [2025-04-24 11:20:05,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.1, 300 sec: 9608.2). Total num frames: 2969600. Throughput: 0: 2419.0. Samples: 739038. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
380
+ [2025-04-24 11:20:05,037][00031] Avg episode reward: [(0, '18.124')]
381
+ [2025-04-24 11:20:07,403][01236] Updated weights for policy 0, policy_version 730 (0.0015)
382
+ [2025-04-24 11:20:10,036][00031] Fps is (10 sec: 9011.2, 60 sec: 9625.6, 300 sec: 9594.4). Total num frames: 3010560. Throughput: 0: 2392.8. Samples: 752448. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
383
+ [2025-04-24 11:20:10,037][00031] Avg episode reward: [(0, '19.417')]
384
+ [2025-04-24 11:20:11,836][01236] Updated weights for policy 0, policy_version 740 (0.0018)
385
+ [2025-04-24 11:20:15,036][00031] Fps is (10 sec: 9011.2, 60 sec: 9625.6, 300 sec: 9608.3). Total num frames: 3059712. Throughput: 0: 2387.1. Samples: 759612. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
386
+ [2025-04-24 11:20:15,037][00031] Avg episode reward: [(0, '21.493')]
387
+ [2025-04-24 11:20:15,040][01223] Saving new best policy, reward=21.493!
388
+ [2025-04-24 11:20:16,030][01236] Updated weights for policy 0, policy_version 750 (0.0015)
389
+ [2025-04-24 11:20:20,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9608.2). Total num frames: 3108864. Throughput: 0: 2388.9. Samples: 774340. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
390
+ [2025-04-24 11:20:20,037][00031] Avg episode reward: [(0, '20.920')]
391
+ [2025-04-24 11:20:20,186][01236] Updated weights for policy 0, policy_version 760 (0.0015)
392
+ [2025-04-24 11:20:24,313][01236] Updated weights for policy 0, policy_version 770 (0.0016)
393
+ [2025-04-24 11:20:25,037][00031] Fps is (10 sec: 9829.6, 60 sec: 9625.5, 300 sec: 9608.2). Total num frames: 3158016. Throughput: 0: 2414.9. Samples: 789092. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
394
+ [2025-04-24 11:20:25,038][00031] Avg episode reward: [(0, '21.647')]
395
+ [2025-04-24 11:20:25,040][01223] Saving new best policy, reward=21.647!
396
+ [2025-04-24 11:20:28,463][01236] Updated weights for policy 0, policy_version 780 (0.0013)
397
+ [2025-04-24 11:20:30,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9608.2). Total num frames: 3207168. Throughput: 0: 2421.1. Samples: 796512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
398
+ [2025-04-24 11:20:30,037][00031] Avg episode reward: [(0, '21.550')]
399
+ [2025-04-24 11:20:30,052][01223] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000783_3207168.pth...
400
+ [2025-04-24 11:20:30,138][01223] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000220_901120.pth
401
+ [2025-04-24 11:20:32,688][01236] Updated weights for policy 0, policy_version 790 (0.0013)
402
+ [2025-04-24 11:20:35,036][00031] Fps is (10 sec: 9830.8, 60 sec: 9694.1, 300 sec: 9608.2). Total num frames: 3256320. Throughput: 0: 2427.4. Samples: 811360. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
403
+ [2025-04-24 11:20:35,039][00031] Avg episode reward: [(0, '22.633')]
404
+ [2025-04-24 11:20:35,093][01223] Saving new best policy, reward=22.633!
405
+ [2025-04-24 11:20:36,798][01236] Updated weights for policy 0, policy_version 800 (0.0016)
406
+ [2025-04-24 11:20:40,036][00031] Fps is (10 sec: 9420.1, 60 sec: 9625.5, 300 sec: 9594.3). Total num frames: 3301376. Throughput: 0: 2416.1. Samples: 825626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
407
+ [2025-04-24 11:20:40,039][00031] Avg episode reward: [(0, '23.292')]
408
+ [2025-04-24 11:20:40,075][01223] Saving new best policy, reward=23.292!
409
+ [2025-04-24 11:20:41,568][01236] Updated weights for policy 0, policy_version 810 (0.0018)
410
+ [2025-04-24 11:20:45,036][00031] Fps is (10 sec: 9421.2, 60 sec: 9625.6, 300 sec: 9608.2). Total num frames: 3350528. Throughput: 0: 2392.4. Samples: 831950. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
411
+ [2025-04-24 11:20:45,038][00031] Avg episode reward: [(0, '22.740')]
412
+ [2025-04-24 11:20:45,678][01236] Updated weights for policy 0, policy_version 820 (0.0015)
413
+ [2025-04-24 11:20:49,735][01236] Updated weights for policy 0, policy_version 830 (0.0013)
414
+ [2025-04-24 11:20:50,036][00031] Fps is (10 sec: 9831.1, 60 sec: 9625.6, 300 sec: 9608.2). Total num frames: 3399680. Throughput: 0: 2398.7. Samples: 846980. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
415
+ [2025-04-24 11:20:50,039][00031] Avg episode reward: [(0, '23.628')]
416
+ [2025-04-24 11:20:50,046][01223] Saving new best policy, reward=23.628!
417
+ [2025-04-24 11:20:53,889][01236] Updated weights for policy 0, policy_version 840 (0.0013)
418
+ [2025-04-24 11:20:55,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9622.1). Total num frames: 3448832. Throughput: 0: 2429.9. Samples: 861792. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
419
+ [2025-04-24 11:20:55,038][00031] Avg episode reward: [(0, '23.406')]
420
+ [2025-04-24 11:20:58,035][01236] Updated weights for policy 0, policy_version 850 (0.0013)
421
+ [2025-04-24 11:21:00,036][00031] Fps is (10 sec: 10240.0, 60 sec: 9693.9, 300 sec: 9636.0). Total num frames: 3502080. Throughput: 0: 2437.4. Samples: 869294. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
422
+ [2025-04-24 11:21:00,037][00031] Avg episode reward: [(0, '21.158')]
423
+ [2025-04-24 11:21:02,279][01236] Updated weights for policy 0, policy_version 860 (0.0015)
424
+ [2025-04-24 11:21:05,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9622.1). Total num frames: 3547136. Throughput: 0: 2436.2. Samples: 883968. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
425
+ [2025-04-24 11:21:05,037][00031] Avg episode reward: [(0, '21.758')]
426
+ [2025-04-24 11:21:06,356][01236] Updated weights for policy 0, policy_version 870 (0.0016)
427
+ [2025-04-24 11:21:10,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9830.4, 300 sec: 9636.0). Total num frames: 3600384. Throughput: 0: 2441.6. Samples: 898962. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
428
+ [2025-04-24 11:21:10,037][00031] Avg episode reward: [(0, '22.299')]
429
+ [2025-04-24 11:21:10,455][01236] Updated weights for policy 0, policy_version 880 (0.0016)
430
+ [2025-04-24 11:21:15,036][00031] Fps is (10 sec: 9420.8, 60 sec: 9693.9, 300 sec: 9622.1). Total num frames: 3641344. Throughput: 0: 2421.4. Samples: 905476. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
431
+ [2025-04-24 11:21:15,037][00031] Avg episode reward: [(0, '23.000')]
432
+ [2025-04-24 11:21:15,210][01236] Updated weights for policy 0, policy_version 890 (0.0015)
433
+ [2025-04-24 11:21:19,296][01236] Updated weights for policy 0, policy_version 900 (0.0014)
434
+ [2025-04-24 11:21:20,036][00031] Fps is (10 sec: 9011.2, 60 sec: 9693.9, 300 sec: 9622.1). Total num frames: 3690496. Throughput: 0: 2410.6. Samples: 919834. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
435
+ [2025-04-24 11:21:20,037][00031] Avg episode reward: [(0, '25.459')]
436
+ [2025-04-24 11:21:20,047][01223] Saving new best policy, reward=25.459!
437
+ [2025-04-24 11:21:23,458][01236] Updated weights for policy 0, policy_version 910 (0.0014)
438
+ [2025-04-24 11:21:25,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9694.0, 300 sec: 9622.1). Total num frames: 3739648. Throughput: 0: 2420.4. Samples: 934542. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
439
+ [2025-04-24 11:21:25,037][00031] Avg episode reward: [(0, '26.397')]
440
+ [2025-04-24 11:21:25,040][01223] Saving new best policy, reward=26.397!
441
+ [2025-04-24 11:21:27,600][01236] Updated weights for policy 0, policy_version 920 (0.0013)
442
+ [2025-04-24 11:21:30,036][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9636.0). Total num frames: 3788800. Throughput: 0: 2446.1. Samples: 942026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
443
+ [2025-04-24 11:21:30,037][00031] Avg episode reward: [(0, '24.623')]
444
+ [2025-04-24 11:21:31,732][01236] Updated weights for policy 0, policy_version 930 (0.0014)
445
+ [2025-04-24 11:21:35,036][00031] Fps is (10 sec: 9830.3, 60 sec: 9693.9, 300 sec: 9636.0). Total num frames: 3837952. Throughput: 0: 2440.1. Samples: 956784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
446
+ [2025-04-24 11:21:35,039][00031] Avg episode reward: [(0, '24.444')]
447
+ [2025-04-24 11:21:35,974][01236] Updated weights for policy 0, policy_version 940 (0.0014)
448
+ [2025-04-24 11:21:39,933][01236] Updated weights for policy 0, policy_version 950 (0.0014)
449
+ [2025-04-24 11:21:40,036][00031] Fps is (10 sec: 10240.1, 60 sec: 9830.5, 300 sec: 9649.9). Total num frames: 3891200. Throughput: 0: 2447.3. Samples: 971922. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
450
+ [2025-04-24 11:21:40,037][00031] Avg episode reward: [(0, '23.657')]
451
+ [2025-04-24 11:21:44,159][01236] Updated weights for policy 0, policy_version 960 (0.0018)
452
+ [2025-04-24 11:21:45,036][00031] Fps is (10 sec: 9830.5, 60 sec: 9762.1, 300 sec: 9636.0). Total num frames: 3936256. Throughput: 0: 2444.3. Samples: 979288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
453
+ [2025-04-24 11:21:45,038][00031] Avg episode reward: [(0, '22.433')]
454
+ [2025-04-24 11:21:48,790][01236] Updated weights for policy 0, policy_version 970 (0.0014)
455
+ [2025-04-24 11:21:50,036][00031] Fps is (10 sec: 9010.6, 60 sec: 9693.8, 300 sec: 9636.0). Total num frames: 3981312. Throughput: 0: 2417.5. Samples: 992756. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
456
+ [2025-04-24 11:21:50,039][00031] Avg episode reward: [(0, '22.986')]
457
+ [2025-04-24 11:21:52,139][01223] Stopping Batcher_0...
458
+ [2025-04-24 11:21:52,140][01223] Loop batcher_evt_loop terminating...
459
+ [2025-04-24 11:21:52,139][00031] Component Batcher_0 stopped!
460
+ [2025-04-24 11:21:52,141][01223] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
461
+ [2025-04-24 11:21:52,142][00031] Component RolloutWorker_w1 process died already! Don't wait for it.
462
+ [2025-04-24 11:21:52,173][01236] Weights refcount: 2 0
463
+ [2025-04-24 11:21:52,176][01236] Stopping InferenceWorker_p0-w0...
464
+ [2025-04-24 11:21:52,176][01236] Loop inference_proc0-0_evt_loop terminating...
465
+ [2025-04-24 11:21:52,175][00031] Component InferenceWorker_p0-w0 stopped!
466
+ [2025-04-24 11:21:52,226][01223] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000500_2048000.pth
467
+ [2025-04-24 11:21:52,230][01244] Stopping RolloutWorker_w7...
468
+ [2025-04-24 11:21:52,231][01244] Loop rollout_proc7_evt_loop terminating...
469
+ [2025-04-24 11:21:52,231][00031] Component RolloutWorker_w7 stopped!
470
+ [2025-04-24 11:21:52,237][01223] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
471
+ [2025-04-24 11:21:52,240][01241] Stopping RolloutWorker_w4...
472
+ [2025-04-24 11:21:52,240][01240] Stopping RolloutWorker_w3...
473
+ [2025-04-24 11:21:52,240][00031] Component RolloutWorker_w4 stopped!
474
+ [2025-04-24 11:21:52,241][01240] Loop rollout_proc3_evt_loop terminating...
475
+ [2025-04-24 11:21:52,242][00031] Component RolloutWorker_w3 stopped!
476
+ [2025-04-24 11:21:52,245][01241] Loop rollout_proc4_evt_loop terminating...
477
+ [2025-04-24 11:21:52,248][01237] Stopping RolloutWorker_w0...
478
+ [2025-04-24 11:21:52,248][01237] Loop rollout_proc0_evt_loop terminating...
479
+ [2025-04-24 11:21:52,249][00031] Component RolloutWorker_w0 stopped!
480
+ [2025-04-24 11:21:52,347][01242] Stopping RolloutWorker_w5...
481
+ [2025-04-24 11:21:52,347][00031] Component RolloutWorker_w5 stopped!
482
+ [2025-04-24 11:21:52,350][01242] Loop rollout_proc5_evt_loop terminating...
483
+ [2025-04-24 11:21:52,375][01223] Stopping LearnerWorker_p0...
484
+ [2025-04-24 11:21:52,375][01223] Loop learner_proc0_evt_loop terminating...
485
+ [2025-04-24 11:21:52,375][00031] Component LearnerWorker_p0 stopped!
486
+ [2025-04-24 11:21:52,415][00031] Component RolloutWorker_w2 stopped!
487
+ [2025-04-24 11:21:52,416][01238] Stopping RolloutWorker_w2...
488
+ [2025-04-24 11:21:52,417][01238] Loop rollout_proc2_evt_loop terminating...
489
+ [2025-04-24 11:21:52,430][00031] Component RolloutWorker_w6 stopped!
490
+ [2025-04-24 11:21:52,431][00031] Waiting for process learner_proc0 to stop...
491
+ [2025-04-24 11:21:52,432][01243] Stopping RolloutWorker_w6...
492
+ [2025-04-24 11:21:52,435][01243] Loop rollout_proc6_evt_loop terminating...
493
+ [2025-04-24 11:21:53,727][00031] Waiting for process inference_proc0-0 to join...
494
+ [2025-04-24 11:21:53,729][00031] Waiting for process rollout_proc0 to join...
495
+ [2025-04-24 11:21:54,073][00031] Waiting for process rollout_proc1 to join...
496
+ [2025-04-24 11:21:54,074][00031] Waiting for process rollout_proc2 to join...
497
+ [2025-04-24 11:21:54,245][00031] Waiting for process rollout_proc3 to join...
498
+ [2025-04-24 11:21:54,246][00031] Waiting for process rollout_proc4 to join...
499
+ [2025-04-24 11:21:54,247][00031] Waiting for process rollout_proc5 to join...
500
+ [2025-04-24 11:21:54,248][00031] Waiting for process rollout_proc6 to join...
501
+ [2025-04-24 11:21:54,249][00031] Waiting for process rollout_proc7 to join...
502
+ [2025-04-24 11:21:54,250][00031] Batcher 0 profile tree view:
503
+ batching: 21.7457, releasing_batches: 0.0229
504
+ [2025-04-24 11:21:54,251][00031] InferenceWorker_p0-w0 profile tree view:
505
+ wait_policy: 0.0023
506
+ wait_policy_total: 12.9770
507
+ update_model: 5.9709
508
+ weight_update: 0.0014
509
+ one_step: 0.0026
510
+ handle_policy_step: 384.4620
511
+ deserialize: 11.1353, stack: 2.4861, obs_to_device_normalize: 95.0258, forward: 186.4066, send_messages: 19.1418
512
+ prepare_outputs: 54.3857
513
+ to_cpu: 35.6634
514
+ [2025-04-24 11:21:54,252][00031] Learner 0 profile tree view:
515
+ misc: 0.0034, prepare_batch: 11.9075
516
+ train: 48.2229
517
+ epoch_init: 0.0046, minibatch_init: 0.0058, losses_postprocess: 0.5200, kl_divergence: 0.5248, after_optimizer: 21.2001
518
+ calculate_losses: 16.3529
519
+ losses_init: 0.0034, forward_head: 0.9508, bptt_initial: 11.4578, tail: 0.6834, advantages_returns: 0.1799, losses: 1.5829
520
+ bptt: 1.3280
521
+ bptt_forward_core: 1.2728
522
+ update: 9.2565
523
+ clip: 0.7626
524
+ [2025-04-24 11:21:54,253][00031] RolloutWorker_w0 profile tree view:
525
+ wait_for_trajectories: 0.1670, enqueue_policy_requests: 8.7871, env_step: 305.9285, overhead: 7.1967, complete_rollouts: 1.0693
526
+ save_policy_outputs: 10.1860
527
+ split_output_tensors: 3.8644
528
+ [2025-04-24 11:21:54,254][00031] RolloutWorker_w7 profile tree view:
529
+ wait_for_trajectories: 0.1746, enqueue_policy_requests: 8.6829, env_step: 294.7375, overhead: 7.6320, complete_rollouts: 1.2495
530
+ save_policy_outputs: 10.3105
531
+ split_output_tensors: 3.9377
532
+ [2025-04-24 11:21:54,255][00031] Loop Runner_EvtLoop terminating...
533
+ [2025-04-24 11:21:54,256][00031] Runner profile tree view:
534
+ main_loop: 439.3208
535
+ [2025-04-24 11:21:54,256][00031] Collected {0: 4005888}, FPS: 9118.4
536
+ [2025-04-24 11:22:09,622][00031] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json
537
+ [2025-04-24 11:22:09,623][00031] Overriding arg 'num_workers' with value 1 passed from command line
538
+ [2025-04-24 11:22:09,624][00031] Adding new argument 'no_render'=True that is not in the saved config file!
539
+ [2025-04-24 11:22:09,624][00031] Adding new argument 'save_video'=True that is not in the saved config file!
540
+ [2025-04-24 11:22:09,625][00031] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
541
+ [2025-04-24 11:22:09,627][00031] Adding new argument 'video_name'=None that is not in the saved config file!
542
+ [2025-04-24 11:22:09,627][00031] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
543
+ [2025-04-24 11:22:09,628][00031] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
544
+ [2025-04-24 11:22:09,629][00031] Adding new argument 'push_to_hub'=False that is not in the saved config file!
545
+ [2025-04-24 11:22:09,630][00031] Adding new argument 'hf_repository'=None that is not in the saved config file!
546
+ [2025-04-24 11:22:09,630][00031] Adding new argument 'policy_index'=0 that is not in the saved config file!
547
+ [2025-04-24 11:22:09,631][00031] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
548
+ [2025-04-24 11:22:09,631][00031] Adding new argument 'train_script'=None that is not in the saved config file!
549
+ [2025-04-24 11:22:09,632][00031] Adding new argument 'enjoy_script'=None that is not in the saved config file!
550
+ [2025-04-24 11:22:09,633][00031] Using frameskip 1 and render_action_repeat=4 for evaluation
551
+ [2025-04-24 11:22:09,660][00031] Doom resolution: 160x120, resize resolution: (128, 72)
552
+ [2025-04-24 11:22:09,663][00031] RunningMeanStd input shape: (3, 72, 128)
553
+ [2025-04-24 11:22:09,664][00031] RunningMeanStd input shape: (1,)
554
+ [2025-04-24 11:22:09,678][00031] ConvEncoder: input_channels=3
555
+ [2025-04-24 11:22:09,783][00031] Conv encoder output size: 512
556
+ [2025-04-24 11:22:09,783][00031] Policy head output size: 512
557
+ [2025-04-24 11:22:09,992][00031] Loading state from checkpoint /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
558
+ [2025-04-24 11:22:10,775][00031] Num frames 100...
559
+ [2025-04-24 11:22:10,887][00031] Num frames 200...
560
+ [2025-04-24 11:22:10,997][00031] Num frames 300...
561
+ [2025-04-24 11:22:11,107][00031] Num frames 400...
562
+ [2025-04-24 11:22:11,224][00031] Num frames 500...
563
+ [2025-04-24 11:22:11,369][00031] Avg episode rewards: #0: 10.760, true rewards: #0: 5.760
564
+ [2025-04-24 11:22:11,370][00031] Avg episode reward: 10.760, avg true_objective: 5.760
565
+ [2025-04-24 11:22:11,408][00031] Num frames 600...
566
+ [2025-04-24 11:22:11,526][00031] Num frames 700...
567
+ [2025-04-24 11:22:11,659][00031] Num frames 800...
568
+ [2025-04-24 11:22:11,771][00031] Num frames 900...
569
+ [2025-04-24 11:22:11,884][00031] Num frames 1000...
570
+ [2025-04-24 11:22:11,998][00031] Num frames 1100...
571
+ [2025-04-24 11:22:12,114][00031] Num frames 1200...
572
+ [2025-04-24 11:22:12,228][00031] Num frames 1300...
573
+ [2025-04-24 11:22:12,338][00031] Num frames 1400...
574
+ [2025-04-24 11:22:12,449][00031] Num frames 1500...
575
+ [2025-04-24 11:22:12,560][00031] Num frames 1600...
576
+ [2025-04-24 11:22:12,673][00031] Num frames 1700...
577
+ [2025-04-24 11:22:12,785][00031] Num frames 1800...
578
+ [2025-04-24 11:22:12,898][00031] Num frames 1900...
579
+ [2025-04-24 11:22:13,012][00031] Num frames 2000...
580
+ [2025-04-24 11:22:13,126][00031] Num frames 2100...
581
+ [2025-04-24 11:22:13,240][00031] Num frames 2200...
582
+ [2025-04-24 11:22:13,353][00031] Num frames 2300...
583
+ [2025-04-24 11:22:13,469][00031] Num frames 2400...
584
+ [2025-04-24 11:22:13,581][00031] Num frames 2500...
585
+ [2025-04-24 11:22:13,696][00031] Num frames 2600...
586
+ [2025-04-24 11:22:13,835][00031] Avg episode rewards: #0: 37.879, true rewards: #0: 13.380
587
+ [2025-04-24 11:22:13,836][00031] Avg episode reward: 37.879, avg true_objective: 13.380
588
+ [2025-04-24 11:22:13,865][00031] Num frames 2700...
589
+ [2025-04-24 11:22:13,983][00031] Num frames 2800...
590
+ [2025-04-24 11:22:14,100][00031] Num frames 2900...
591
+ [2025-04-24 11:22:14,217][00031] Num frames 3000...
592
+ [2025-04-24 11:22:14,336][00031] Num frames 3100...
593
+ [2025-04-24 11:22:14,443][00031] Num frames 3200...
594
+ [2025-04-24 11:22:14,553][00031] Num frames 3300...
595
+ [2025-04-24 11:22:14,670][00031] Num frames 3400...
596
+ [2025-04-24 11:22:14,791][00031] Num frames 3500...
597
+ [2025-04-24 11:22:14,911][00031] Num frames 3600...
598
+ [2025-04-24 11:22:15,029][00031] Num frames 3700...
599
+ [2025-04-24 11:22:15,149][00031] Num frames 3800...
600
+ [2025-04-24 11:22:15,269][00031] Num frames 3900...
601
+ [2025-04-24 11:22:15,389][00031] Num frames 4000...
602
+ [2025-04-24 11:22:15,508][00031] Num frames 4100...
603
+ [2025-04-24 11:22:15,656][00031] Avg episode rewards: #0: 34.933, true rewards: #0: 13.933
604
+ [2025-04-24 11:22:15,657][00031] Avg episode reward: 34.933, avg true_objective: 13.933
605
+ [2025-04-24 11:22:15,680][00031] Num frames 4200...
606
+ [2025-04-24 11:22:15,789][00031] Num frames 4300...
607
+ [2025-04-24 11:22:15,898][00031] Num frames 4400...
608
+ [2025-04-24 11:22:16,009][00031] Num frames 4500...
609
+ [2025-04-24 11:22:16,123][00031] Num frames 4600...
610
+ [2025-04-24 11:22:16,244][00031] Num frames 4700...
611
+ [2025-04-24 11:22:16,361][00031] Num frames 4800...
612
+ [2025-04-24 11:22:16,479][00031] Num frames 4900...
613
+ [2025-04-24 11:22:16,626][00031] Avg episode rewards: #0: 30.200, true rewards: #0: 12.450
614
+ [2025-04-24 11:22:16,626][00031] Avg episode reward: 30.200, avg true_objective: 12.450
615
+ [2025-04-24 11:22:16,651][00031] Num frames 5000...
616
+ [2025-04-24 11:22:16,780][00031] Num frames 5100...
617
+ [2025-04-24 11:22:16,907][00031] Num frames 5200...
618
+ [2025-04-24 11:22:17,032][00031] Num frames 5300...
619
+ [2025-04-24 11:22:17,156][00031] Num frames 5400...
620
+ [2025-04-24 11:22:17,287][00031] Num frames 5500...
621
+ [2025-04-24 11:22:17,409][00031] Num frames 5600...
622
+ [2025-04-24 11:22:17,529][00031] Num frames 5700...
623
+ [2025-04-24 11:22:17,649][00031] Num frames 5800...
624
+ [2025-04-24 11:22:17,769][00031] Num frames 5900...
625
+ [2025-04-24 11:22:17,892][00031] Num frames 6000...
626
+ [2025-04-24 11:22:18,011][00031] Num frames 6100...
627
+ [2025-04-24 11:22:18,125][00031] Num frames 6200...
628
+ [2025-04-24 11:22:18,242][00031] Num frames 6300...
629
+ [2025-04-24 11:22:18,363][00031] Avg episode rewards: #0: 30.112, true rewards: #0: 12.712
630
+ [2025-04-24 11:22:18,364][00031] Avg episode reward: 30.112, avg true_objective: 12.712
631
+ [2025-04-24 11:22:18,417][00031] Num frames 6400...
632
+ [2025-04-24 11:22:18,525][00031] Num frames 6500...
633
+ [2025-04-24 11:22:18,638][00031] Num frames 6600...
634
+ [2025-04-24 11:22:18,755][00031] Num frames 6700...
635
+ [2025-04-24 11:22:18,872][00031] Num frames 6800...
636
+ [2025-04-24 11:22:18,990][00031] Num frames 6900...
637
+ [2025-04-24 11:22:19,101][00031] Num frames 7000...
638
+ [2025-04-24 11:22:19,217][00031] Num frames 7100...
639
+ [2025-04-24 11:22:19,332][00031] Num frames 7200...
640
+ [2025-04-24 11:22:19,442][00031] Num frames 7300...
641
+ [2025-04-24 11:22:19,552][00031] Num frames 7400...
642
+ [2025-04-24 11:22:19,678][00031] Num frames 7500...
643
+ [2025-04-24 11:22:19,790][00031] Num frames 7600...
644
+ [2025-04-24 11:22:19,899][00031] Num frames 7700...
645
+ [2025-04-24 11:22:20,017][00031] Num frames 7800...
646
+ [2025-04-24 11:22:20,135][00031] Num frames 7900...
647
+ [2025-04-24 11:22:20,216][00031] Avg episode rewards: #0: 31.707, true rewards: #0: 13.207
648
+ [2025-04-24 11:22:20,217][00031] Avg episode reward: 31.707, avg true_objective: 13.207
649
+ [2025-04-24 11:22:20,302][00031] Num frames 8000...
650
+ [2025-04-24 11:22:20,410][00031] Num frames 8100...
651
+ [2025-04-24 11:22:20,525][00031] Num frames 8200...
652
+ [2025-04-24 11:22:20,644][00031] Num frames 8300...
653
+ [2025-04-24 11:22:20,761][00031] Num frames 8400...
654
+ [2025-04-24 11:22:20,883][00031] Num frames 8500...
655
+ [2025-04-24 11:22:21,003][00031] Num frames 8600...
656
+ [2025-04-24 11:22:21,124][00031] Num frames 8700...
657
+ [2025-04-24 11:22:21,243][00031] Num frames 8800...
658
+ [2025-04-24 11:22:21,363][00031] Num frames 8900...
659
+ [2025-04-24 11:22:21,428][00031] Avg episode rewards: #0: 30.868, true rewards: #0: 12.726
660
+ [2025-04-24 11:22:21,429][00031] Avg episode reward: 30.868, avg true_objective: 12.726
661
+ [2025-04-24 11:22:21,542][00031] Num frames 9000...
662
+ [2025-04-24 11:22:21,675][00031] Num frames 9100...
663
+ [2025-04-24 11:22:21,785][00031] Num frames 9200...
664
+ [2025-04-24 11:22:21,894][00031] Num frames 9300...
665
+ [2025-04-24 11:22:22,004][00031] Num frames 9400...
666
+ [2025-04-24 11:22:22,114][00031] Num frames 9500...
667
+ [2025-04-24 11:22:22,223][00031] Num frames 9600...
668
+ [2025-04-24 11:22:22,333][00031] Num frames 9700...
669
+ [2025-04-24 11:22:22,450][00031] Num frames 9800...
670
+ [2025-04-24 11:22:22,542][00031] Avg episode rewards: #0: 29.665, true rewards: #0: 12.290
671
+ [2025-04-24 11:22:22,543][00031] Avg episode reward: 29.665, avg true_objective: 12.290
672
+ [2025-04-24 11:22:22,620][00031] Num frames 9900...
673
+ [2025-04-24 11:22:22,729][00031] Num frames 10000...
674
+ [2025-04-24 11:22:22,841][00031] Num frames 10100...
675
+ [2025-04-24 11:22:22,955][00031] Num frames 10200...
676
+ [2025-04-24 11:22:23,074][00031] Num frames 10300...
677
+ [2025-04-24 11:22:23,192][00031] Num frames 10400...
678
+ [2025-04-24 11:22:23,310][00031] Num frames 10500...
679
+ [2025-04-24 11:22:23,427][00031] Num frames 10600...
680
+ [2025-04-24 11:22:23,539][00031] Num frames 10700...
681
+ [2025-04-24 11:22:23,656][00031] Num frames 10800...
682
+ [2025-04-24 11:22:23,768][00031] Num frames 10900...
683
+ [2025-04-24 11:22:23,883][00031] Num frames 11000...
684
+ [2025-04-24 11:22:23,997][00031] Num frames 11100...
685
+ [2025-04-24 11:22:24,111][00031] Num frames 11200...
686
+ [2025-04-24 11:22:24,225][00031] Num frames 11300...
687
+ [2025-04-24 11:22:24,345][00031] Num frames 11400...
688
+ [2025-04-24 11:22:24,465][00031] Num frames 11500...
689
+ [2025-04-24 11:22:24,582][00031] Num frames 11600...
690
+ [2025-04-24 11:22:24,695][00031] Num frames 11700...
691
+ [2025-04-24 11:22:24,806][00031] Num frames 11800...
692
+ [2025-04-24 11:22:24,916][00031] Num frames 11900...
693
+ [2025-04-24 11:22:25,005][00031] Avg episode rewards: #0: 32.702, true rewards: #0: 13.258
694
+ [2025-04-24 11:22:25,006][00031] Avg episode reward: 32.702, avg true_objective: 13.258
695
+ [2025-04-24 11:22:25,080][00031] Num frames 12000...
696
+ [2025-04-24 11:22:25,198][00031] Num frames 12100...
697
+ [2025-04-24 11:22:25,316][00031] Num frames 12200...
698
+ [2025-04-24 11:22:25,433][00031] Num frames 12300...
699
+ [2025-04-24 11:22:25,551][00031] Num frames 12400...
700
+ [2025-04-24 11:22:25,671][00031] Num frames 12500...
701
+ [2025-04-24 11:22:25,837][00031] Avg episode rewards: #0: 30.992, true rewards: #0: 12.592
702
+ [2025-04-24 11:22:25,838][00031] Avg episode reward: 30.992, avg true_objective: 12.592
703
+ [2025-04-24 11:23:04,829][00031] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4!
704
+ [2025-04-24 11:25:18,101][00031] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json
705
+ [2025-04-24 11:25:18,102][00031] Overriding arg 'num_workers' with value 1 passed from command line
706
+ [2025-04-24 11:25:18,103][00031] Adding new argument 'no_render'=True that is not in the saved config file!
707
+ [2025-04-24 11:25:18,104][00031] Adding new argument 'save_video'=True that is not in the saved config file!
708
+ [2025-04-24 11:25:18,105][00031] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
709
+ [2025-04-24 11:25:18,105][00031] Adding new argument 'video_name'=None that is not in the saved config file!
710
+ [2025-04-24 11:25:18,106][00031] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
711
+ [2025-04-24 11:25:18,107][00031] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
712
+ [2025-04-24 11:25:18,107][00031] Adding new argument 'push_to_hub'=True that is not in the saved config file!
713
+ [2025-04-24 11:25:18,108][00031] Adding new argument 'hf_repository'='ezrab/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
714
+ [2025-04-24 11:25:18,108][00031] Adding new argument 'policy_index'=0 that is not in the saved config file!
715
+ [2025-04-24 11:25:18,109][00031] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
716
+ [2025-04-24 11:25:18,111][00031] Adding new argument 'train_script'=None that is not in the saved config file!
717
+ [2025-04-24 11:25:18,112][00031] Adding new argument 'enjoy_script'=None that is not in the saved config file!
718
+ [2025-04-24 11:25:18,113][00031] Using frameskip 1 and render_action_repeat=4 for evaluation
719
+ [2025-04-24 11:25:18,143][00031] RunningMeanStd input shape: (3, 72, 128)
720
+ [2025-04-24 11:25:18,144][00031] RunningMeanStd input shape: (1,)
721
+ [2025-04-24 11:25:18,154][00031] ConvEncoder: input_channels=3
722
+ [2025-04-24 11:25:18,190][00031] Conv encoder output size: 512
723
+ [2025-04-24 11:25:18,191][00031] Policy head output size: 512
724
+ [2025-04-24 11:25:18,209][00031] Loading state from checkpoint /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
725
+ [2025-04-24 11:25:18,654][00031] Num frames 100...
726
+ [2025-04-24 11:25:18,765][00031] Num frames 200...
727
+ [2025-04-24 11:25:18,874][00031] Num frames 300...
728
+ [2025-04-24 11:25:18,984][00031] Num frames 400...
729
+ [2025-04-24 11:25:19,094][00031] Num frames 500...
730
+ [2025-04-24 11:25:19,205][00031] Avg episode rewards: #0: 12.510, true rewards: #0: 5.510
731
+ [2025-04-24 11:25:19,206][00031] Avg episode reward: 12.510, avg true_objective: 5.510
732
+ [2025-04-24 11:25:19,259][00031] Num frames 600...
733
+ [2025-04-24 11:25:19,368][00031] Num frames 700...
734
+ [2025-04-24 11:25:19,477][00031] Num frames 800...
735
+ [2025-04-24 11:25:19,586][00031] Num frames 900...
736
+ [2025-04-24 11:25:19,701][00031] Num frames 1000...
737
+ [2025-04-24 11:25:19,815][00031] Num frames 1100...
738
+ [2025-04-24 11:25:19,924][00031] Num frames 1200...
739
+ [2025-04-24 11:25:20,054][00031] Avg episode rewards: #0: 12.330, true rewards: #0: 6.330
740
+ [2025-04-24 11:25:20,055][00031] Avg episode reward: 12.330, avg true_objective: 6.330
741
+ [2025-04-24 11:25:20,097][00031] Num frames 1300...
742
+ [2025-04-24 11:25:20,205][00031] Num frames 1400...
743
+ [2025-04-24 11:25:20,321][00031] Num frames 1500...
744
+ [2025-04-24 11:25:20,438][00031] Num frames 1600...
745
+ [2025-04-24 11:25:20,556][00031] Num frames 1700...
746
+ [2025-04-24 11:25:20,673][00031] Num frames 1800...
747
+ [2025-04-24 11:25:20,790][00031] Num frames 1900...
748
+ [2025-04-24 11:25:20,909][00031] Num frames 2000...
749
+ [2025-04-24 11:25:21,028][00031] Num frames 2100...
750
+ [2025-04-24 11:25:21,139][00031] Num frames 2200...
751
+ [2025-04-24 11:25:21,250][00031] Num frames 2300...
752
+ [2025-04-24 11:25:21,362][00031] Num frames 2400...
753
+ [2025-04-24 11:25:21,520][00031] Avg episode rewards: #0: 16.940, true rewards: #0: 8.273
754
+ [2025-04-24 11:25:21,521][00031] Avg episode reward: 16.940, avg true_objective: 8.273
755
+ [2025-04-24 11:25:21,547][00031] Num frames 2500...
756
+ [2025-04-24 11:25:21,668][00031] Num frames 2600...
757
+ [2025-04-24 11:25:21,775][00031] Num frames 2700...
758
+ [2025-04-24 11:25:21,883][00031] Num frames 2800...
759
+ [2025-04-24 11:25:21,993][00031] Num frames 2900...
760
+ [2025-04-24 11:25:22,109][00031] Num frames 3000...
761
+ [2025-04-24 11:25:22,225][00031] Num frames 3100...
762
+ [2025-04-24 11:25:22,341][00031] Num frames 3200...
763
+ [2025-04-24 11:25:22,461][00031] Num frames 3300...
764
+ [2025-04-24 11:25:22,574][00031] Num frames 3400...
765
+ [2025-04-24 11:25:22,686][00031] Num frames 3500...
766
+ [2025-04-24 11:25:22,798][00031] Num frames 3600...
767
+ [2025-04-24 11:25:22,906][00031] Num frames 3700...
768
+ [2025-04-24 11:25:23,015][00031] Num frames 3800...
769
+ [2025-04-24 11:25:23,133][00031] Num frames 3900...
770
+ [2025-04-24 11:25:23,246][00031] Num frames 4000...
771
+ [2025-04-24 11:25:23,357][00031] Num frames 4100...
772
+ [2025-04-24 11:25:23,470][00031] Num frames 4200...
773
+ [2025-04-24 11:25:23,586][00031] Num frames 4300...
774
+ [2025-04-24 11:25:23,708][00031] Num frames 4400...
775
+ [2025-04-24 11:25:23,828][00031] Num frames 4500...
776
+ [2025-04-24 11:25:23,979][00031] Avg episode rewards: #0: 25.955, true rewards: #0: 11.455
777
+ [2025-04-24 11:25:23,979][00031] Avg episode reward: 25.955, avg true_objective: 11.455
778
+ [2025-04-24 11:25:24,000][00031] Num frames 4600...
779
+ [2025-04-24 11:25:24,112][00031] Num frames 4700...
780
+ [2025-04-24 11:25:24,222][00031] Num frames 4800...
781
+ [2025-04-24 11:25:24,336][00031] Num frames 4900...
782
+ [2025-04-24 11:25:24,454][00031] Num frames 5000...
783
+ [2025-04-24 11:25:24,571][00031] Num frames 5100...
784
+ [2025-04-24 11:25:24,678][00031] Num frames 5200...
785
+ [2025-04-24 11:25:24,826][00031] Avg episode rewards: #0: 23.372, true rewards: #0: 10.572
786
+ [2025-04-24 11:25:24,827][00031] Avg episode reward: 23.372, avg true_objective: 10.572
787
+ [2025-04-24 11:25:24,843][00031] Num frames 5300...
788
+ [2025-04-24 11:25:24,951][00031] Num frames 5400...
789
+ [2025-04-24 11:25:25,059][00031] Num frames 5500...
790
+ [2025-04-24 11:25:25,168][00031] Num frames 5600...
791
+ [2025-04-24 11:25:25,278][00031] Num frames 5700...
792
+ [2025-04-24 11:25:25,390][00031] Num frames 5800...
793
+ [2025-04-24 11:25:25,479][00031] Avg episode rewards: #0: 21.217, true rewards: #0: 9.717
794
+ [2025-04-24 11:25:25,479][00031] Avg episode reward: 21.217, avg true_objective: 9.717
795
+ [2025-04-24 11:25:25,561][00031] Num frames 5900...
796
+ [2025-04-24 11:25:25,668][00031] Num frames 6000...
797
+ [2025-04-24 11:25:25,775][00031] Num frames 6100...
798
+ [2025-04-24 11:25:25,881][00031] Num frames 6200...
799
+ [2025-04-24 11:25:25,990][00031] Num frames 6300...
800
+ [2025-04-24 11:25:26,100][00031] Num frames 6400...
801
+ [2025-04-24 11:25:26,210][00031] Num frames 6500...
802
+ [2025-04-24 11:25:26,319][00031] Num frames 6600...
803
+ [2025-04-24 11:25:26,427][00031] Num frames 6700...
804
+ [2025-04-24 11:25:26,536][00031] Num frames 6800...
805
+ [2025-04-24 11:25:26,684][00031] Avg episode rewards: #0: 21.980, true rewards: #0: 9.837
806
+ [2025-04-24 11:25:26,685][00031] Avg episode reward: 21.980, avg true_objective: 9.837
807
+ [2025-04-24 11:25:26,700][00031] Num frames 6900...
808
+ [2025-04-24 11:25:26,810][00031] Num frames 7000...
809
+ [2025-04-24 11:25:26,918][00031] Num frames 7100...
810
+ [2025-04-24 11:25:27,037][00031] Num frames 7200...
811
+ [2025-04-24 11:25:27,179][00031] Num frames 7300...
812
+ [2025-04-24 11:25:27,307][00031] Num frames 7400...
813
+ [2025-04-24 11:25:27,431][00031] Num frames 7500...
814
+ [2025-04-24 11:25:27,563][00031] Avg episode rewards: #0: 20.822, true rewards: #0: 9.447
815
+ [2025-04-24 11:25:27,564][00031] Avg episode reward: 20.822, avg true_objective: 9.447
816
+ [2025-04-24 11:25:27,615][00031] Num frames 7600...
817
+ [2025-04-24 11:25:27,736][00031] Num frames 7700...
818
+ [2025-04-24 11:25:27,852][00031] Num frames 7800...
819
+ [2025-04-24 11:25:27,972][00031] Num frames 7900...
820
+ [2025-04-24 11:25:28,085][00031] Num frames 8000...
821
+ [2025-04-24 11:25:28,198][00031] Num frames 8100...
822
+ [2025-04-24 11:25:28,315][00031] Num frames 8200...
823
+ [2025-04-24 11:25:28,437][00031] Num frames 8300...
824
+ [2025-04-24 11:25:28,597][00031] Avg episode rewards: #0: 20.211, true rewards: #0: 9.322
825
+ [2025-04-24 11:25:28,598][00031] Avg episode reward: 20.211, avg true_objective: 9.322
826
+ [2025-04-24 11:25:28,609][00031] Num frames 8400...
827
+ [2025-04-24 11:25:28,726][00031] Num frames 8500...
828
+ [2025-04-24 11:25:28,843][00031] Num frames 8600...
829
+ [2025-04-24 11:25:28,955][00031] Num frames 8700...
830
+ [2025-04-24 11:25:29,064][00031] Num frames 8800...
831
+ [2025-04-24 11:25:29,181][00031] Num frames 8900...
832
+ [2025-04-24 11:25:29,296][00031] Num frames 9000...
833
+ [2025-04-24 11:25:29,453][00031] Avg episode rewards: #0: 19.994, true rewards: #0: 9.094
834
+ [2025-04-24 11:25:29,454][00031] Avg episode reward: 19.994, avg true_objective: 9.094
835
+ [2025-04-24 11:25:57,388][00031] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4!