[2025-05-12 23:33:36,833][40219] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-05-12 23:33:36,836][40219] Rollout worker 0 uses device cpu [2025-05-12 23:33:36,837][40219] Rollout worker 1 uses device cpu [2025-05-12 23:33:36,838][40219] Rollout worker 2 uses device cpu [2025-05-12 23:33:36,840][40219] Rollout worker 3 uses device cpu [2025-05-12 23:33:36,841][40219] Rollout worker 4 uses device cpu [2025-05-12 23:33:36,842][40219] Rollout worker 5 uses device cpu [2025-05-12 23:33:36,844][40219] Rollout worker 6 uses device cpu [2025-05-12 23:33:36,845][40219] Rollout worker 7 uses device cpu [2025-05-12 23:35:57,093][40219] Environment doom_basic already registered, overwriting... [2025-05-12 23:35:57,099][40219] Environment doom_two_colors_easy already registered, overwriting... [2025-05-12 23:35:57,100][40219] Environment doom_two_colors_hard already registered, overwriting... [2025-05-12 23:35:57,109][40219] Environment doom_dm already registered, overwriting... [2025-05-12 23:35:57,113][40219] Environment doom_dwango5 already registered, overwriting... [2025-05-12 23:35:57,116][40219] Environment doom_my_way_home_flat_actions already registered, overwriting... [2025-05-12 23:35:57,118][40219] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2025-05-12 23:35:57,124][40219] Environment doom_my_way_home already registered, overwriting... [2025-05-12 23:35:57,127][40219] Environment doom_deadly_corridor already registered, overwriting... [2025-05-12 23:35:57,130][40219] Environment doom_defend_the_center already registered, overwriting... [2025-05-12 23:35:57,134][40219] Environment doom_defend_the_line already registered, overwriting... [2025-05-12 23:35:57,144][40219] Environment doom_health_gathering already registered, overwriting... [2025-05-12 23:35:57,149][40219] Environment doom_health_gathering_supreme already registered, overwriting... [2025-05-12 23:35:57,152][40219] Environment doom_battle already registered, overwriting... [2025-05-12 23:35:57,153][40219] Environment doom_battle2 already registered, overwriting... [2025-05-12 23:35:57,153][40219] Environment doom_duel_bots already registered, overwriting... [2025-05-12 23:35:57,159][40219] Environment doom_deathmatch_bots already registered, overwriting... [2025-05-12 23:35:57,160][40219] Environment doom_duel already registered, overwriting... [2025-05-12 23:35:57,161][40219] Environment doom_deathmatch_full already registered, overwriting... [2025-05-12 23:35:57,161][40219] Environment doom_benchmark already registered, overwriting... [2025-05-12 23:35:57,162][40219] register_encoder_factory: [2025-05-12 23:35:57,225][40219] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-05-12 23:35:57,227][40219] Overriding arg 'device' with value 'cpu' passed from command line [2025-05-12 23:35:57,248][40219] Experiment dir /content/train_dir/default_experiment already exists! [2025-05-12 23:35:57,249][40219] Resuming existing experiment from /content/train_dir/default_experiment... [2025-05-12 23:35:57,259][40219] Weights and Biases integration disabled [2025-05-12 23:35:57,278][40219] Environment var CUDA_VISIBLE_DEVICES is [2025-05-12 23:36:00,337][40219] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=cpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=40000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=40000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 40000} git_hash=unknown git_repo_name=not a git repository [2025-05-12 23:36:00,339][40219] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-05-12 23:36:00,342][40219] Rollout worker 0 uses device cpu [2025-05-12 23:36:00,343][40219] Rollout worker 1 uses device cpu [2025-05-12 23:36:00,345][40219] Rollout worker 2 uses device cpu [2025-05-12 23:36:00,347][40219] Rollout worker 3 uses device cpu [2025-05-12 23:36:00,348][40219] Rollout worker 4 uses device cpu [2025-05-12 23:36:00,349][40219] Rollout worker 5 uses device cpu [2025-05-12 23:36:00,350][40219] Rollout worker 6 uses device cpu [2025-05-12 23:36:00,352][40219] Rollout worker 7 uses device cpu [2025-05-12 23:36:00,455][40219] InferenceWorker_p0-w0: min num requests: 2 [2025-05-12 23:36:00,490][40219] Starting all processes... [2025-05-12 23:36:00,491][40219] Starting process learner_proc0 [2025-05-12 23:36:00,559][40219] Starting all processes... [2025-05-12 23:36:00,569][40219] Starting process inference_proc0-0 [2025-05-12 23:36:00,570][40219] Starting process rollout_proc0 [2025-05-12 23:36:00,570][40219] Starting process rollout_proc1 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc2 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc3 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc4 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc5 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc6 [2025-05-12 23:36:00,573][40219] Starting process rollout_proc7 [2025-05-12 23:36:19,881][41000] Worker 2 uses CPU cores [0] [2025-05-12 23:36:20,023][41005] Worker 7 uses CPU cores [1] [2025-05-12 23:36:20,094][40985] Starting seed is not provided [2025-05-12 23:36:20,095][40985] Initializing actor-critic model on device cpu [2025-05-12 23:36:20,096][40985] RunningMeanStd input shape: (3, 72, 128) [2025-05-12 23:36:20,099][40985] RunningMeanStd input shape: (1,) [2025-05-12 23:36:20,157][41004] Worker 3 uses CPU cores [1] [2025-05-12 23:36:20,161][40985] ConvEncoder: input_channels=3 [2025-05-12 23:36:20,401][41002] Worker 5 uses CPU cores [1] [2025-05-12 23:36:20,442][41001] Worker 1 uses CPU cores [1] [2025-05-12 23:36:20,460][40219] Heartbeat connected on Batcher_0 [2025-05-12 23:36:20,471][40219] Heartbeat connected on RolloutWorker_w2 [2025-05-12 23:36:20,475][40219] Heartbeat connected on RolloutWorker_w3 [2025-05-12 23:36:20,490][40219] Heartbeat connected on RolloutWorker_w7 [2025-05-12 23:36:20,499][40219] Heartbeat connected on RolloutWorker_w5 [2025-05-12 23:36:20,522][40999] Worker 0 uses CPU cores [0] [2025-05-12 23:36:20,529][40219] Heartbeat connected on RolloutWorker_w1 [2025-05-12 23:36:20,555][41003] Worker 4 uses CPU cores [0] [2025-05-12 23:36:20,606][40219] Heartbeat connected on RolloutWorker_w0 [2025-05-12 23:36:20,620][40219] Heartbeat connected on InferenceWorker_p0-w0 [2025-05-12 23:36:20,632][40219] Heartbeat connected on RolloutWorker_w4 [2025-05-12 23:36:20,719][41006] Worker 6 uses CPU cores [0] [2025-05-12 23:36:20,727][40985] Conv encoder output size: 512 [2025-05-12 23:36:20,728][40985] Policy head output size: 512 [2025-05-12 23:36:20,729][40219] Heartbeat connected on RolloutWorker_w6 [2025-05-12 23:36:20,786][40985] Created Actor Critic model with architecture: [2025-05-12 23:36:20,786][40985] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-05-12 23:36:21,452][40985] Using optimizer [2025-05-12 23:36:21,454][40985] No checkpoints found [2025-05-12 23:36:21,454][40985] Did not load from checkpoint, starting from scratch! [2025-05-12 23:36:21,454][40985] Initialized policy 0 weights for model version 0 [2025-05-12 23:36:21,457][40985] LearnerWorker_p0 finished initialization! [2025-05-12 23:36:21,460][40219] Heartbeat connected on LearnerWorker_p0 [2025-05-12 23:36:21,462][40998] RunningMeanStd input shape: (3, 72, 128) [2025-05-12 23:36:21,465][40998] RunningMeanStd input shape: (1,) [2025-05-12 23:36:21,488][40998] ConvEncoder: input_channels=3 [2025-05-12 23:36:21,677][40998] Conv encoder output size: 512 [2025-05-12 23:36:21,678][40998] Policy head output size: 512 [2025-05-12 23:36:21,707][40219] Inference worker 0-0 is ready! [2025-05-12 23:36:21,708][40219] All inference workers are ready! Signal rollout workers to start! [2025-05-12 23:36:22,061][41002] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,081][41004] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,208][41000] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,210][41003] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,279][40219] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:36:22,306][41005] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,317][41001] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,363][40999] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:22,365][41006] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:36:25,002][41001] Decorrelating experience for 0 frames... [2025-05-12 23:36:25,004][41004] Decorrelating experience for 0 frames... [2025-05-12 23:36:25,008][41005] Decorrelating experience for 0 frames... [2025-05-12 23:36:25,005][41000] Decorrelating experience for 0 frames... [2025-05-12 23:36:25,005][41006] Decorrelating experience for 0 frames... [2025-05-12 23:36:25,005][41003] Decorrelating experience for 0 frames... [2025-05-12 23:36:27,087][41004] Decorrelating experience for 32 frames... [2025-05-12 23:36:27,100][41005] Decorrelating experience for 32 frames... [2025-05-12 23:36:27,104][41002] Decorrelating experience for 0 frames... [2025-05-12 23:36:27,103][41001] Decorrelating experience for 32 frames... [2025-05-12 23:36:27,155][40999] Decorrelating experience for 0 frames... [2025-05-12 23:36:27,185][41003] Decorrelating experience for 32 frames... [2025-05-12 23:36:27,189][41000] Decorrelating experience for 32 frames... [2025-05-12 23:36:27,279][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:36:28,367][41002] Decorrelating experience for 32 frames... [2025-05-12 23:36:28,779][41004] Decorrelating experience for 64 frames... [2025-05-12 23:36:29,119][41006] Decorrelating experience for 32 frames... [2025-05-12 23:36:29,183][41001] Decorrelating experience for 64 frames... [2025-05-12 23:36:29,207][40999] Decorrelating experience for 32 frames... [2025-05-12 23:36:29,761][41003] Decorrelating experience for 64 frames... [2025-05-12 23:36:29,765][41000] Decorrelating experience for 64 frames... [2025-05-12 23:36:30,925][41002] Decorrelating experience for 64 frames... [2025-05-12 23:36:30,993][41005] Decorrelating experience for 64 frames... [2025-05-12 23:36:31,011][41001] Decorrelating experience for 96 frames... [2025-05-12 23:36:31,410][41006] Decorrelating experience for 64 frames... [2025-05-12 23:36:31,557][40999] Decorrelating experience for 64 frames... [2025-05-12 23:36:31,783][41003] Decorrelating experience for 96 frames... [2025-05-12 23:36:31,782][41004] Decorrelating experience for 96 frames... [2025-05-12 23:36:32,279][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:36:33,603][41000] Decorrelating experience for 96 frames... [2025-05-12 23:36:33,680][41002] Decorrelating experience for 96 frames... [2025-05-12 23:36:34,991][40999] Decorrelating experience for 96 frames... [2025-05-12 23:36:35,353][41005] Decorrelating experience for 96 frames... [2025-05-12 23:36:35,986][41006] Decorrelating experience for 96 frames... [2025-05-12 23:36:37,279][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 104.3. Samples: 1564. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:36:37,286][40219] Avg episode reward: [(0, '2.054')] [2025-05-12 23:36:40,836][40985] Signal inference workers to stop experience collection... [2025-05-12 23:36:40,926][40998] InferenceWorker_p0-w0: stopping experience collection [2025-05-12 23:36:42,279][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 112.4. Samples: 2248. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:36:42,284][40219] Avg episode reward: [(0, '2.563')] [2025-05-12 23:36:42,364][40985] Signal inference workers to resume experience collection... [2025-05-12 23:36:42,366][40998] InferenceWorker_p0-w0: resuming experience collection [2025-05-12 23:36:47,279][40219] Fps is (10 sec: 819.2, 60 sec: 327.7, 300 sec: 327.7). Total num frames: 8192. Throughput: 0: 141.9. Samples: 3548. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2025-05-12 23:36:47,280][40219] Avg episode reward: [(0, '2.752')] [2025-05-12 23:36:52,280][40219] Fps is (10 sec: 1228.7, 60 sec: 409.6, 300 sec: 409.6). Total num frames: 12288. Throughput: 0: 180.1. Samples: 5402. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:36:52,284][40219] Avg episode reward: [(0, '3.379')] [2025-05-12 23:36:57,279][40219] Fps is (10 sec: 819.2, 60 sec: 468.1, 300 sec: 468.1). Total num frames: 16384. Throughput: 0: 162.7. Samples: 5696. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:36:57,283][40219] Avg episode reward: [(0, '3.489')] [2025-05-12 23:37:02,281][40219] Fps is (10 sec: 819.1, 60 sec: 512.0, 300 sec: 512.0). Total num frames: 20480. Throughput: 0: 171.1. Samples: 6844. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:37:02,282][40219] Avg episode reward: [(0, '3.759')] [2025-05-12 23:37:07,279][40219] Fps is (10 sec: 819.2, 60 sec: 546.1, 300 sec: 546.1). Total num frames: 24576. Throughput: 0: 190.1. Samples: 8554. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:37:07,281][40219] Avg episode reward: [(0, '3.833')] [2025-05-12 23:37:12,279][40219] Fps is (10 sec: 819.4, 60 sec: 573.4, 300 sec: 573.4). Total num frames: 28672. Throughput: 0: 195.6. Samples: 8804. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:37:12,280][40219] Avg episode reward: [(0, '3.933')] [2025-05-12 23:37:17,279][40219] Fps is (10 sec: 1228.8, 60 sec: 670.3, 300 sec: 670.3). Total num frames: 36864. Throughput: 0: 238.0. Samples: 10708. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:37:17,281][40219] Avg episode reward: [(0, '4.122')] [2025-05-12 23:37:20,498][40998] Updated weights for policy 0, policy_version 10 (0.0036) [2025-05-12 23:37:22,279][40219] Fps is (10 sec: 1228.8, 60 sec: 682.7, 300 sec: 682.7). Total num frames: 40960. Throughput: 0: 235.6. Samples: 12168. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:37:22,283][40219] Avg episode reward: [(0, '4.286')] [2025-05-12 23:37:25,689][40985] Stopping Batcher_0... [2025-05-12 23:37:25,691][40219] Component Batcher_0 stopped! [2025-05-12 23:37:25,695][40985] Loop batcher_evt_loop terminating... [2025-05-12 23:37:25,850][40998] Weights refcount: 2 0 [2025-05-12 23:37:25,853][40219] Component InferenceWorker_p0-w0 stopped! [2025-05-12 23:37:25,858][40998] Stopping InferenceWorker_p0-w0... [2025-05-12 23:37:25,859][40998] Loop inference_proc0-0_evt_loop terminating... [2025-05-12 23:37:26,014][40219] Component RolloutWorker_w5 stopped! [2025-05-12 23:37:26,018][41002] Stopping RolloutWorker_w5... [2025-05-12 23:37:26,022][41002] Loop rollout_proc5_evt_loop terminating... [2025-05-12 23:37:26,102][40219] Component RolloutWorker_w1 stopped! [2025-05-12 23:37:26,105][40999] Stopping RolloutWorker_w0... [2025-05-12 23:37:26,107][40219] Component RolloutWorker_w0 stopped! [2025-05-12 23:37:26,114][41003] Stopping RolloutWorker_w4... [2025-05-12 23:37:26,115][41003] Loop rollout_proc4_evt_loop terminating... [2025-05-12 23:37:26,116][40219] Component RolloutWorker_w4 stopped! [2025-05-12 23:37:26,110][41001] Stopping RolloutWorker_w1... [2025-05-12 23:37:26,120][41001] Loop rollout_proc1_evt_loop terminating... [2025-05-12 23:37:26,110][40999] Loop rollout_proc0_evt_loop terminating... [2025-05-12 23:37:26,124][40219] Component RolloutWorker_w7 stopped! [2025-05-12 23:37:26,128][41005] Stopping RolloutWorker_w7... [2025-05-12 23:37:26,134][40219] Component RolloutWorker_w3 stopped! [2025-05-12 23:37:26,131][41005] Loop rollout_proc7_evt_loop terminating... [2025-05-12 23:37:26,138][41004] Stopping RolloutWorker_w3... [2025-05-12 23:37:26,142][41004] Loop rollout_proc3_evt_loop terminating... [2025-05-12 23:37:26,155][41000] Stopping RolloutWorker_w2... [2025-05-12 23:37:26,155][40219] Component RolloutWorker_w2 stopped! [2025-05-12 23:37:26,156][41000] Loop rollout_proc2_evt_loop terminating... [2025-05-12 23:37:26,241][41006] Stopping RolloutWorker_w6... [2025-05-12 23:37:26,241][40219] Component RolloutWorker_w6 stopped! [2025-05-12 23:37:26,242][41006] Loop rollout_proc6_evt_loop terminating... [2025-05-12 23:37:30,633][40985] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000012_49152.pth... [2025-05-12 23:37:30,727][40985] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000012_49152.pth... [2025-05-12 23:37:30,887][40985] Stopping LearnerWorker_p0... [2025-05-12 23:37:30,888][40985] Loop learner_proc0_evt_loop terminating... [2025-05-12 23:37:30,887][40219] Component LearnerWorker_p0 stopped! [2025-05-12 23:37:30,890][40219] Waiting for process learner_proc0 to stop... [2025-05-12 23:37:32,090][40219] Waiting for process inference_proc0-0 to join... [2025-05-12 23:37:32,092][40219] Waiting for process rollout_proc0 to join... [2025-05-12 23:37:32,464][40219] Waiting for process rollout_proc1 to join... [2025-05-12 23:37:32,465][40219] Waiting for process rollout_proc2 to join... [2025-05-12 23:37:32,466][40219] Waiting for process rollout_proc3 to join... [2025-05-12 23:37:32,467][40219] Waiting for process rollout_proc4 to join... [2025-05-12 23:37:32,468][40219] Waiting for process rollout_proc5 to join... [2025-05-12 23:37:32,469][40219] Waiting for process rollout_proc6 to join... [2025-05-12 23:37:32,470][40219] Waiting for process rollout_proc7 to join... [2025-05-12 23:37:32,472][40219] Batcher 0 profile tree view: batching: 0.1970, releasing_batches: 0.0026 [2025-05-12 23:37:32,474][40219] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0051 wait_policy_total: 10.6378 update_model: 0.3321 weight_update: 0.0060 one_step: 0.0696 handle_policy_step: 35.3273 deserialize: 0.9221, stack: 0.1720, obs_to_device_normalize: 5.9680, forward: 26.3367, send_messages: 0.7474 prepare_outputs: 0.4052 to_cpu: 0.0345 [2025-05-12 23:37:32,476][40219] Learner 0 profile tree view: misc: 0.0001, prepare_batch: 16.3002 train: 37.7421 epoch_init: 0.0001, minibatch_init: 0.0001, losses_postprocess: 0.0011, kl_divergence: 0.0041, after_optimizer: 0.0299 calculate_losses: 18.9488 losses_init: 0.0000, forward_head: 17.0542, bptt_initial: 0.0547, tail: 0.0426, advantages_returns: 0.0048, losses: 0.0233 bptt: 1.7588 bptt_forward_core: 1.7509 update: 18.7449 clip: 0.0490 [2025-05-12 23:37:32,478][40219] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.0025, enqueue_policy_requests: 0.5907, env_step: 17.0171, overhead: 0.5400, complete_rollouts: 0.1576 save_policy_outputs: 0.2373 split_output_tensors: 0.0856 [2025-05-12 23:37:32,480][40219] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.0267, enqueue_policy_requests: 0.4142, env_step: 17.7747, overhead: 0.5730, complete_rollouts: 0.1611 save_policy_outputs: 0.2014 split_output_tensors: 0.0907 [2025-05-12 23:37:32,481][40219] Loop Runner_EvtLoop terminating... [2025-05-12 23:37:32,483][40219] Runner profile tree view: main_loop: 91.9930 [2025-05-12 23:37:32,484][40219] Collected {0: 49152}, FPS: 534.3 [2025-05-12 23:39:03,619][40219] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-05-12 23:39:03,621][40219] Overriding arg 'num_workers' with value 1 passed from command line [2025-05-12 23:39:03,622][40219] Adding new argument 'no_render'=True that is not in the saved config file! [2025-05-12 23:39:03,624][40219] Adding new argument 'save_video'=True that is not in the saved config file! [2025-05-12 23:39:03,626][40219] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-05-12 23:39:03,627][40219] Adding new argument 'video_name'=None that is not in the saved config file! [2025-05-12 23:39:03,628][40219] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-05-12 23:39:03,629][40219] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-05-12 23:39:03,630][40219] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-05-12 23:39:03,631][40219] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-05-12 23:39:03,632][40219] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-05-12 23:39:03,633][40219] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-05-12 23:39:03,634][40219] Adding new argument 'train_script'=None that is not in the saved config file! [2025-05-12 23:39:03,635][40219] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-05-12 23:39:03,636][40219] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-05-12 23:39:03,670][40219] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:39:03,673][40219] RunningMeanStd input shape: (3, 72, 128) [2025-05-12 23:39:03,675][40219] RunningMeanStd input shape: (1,) [2025-05-12 23:39:03,693][40219] ConvEncoder: input_channels=3 [2025-05-12 23:39:03,807][40219] Conv encoder output size: 512 [2025-05-12 23:39:03,808][40219] Policy head output size: 512 [2025-05-12 23:39:03,827][40219] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000012_49152.pth... [2025-05-12 23:39:04,334][40219] Num frames 100... [2025-05-12 23:39:04,510][40219] Num frames 200... [2025-05-12 23:39:04,689][40219] Num frames 300... [2025-05-12 23:39:04,863][40219] Num frames 400... [2025-05-12 23:39:04,946][40219] Avg episode rewards: #0: 5.160, true rewards: #0: 4.160 [2025-05-12 23:39:04,948][40219] Avg episode reward: 5.160, avg true_objective: 4.160 [2025-05-12 23:39:05,098][40219] Num frames 500... [2025-05-12 23:39:05,284][40219] Num frames 600... [2025-05-12 23:39:05,465][40219] Num frames 700... [2025-05-12 23:39:05,661][40219] Num frames 800... [2025-05-12 23:39:05,834][40219] Avg episode rewards: #0: 5.320, true rewards: #0: 4.320 [2025-05-12 23:39:05,835][40219] Avg episode reward: 5.320, avg true_objective: 4.320 [2025-05-12 23:39:05,921][40219] Num frames 900... [2025-05-12 23:39:06,095][40219] Num frames 1000... [2025-05-12 23:39:06,495][40219] Num frames 1100... [2025-05-12 23:39:06,674][40219] Num frames 1200... [2025-05-12 23:39:06,856][40219] Num frames 1300... [2025-05-12 23:39:06,934][40219] Avg episode rewards: #0: 5.373, true rewards: #0: 4.373 [2025-05-12 23:39:06,935][40219] Avg episode reward: 5.373, avg true_objective: 4.373 [2025-05-12 23:39:07,097][40219] Num frames 1400... [2025-05-12 23:39:07,289][40219] Num frames 1500... [2025-05-12 23:39:07,465][40219] Num frames 1600... [2025-05-12 23:39:07,689][40219] Avg episode rewards: #0: 4.990, true rewards: #0: 4.240 [2025-05-12 23:39:07,690][40219] Avg episode reward: 4.990, avg true_objective: 4.240 [2025-05-12 23:39:07,700][40219] Num frames 1700... [2025-05-12 23:39:07,874][40219] Num frames 1800... [2025-05-12 23:39:08,059][40219] Num frames 1900... [2025-05-12 23:39:08,236][40219] Num frames 2000... [2025-05-12 23:39:08,471][40219] Avg episode rewards: #0: 4.760, true rewards: #0: 4.160 [2025-05-12 23:39:08,473][40219] Avg episode reward: 4.760, avg true_objective: 4.160 [2025-05-12 23:39:08,530][40219] Num frames 2100... [2025-05-12 23:39:08,785][40219] Num frames 2200... [2025-05-12 23:39:09,035][40219] Num frames 2300... [2025-05-12 23:39:09,283][40219] Num frames 2400... [2025-05-12 23:39:09,506][40219] Avg episode rewards: #0: 4.607, true rewards: #0: 4.107 [2025-05-12 23:39:09,510][40219] Avg episode reward: 4.607, avg true_objective: 4.107 [2025-05-12 23:39:09,606][40219] Num frames 2500... [2025-05-12 23:39:09,856][40219] Num frames 2600... [2025-05-12 23:39:10,125][40219] Num frames 2700... [2025-05-12 23:39:10,386][40219] Num frames 2800... [2025-05-12 23:39:10,576][40219] Avg episode rewards: #0: 4.497, true rewards: #0: 4.069 [2025-05-12 23:39:10,578][40219] Avg episode reward: 4.497, avg true_objective: 4.069 [2025-05-12 23:39:10,724][40219] Num frames 2900... [2025-05-12 23:39:10,935][40219] Num frames 3000... [2025-05-12 23:39:11,113][40219] Num frames 3100... [2025-05-12 23:39:11,299][40219] Num frames 3200... [2025-05-12 23:39:11,412][40219] Avg episode rewards: #0: 4.415, true rewards: #0: 4.040 [2025-05-12 23:39:11,414][40219] Avg episode reward: 4.415, avg true_objective: 4.040 [2025-05-12 23:39:11,540][40219] Num frames 3300... [2025-05-12 23:39:11,716][40219] Num frames 3400... [2025-05-12 23:39:11,898][40219] Num frames 3500... [2025-05-12 23:39:12,067][40219] Num frames 3600... [2025-05-12 23:39:12,156][40219] Avg episode rewards: #0: 4.351, true rewards: #0: 4.018 [2025-05-12 23:39:12,158][40219] Avg episode reward: 4.351, avg true_objective: 4.018 [2025-05-12 23:39:12,307][40219] Num frames 3700... [2025-05-12 23:39:12,496][40219] Num frames 3800... [2025-05-12 23:39:12,672][40219] Num frames 3900... [2025-05-12 23:39:12,855][40219] Num frames 4000... [2025-05-12 23:39:12,907][40219] Avg episode rewards: #0: 4.300, true rewards: #0: 4.000 [2025-05-12 23:39:12,909][40219] Avg episode reward: 4.300, avg true_objective: 4.000 [2025-05-12 23:39:37,326][40219] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-05-12 23:40:05,245][40219] Environment doom_basic already registered, overwriting... [2025-05-12 23:40:05,247][40219] Environment doom_two_colors_easy already registered, overwriting... [2025-05-12 23:40:05,248][40219] Environment doom_two_colors_hard already registered, overwriting... [2025-05-12 23:40:05,249][40219] Environment doom_dm already registered, overwriting... [2025-05-12 23:40:05,251][40219] Environment doom_dwango5 already registered, overwriting... [2025-05-12 23:40:05,252][40219] Environment doom_my_way_home_flat_actions already registered, overwriting... [2025-05-12 23:40:05,253][40219] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2025-05-12 23:40:05,254][40219] Environment doom_my_way_home already registered, overwriting... [2025-05-12 23:40:05,255][40219] Environment doom_deadly_corridor already registered, overwriting... [2025-05-12 23:40:05,256][40219] Environment doom_defend_the_center already registered, overwriting... [2025-05-12 23:40:05,256][40219] Environment doom_defend_the_line already registered, overwriting... [2025-05-12 23:40:05,257][40219] Environment doom_health_gathering already registered, overwriting... [2025-05-12 23:40:05,258][40219] Environment doom_health_gathering_supreme already registered, overwriting... [2025-05-12 23:40:05,259][40219] Environment doom_battle already registered, overwriting... [2025-05-12 23:40:05,260][40219] Environment doom_battle2 already registered, overwriting... [2025-05-12 23:40:05,261][40219] Environment doom_duel_bots already registered, overwriting... [2025-05-12 23:40:05,261][40219] Environment doom_deathmatch_bots already registered, overwriting... [2025-05-12 23:40:05,262][40219] Environment doom_duel already registered, overwriting... [2025-05-12 23:40:05,263][40219] Environment doom_deathmatch_full already registered, overwriting... [2025-05-12 23:40:05,264][40219] Environment doom_benchmark already registered, overwriting... [2025-05-12 23:40:05,264][40219] register_encoder_factory: [2025-05-12 23:40:05,277][40219] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-05-12 23:40:05,278][40219] Overriding arg 'train_for_env_steps' with value 4000000 passed from command line [2025-05-12 23:40:05,284][40219] Experiment dir /content/train_dir/default_experiment already exists! [2025-05-12 23:40:05,284][40219] Resuming existing experiment from /content/train_dir/default_experiment... [2025-05-12 23:40:05,286][40219] Weights and Biases integration disabled [2025-05-12 23:40:05,289][40219] Environment var CUDA_VISIBLE_DEVICES is [2025-05-12 23:40:07,914][40219] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=cpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=4000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=40000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 40000} git_hash=unknown git_repo_name=not a git repository [2025-05-12 23:40:07,915][40219] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-05-12 23:40:07,918][40219] Rollout worker 0 uses device cpu [2025-05-12 23:40:07,919][40219] Rollout worker 1 uses device cpu [2025-05-12 23:40:07,920][40219] Rollout worker 2 uses device cpu [2025-05-12 23:40:07,921][40219] Rollout worker 3 uses device cpu [2025-05-12 23:40:07,922][40219] Rollout worker 4 uses device cpu [2025-05-12 23:40:07,924][40219] Rollout worker 5 uses device cpu [2025-05-12 23:40:07,924][40219] Rollout worker 6 uses device cpu [2025-05-12 23:40:07,925][40219] Rollout worker 7 uses device cpu [2025-05-12 23:40:08,025][40219] InferenceWorker_p0-w0: min num requests: 2 [2025-05-12 23:40:08,060][40219] Starting all processes... [2025-05-12 23:40:08,061][40219] Starting process learner_proc0 [2025-05-12 23:40:08,122][40219] Starting all processes... [2025-05-12 23:40:08,130][40219] Starting process inference_proc0-0 [2025-05-12 23:40:08,130][40219] Starting process rollout_proc0 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc1 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc2 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc3 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc4 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc5 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc6 [2025-05-12 23:40:08,134][40219] Starting process rollout_proc7 [2025-05-12 23:40:26,538][46917] Worker 7 uses CPU cores [1] [2025-05-12 23:40:26,815][46912] Worker 0 uses CPU cores [0] [2025-05-12 23:40:27,221][46911] Worker 2 uses CPU cores [0] [2025-05-12 23:40:27,263][46896] Starting seed is not provided [2025-05-12 23:40:27,263][46896] Initializing actor-critic model on device cpu [2025-05-12 23:40:27,264][46896] RunningMeanStd input shape: (3, 72, 128) [2025-05-12 23:40:27,268][46896] RunningMeanStd input shape: (1,) [2025-05-12 23:40:27,362][46913] Worker 3 uses CPU cores [1] [2025-05-12 23:40:27,363][46896] ConvEncoder: input_channels=3 [2025-05-12 23:40:27,496][46916] Worker 4 uses CPU cores [0] [2025-05-12 23:40:27,533][46915] Worker 5 uses CPU cores [1] [2025-05-12 23:40:27,559][46909] Worker 1 uses CPU cores [1] [2025-05-12 23:40:27,562][46914] Worker 6 uses CPU cores [0] [2025-05-12 23:40:27,707][46896] Conv encoder output size: 512 [2025-05-12 23:40:27,708][46896] Policy head output size: 512 [2025-05-12 23:40:27,726][46896] Created Actor Critic model with architecture: [2025-05-12 23:40:27,726][46896] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-05-12 23:40:28,026][40219] Heartbeat connected on InferenceWorker_p0-w0 [2025-05-12 23:40:28,037][40219] Heartbeat connected on RolloutWorker_w0 [2025-05-12 23:40:28,038][40219] Heartbeat connected on RolloutWorker_w1 [2025-05-12 23:40:28,044][40219] Heartbeat connected on RolloutWorker_w2 [2025-05-12 23:40:28,045][40219] Heartbeat connected on RolloutWorker_w3 [2025-05-12 23:40:28,049][40219] Heartbeat connected on RolloutWorker_w4 [2025-05-12 23:40:28,053][40219] Heartbeat connected on RolloutWorker_w5 [2025-05-12 23:40:28,056][40219] Heartbeat connected on RolloutWorker_w6 [2025-05-12 23:40:28,064][40219] Heartbeat connected on RolloutWorker_w7 [2025-05-12 23:40:28,230][40219] Heartbeat connected on Batcher_0 [2025-05-12 23:40:28,245][46896] Using optimizer [2025-05-12 23:40:28,246][46896] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000012_49152.pth... [2025-05-12 23:40:28,278][46896] Loading model from checkpoint [2025-05-12 23:40:28,300][46896] Loaded experiment state at self.train_step=12, self.env_steps=49152 [2025-05-12 23:40:28,301][46896] Initialized policy 0 weights for model version 12 [2025-05-12 23:40:28,306][46910] RunningMeanStd input shape: (3, 72, 128) [2025-05-12 23:40:28,303][46896] LearnerWorker_p0 finished initialization! [2025-05-12 23:40:28,308][46910] RunningMeanStd input shape: (1,) [2025-05-12 23:40:28,308][40219] Heartbeat connected on LearnerWorker_p0 [2025-05-12 23:40:28,332][46910] ConvEncoder: input_channels=3 [2025-05-12 23:40:28,457][46910] Conv encoder output size: 512 [2025-05-12 23:40:28,458][46910] Policy head output size: 512 [2025-05-12 23:40:28,479][40219] Inference worker 0-0 is ready! [2025-05-12 23:40:28,480][40219] All inference workers are ready! Signal rollout workers to start! [2025-05-12 23:40:28,727][46916] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,732][46911] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,745][46915] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,754][46914] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,772][46913] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,776][46917] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,784][46909] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:28,893][46912] Doom resolution: 160x120, resize resolution: (128, 72) [2025-05-12 23:40:30,289][40219] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 49152. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:40:30,372][46916] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,374][46911] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,455][46912] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,831][46915] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,840][46917] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,852][46909] Decorrelating experience for 0 frames... [2025-05-12 23:40:30,857][46913] Decorrelating experience for 0 frames... [2025-05-12 23:40:31,733][46911] Decorrelating experience for 32 frames... [2025-05-12 23:40:31,811][46914] Decorrelating experience for 0 frames... [2025-05-12 23:40:31,859][46912] Decorrelating experience for 32 frames... [2025-05-12 23:40:32,721][46915] Decorrelating experience for 32 frames... [2025-05-12 23:40:32,739][46909] Decorrelating experience for 32 frames... [2025-05-12 23:40:32,765][46913] Decorrelating experience for 32 frames... [2025-05-12 23:40:32,820][46917] Decorrelating experience for 32 frames... [2025-05-12 23:40:33,028][46914] Decorrelating experience for 32 frames... [2025-05-12 23:40:33,326][46911] Decorrelating experience for 64 frames... [2025-05-12 23:40:33,972][46916] Decorrelating experience for 32 frames... [2025-05-12 23:40:34,500][46912] Decorrelating experience for 64 frames... [2025-05-12 23:40:34,844][46911] Decorrelating experience for 96 frames... [2025-05-12 23:40:35,186][46915] Decorrelating experience for 64 frames... [2025-05-12 23:40:35,211][46909] Decorrelating experience for 64 frames... [2025-05-12 23:40:35,221][46913] Decorrelating experience for 64 frames... [2025-05-12 23:40:35,289][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 49152. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:40:35,312][46917] Decorrelating experience for 64 frames... [2025-05-12 23:40:37,258][46915] Decorrelating experience for 96 frames... [2025-05-12 23:40:37,303][46913] Decorrelating experience for 96 frames... [2025-05-12 23:40:37,294][46909] Decorrelating experience for 96 frames... [2025-05-12 23:40:38,377][46914] Decorrelating experience for 64 frames... [2025-05-12 23:40:39,439][46916] Decorrelating experience for 64 frames... [2025-05-12 23:40:39,646][46912] Decorrelating experience for 96 frames... [2025-05-12 23:40:40,290][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 49152. Throughput: 0: 29.4. Samples: 294. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:40:40,291][40219] Avg episode reward: [(0, '2.320')] [2025-05-12 23:40:42,438][46917] Decorrelating experience for 96 frames... [2025-05-12 23:40:45,289][40219] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 49152. Throughput: 0: 115.3. Samples: 1730. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-05-12 23:40:45,291][40219] Avg episode reward: [(0, '2.435')] [2025-05-12 23:40:45,448][46896] Signal inference workers to stop experience collection... [2025-05-12 23:40:45,482][46910] InferenceWorker_p0-w0: stopping experience collection [2025-05-12 23:40:45,485][46916] Decorrelating experience for 96 frames... [2025-05-12 23:40:45,853][46914] Decorrelating experience for 96 frames... [2025-05-12 23:40:46,956][46896] Signal inference workers to resume experience collection... [2025-05-12 23:40:46,957][46910] InferenceWorker_p0-w0: resuming experience collection [2025-05-12 23:40:50,290][40219] Fps is (10 sec: 409.6, 60 sec: 204.8, 300 sec: 204.8). Total num frames: 53248. Throughput: 0: 169.6. Samples: 3392. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2025-05-12 23:40:50,291][40219] Avg episode reward: [(0, '3.025')] [2025-05-12 23:40:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 491.5, 300 sec: 491.5). Total num frames: 61440. Throughput: 0: 162.7. Samples: 4068. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-12 23:40:55,299][40219] Avg episode reward: [(0, '3.183')] [2025-05-12 23:41:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 409.6, 300 sec: 409.6). Total num frames: 61440. Throughput: 0: 177.7. Samples: 5330. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-12 23:41:00,291][40219] Avg episode reward: [(0, '3.518')] [2025-05-12 23:41:05,289][40219] Fps is (10 sec: 409.6, 60 sec: 468.1, 300 sec: 468.1). Total num frames: 65536. Throughput: 0: 186.8. Samples: 6538. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-12 23:41:05,291][40219] Avg episode reward: [(0, '3.703')] [2025-05-12 23:41:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 614.4, 300 sec: 614.4). Total num frames: 73728. Throughput: 0: 187.5. Samples: 7500. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:41:10,293][40219] Avg episode reward: [(0, '3.749')] [2025-05-12 23:41:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 637.2, 300 sec: 637.2). Total num frames: 77824. Throughput: 0: 192.7. Samples: 8672. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:15,296][40219] Avg episode reward: [(0, '3.831')] [2025-05-12 23:41:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 655.4, 300 sec: 655.4). Total num frames: 81920. Throughput: 0: 227.0. Samples: 10214. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:20,291][40219] Avg episode reward: [(0, '4.027')] [2025-05-12 23:41:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 744.7, 300 sec: 744.7). Total num frames: 90112. Throughput: 0: 237.0. Samples: 10958. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:25,295][40219] Avg episode reward: [(0, '4.238')] [2025-05-12 23:41:25,569][46910] Updated weights for policy 0, policy_version 22 (0.1403) [2025-05-12 23:41:30,293][40219] Fps is (10 sec: 1228.4, 60 sec: 750.9, 300 sec: 750.9). Total num frames: 94208. Throughput: 0: 241.9. Samples: 12616. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:30,295][40219] Avg episode reward: [(0, '4.359')] [2025-05-12 23:41:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 756.2). Total num frames: 98304. Throughput: 0: 236.7. Samples: 14044. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:35,293][40219] Avg episode reward: [(0, '4.481')] [2025-05-12 23:41:40,289][40219] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 760.7). Total num frames: 102400. Throughput: 0: 239.2. Samples: 14830. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-12 23:41:40,294][40219] Avg episode reward: [(0, '4.446')] [2025-05-12 23:41:45,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 764.6). Total num frames: 106496. Throughput: 0: 244.1. Samples: 16314. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-12 23:41:45,291][40219] Avg episode reward: [(0, '4.426')] [2025-05-12 23:41:46,207][46896] Saving new best policy, reward=4.446! [2025-05-12 23:41:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 819.2). Total num frames: 114688. Throughput: 0: 250.8. Samples: 17824. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:50,291][40219] Avg episode reward: [(0, '4.410')] [2025-05-12 23:41:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 819.2). Total num frames: 118784. Throughput: 0: 252.1. Samples: 18844. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:41:55,291][40219] Avg episode reward: [(0, '4.397')] [2025-05-12 23:42:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 819.2). Total num frames: 122880. Throughput: 0: 257.4. Samples: 20254. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:42:00,291][40219] Avg episode reward: [(0, '4.351')] [2025-05-12 23:42:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 819.2). Total num frames: 126976. Throughput: 0: 262.6. Samples: 22030. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-12 23:42:05,293][40219] Avg episode reward: [(0, '4.400')] [2025-05-12 23:42:05,683][46910] Updated weights for policy 0, policy_version 32 (0.0574) [2025-05-12 23:42:09,252][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000033_135168.pth... [2025-05-12 23:42:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 860.2). Total num frames: 135168. Throughput: 0: 265.0. Samples: 22884. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:42:10,291][40219] Avg episode reward: [(0, '4.383')] [2025-05-12 23:42:15,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 858.2). Total num frames: 139264. Throughput: 0: 256.2. Samples: 24144. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:42:15,291][40219] Avg episode reward: [(0, '4.399')] [2025-05-12 23:42:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 856.4). Total num frames: 143360. Throughput: 0: 260.2. Samples: 25754. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:42:20,294][40219] Avg episode reward: [(0, '4.374')] [2025-05-12 23:42:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 854.8). Total num frames: 147456. Throughput: 0: 255.7. Samples: 26338. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:42:25,293][40219] Avg episode reward: [(0, '4.367')] [2025-05-12 23:42:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 853.3). Total num frames: 151552. Throughput: 0: 260.7. Samples: 28044. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:42:30,291][40219] Avg episode reward: [(0, '4.383')] [2025-05-12 23:42:35,292][40219] Fps is (10 sec: 1228.5, 60 sec: 1024.0, 300 sec: 884.7). Total num frames: 159744. Throughput: 0: 256.7. Samples: 29376. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:42:35,293][40219] Avg episode reward: [(0, '4.353')] [2025-05-12 23:42:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 882.2). Total num frames: 163840. Throughput: 0: 253.9. Samples: 30270. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:42:40,295][40219] Avg episode reward: [(0, '4.318')] [2025-05-12 23:42:45,289][40219] Fps is (10 sec: 819.4, 60 sec: 1024.0, 300 sec: 879.9). Total num frames: 167936. Throughput: 0: 255.1. Samples: 31732. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:42:45,294][40219] Avg episode reward: [(0, '4.318')] [2025-05-12 23:42:46,722][46910] Updated weights for policy 0, policy_version 42 (0.0493) [2025-05-12 23:42:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 907.0). Total num frames: 176128. Throughput: 0: 252.3. Samples: 33384. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:42:50,292][40219] Avg episode reward: [(0, '4.300')] [2025-05-12 23:42:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 903.9). Total num frames: 180224. Throughput: 0: 254.8. Samples: 34352. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:42:55,291][40219] Avg episode reward: [(0, '4.239')] [2025-05-12 23:43:00,291][40219] Fps is (10 sec: 819.1, 60 sec: 1024.0, 300 sec: 901.1). Total num frames: 184320. Throughput: 0: 252.2. Samples: 35492. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:43:00,296][40219] Avg episode reward: [(0, '4.242')] [2025-05-12 23:43:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 898.5). Total num frames: 188416. Throughput: 0: 254.1. Samples: 37190. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:43:05,291][40219] Avg episode reward: [(0, '4.372')] [2025-05-12 23:43:10,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 921.6). Total num frames: 196608. Throughput: 0: 262.6. Samples: 38156. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:43:10,295][40219] Avg episode reward: [(0, '4.343')] [2025-05-12 23:43:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 918.5). Total num frames: 200704. Throughput: 0: 256.5. Samples: 39586. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:43:15,291][40219] Avg episode reward: [(0, '4.408')] [2025-05-12 23:43:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 915.6). Total num frames: 204800. Throughput: 0: 260.5. Samples: 41100. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:43:20,291][40219] Avg episode reward: [(0, '4.416')] [2025-05-12 23:43:25,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 912.8). Total num frames: 208896. Throughput: 0: 258.6. Samples: 41908. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:43:25,294][40219] Avg episode reward: [(0, '4.302')] [2025-05-12 23:43:25,644][46910] Updated weights for policy 0, policy_version 52 (0.0947) [2025-05-12 23:43:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 910.2). Total num frames: 212992. Throughput: 0: 259.1. Samples: 43390. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:43:30,293][40219] Avg episode reward: [(0, '4.346')] [2025-05-12 23:43:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 929.9). Total num frames: 221184. Throughput: 0: 253.9. Samples: 44810. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:43:35,294][40219] Avg episode reward: [(0, '4.444')] [2025-05-12 23:43:40,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 927.0). Total num frames: 225280. Throughput: 0: 251.0. Samples: 45646. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:43:40,291][40219] Avg episode reward: [(0, '4.507')] [2025-05-12 23:43:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 924.2). Total num frames: 229376. Throughput: 0: 259.1. Samples: 47152. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:43:45,291][40219] Avg episode reward: [(0, '4.461')] [2025-05-12 23:43:46,386][46896] Saving new best policy, reward=4.507! [2025-05-12 23:43:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 942.1). Total num frames: 237568. Throughput: 0: 254.8. Samples: 48658. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:43:50,297][40219] Avg episode reward: [(0, '4.484')] [2025-05-12 23:43:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 939.1). Total num frames: 241664. Throughput: 0: 251.2. Samples: 49460. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:43:55,293][40219] Avg episode reward: [(0, '4.416')] [2025-05-12 23:44:00,289][40219] Fps is (10 sec: 409.6, 60 sec: 955.8, 300 sec: 916.7). Total num frames: 241664. Throughput: 0: 244.4. Samples: 50582. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:44:00,291][40219] Avg episode reward: [(0, '4.438')] [2025-05-12 23:44:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 933.5). Total num frames: 249856. Throughput: 0: 243.3. Samples: 52050. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:44:05,291][40219] Avg episode reward: [(0, '4.429')] [2025-05-12 23:44:07,656][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000062_253952.pth... [2025-05-12 23:44:07,662][46910] Updated weights for policy 0, policy_version 62 (0.0046) [2025-05-12 23:44:07,754][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000012_49152.pth [2025-05-12 23:44:09,883][46896] Signal inference workers to stop experience collection... (50 times) [2025-05-12 23:44:09,938][46910] InferenceWorker_p0-w0: stopping experience collection (50 times) [2025-05-12 23:44:10,290][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 930.9). Total num frames: 253952. Throughput: 0: 243.8. Samples: 52880. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:44:10,295][40219] Avg episode reward: [(0, '4.491')] [2025-05-12 23:44:11,810][46896] Signal inference workers to resume experience collection... (50 times) [2025-05-12 23:44:11,811][46910] InferenceWorker_p0-w0: resuming experience collection (50 times) [2025-05-12 23:44:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 928.4). Total num frames: 258048. Throughput: 0: 240.9. Samples: 54232. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:44:15,291][40219] Avg episode reward: [(0, '4.521')] [2025-05-12 23:44:16,497][46896] Saving new best policy, reward=4.521! [2025-05-12 23:44:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 943.9). Total num frames: 266240. Throughput: 0: 247.9. Samples: 55966. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:44:20,294][40219] Avg episode reward: [(0, '4.433')] [2025-05-12 23:44:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 941.2). Total num frames: 270336. Throughput: 0: 253.1. Samples: 57034. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:44:25,297][40219] Avg episode reward: [(0, '4.403')] [2025-05-12 23:44:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 938.7). Total num frames: 274432. Throughput: 0: 241.3. Samples: 58012. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:44:30,291][40219] Avg episode reward: [(0, '4.334')] [2025-05-12 23:44:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 936.2). Total num frames: 278528. Throughput: 0: 252.6. Samples: 60026. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:44:35,291][40219] Avg episode reward: [(0, '4.403')] [2025-05-12 23:44:40,297][40219] Fps is (10 sec: 1227.9, 60 sec: 1023.9, 300 sec: 950.2). Total num frames: 286720. Throughput: 0: 252.4. Samples: 60822. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:44:40,300][40219] Avg episode reward: [(0, '4.399')] [2025-05-12 23:44:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 947.7). Total num frames: 290816. Throughput: 0: 256.1. Samples: 62106. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:44:45,293][40219] Avg episode reward: [(0, '4.393')] [2025-05-12 23:44:48,368][46910] Updated weights for policy 0, policy_version 72 (0.1095) [2025-05-12 23:44:50,289][40219] Fps is (10 sec: 819.8, 60 sec: 955.7, 300 sec: 945.2). Total num frames: 294912. Throughput: 0: 258.8. Samples: 63698. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:44:50,291][40219] Avg episode reward: [(0, '4.303')] [2025-05-12 23:44:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 942.9). Total num frames: 299008. Throughput: 0: 257.6. Samples: 64470. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:44:55,293][40219] Avg episode reward: [(0, '4.333')] [2025-05-12 23:45:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 940.6). Total num frames: 303104. Throughput: 0: 258.3. Samples: 65854. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:00,294][40219] Avg episode reward: [(0, '4.345')] [2025-05-12 23:45:05,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 953.3). Total num frames: 311296. Throughput: 0: 253.6. Samples: 67378. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:05,295][40219] Avg episode reward: [(0, '4.303')] [2025-05-12 23:45:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 950.9). Total num frames: 315392. Throughput: 0: 249.7. Samples: 68270. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:45:10,298][40219] Avg episode reward: [(0, '4.424')] [2025-05-12 23:45:15,295][40219] Fps is (10 sec: 818.8, 60 sec: 1023.9, 300 sec: 948.5). Total num frames: 319488. Throughput: 0: 257.3. Samples: 69594. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:45:15,296][40219] Avg episode reward: [(0, '4.424')] [2025-05-12 23:45:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 960.4). Total num frames: 327680. Throughput: 0: 251.0. Samples: 71322. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:20,291][40219] Avg episode reward: [(0, '4.419')] [2025-05-12 23:45:25,289][40219] Fps is (10 sec: 1229.5, 60 sec: 1024.0, 300 sec: 958.0). Total num frames: 331776. Throughput: 0: 255.2. Samples: 72306. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:25,291][40219] Avg episode reward: [(0, '4.444')] [2025-05-12 23:45:28,996][46910] Updated weights for policy 0, policy_version 82 (0.0496) [2025-05-12 23:45:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 971.9). Total num frames: 335872. Throughput: 0: 250.8. Samples: 73394. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:30,293][40219] Avg episode reward: [(0, '4.468')] [2025-05-12 23:45:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 339968. Throughput: 0: 257.6. Samples: 75288. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:45:35,295][40219] Avg episode reward: [(0, '4.517')] [2025-05-12 23:45:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 348160. Throughput: 0: 257.2. Samples: 76046. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:40,292][40219] Avg episode reward: [(0, '4.496')] [2025-05-12 23:45:45,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 352256. Throughput: 0: 251.9. Samples: 77190. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:45:45,292][40219] Avg episode reward: [(0, '4.549')] [2025-05-12 23:45:48,658][46896] Saving new best policy, reward=4.549! [2025-05-12 23:45:50,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 356352. Throughput: 0: 255.3. Samples: 78868. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:45:50,294][40219] Avg episode reward: [(0, '4.526')] [2025-05-12 23:45:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 360448. Throughput: 0: 249.8. Samples: 79510. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-12 23:45:55,292][40219] Avg episode reward: [(0, '4.480')] [2025-05-12 23:46:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 364544. Throughput: 0: 252.3. Samples: 80948. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:46:00,294][40219] Avg episode reward: [(0, '4.531')] [2025-05-12 23:46:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 372736. Throughput: 0: 248.4. Samples: 82502. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:05,291][40219] Avg episode reward: [(0, '4.568')] [2025-05-12 23:46:08,145][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000092_376832.pth... [2025-05-12 23:46:08,150][46910] Updated weights for policy 0, policy_version 92 (0.1095) [2025-05-12 23:46:08,233][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000033_135168.pth [2025-05-12 23:46:08,251][46896] Saving new best policy, reward=4.568! [2025-05-12 23:46:10,294][40219] Fps is (10 sec: 1228.3, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 376832. Throughput: 0: 248.3. Samples: 83480. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:10,296][40219] Avg episode reward: [(0, '4.506')] [2025-05-12 23:46:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 380928. Throughput: 0: 248.2. Samples: 84564. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:15,295][40219] Avg episode reward: [(0, '4.480')] [2025-05-12 23:46:20,289][40219] Fps is (10 sec: 819.5, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 385024. Throughput: 0: 251.6. Samples: 86608. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:20,294][40219] Avg episode reward: [(0, '4.471')] [2025-05-12 23:46:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 393216. Throughput: 0: 252.3. Samples: 87398. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:25,293][40219] Avg episode reward: [(0, '4.477')] [2025-05-12 23:46:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 397312. Throughput: 0: 253.9. Samples: 88614. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:30,294][40219] Avg episode reward: [(0, '4.384')] [2025-05-12 23:46:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 401408. Throughput: 0: 249.9. Samples: 90114. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:35,291][40219] Avg episode reward: [(0, '4.358')] [2025-05-12 23:46:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 405504. Throughput: 0: 256.8. Samples: 91068. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:46:40,291][40219] Avg episode reward: [(0, '4.221')] [2025-05-12 23:46:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 409600. Throughput: 0: 250.3. Samples: 92210. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:46:45,294][40219] Avg episode reward: [(0, '4.261')] [2025-05-12 23:46:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 413696. Throughput: 0: 248.4. Samples: 93678. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:46:50,295][40219] Avg episode reward: [(0, '4.265')] [2025-05-12 23:46:50,802][46910] Updated weights for policy 0, policy_version 102 (0.1024) [2025-05-12 23:46:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 421888. Throughput: 0: 244.2. Samples: 94468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:46:55,295][40219] Avg episode reward: [(0, '4.394')] [2025-05-12 23:47:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 425984. Throughput: 0: 247.9. Samples: 95720. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:47:00,292][40219] Avg episode reward: [(0, '4.374')] [2025-05-12 23:47:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 430080. Throughput: 0: 239.6. Samples: 97392. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:47:05,294][40219] Avg episode reward: [(0, '4.524')] [2025-05-12 23:47:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 434176. Throughput: 0: 239.1. Samples: 98158. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:47:10,291][40219] Avg episode reward: [(0, '4.553')] [2025-05-12 23:47:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 438272. Throughput: 0: 245.0. Samples: 99640. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:47:15,293][40219] Avg episode reward: [(0, '4.537')] [2025-05-12 23:47:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 446464. Throughput: 0: 239.5. Samples: 100890. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:47:20,291][40219] Avg episode reward: [(0, '4.593')] [2025-05-12 23:47:22,842][46896] Saving new best policy, reward=4.593! [2025-05-12 23:47:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 450560. Throughput: 0: 239.8. Samples: 101858. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:47:25,293][40219] Avg episode reward: [(0, '4.675')] [2025-05-12 23:47:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 454656. Throughput: 0: 244.3. Samples: 103202. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:47:30,293][40219] Avg episode reward: [(0, '4.635')] [2025-05-12 23:47:31,560][46896] Saving new best policy, reward=4.675! [2025-05-12 23:47:31,564][46910] Updated weights for policy 0, policy_version 112 (0.0507) [2025-05-12 23:47:33,860][46896] Signal inference workers to stop experience collection... (100 times) [2025-05-12 23:47:33,895][46910] InferenceWorker_p0-w0: stopping experience collection (100 times) [2025-05-12 23:47:35,214][46896] Signal inference workers to resume experience collection... (100 times) [2025-05-12 23:47:35,216][46910] InferenceWorker_p0-w0: resuming experience collection (100 times) [2025-05-12 23:47:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 462848. Throughput: 0: 249.4. Samples: 104902. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:47:35,295][40219] Avg episode reward: [(0, '4.530')] [2025-05-12 23:47:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 466944. Throughput: 0: 253.8. Samples: 105888. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:47:40,291][40219] Avg episode reward: [(0, '4.479')] [2025-05-12 23:47:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 471040. Throughput: 0: 248.2. Samples: 106890. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:47:45,293][40219] Avg episode reward: [(0, '4.483')] [2025-05-12 23:47:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 475136. Throughput: 0: 253.4. Samples: 108796. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:47:50,295][40219] Avg episode reward: [(0, '4.581')] [2025-05-12 23:47:55,293][40219] Fps is (10 sec: 1228.4, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 483328. Throughput: 0: 253.3. Samples: 109556. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:47:55,297][40219] Avg episode reward: [(0, '4.499')] [2025-05-12 23:48:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 487424. Throughput: 0: 249.8. Samples: 110882. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:00,293][40219] Avg episode reward: [(0, '4.510')] [2025-05-12 23:48:05,291][40219] Fps is (10 sec: 819.4, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 491520. Throughput: 0: 252.8. Samples: 112266. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:05,292][40219] Avg episode reward: [(0, '4.452')] [2025-05-12 23:48:07,047][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000121_495616.pth... [2025-05-12 23:48:07,144][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000062_253952.pth [2025-05-12 23:48:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 495616. Throughput: 0: 251.9. Samples: 113194. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:10,297][40219] Avg episode reward: [(0, '4.387')] [2025-05-12 23:48:11,375][46910] Updated weights for policy 0, policy_version 122 (0.0523) [2025-05-12 23:48:15,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 499712. Throughput: 0: 253.3. Samples: 114600. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:15,292][40219] Avg episode reward: [(0, '4.439')] [2025-05-12 23:48:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 507904. Throughput: 0: 248.6. Samples: 116088. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:20,294][40219] Avg episode reward: [(0, '4.565')] [2025-05-12 23:48:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 512000. Throughput: 0: 248.0. Samples: 117046. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:25,291][40219] Avg episode reward: [(0, '4.495')] [2025-05-12 23:48:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 516096. Throughput: 0: 251.0. Samples: 118184. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:48:30,291][40219] Avg episode reward: [(0, '4.377')] [2025-05-12 23:48:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 520192. Throughput: 0: 251.5. Samples: 120114. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:35,295][40219] Avg episode reward: [(0, '4.492')] [2025-05-12 23:48:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 528384. Throughput: 0: 255.1. Samples: 121036. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:40,291][40219] Avg episode reward: [(0, '4.495')] [2025-05-12 23:48:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 532480. Throughput: 0: 251.0. Samples: 122176. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:45,293][40219] Avg episode reward: [(0, '4.510')] [2025-05-12 23:48:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 536576. Throughput: 0: 257.3. Samples: 123842. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:50,291][40219] Avg episode reward: [(0, '4.626')] [2025-05-12 23:48:51,662][46910] Updated weights for policy 0, policy_version 132 (0.0969) [2025-05-12 23:48:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 1013.6). Total num frames: 540672. Throughput: 0: 254.0. Samples: 124626. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:48:55,293][40219] Avg episode reward: [(0, '4.623')] [2025-05-12 23:49:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 544768. Throughput: 0: 253.6. Samples: 126012. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:00,294][40219] Avg episode reward: [(0, '4.612')] [2025-05-12 23:49:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 552960. Throughput: 0: 255.2. Samples: 127570. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:05,293][40219] Avg episode reward: [(0, '4.564')] [2025-05-12 23:49:10,296][40219] Fps is (10 sec: 1228.0, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 557056. Throughput: 0: 251.4. Samples: 128362. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:10,298][40219] Avg episode reward: [(0, '4.676')] [2025-05-12 23:49:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 561152. Throughput: 0: 255.2. Samples: 129668. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:15,291][40219] Avg episode reward: [(0, '4.600')] [2025-05-12 23:49:20,289][40219] Fps is (10 sec: 1229.6, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 569344. Throughput: 0: 252.3. Samples: 131468. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:49:20,291][40219] Avg episode reward: [(0, '4.610')] [2025-05-12 23:49:25,296][40219] Fps is (10 sec: 1228.0, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 573440. Throughput: 0: 253.8. Samples: 132458. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:49:25,298][40219] Avg episode reward: [(0, '4.659')] [2025-05-12 23:49:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 577536. Throughput: 0: 251.1. Samples: 133474. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:49:30,295][40219] Avg episode reward: [(0, '4.618')] [2025-05-12 23:49:34,782][46910] Updated weights for policy 0, policy_version 142 (0.1110) [2025-05-12 23:49:35,289][40219] Fps is (10 sec: 819.7, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 581632. Throughput: 0: 237.2. Samples: 134518. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:49:35,291][40219] Avg episode reward: [(0, '4.568')] [2025-05-12 23:49:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 585728. Throughput: 0: 242.1. Samples: 135520. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:40,291][40219] Avg episode reward: [(0, '4.555')] [2025-05-12 23:49:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 589824. Throughput: 0: 236.6. Samples: 136658. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:45,293][40219] Avg episode reward: [(0, '4.589')] [2025-05-12 23:49:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 593920. Throughput: 0: 248.1. Samples: 138734. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:49:50,294][40219] Avg episode reward: [(0, '4.532')] [2025-05-12 23:49:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 602112. Throughput: 0: 248.1. Samples: 139526. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:49:55,295][40219] Avg episode reward: [(0, '4.502')] [2025-05-12 23:50:00,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 606208. Throughput: 0: 247.1. Samples: 140786. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:00,296][40219] Avg episode reward: [(0, '4.565')] [2025-05-12 23:50:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 610304. Throughput: 0: 244.6. Samples: 142474. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:05,291][40219] Avg episode reward: [(0, '4.382')] [2025-05-12 23:50:10,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 614400. Throughput: 0: 235.3. Samples: 143044. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:10,291][40219] Avg episode reward: [(0, '4.431')] [2025-05-12 23:50:11,640][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000151_618496.pth... [2025-05-12 23:50:11,744][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000092_376832.pth [2025-05-12 23:50:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 618496. Throughput: 0: 244.8. Samples: 144488. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:15,294][40219] Avg episode reward: [(0, '4.365')] [2025-05-12 23:50:15,654][46910] Updated weights for policy 0, policy_version 152 (0.0964) [2025-05-12 23:50:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 626688. Throughput: 0: 254.7. Samples: 145978. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:20,291][40219] Avg episode reward: [(0, '4.451')] [2025-05-12 23:50:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 630784. Throughput: 0: 253.7. Samples: 146938. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:25,294][40219] Avg episode reward: [(0, '4.427')] [2025-05-12 23:50:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 634880. Throughput: 0: 255.4. Samples: 148152. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:30,291][40219] Avg episode reward: [(0, '4.436')] [2025-05-12 23:50:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 643072. Throughput: 0: 250.3. Samples: 149998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:35,291][40219] Avg episode reward: [(0, '4.438')] [2025-05-12 23:50:40,292][40219] Fps is (10 sec: 1228.5, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 647168. Throughput: 0: 255.5. Samples: 151026. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:50:40,300][40219] Avg episode reward: [(0, '4.507')] [2025-05-12 23:50:45,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 651264. Throughput: 0: 250.7. Samples: 152066. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:45,294][40219] Avg episode reward: [(0, '4.582')] [2025-05-12 23:50:50,289][40219] Fps is (10 sec: 819.4, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 655360. Throughput: 0: 255.9. Samples: 153990. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:50,291][40219] Avg episode reward: [(0, '4.653')] [2025-05-12 23:50:55,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 659456. Throughput: 0: 255.8. Samples: 154556. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:50:55,291][40219] Avg episode reward: [(0, '4.625')] [2025-05-12 23:50:56,478][46910] Updated weights for policy 0, policy_version 162 (0.2134) [2025-05-12 23:50:58,917][46896] Signal inference workers to stop experience collection... (150 times) [2025-05-12 23:50:58,954][46910] InferenceWorker_p0-w0: stopping experience collection (150 times) [2025-05-12 23:50:59,833][46896] Signal inference workers to resume experience collection... (150 times) [2025-05-12 23:50:59,835][46910] InferenceWorker_p0-w0: resuming experience collection (150 times) [2025-05-12 23:51:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 667648. Throughput: 0: 253.9. Samples: 155912. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:00,295][40219] Avg episode reward: [(0, '4.605')] [2025-05-12 23:51:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 671744. Throughput: 0: 258.4. Samples: 157608. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:05,294][40219] Avg episode reward: [(0, '4.655')] [2025-05-12 23:51:10,294][40219] Fps is (10 sec: 818.8, 60 sec: 1023.9, 300 sec: 999.7). Total num frames: 675840. Throughput: 0: 249.9. Samples: 158184. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:10,298][40219] Avg episode reward: [(0, '4.720')] [2025-05-12 23:51:12,371][46896] Saving new best policy, reward=4.720! [2025-05-12 23:51:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 679936. Throughput: 0: 256.9. Samples: 159714. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:15,295][40219] Avg episode reward: [(0, '4.740')] [2025-05-12 23:51:19,564][46896] Saving new best policy, reward=4.740! [2025-05-12 23:51:20,289][40219] Fps is (10 sec: 1229.3, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 688128. Throughput: 0: 249.0. Samples: 161202. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:51:20,291][40219] Avg episode reward: [(0, '4.662')] [2025-05-12 23:51:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 692224. Throughput: 0: 249.1. Samples: 162236. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:25,293][40219] Avg episode reward: [(0, '4.662')] [2025-05-12 23:51:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 696320. Throughput: 0: 252.9. Samples: 163446. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:30,295][40219] Avg episode reward: [(0, '4.710')] [2025-05-12 23:51:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 700416. Throughput: 0: 254.0. Samples: 165420. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:51:35,291][40219] Avg episode reward: [(0, '4.842')] [2025-05-12 23:51:35,520][46910] Updated weights for policy 0, policy_version 172 (0.1039) [2025-05-12 23:51:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 704512. Throughput: 0: 255.9. Samples: 166072. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:51:40,291][40219] Avg episode reward: [(0, '4.852')] [2025-05-12 23:51:40,577][46896] Saving new best policy, reward=4.842! [2025-05-12 23:51:44,248][46896] Saving new best policy, reward=4.852! [2025-05-12 23:51:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 712704. Throughput: 0: 256.8. Samples: 167466. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:51:45,293][40219] Avg episode reward: [(0, '4.901')] [2025-05-12 23:51:48,034][46896] Saving new best policy, reward=4.901! [2025-05-12 23:51:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 716800. Throughput: 0: 257.6. Samples: 169198. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:50,293][40219] Avg episode reward: [(0, '4.931')] [2025-05-12 23:51:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 720896. Throughput: 0: 255.3. Samples: 169670. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:51:55,291][40219] Avg episode reward: [(0, '4.895')] [2025-05-12 23:51:56,700][46896] Saving new best policy, reward=4.931! [2025-05-12 23:52:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 724992. Throughput: 0: 258.8. Samples: 171360. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:52:00,295][40219] Avg episode reward: [(0, '4.870')] [2025-05-12 23:52:05,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 733184. Throughput: 0: 260.1. Samples: 172906. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:05,293][40219] Avg episode reward: [(0, '4.807')] [2025-05-12 23:52:08,742][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000180_737280.pth... [2025-05-12 23:52:08,865][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000121_495616.pth [2025-05-12 23:52:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 737280. Throughput: 0: 253.5. Samples: 173642. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:10,293][40219] Avg episode reward: [(0, '4.753')] [2025-05-12 23:52:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 741376. Throughput: 0: 259.2. Samples: 175110. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:15,299][40219] Avg episode reward: [(0, '4.699')] [2025-05-12 23:52:16,137][46910] Updated weights for policy 0, policy_version 182 (0.0048) [2025-05-12 23:52:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 749568. Throughput: 0: 251.2. Samples: 176726. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:20,291][40219] Avg episode reward: [(0, '4.646')] [2025-05-12 23:52:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 753664. Throughput: 0: 250.8. Samples: 177356. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:25,295][40219] Avg episode reward: [(0, '4.563')] [2025-05-12 23:52:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 757760. Throughput: 0: 252.5. Samples: 178828. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:52:30,292][40219] Avg episode reward: [(0, '4.454')] [2025-05-12 23:52:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 761856. Throughput: 0: 243.8. Samples: 180170. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:52:35,296][40219] Avg episode reward: [(0, '4.474')] [2025-05-12 23:52:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 765952. Throughput: 0: 249.4. Samples: 180894. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:40,291][40219] Avg episode reward: [(0, '4.445')] [2025-05-12 23:52:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 770048. Throughput: 0: 241.4. Samples: 182222. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:45,291][40219] Avg episode reward: [(0, '4.510')] [2025-05-12 23:52:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 778240. Throughput: 0: 244.1. Samples: 183890. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-12 23:52:50,301][40219] Avg episode reward: [(0, '4.551')] [2025-05-12 23:52:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 782336. Throughput: 0: 241.5. Samples: 184510. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:52:55,295][40219] Avg episode reward: [(0, '4.609')] [2025-05-12 23:52:58,185][46910] Updated weights for policy 0, policy_version 192 (0.0493) [2025-05-12 23:53:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 786432. Throughput: 0: 242.1. Samples: 186004. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:53:00,291][40219] Avg episode reward: [(0, '4.633')] [2025-05-12 23:53:05,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 790528. Throughput: 0: 247.1. Samples: 187846. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:05,292][40219] Avg episode reward: [(0, '4.721')] [2025-05-12 23:53:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 794624. Throughput: 0: 241.2. Samples: 188208. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:10,295][40219] Avg episode reward: [(0, '4.793')] [2025-05-12 23:53:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 802816. Throughput: 0: 246.6. Samples: 189926. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:53:15,291][40219] Avg episode reward: [(0, '4.736')] [2025-05-12 23:53:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 806912. Throughput: 0: 250.8. Samples: 191456. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:20,293][40219] Avg episode reward: [(0, '4.851')] [2025-05-12 23:53:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 811008. Throughput: 0: 246.4. Samples: 191982. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:25,296][40219] Avg episode reward: [(0, '4.977')] [2025-05-12 23:53:30,061][46896] Saving new best policy, reward=4.977! [2025-05-12 23:53:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 819200. Throughput: 0: 257.2. Samples: 193798. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:53:30,291][40219] Avg episode reward: [(0, '4.919')] [2025-05-12 23:53:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 823296. Throughput: 0: 249.4. Samples: 195114. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:35,291][40219] Avg episode reward: [(0, '5.004')] [2025-05-12 23:53:38,794][46896] Saving new best policy, reward=5.004! [2025-05-12 23:53:38,799][46910] Updated weights for policy 0, policy_version 202 (0.0507) [2025-05-12 23:53:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 827392. Throughput: 0: 255.2. Samples: 195992. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:40,291][40219] Avg episode reward: [(0, '4.980')] [2025-05-12 23:53:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 831488. Throughput: 0: 258.2. Samples: 197624. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:45,295][40219] Avg episode reward: [(0, '5.174')] [2025-05-12 23:53:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 835584. Throughput: 0: 248.4. Samples: 199026. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:50,291][40219] Avg episode reward: [(0, '5.198')] [2025-05-12 23:53:51,285][46896] Saving new best policy, reward=5.174! [2025-05-12 23:53:51,455][46896] Saving new best policy, reward=5.198! [2025-05-12 23:53:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 843776. Throughput: 0: 256.4. Samples: 199746. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:53:55,295][40219] Avg episode reward: [(0, '5.335')] [2025-05-12 23:53:58,686][46896] Saving new best policy, reward=5.335! [2025-05-12 23:54:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 847872. Throughput: 0: 253.2. Samples: 201318. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:00,291][40219] Avg episode reward: [(0, '5.565')] [2025-05-12 23:54:02,653][46896] Saving new best policy, reward=5.565! [2025-05-12 23:54:05,291][40219] Fps is (10 sec: 819.1, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 851968. Throughput: 0: 252.5. Samples: 202820. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:05,292][40219] Avg episode reward: [(0, '5.548')] [2025-05-12 23:54:07,442][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000209_856064.pth... [2025-05-12 23:54:07,516][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000151_618496.pth [2025-05-12 23:54:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 856064. Throughput: 0: 253.6. Samples: 203392. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:10,291][40219] Avg episode reward: [(0, '5.381')] [2025-05-12 23:54:15,289][40219] Fps is (10 sec: 1229.0, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 864256. Throughput: 0: 256.6. Samples: 205344. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:15,291][40219] Avg episode reward: [(0, '5.460')] [2025-05-12 23:54:19,280][46910] Updated weights for policy 0, policy_version 212 (0.1420) [2025-05-12 23:54:20,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 868352. Throughput: 0: 252.3. Samples: 206468. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:20,292][40219] Avg episode reward: [(0, '5.421')] [2025-05-12 23:54:21,950][46896] Signal inference workers to stop experience collection... (200 times) [2025-05-12 23:54:22,014][46910] InferenceWorker_p0-w0: stopping experience collection (200 times) [2025-05-12 23:54:23,219][46896] Signal inference workers to resume experience collection... (200 times) [2025-05-12 23:54:23,220][46910] InferenceWorker_p0-w0: resuming experience collection (200 times) [2025-05-12 23:54:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 872448. Throughput: 0: 253.5. Samples: 207400. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:54:25,291][40219] Avg episode reward: [(0, '5.208')] [2025-05-12 23:54:30,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 876544. Throughput: 0: 257.1. Samples: 209192. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:54:30,295][40219] Avg episode reward: [(0, '5.234')] [2025-05-12 23:54:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 880640. Throughput: 0: 254.0. Samples: 210454. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:54:35,299][40219] Avg episode reward: [(0, '5.084')] [2025-05-12 23:54:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 888832. Throughput: 0: 258.2. Samples: 211364. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:54:40,291][40219] Avg episode reward: [(0, '4.996')] [2025-05-12 23:54:45,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 892928. Throughput: 0: 258.0. Samples: 212928. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:54:45,292][40219] Avg episode reward: [(0, '4.974')] [2025-05-12 23:54:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 897024. Throughput: 0: 254.1. Samples: 214256. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:54:50,291][40219] Avg episode reward: [(0, '5.072')] [2025-05-12 23:54:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 905216. Throughput: 0: 260.6. Samples: 215118. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:54:55,291][40219] Avg episode reward: [(0, '5.194')] [2025-05-12 23:54:58,364][46910] Updated weights for policy 0, policy_version 222 (0.0963) [2025-05-12 23:55:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 909312. Throughput: 0: 252.6. Samples: 216712. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:55:00,293][40219] Avg episode reward: [(0, '5.056')] [2025-05-12 23:55:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 913408. Throughput: 0: 258.4. Samples: 218096. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:55:05,293][40219] Avg episode reward: [(0, '5.101')] [2025-05-12 23:55:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 917504. Throughput: 0: 256.2. Samples: 218928. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:55:10,295][40219] Avg episode reward: [(0, '5.039')] [2025-05-12 23:55:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 925696. Throughput: 0: 256.5. Samples: 220736. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:15,296][40219] Avg episode reward: [(0, '5.131')] [2025-05-12 23:55:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 929792. Throughput: 0: 249.7. Samples: 221692. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:20,291][40219] Avg episode reward: [(0, '5.105')] [2025-05-12 23:55:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 933888. Throughput: 0: 246.8. Samples: 222468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:25,291][40219] Avg episode reward: [(0, '5.153')] [2025-05-12 23:55:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 937984. Throughput: 0: 246.3. Samples: 224010. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:30,291][40219] Avg episode reward: [(0, '5.144')] [2025-05-12 23:55:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 942080. Throughput: 0: 247.1. Samples: 225374. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:35,291][40219] Avg episode reward: [(0, '5.277')] [2025-05-12 23:55:39,994][46910] Updated weights for policy 0, policy_version 232 (0.0036) [2025-05-12 23:55:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 950272. Throughput: 0: 247.2. Samples: 226242. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:40,291][40219] Avg episode reward: [(0, '5.255')] [2025-05-12 23:55:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 954368. Throughput: 0: 246.8. Samples: 227820. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:45,291][40219] Avg episode reward: [(0, '5.331')] [2025-05-12 23:55:50,292][40219] Fps is (10 sec: 819.0, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 958464. Throughput: 0: 247.1. Samples: 229214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:50,295][40219] Avg episode reward: [(0, '5.500')] [2025-05-12 23:55:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 962560. Throughput: 0: 246.4. Samples: 230014. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:55:55,295][40219] Avg episode reward: [(0, '5.287')] [2025-05-12 23:56:00,289][40219] Fps is (10 sec: 1229.1, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 970752. Throughput: 0: 249.4. Samples: 231958. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:56:00,294][40219] Avg episode reward: [(0, '5.248')] [2025-05-12 23:56:05,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 974848. Throughput: 0: 251.3. Samples: 233002. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:56:05,294][40219] Avg episode reward: [(0, '5.234')] [2025-05-12 23:56:07,945][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000239_978944.pth... [2025-05-12 23:56:08,040][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000180_737280.pth [2025-05-12 23:56:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 978944. Throughput: 0: 255.4. Samples: 233960. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:10,296][40219] Avg episode reward: [(0, '5.281')] [2025-05-12 23:56:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 983040. Throughput: 0: 260.3. Samples: 235722. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:15,291][40219] Avg episode reward: [(0, '5.100')] [2025-05-12 23:56:20,279][46910] Updated weights for policy 0, policy_version 242 (0.0645) [2025-05-12 23:56:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 991232. Throughput: 0: 259.8. Samples: 237066. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:20,292][40219] Avg episode reward: [(0, '5.128')] [2025-05-12 23:56:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 995328. Throughput: 0: 263.6. Samples: 238104. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:25,291][40219] Avg episode reward: [(0, '5.120')] [2025-05-12 23:56:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 999424. Throughput: 0: 256.8. Samples: 239376. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:30,293][40219] Avg episode reward: [(0, '5.122')] [2025-05-12 23:56:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1003520. Throughput: 0: 262.4. Samples: 241022. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:56:35,291][40219] Avg episode reward: [(0, '5.282')] [2025-05-12 23:56:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1011712. Throughput: 0: 257.8. Samples: 241614. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:56:40,294][40219] Avg episode reward: [(0, '5.324')] [2025-05-12 23:56:45,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1015808. Throughput: 0: 245.7. Samples: 243016. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:56:45,292][40219] Avg episode reward: [(0, '5.400')] [2025-05-12 23:56:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1019904. Throughput: 0: 259.6. Samples: 244686. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:56:50,291][40219] Avg episode reward: [(0, '5.657')] [2025-05-12 23:56:51,718][46896] Saving new best policy, reward=5.657! [2025-05-12 23:56:55,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1024000. Throughput: 0: 256.0. Samples: 245480. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-12 23:56:55,295][40219] Avg episode reward: [(0, '5.642')] [2025-05-12 23:57:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1028096. Throughput: 0: 253.4. Samples: 247126. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:00,294][40219] Avg episode reward: [(0, '5.749')] [2025-05-12 23:57:00,470][46910] Updated weights for policy 0, policy_version 252 (0.0951) [2025-05-12 23:57:03,962][46896] Saving new best policy, reward=5.749! [2025-05-12 23:57:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1036288. Throughput: 0: 253.0. Samples: 248452. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:05,295][40219] Avg episode reward: [(0, '5.701')] [2025-05-12 23:57:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1040384. Throughput: 0: 247.1. Samples: 249224. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:10,295][40219] Avg episode reward: [(0, '5.762')] [2025-05-12 23:57:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1044480. Throughput: 0: 249.2. Samples: 250588. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:15,294][40219] Avg episode reward: [(0, '5.735')] [2025-05-12 23:57:16,244][46896] Saving new best policy, reward=5.762! [2025-05-12 23:57:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1052672. Throughput: 0: 250.0. Samples: 252272. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-12 23:57:20,294][40219] Avg episode reward: [(0, '5.752')] [2025-05-12 23:57:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1056768. Throughput: 0: 259.1. Samples: 253274. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-12 23:57:25,293][40219] Avg episode reward: [(0, '5.659')] [2025-05-12 23:57:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1060864. Throughput: 0: 252.8. Samples: 254392. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:30,297][40219] Avg episode reward: [(0, '5.530')] [2025-05-12 23:57:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1064960. Throughput: 0: 257.1. Samples: 256256. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:35,291][40219] Avg episode reward: [(0, '5.344')] [2025-05-12 23:57:39,500][46910] Updated weights for policy 0, policy_version 262 (0.2015) [2025-05-12 23:57:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 1073152. Throughput: 0: 260.0. Samples: 257182. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:40,291][40219] Avg episode reward: [(0, '5.527')] [2025-05-12 23:57:43,433][46896] Signal inference workers to stop experience collection... (250 times) [2025-05-12 23:57:43,576][46910] InferenceWorker_p0-w0: stopping experience collection (250 times) [2025-05-12 23:57:44,615][46896] Signal inference workers to resume experience collection... (250 times) [2025-05-12 23:57:44,616][46910] InferenceWorker_p0-w0: resuming experience collection (250 times) [2025-05-12 23:57:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1077248. Throughput: 0: 251.2. Samples: 258432. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:45,291][40219] Avg episode reward: [(0, '5.620')] [2025-05-12 23:57:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1081344. Throughput: 0: 257.8. Samples: 260052. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:50,294][40219] Avg episode reward: [(0, '5.541')] [2025-05-12 23:57:55,293][40219] Fps is (10 sec: 818.9, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 1085440. Throughput: 0: 256.1. Samples: 260748. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:57:55,295][40219] Avg episode reward: [(0, '5.253')] [2025-05-12 23:58:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1089536. Throughput: 0: 255.9. Samples: 262102. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:58:00,295][40219] Avg episode reward: [(0, '5.243')] [2025-05-12 23:58:05,289][40219] Fps is (10 sec: 1229.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 1097728. Throughput: 0: 252.7. Samples: 263642. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:58:05,291][40219] Avg episode reward: [(0, '5.474')] [2025-05-12 23:58:08,101][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000269_1101824.pth... [2025-05-12 23:58:08,212][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000209_856064.pth [2025-05-12 23:58:10,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1101824. Throughput: 0: 253.7. Samples: 264692. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:58:10,293][40219] Avg episode reward: [(0, '5.484')] [2025-05-12 23:58:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1105920. Throughput: 0: 251.8. Samples: 265722. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-12 23:58:15,303][40219] Avg episode reward: [(0, '5.554')] [2025-05-12 23:58:20,289][40219] Fps is (10 sec: 819.4, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 1110016. Throughput: 0: 241.8. Samples: 267138. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:20,291][40219] Avg episode reward: [(0, '5.992')] [2025-05-12 23:58:22,139][46896] Saving new best policy, reward=5.992! [2025-05-12 23:58:22,143][46910] Updated weights for policy 0, policy_version 272 (0.0508) [2025-05-12 23:58:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1114112. Throughput: 0: 238.1. Samples: 267898. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:25,291][40219] Avg episode reward: [(0, '5.778')] [2025-05-12 23:58:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1118208. Throughput: 0: 247.5. Samples: 269570. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:30,291][40219] Avg episode reward: [(0, '5.715')] [2025-05-12 23:58:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1126400. Throughput: 0: 240.5. Samples: 270874. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:35,291][40219] Avg episode reward: [(0, '5.590')] [2025-05-12 23:58:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 1130496. Throughput: 0: 248.0. Samples: 271906. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:40,295][40219] Avg episode reward: [(0, '5.976')] [2025-05-12 23:58:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 1134592. Throughput: 0: 249.0. Samples: 273308. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:45,292][40219] Avg episode reward: [(0, '5.988')] [2025-05-12 23:58:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1138688. Throughput: 0: 252.7. Samples: 275012. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:50,291][40219] Avg episode reward: [(0, '5.901')] [2025-05-12 23:58:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 1146880. Throughput: 0: 250.9. Samples: 275982. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:58:55,291][40219] Avg episode reward: [(0, '6.004')] [2025-05-12 23:58:58,626][46896] Saving new best policy, reward=6.004! [2025-05-12 23:59:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1150976. Throughput: 0: 251.6. Samples: 277042. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:59:00,295][40219] Avg episode reward: [(0, '5.996')] [2025-05-12 23:59:03,222][46910] Updated weights for policy 0, policy_version 282 (0.1425) [2025-05-12 23:59:05,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 1155072. Throughput: 0: 258.8. Samples: 278786. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-12 23:59:05,304][40219] Avg episode reward: [(0, '5.988')] [2025-05-12 23:59:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1163264. Throughput: 0: 261.9. Samples: 279684. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:59:10,291][40219] Avg episode reward: [(0, '6.166')] [2025-05-12 23:59:15,071][46896] Saving new best policy, reward=6.166! [2025-05-12 23:59:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1167360. Throughput: 0: 252.9. Samples: 280950. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:15,291][40219] Avg episode reward: [(0, '6.330')] [2025-05-12 23:59:18,576][46896] Saving new best policy, reward=6.330! [2025-05-12 23:59:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1171456. Throughput: 0: 260.2. Samples: 282584. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:20,291][40219] Avg episode reward: [(0, '6.420')] [2025-05-12 23:59:22,228][46896] Saving new best policy, reward=6.420! [2025-05-12 23:59:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1175552. Throughput: 0: 253.4. Samples: 283308. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:25,291][40219] Avg episode reward: [(0, '6.426')] [2025-05-12 23:59:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1179648. Throughput: 0: 253.3. Samples: 284708. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:30,296][40219] Avg episode reward: [(0, '6.367')] [2025-05-12 23:59:30,865][46896] Saving new best policy, reward=6.426! [2025-05-12 23:59:35,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1187840. Throughput: 0: 250.6. Samples: 286290. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:59:35,291][40219] Avg episode reward: [(0, '6.711')] [2025-05-12 23:59:37,940][46896] Saving new best policy, reward=6.711! [2025-05-12 23:59:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1191936. Throughput: 0: 250.3. Samples: 287246. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:40,291][40219] Avg episode reward: [(0, '6.709')] [2025-05-12 23:59:43,473][46910] Updated weights for policy 0, policy_version 292 (0.1109) [2025-05-12 23:59:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1196032. Throughput: 0: 250.7. Samples: 288324. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-12 23:59:45,291][40219] Avg episode reward: [(0, '6.757')] [2025-05-12 23:59:46,563][46896] Saving new best policy, reward=6.757! [2025-05-12 23:59:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1013.6). Total num frames: 1204224. Throughput: 0: 255.8. Samples: 290296. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-12 23:59:50,292][40219] Avg episode reward: [(0, '7.027')] [2025-05-12 23:59:54,232][46896] Saving new best policy, reward=7.027! [2025-05-12 23:59:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1208320. Throughput: 0: 254.1. Samples: 291120. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-12 23:59:55,297][40219] Avg episode reward: [(0, '7.171')] [2025-05-12 23:59:59,093][46896] Saving new best policy, reward=7.171! [2025-05-13 00:00:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1212416. Throughput: 0: 252.9. Samples: 292330. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:00:00,296][40219] Avg episode reward: [(0, '7.458')] [2025-05-13 00:00:02,745][46896] Saving new best policy, reward=7.458! [2025-05-13 00:00:05,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1216512. Throughput: 0: 256.7. Samples: 294136. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:05,291][40219] Avg episode reward: [(0, '7.676')] [2025-05-13 00:00:10,292][40219] Fps is (10 sec: 819.0, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1220608. Throughput: 0: 252.3. Samples: 294662. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:10,300][40219] Avg episode reward: [(0, '7.743')] [2025-05-13 00:00:11,491][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000299_1224704.pth... [2025-05-13 00:00:11,593][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000239_978944.pth [2025-05-13 00:00:11,602][46896] Saving new best policy, reward=7.676! [2025-05-13 00:00:11,738][46896] Saving new best policy, reward=7.743! [2025-05-13 00:00:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1224704. Throughput: 0: 253.8. Samples: 296128. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:15,295][40219] Avg episode reward: [(0, '7.853')] [2025-05-13 00:00:19,193][46896] Saving new best policy, reward=7.853! [2025-05-13 00:00:20,289][40219] Fps is (10 sec: 1229.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1232896. Throughput: 0: 252.7. Samples: 297662. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:20,291][40219] Avg episode reward: [(0, '7.704')] [2025-05-13 00:00:23,562][46910] Updated weights for policy 0, policy_version 302 (0.1623) [2025-05-13 00:00:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1236992. Throughput: 0: 249.9. Samples: 298490. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:25,291][40219] Avg episode reward: [(0, '7.681')] [2025-05-13 00:00:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1241088. Throughput: 0: 253.5. Samples: 299732. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:00:30,291][40219] Avg episode reward: [(0, '7.478')] [2025-05-13 00:00:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1249280. Throughput: 0: 251.8. Samples: 301626. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:00:35,292][40219] Avg episode reward: [(0, '7.609')] [2025-05-13 00:00:40,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1253376. Throughput: 0: 256.3. Samples: 302652. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:00:40,291][40219] Avg episode reward: [(0, '7.478')] [2025-05-13 00:00:45,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1257472. Throughput: 0: 250.8. Samples: 303618. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:00:45,293][40219] Avg episode reward: [(0, '7.152')] [2025-05-13 00:00:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 1261568. Throughput: 0: 255.1. Samples: 305616. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:00:50,297][40219] Avg episode reward: [(0, '6.874')] [2025-05-13 00:00:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1265664. Throughput: 0: 255.6. Samples: 306164. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:00:55,292][40219] Avg episode reward: [(0, '6.809')] [2025-05-13 00:01:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1273856. Throughput: 0: 255.7. Samples: 307636. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:00,291][40219] Avg episode reward: [(0, '6.606')] [2025-05-13 00:01:03,270][46910] Updated weights for policy 0, policy_version 312 (0.0486) [2025-05-13 00:01:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1277952. Throughput: 0: 258.7. Samples: 309302. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:05,291][40219] Avg episode reward: [(0, '6.726')] [2025-05-13 00:01:05,529][46896] Signal inference workers to stop experience collection... (300 times) [2025-05-13 00:01:05,573][46910] InferenceWorker_p0-w0: stopping experience collection (300 times) [2025-05-13 00:01:07,452][46896] Signal inference workers to resume experience collection... (300 times) [2025-05-13 00:01:07,455][46910] InferenceWorker_p0-w0: resuming experience collection (300 times) [2025-05-13 00:01:10,292][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 1282048. Throughput: 0: 252.2. Samples: 309840. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:01:10,294][40219] Avg episode reward: [(0, '6.893')] [2025-05-13 00:01:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1286144. Throughput: 0: 246.7. Samples: 310832. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:01:15,291][40219] Avg episode reward: [(0, '7.044')] [2025-05-13 00:01:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1290240. Throughput: 0: 250.8. Samples: 312910. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:20,293][40219] Avg episode reward: [(0, '7.201')] [2025-05-13 00:01:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1294336. Throughput: 0: 243.4. Samples: 313604. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:25,291][40219] Avg episode reward: [(0, '6.929')] [2025-05-13 00:01:30,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1302528. Throughput: 0: 251.8. Samples: 314950. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:01:30,291][40219] Avg episode reward: [(0, '7.062')] [2025-05-13 00:01:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1306624. Throughput: 0: 242.1. Samples: 316510. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:01:35,291][40219] Avg episode reward: [(0, '7.138')] [2025-05-13 00:01:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1310720. Throughput: 0: 246.9. Samples: 317276. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:01:40,291][40219] Avg episode reward: [(0, '7.040')] [2025-05-13 00:01:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1314816. Throughput: 0: 243.1. Samples: 318576. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:45,293][40219] Avg episode reward: [(0, '7.268')] [2025-05-13 00:01:45,966][46910] Updated weights for policy 0, policy_version 322 (0.1434) [2025-05-13 00:01:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1323008. Throughput: 0: 239.5. Samples: 320078. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:50,296][40219] Avg episode reward: [(0, '7.474')] [2025-05-13 00:01:55,293][40219] Fps is (10 sec: 1228.4, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 1327104. Throughput: 0: 248.8. Samples: 321038. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:01:55,294][40219] Avg episode reward: [(0, '7.578')] [2025-05-13 00:02:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1331200. Throughput: 0: 251.1. Samples: 322132. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:02:00,291][40219] Avg episode reward: [(0, '7.720')] [2025-05-13 00:02:05,289][40219] Fps is (10 sec: 1229.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1339392. Throughput: 0: 247.2. Samples: 324032. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:05,291][40219] Avg episode reward: [(0, '7.553')] [2025-05-13 00:02:09,912][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000328_1343488.pth... [2025-05-13 00:02:10,089][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000269_1101824.pth [2025-05-13 00:02:10,294][40219] Fps is (10 sec: 1228.3, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 1343488. Throughput: 0: 251.4. Samples: 324918. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:10,296][40219] Avg episode reward: [(0, '7.609')] [2025-05-13 00:02:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1347584. Throughput: 0: 246.8. Samples: 326058. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:02:15,303][40219] Avg episode reward: [(0, '7.747')] [2025-05-13 00:02:20,289][40219] Fps is (10 sec: 819.6, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1351680. Throughput: 0: 255.6. Samples: 328010. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:02:20,296][40219] Avg episode reward: [(0, '7.630')] [2025-05-13 00:02:25,290][40219] Fps is (10 sec: 819.1, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1355776. Throughput: 0: 248.8. Samples: 328474. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:02:25,296][40219] Avg episode reward: [(0, '7.717')] [2025-05-13 00:02:26,540][46910] Updated weights for policy 0, policy_version 332 (0.0935) [2025-05-13 00:02:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1363968. Throughput: 0: 255.9. Samples: 330092. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:30,295][40219] Avg episode reward: [(0, '7.815')] [2025-05-13 00:02:35,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1368064. Throughput: 0: 255.4. Samples: 331570. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:35,291][40219] Avg episode reward: [(0, '8.197')] [2025-05-13 00:02:38,280][46896] Saving new best policy, reward=8.197! [2025-05-13 00:02:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1372160. Throughput: 0: 249.5. Samples: 332264. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:40,291][40219] Avg episode reward: [(0, '8.179')] [2025-05-13 00:02:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1376256. Throughput: 0: 257.4. Samples: 333716. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:45,300][40219] Avg episode reward: [(0, '8.000')] [2025-05-13 00:02:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1384448. Throughput: 0: 250.4. Samples: 335300. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:02:50,294][40219] Avg episode reward: [(0, '8.016')] [2025-05-13 00:02:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 1388544. Throughput: 0: 244.5. Samples: 335920. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:02:55,291][40219] Avg episode reward: [(0, '8.356')] [2025-05-13 00:02:58,966][46896] Saving new best policy, reward=8.356! [2025-05-13 00:03:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1392640. Throughput: 0: 251.6. Samples: 337380. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:03:00,291][40219] Avg episode reward: [(0, '8.436')] [2025-05-13 00:03:02,657][46896] Saving new best policy, reward=8.436! [2025-05-13 00:03:05,292][40219] Fps is (10 sec: 819.0, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1396736. Throughput: 0: 249.1. Samples: 339218. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:05,293][40219] Avg episode reward: [(0, '8.620')] [2025-05-13 00:03:07,242][46910] Updated weights for policy 0, policy_version 342 (0.1468) [2025-05-13 00:03:10,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 1400832. Throughput: 0: 247.1. Samples: 339592. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:10,292][40219] Avg episode reward: [(0, '8.794')] [2025-05-13 00:03:11,677][46896] Saving new best policy, reward=8.620! [2025-05-13 00:03:11,786][46896] Saving new best policy, reward=8.794! [2025-05-13 00:03:15,289][40219] Fps is (10 sec: 819.4, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1404928. Throughput: 0: 249.8. Samples: 341332. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:15,295][40219] Avg episode reward: [(0, '8.785')] [2025-05-13 00:03:20,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1413120. Throughput: 0: 243.8. Samples: 342542. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:03:20,291][40219] Avg episode reward: [(0, '8.769')] [2025-05-13 00:03:25,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1417216. Throughput: 0: 243.4. Samples: 343218. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:03:25,295][40219] Avg episode reward: [(0, '8.783')] [2025-05-13 00:03:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1421312. Throughput: 0: 240.4. Samples: 344536. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:30,294][40219] Avg episode reward: [(0, '9.371')] [2025-05-13 00:03:31,807][46896] Saving new best policy, reward=9.371! [2025-05-13 00:03:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1425408. Throughput: 0: 250.6. Samples: 346578. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:35,291][40219] Avg episode reward: [(0, '9.205')] [2025-05-13 00:03:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1429504. Throughput: 0: 245.3. Samples: 346960. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:03:40,296][40219] Avg episode reward: [(0, '9.587')] [2025-05-13 00:03:44,392][46896] Saving new best policy, reward=9.587! [2025-05-13 00:03:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 1437696. Throughput: 0: 252.7. Samples: 348752. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:45,291][40219] Avg episode reward: [(0, '9.525')] [2025-05-13 00:03:48,474][46910] Updated weights for policy 0, policy_version 352 (0.1463) [2025-05-13 00:03:50,290][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1441792. Throughput: 0: 241.9. Samples: 350104. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:03:50,291][40219] Avg episode reward: [(0, '9.234')] [2025-05-13 00:03:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1445888. Throughput: 0: 248.6. Samples: 350780. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:03:55,293][40219] Avg episode reward: [(0, '9.112')] [2025-05-13 00:04:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1449984. Throughput: 0: 246.1. Samples: 352408. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:04:00,291][40219] Avg episode reward: [(0, '9.049')] [2025-05-13 00:04:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 985.8). Total num frames: 1454080. Throughput: 0: 249.6. Samples: 353774. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:04:05,291][40219] Avg episode reward: [(0, '8.989')] [2025-05-13 00:04:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1458176. Throughput: 0: 241.8. Samples: 354100. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:04:10,291][40219] Avg episode reward: [(0, '8.816')] [2025-05-13 00:04:11,181][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000357_1462272.pth... [2025-05-13 00:04:11,276][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000299_1224704.pth [2025-05-13 00:04:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1466368. Throughput: 0: 251.6. Samples: 355856. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:04:15,298][40219] Avg episode reward: [(0, '9.186')] [2025-05-13 00:04:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1470464. Throughput: 0: 239.1. Samples: 357336. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:20,295][40219] Avg episode reward: [(0, '9.318')] [2025-05-13 00:04:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1474560. Throughput: 0: 244.0. Samples: 357938. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:25,291][40219] Avg episode reward: [(0, '9.113')] [2025-05-13 00:04:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1478656. Throughput: 0: 237.6. Samples: 359446. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:30,295][40219] Avg episode reward: [(0, '8.414')] [2025-05-13 00:04:30,963][46910] Updated weights for policy 0, policy_version 362 (0.0038) [2025-05-13 00:04:33,152][46896] Signal inference workers to stop experience collection... (350 times) [2025-05-13 00:04:33,186][46910] InferenceWorker_p0-w0: stopping experience collection (350 times) [2025-05-13 00:04:34,542][46896] Signal inference workers to resume experience collection... (350 times) [2025-05-13 00:04:34,545][46910] InferenceWorker_p0-w0: resuming experience collection (350 times) [2025-05-13 00:04:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1486848. Throughput: 0: 241.6. Samples: 360978. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:35,291][40219] Avg episode reward: [(0, '8.674')] [2025-05-13 00:04:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1490944. Throughput: 0: 239.9. Samples: 361574. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:40,294][40219] Avg episode reward: [(0, '9.264')] [2025-05-13 00:04:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1495040. Throughput: 0: 236.6. Samples: 363056. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:45,295][40219] Avg episode reward: [(0, '9.347')] [2025-05-13 00:04:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1499136. Throughput: 0: 251.2. Samples: 365078. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:04:50,295][40219] Avg episode reward: [(0, '9.469')] [2025-05-13 00:04:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1503232. Throughput: 0: 249.6. Samples: 365334. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:04:55,294][40219] Avg episode reward: [(0, '9.583')] [2025-05-13 00:05:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1511424. Throughput: 0: 248.8. Samples: 367050. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:05:00,291][40219] Avg episode reward: [(0, '9.763')] [2025-05-13 00:05:03,025][46896] Saving new best policy, reward=9.763! [2025-05-13 00:05:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1515520. Throughput: 0: 250.0. Samples: 368586. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:05:05,293][40219] Avg episode reward: [(0, '9.760')] [2025-05-13 00:05:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1519616. Throughput: 0: 247.6. Samples: 369082. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:05:10,291][40219] Avg episode reward: [(0, '9.965')] [2025-05-13 00:05:11,968][46896] Saving new best policy, reward=9.965! [2025-05-13 00:05:11,982][46910] Updated weights for policy 0, policy_version 372 (0.0631) [2025-05-13 00:05:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1523712. Throughput: 0: 253.0. Samples: 370832. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:05:15,297][40219] Avg episode reward: [(0, '9.839')] [2025-05-13 00:05:20,296][40219] Fps is (10 sec: 1228.0, 60 sec: 1023.9, 300 sec: 999.7). Total num frames: 1531904. Throughput: 0: 249.0. Samples: 372186. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:05:20,297][40219] Avg episode reward: [(0, '9.778')] [2025-05-13 00:05:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1536000. Throughput: 0: 250.1. Samples: 372828. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:05:25,291][40219] Avg episode reward: [(0, '9.854')] [2025-05-13 00:05:30,289][40219] Fps is (10 sec: 819.7, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1540096. Throughput: 0: 251.6. Samples: 374376. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:05:30,291][40219] Avg episode reward: [(0, '10.411')] [2025-05-13 00:05:31,638][46896] Saving new best policy, reward=10.411! [2025-05-13 00:05:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1544192. Throughput: 0: 244.4. Samples: 376074. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:05:35,291][40219] Avg episode reward: [(0, '10.148')] [2025-05-13 00:05:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1548288. Throughput: 0: 245.6. Samples: 376384. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:05:40,291][40219] Avg episode reward: [(0, '10.227')] [2025-05-13 00:05:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1556480. Throughput: 0: 251.1. Samples: 378348. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:05:45,293][40219] Avg episode reward: [(0, '9.767')] [2025-05-13 00:05:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1560576. Throughput: 0: 247.2. Samples: 379708. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:05:50,294][40219] Avg episode reward: [(0, '9.681')] [2025-05-13 00:05:52,890][46910] Updated weights for policy 0, policy_version 382 (0.1058) [2025-05-13 00:05:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1564672. Throughput: 0: 251.7. Samples: 380410. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:05:55,295][40219] Avg episode reward: [(0, '9.681')] [2025-05-13 00:06:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1572864. Throughput: 0: 254.3. Samples: 382274. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:00,291][40219] Avg episode reward: [(0, '9.662')] [2025-05-13 00:06:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1576960. Throughput: 0: 251.4. Samples: 383498. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:05,294][40219] Avg episode reward: [(0, '9.951')] [2025-05-13 00:06:09,350][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000386_1581056.pth... [2025-05-13 00:06:09,448][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000328_1343488.pth [2025-05-13 00:06:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1581056. Throughput: 0: 251.8. Samples: 384160. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:10,295][40219] Avg episode reward: [(0, '9.824')] [2025-05-13 00:06:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1585152. Throughput: 0: 253.1. Samples: 385764. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:15,295][40219] Avg episode reward: [(0, '9.806')] [2025-05-13 00:06:20,293][40219] Fps is (10 sec: 818.9, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 1589248. Throughput: 0: 249.0. Samples: 387280. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:20,297][40219] Avg episode reward: [(0, '9.956')] [2025-05-13 00:06:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1593344. Throughput: 0: 253.6. Samples: 387794. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:06:25,291][40219] Avg episode reward: [(0, '10.075')] [2025-05-13 00:06:30,289][40219] Fps is (10 sec: 1229.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1601536. Throughput: 0: 254.2. Samples: 389788. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:06:30,295][40219] Avg episode reward: [(0, '10.013')] [2025-05-13 00:06:34,491][46910] Updated weights for policy 0, policy_version 392 (0.2091) [2025-05-13 00:06:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1605632. Throughput: 0: 246.8. Samples: 390812. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:06:35,294][40219] Avg episode reward: [(0, '10.158')] [2025-05-13 00:06:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1609728. Throughput: 0: 254.0. Samples: 391842. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:06:40,292][40219] Avg episode reward: [(0, '9.911')] [2025-05-13 00:06:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1613824. Throughput: 0: 248.4. Samples: 393454. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:06:45,291][40219] Avg episode reward: [(0, '10.301')] [2025-05-13 00:06:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1617920. Throughput: 0: 252.3. Samples: 394850. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:06:50,291][40219] Avg episode reward: [(0, '10.305')] [2025-05-13 00:06:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1626112. Throughput: 0: 253.2. Samples: 395552. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:06:55,292][40219] Avg episode reward: [(0, '10.510')] [2025-05-13 00:06:59,518][46896] Saving new best policy, reward=10.510! [2025-05-13 00:07:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1630208. Throughput: 0: 248.4. Samples: 396942. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:07:00,297][40219] Avg episode reward: [(0, '10.472')] [2025-05-13 00:07:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1634304. Throughput: 0: 237.4. Samples: 397960. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:07:05,294][40219] Avg episode reward: [(0, '10.862')] [2025-05-13 00:07:08,769][46896] Saving new best policy, reward=10.862! [2025-05-13 00:07:10,291][40219] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1638400. Throughput: 0: 248.7. Samples: 398986. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:07:10,296][40219] Avg episode reward: [(0, '10.659')] [2025-05-13 00:07:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1642496. Throughput: 0: 233.9. Samples: 400312. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:15,291][40219] Avg episode reward: [(0, '10.817')] [2025-05-13 00:07:16,418][46910] Updated weights for policy 0, policy_version 402 (0.0511) [2025-05-13 00:07:20,294][40219] Fps is (10 sec: 818.9, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1646592. Throughput: 0: 244.5. Samples: 401814. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:20,299][40219] Avg episode reward: [(0, '10.969')] [2025-05-13 00:07:21,420][46896] Saving new best policy, reward=10.969! [2025-05-13 00:07:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1654784. Throughput: 0: 238.0. Samples: 402550. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:07:25,292][40219] Avg episode reward: [(0, '11.083')] [2025-05-13 00:07:28,621][46896] Saving new best policy, reward=11.083! [2025-05-13 00:07:30,289][40219] Fps is (10 sec: 1229.3, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1658880. Throughput: 0: 236.5. Samples: 404098. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:07:30,291][40219] Avg episode reward: [(0, '10.978')] [2025-05-13 00:07:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1662976. Throughput: 0: 235.6. Samples: 405454. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:07:35,293][40219] Avg episode reward: [(0, '10.992')] [2025-05-13 00:07:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1667072. Throughput: 0: 237.2. Samples: 406224. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:40,295][40219] Avg episode reward: [(0, '11.199')] [2025-05-13 00:07:44,471][46896] Saving new best policy, reward=11.199! [2025-05-13 00:07:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1675264. Throughput: 0: 250.0. Samples: 408190. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:45,291][40219] Avg episode reward: [(0, '11.401')] [2025-05-13 00:07:49,638][46896] Saving new best policy, reward=11.401! [2025-05-13 00:07:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1679360. Throughput: 0: 248.8. Samples: 409158. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:50,291][40219] Avg episode reward: [(0, '11.398')] [2025-05-13 00:07:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1683456. Throughput: 0: 249.3. Samples: 410204. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:07:55,291][40219] Avg episode reward: [(0, '11.319')] [2025-05-13 00:07:56,942][46910] Updated weights for policy 0, policy_version 412 (0.0052) [2025-05-13 00:07:59,088][46896] Signal inference workers to stop experience collection... (400 times) [2025-05-13 00:07:59,126][46910] InferenceWorker_p0-w0: stopping experience collection (400 times) [2025-05-13 00:08:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1687552. Throughput: 0: 256.9. Samples: 411872. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:08:00,291][40219] Avg episode reward: [(0, '11.300')] [2025-05-13 00:08:01,096][46896] Signal inference workers to resume experience collection... (400 times) [2025-05-13 00:08:01,098][46910] InferenceWorker_p0-w0: resuming experience collection (400 times) [2025-05-13 00:08:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1691648. Throughput: 0: 250.2. Samples: 413070. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:08:05,295][40219] Avg episode reward: [(0, '11.731')] [2025-05-13 00:08:09,041][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000415_1699840.pth... [2025-05-13 00:08:09,142][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000357_1462272.pth [2025-05-13 00:08:09,153][46896] Saving new best policy, reward=11.731! [2025-05-13 00:08:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1699840. Throughput: 0: 253.9. Samples: 413976. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:08:10,291][40219] Avg episode reward: [(0, '11.941')] [2025-05-13 00:08:12,691][46896] Saving new best policy, reward=11.941! [2025-05-13 00:08:15,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1703936. Throughput: 0: 251.2. Samples: 415404. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:08:15,295][40219] Avg episode reward: [(0, '11.887')] [2025-05-13 00:08:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 985.8). Total num frames: 1708032. Throughput: 0: 251.3. Samples: 416762. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:08:20,292][40219] Avg episode reward: [(0, '11.972')] [2025-05-13 00:08:25,054][46896] Saving new best policy, reward=11.972! [2025-05-13 00:08:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1716224. Throughput: 0: 250.5. Samples: 417498. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:25,291][40219] Avg episode reward: [(0, '12.273')] [2025-05-13 00:08:28,952][46896] Saving new best policy, reward=12.273! [2025-05-13 00:08:30,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1720320. Throughput: 0: 241.4. Samples: 419052. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:30,291][40219] Avg episode reward: [(0, '11.875')] [2025-05-13 00:08:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1724416. Throughput: 0: 247.6. Samples: 420300. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:35,291][40219] Avg episode reward: [(0, '11.985')] [2025-05-13 00:08:38,127][46910] Updated weights for policy 0, policy_version 422 (0.0515) [2025-05-13 00:08:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1728512. Throughput: 0: 241.9. Samples: 421088. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:40,291][40219] Avg episode reward: [(0, '11.865')] [2025-05-13 00:08:45,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1732608. Throughput: 0: 248.9. Samples: 423074. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:08:45,295][40219] Avg episode reward: [(0, '12.074')] [2025-05-13 00:08:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1740800. Throughput: 0: 247.2. Samples: 424194. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:50,296][40219] Avg episode reward: [(0, '11.616')] [2025-05-13 00:08:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1744896. Throughput: 0: 247.4. Samples: 425108. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:08:55,291][40219] Avg episode reward: [(0, '11.519')] [2025-05-13 00:09:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1748992. Throughput: 0: 248.0. Samples: 426564. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:00,291][40219] Avg episode reward: [(0, '11.548')] [2025-05-13 00:09:05,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1753088. Throughput: 0: 251.2. Samples: 428066. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:05,291][40219] Avg episode reward: [(0, '12.302')] [2025-05-13 00:09:09,981][46896] Saving new best policy, reward=12.302! [2025-05-13 00:09:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1761280. Throughput: 0: 247.2. Samples: 428622. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:10,295][40219] Avg episode reward: [(0, '12.405')] [2025-05-13 00:09:14,473][46896] Saving new best policy, reward=12.405! [2025-05-13 00:09:15,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1765376. Throughput: 0: 249.7. Samples: 430290. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:15,293][40219] Avg episode reward: [(0, '12.320')] [2025-05-13 00:09:18,889][46910] Updated weights for policy 0, policy_version 432 (0.0972) [2025-05-13 00:09:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1769472. Throughput: 0: 252.8. Samples: 431678. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:20,294][40219] Avg episode reward: [(0, '11.839')] [2025-05-13 00:09:25,291][40219] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1773568. Throughput: 0: 252.2. Samples: 432438. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:09:25,297][40219] Avg episode reward: [(0, '12.595')] [2025-05-13 00:09:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1777664. Throughput: 0: 243.1. Samples: 434012. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:09:30,291][40219] Avg episode reward: [(0, '12.462')] [2025-05-13 00:09:31,415][46896] Saving new best policy, reward=12.595! [2025-05-13 00:09:35,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1785856. Throughput: 0: 249.9. Samples: 435440. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:35,291][40219] Avg episode reward: [(0, '11.975')] [2025-05-13 00:09:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1789952. Throughput: 0: 252.5. Samples: 436470. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:09:40,295][40219] Avg episode reward: [(0, '12.439')] [2025-05-13 00:09:45,290][40219] Fps is (10 sec: 819.1, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1794048. Throughput: 0: 246.0. Samples: 437636. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:09:45,292][40219] Avg episode reward: [(0, '12.550')] [2025-05-13 00:09:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1798144. Throughput: 0: 253.6. Samples: 439480. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2025-05-13 00:09:50,296][40219] Avg episode reward: [(0, '13.016')] [2025-05-13 00:09:54,756][46896] Saving new best policy, reward=13.016! [2025-05-13 00:09:55,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1806336. Throughput: 0: 257.7. Samples: 440220. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:09:55,295][40219] Avg episode reward: [(0, '12.955')] [2025-05-13 00:10:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1806336. Throughput: 0: 250.0. Samples: 441540. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:10:00,292][40219] Avg episode reward: [(0, '13.514')] [2025-05-13 00:10:01,049][46910] Updated weights for policy 0, policy_version 442 (0.0515) [2025-05-13 00:10:05,289][40219] Fps is (10 sec: 409.6, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1810432. Throughput: 0: 244.7. Samples: 442690. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:10:05,296][40219] Avg episode reward: [(0, '13.783')] [2025-05-13 00:10:05,751][46896] Saving new best policy, reward=13.514! [2025-05-13 00:10:09,394][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000444_1818624.pth... [2025-05-13 00:10:09,496][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000386_1581056.pth [2025-05-13 00:10:09,507][46896] Saving new best policy, reward=13.783! [2025-05-13 00:10:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1818624. Throughput: 0: 246.3. Samples: 443522. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:10:10,291][40219] Avg episode reward: [(0, '13.506')] [2025-05-13 00:10:15,294][40219] Fps is (10 sec: 1228.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1822720. Throughput: 0: 244.1. Samples: 444998. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:10:15,296][40219] Avg episode reward: [(0, '12.622')] [2025-05-13 00:10:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1826816. Throughput: 0: 243.1. Samples: 446380. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:10:20,291][40219] Avg episode reward: [(0, '12.758')] [2025-05-13 00:10:25,289][40219] Fps is (10 sec: 819.6, 60 sec: 955.8, 300 sec: 985.8). Total num frames: 1830912. Throughput: 0: 233.1. Samples: 446960. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:10:25,296][40219] Avg episode reward: [(0, '12.541')] [2025-05-13 00:10:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1839104. Throughput: 0: 250.9. Samples: 448926. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:10:30,291][40219] Avg episode reward: [(0, '13.024')] [2025-05-13 00:10:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1843200. Throughput: 0: 234.7. Samples: 450040. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:10:35,291][40219] Avg episode reward: [(0, '12.649')] [2025-05-13 00:10:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1847296. Throughput: 0: 238.9. Samples: 450972. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:10:40,291][40219] Avg episode reward: [(0, '12.724')] [2025-05-13 00:10:41,570][46910] Updated weights for policy 0, policy_version 452 (0.1432) [2025-05-13 00:10:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1851392. Throughput: 0: 248.7. Samples: 452730. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:10:45,295][40219] Avg episode reward: [(0, '12.450')] [2025-05-13 00:10:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1855488. Throughput: 0: 253.7. Samples: 454108. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:10:50,296][40219] Avg episode reward: [(0, '12.505')] [2025-05-13 00:10:55,290][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1863680. Throughput: 0: 252.5. Samples: 454884. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:10:55,292][40219] Avg episode reward: [(0, '12.365')] [2025-05-13 00:11:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1867776. Throughput: 0: 252.6. Samples: 456366. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:11:00,294][40219] Avg episode reward: [(0, '12.474')] [2025-05-13 00:11:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1871872. Throughput: 0: 254.3. Samples: 457822. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:11:05,292][40219] Avg episode reward: [(0, '12.187')] [2025-05-13 00:11:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1875968. Throughput: 0: 258.6. Samples: 458596. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:11:10,296][40219] Avg episode reward: [(0, '12.553')] [2025-05-13 00:11:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 999.7). Total num frames: 1884160. Throughput: 0: 253.0. Samples: 460312. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:15,295][40219] Avg episode reward: [(0, '12.581')] [2025-05-13 00:11:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1888256. Throughput: 0: 256.5. Samples: 461582. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:20,291][40219] Avg episode reward: [(0, '12.339')] [2025-05-13 00:11:23,113][46910] Updated weights for policy 0, policy_version 462 (0.2025) [2025-05-13 00:11:25,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1892352. Throughput: 0: 254.4. Samples: 462420. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:25,291][40219] Avg episode reward: [(0, '12.555')] [2025-05-13 00:11:25,325][46896] Signal inference workers to stop experience collection... (450 times) [2025-05-13 00:11:25,368][46910] InferenceWorker_p0-w0: stopping experience collection (450 times) [2025-05-13 00:11:26,336][46896] Signal inference workers to resume experience collection... (450 times) [2025-05-13 00:11:26,337][46910] InferenceWorker_p0-w0: resuming experience collection (450 times) [2025-05-13 00:11:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1896448. Throughput: 0: 248.7. Samples: 463922. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:11:30,297][40219] Avg episode reward: [(0, '12.890')] [2025-05-13 00:11:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1900544. Throughput: 0: 253.5. Samples: 465516. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:11:35,295][40219] Avg episode reward: [(0, '13.349')] [2025-05-13 00:11:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1908736. Throughput: 0: 258.1. Samples: 466500. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:40,294][40219] Avg episode reward: [(0, '13.562')] [2025-05-13 00:11:45,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1912832. Throughput: 0: 250.3. Samples: 467628. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:45,291][40219] Avg episode reward: [(0, '13.230')] [2025-05-13 00:11:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 1916928. Throughput: 0: 256.4. Samples: 469362. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:50,291][40219] Avg episode reward: [(0, '14.225')] [2025-05-13 00:11:55,096][46896] Saving new best policy, reward=14.225! [2025-05-13 00:11:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1925120. Throughput: 0: 252.0. Samples: 469938. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:11:55,293][40219] Avg episode reward: [(0, '14.519')] [2025-05-13 00:12:00,106][46896] Saving new best policy, reward=14.519! [2025-05-13 00:12:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1929216. Throughput: 0: 252.0. Samples: 471654. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:00,295][40219] Avg episode reward: [(0, '14.653')] [2025-05-13 00:12:04,118][46896] Saving new best policy, reward=14.653! [2025-05-13 00:12:04,123][46910] Updated weights for policy 0, policy_version 472 (0.1500) [2025-05-13 00:12:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1933312. Throughput: 0: 252.4. Samples: 472942. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:05,291][40219] Avg episode reward: [(0, '14.720')] [2025-05-13 00:12:07,759][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000473_1937408.pth... [2025-05-13 00:12:07,863][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000415_1699840.pth [2025-05-13 00:12:07,872][46896] Saving new best policy, reward=14.720! [2025-05-13 00:12:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1937408. Throughput: 0: 252.0. Samples: 473758. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:12:10,291][40219] Avg episode reward: [(0, '14.537')] [2025-05-13 00:12:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1941504. Throughput: 0: 250.8. Samples: 475206. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:12:15,297][40219] Avg episode reward: [(0, '14.770')] [2025-05-13 00:12:16,799][46896] Saving new best policy, reward=14.770! [2025-05-13 00:12:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1945600. Throughput: 0: 251.2. Samples: 476822. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:12:20,296][40219] Avg episode reward: [(0, '15.117')] [2025-05-13 00:12:24,006][46896] Saving new best policy, reward=15.117! [2025-05-13 00:12:25,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1953792. Throughput: 0: 250.0. Samples: 477750. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:12:25,293][40219] Avg episode reward: [(0, '15.371')] [2025-05-13 00:12:28,870][46896] Saving new best policy, reward=15.371! [2025-05-13 00:12:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1957888. Throughput: 0: 250.6. Samples: 478904. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:12:30,294][40219] Avg episode reward: [(0, '15.527')] [2025-05-13 00:12:32,981][46896] Saving new best policy, reward=15.527! [2025-05-13 00:12:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1961984. Throughput: 0: 249.8. Samples: 480602. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:12:35,291][40219] Avg episode reward: [(0, '15.834')] [2025-05-13 00:12:40,178][46896] Saving new best policy, reward=15.834! [2025-05-13 00:12:40,294][40219] Fps is (10 sec: 1228.3, 60 sec: 1023.9, 300 sec: 999.7). Total num frames: 1970176. Throughput: 0: 253.8. Samples: 481358. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:40,296][40219] Avg episode reward: [(0, '15.742')] [2025-05-13 00:12:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1970176. Throughput: 0: 250.1. Samples: 482908. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:45,296][40219] Avg episode reward: [(0, '14.685')] [2025-05-13 00:12:45,454][46910] Updated weights for policy 0, policy_version 482 (0.0059) [2025-05-13 00:12:50,289][40219] Fps is (10 sec: 819.6, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 1978368. Throughput: 0: 247.3. Samples: 484070. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:50,291][40219] Avg episode reward: [(0, '15.099')] [2025-05-13 00:12:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1982464. Throughput: 0: 251.5. Samples: 485074. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:12:55,291][40219] Avg episode reward: [(0, '14.887')] [2025-05-13 00:13:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 1986560. Throughput: 0: 247.0. Samples: 486320. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:13:00,291][40219] Avg episode reward: [(0, '14.790')] [2025-05-13 00:13:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1990656. Throughput: 0: 240.3. Samples: 487634. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:13:05,291][40219] Avg episode reward: [(0, '14.393')] [2025-05-13 00:13:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 1994752. Throughput: 0: 232.4. Samples: 488206. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:10,297][40219] Avg episode reward: [(0, '14.102')] [2025-05-13 00:13:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2002944. Throughput: 0: 249.9. Samples: 490148. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:15,295][40219] Avg episode reward: [(0, '14.005')] [2025-05-13 00:13:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2007040. Throughput: 0: 238.9. Samples: 491352. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:20,293][40219] Avg episode reward: [(0, '13.833')] [2025-05-13 00:13:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2011136. Throughput: 0: 241.1. Samples: 492208. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:25,292][40219] Avg episode reward: [(0, '14.049')] [2025-05-13 00:13:26,986][46910] Updated weights for policy 0, policy_version 492 (0.1932) [2025-05-13 00:13:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2015232. Throughput: 0: 239.4. Samples: 493680. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:30,295][40219] Avg episode reward: [(0, '14.651')] [2025-05-13 00:13:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2019328. Throughput: 0: 249.6. Samples: 495304. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:35,295][40219] Avg episode reward: [(0, '14.464')] [2025-05-13 00:13:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 2027520. Throughput: 0: 245.3. Samples: 496114. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:13:40,291][40219] Avg episode reward: [(0, '15.265')] [2025-05-13 00:13:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2031616. Throughput: 0: 245.3. Samples: 497358. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:13:45,292][40219] Avg episode reward: [(0, '14.655')] [2025-05-13 00:13:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2035712. Throughput: 0: 248.5. Samples: 498818. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:13:50,298][40219] Avg episode reward: [(0, '14.367')] [2025-05-13 00:13:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2043904. Throughput: 0: 251.6. Samples: 499526. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:13:55,291][40219] Avg episode reward: [(0, '14.586')] [2025-05-13 00:14:00,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2048000. Throughput: 0: 248.8. Samples: 501344. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:00,291][40219] Avg episode reward: [(0, '14.343')] [2025-05-13 00:14:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2052096. Throughput: 0: 245.6. Samples: 502402. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:05,291][40219] Avg episode reward: [(0, '14.582')] [2025-05-13 00:14:07,726][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000502_2056192.pth... [2025-05-13 00:14:07,731][46910] Updated weights for policy 0, policy_version 502 (0.1457) [2025-05-13 00:14:07,820][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000444_1818624.pth [2025-05-13 00:14:10,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2056192. Throughput: 0: 249.6. Samples: 503438. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:10,296][40219] Avg episode reward: [(0, '15.117')] [2025-05-13 00:14:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2060288. Throughput: 0: 252.8. Samples: 505054. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:15,291][40219] Avg episode reward: [(0, '15.392')] [2025-05-13 00:14:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2068480. Throughput: 0: 247.7. Samples: 506452. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:20,291][40219] Avg episode reward: [(0, '14.693')] [2025-05-13 00:14:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2072576. Throughput: 0: 253.3. Samples: 507514. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:25,294][40219] Avg episode reward: [(0, '14.921')] [2025-05-13 00:14:30,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2076672. Throughput: 0: 251.7. Samples: 508684. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:30,294][40219] Avg episode reward: [(0, '15.286')] [2025-05-13 00:14:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2080768. Throughput: 0: 258.9. Samples: 510468. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:35,296][40219] Avg episode reward: [(0, '15.897')] [2025-05-13 00:14:39,611][46896] Saving new best policy, reward=15.897! [2025-05-13 00:14:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2088960. Throughput: 0: 259.5. Samples: 511202. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:40,291][40219] Avg episode reward: [(0, '15.730')] [2025-05-13 00:14:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2093056. Throughput: 0: 250.6. Samples: 512620. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:45,295][40219] Avg episode reward: [(0, '15.328')] [2025-05-13 00:14:48,969][46910] Updated weights for policy 0, policy_version 512 (0.0516) [2025-05-13 00:14:50,291][40219] Fps is (10 sec: 819.1, 60 sec: 1024.0, 300 sec: 985.8). Total num frames: 2097152. Throughput: 0: 255.3. Samples: 513890. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:50,292][40219] Avg episode reward: [(0, '15.130')] [2025-05-13 00:14:51,056][46896] Signal inference workers to stop experience collection... (500 times) [2025-05-13 00:14:51,116][46910] InferenceWorker_p0-w0: stopping experience collection (500 times) [2025-05-13 00:14:52,095][46896] Signal inference workers to resume experience collection... (500 times) [2025-05-13 00:14:52,096][46910] InferenceWorker_p0-w0: resuming experience collection (500 times) [2025-05-13 00:14:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2101248. Throughput: 0: 252.5. Samples: 514802. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:14:55,291][40219] Avg episode reward: [(0, '14.144')] [2025-05-13 00:15:00,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2105344. Throughput: 0: 245.9. Samples: 516118. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:15:00,293][40219] Avg episode reward: [(0, '14.459')] [2025-05-13 00:15:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2113536. Throughput: 0: 250.5. Samples: 517726. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:05,291][40219] Avg episode reward: [(0, '14.366')] [2025-05-13 00:15:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2117632. Throughput: 0: 249.6. Samples: 518744. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:15:10,291][40219] Avg episode reward: [(0, '14.541')] [2025-05-13 00:15:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2121728. Throughput: 0: 250.5. Samples: 519958. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:15:15,293][40219] Avg episode reward: [(0, '14.754')] [2025-05-13 00:15:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2125824. Throughput: 0: 250.3. Samples: 521732. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:15:20,296][40219] Avg episode reward: [(0, '14.738')] [2025-05-13 00:15:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2134016. Throughput: 0: 254.7. Samples: 522662. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:25,292][40219] Avg episode reward: [(0, '14.325')] [2025-05-13 00:15:29,862][46910] Updated weights for policy 0, policy_version 522 (0.1139) [2025-05-13 00:15:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2138112. Throughput: 0: 248.3. Samples: 523792. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:30,291][40219] Avg episode reward: [(0, '14.228')] [2025-05-13 00:15:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2142208. Throughput: 0: 255.5. Samples: 525386. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:35,291][40219] Avg episode reward: [(0, '14.787')] [2025-05-13 00:15:40,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2146304. Throughput: 0: 252.4. Samples: 526162. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:40,291][40219] Avg episode reward: [(0, '15.215')] [2025-05-13 00:15:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2154496. Throughput: 0: 254.1. Samples: 527554. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:45,299][40219] Avg episode reward: [(0, '15.613')] [2025-05-13 00:15:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2158592. Throughput: 0: 253.6. Samples: 529140. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:50,291][40219] Avg episode reward: [(0, '15.516')] [2025-05-13 00:15:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2162688. Throughput: 0: 249.0. Samples: 529950. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:15:55,291][40219] Avg episode reward: [(0, '15.333')] [2025-05-13 00:16:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2166784. Throughput: 0: 246.2. Samples: 531038. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:00,298][40219] Avg episode reward: [(0, '15.374')] [2025-05-13 00:16:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2170880. Throughput: 0: 240.9. Samples: 532572. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:05,291][40219] Avg episode reward: [(0, '15.586')] [2025-05-13 00:16:10,013][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000532_2179072.pth... [2025-05-13 00:16:10,042][46910] Updated weights for policy 0, policy_version 532 (0.0543) [2025-05-13 00:16:10,081][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000473_1937408.pth [2025-05-13 00:16:10,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2179072. Throughput: 0: 237.6. Samples: 533354. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:10,296][40219] Avg episode reward: [(0, '15.574')] [2025-05-13 00:16:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2183168. Throughput: 0: 244.3. Samples: 534786. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:15,295][40219] Avg episode reward: [(0, '15.562')] [2025-05-13 00:16:20,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2187264. Throughput: 0: 237.2. Samples: 536062. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:20,295][40219] Avg episode reward: [(0, '15.922')] [2025-05-13 00:16:22,398][46896] Saving new best policy, reward=15.922! [2025-05-13 00:16:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2191360. Throughput: 0: 241.6. Samples: 537034. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:25,291][40219] Avg episode reward: [(0, '15.536')] [2025-05-13 00:16:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2195456. Throughput: 0: 243.2. Samples: 538500. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:30,296][40219] Avg episode reward: [(0, '16.233')] [2025-05-13 00:16:34,675][46896] Saving new best policy, reward=16.233! [2025-05-13 00:16:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2203648. Throughput: 0: 242.5. Samples: 540052. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:35,295][40219] Avg episode reward: [(0, '16.549')] [2025-05-13 00:16:38,308][46896] Saving new best policy, reward=16.549! [2025-05-13 00:16:40,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2207744. Throughput: 0: 248.0. Samples: 541112. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:40,301][40219] Avg episode reward: [(0, '16.283')] [2025-05-13 00:16:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2211840. Throughput: 0: 247.7. Samples: 542186. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:45,295][40219] Avg episode reward: [(0, '16.664')] [2025-05-13 00:16:47,081][46896] Saving new best policy, reward=16.664! [2025-05-13 00:16:50,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 985.8). Total num frames: 2215936. Throughput: 0: 258.8. Samples: 544218. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:50,298][40219] Avg episode reward: [(0, '16.718')] [2025-05-13 00:16:51,192][46910] Updated weights for policy 0, policy_version 542 (0.1459) [2025-05-13 00:16:54,657][46896] Saving new best policy, reward=16.718! [2025-05-13 00:16:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2224128. Throughput: 0: 255.3. Samples: 544844. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:16:55,298][40219] Avg episode reward: [(0, '16.714')] [2025-05-13 00:17:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2228224. Throughput: 0: 251.6. Samples: 546108. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:00,295][40219] Avg episode reward: [(0, '16.765')] [2025-05-13 00:17:02,964][46896] Saving new best policy, reward=16.765! [2025-05-13 00:17:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2232320. Throughput: 0: 262.7. Samples: 547882. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:05,292][40219] Avg episode reward: [(0, '16.509')] [2025-05-13 00:17:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2236416. Throughput: 0: 258.0. Samples: 548642. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:10,291][40219] Avg episode reward: [(0, '16.614')] [2025-05-13 00:17:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2244608. Throughput: 0: 257.0. Samples: 550066. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:17:15,291][40219] Avg episode reward: [(0, '16.847')] [2025-05-13 00:17:18,758][46896] Saving new best policy, reward=16.847! [2025-05-13 00:17:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2248704. Throughput: 0: 257.2. Samples: 551626. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:20,291][40219] Avg episode reward: [(0, '17.068')] [2025-05-13 00:17:22,437][46896] Saving new best policy, reward=17.068! [2025-05-13 00:17:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2252800. Throughput: 0: 251.2. Samples: 552414. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:25,291][40219] Avg episode reward: [(0, '17.465')] [2025-05-13 00:17:27,811][46896] Saving new best policy, reward=17.465! [2025-05-13 00:17:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2256896. Throughput: 0: 257.4. Samples: 553770. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:17:30,295][40219] Avg episode reward: [(0, '17.634')] [2025-05-13 00:17:31,349][46910] Updated weights for policy 0, policy_version 552 (0.0045) [2025-05-13 00:17:35,090][46896] Saving new best policy, reward=17.634! [2025-05-13 00:17:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2265088. Throughput: 0: 250.7. Samples: 555498. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:17:35,291][40219] Avg episode reward: [(0, '17.446')] [2025-05-13 00:17:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2269184. Throughput: 0: 258.5. Samples: 556478. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:17:40,291][40219] Avg episode reward: [(0, '17.579')] [2025-05-13 00:17:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2273280. Throughput: 0: 253.6. Samples: 557518. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:17:45,296][40219] Avg episode reward: [(0, '17.614')] [2025-05-13 00:17:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2277376. Throughput: 0: 260.0. Samples: 559580. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:17:50,297][40219] Avg episode reward: [(0, '17.757')] [2025-05-13 00:17:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2281472. Throughput: 0: 256.5. Samples: 560184. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:17:55,291][40219] Avg episode reward: [(0, '17.770')] [2025-05-13 00:17:55,848][46896] Saving new best policy, reward=17.757! [2025-05-13 00:17:59,390][46896] Saving new best policy, reward=17.770! [2025-05-13 00:18:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2289664. Throughput: 0: 256.7. Samples: 561616. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:00,293][40219] Avg episode reward: [(0, '17.908')] [2025-05-13 00:18:03,028][46896] Saving new best policy, reward=17.908! [2025-05-13 00:18:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2293760. Throughput: 0: 259.6. Samples: 563306. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:05,291][40219] Avg episode reward: [(0, '17.749')] [2025-05-13 00:18:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2297856. Throughput: 0: 252.4. Samples: 563772. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:10,297][40219] Avg episode reward: [(0, '18.202')] [2025-05-13 00:18:11,751][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000562_2301952.pth... [2025-05-13 00:18:11,757][46910] Updated weights for policy 0, policy_version 562 (0.0054) [2025-05-13 00:18:11,848][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000502_2056192.pth [2025-05-13 00:18:11,856][46896] Saving new best policy, reward=18.202! [2025-05-13 00:18:13,936][46896] Signal inference workers to stop experience collection... (550 times) [2025-05-13 00:18:13,979][46910] InferenceWorker_p0-w0: stopping experience collection (550 times) [2025-05-13 00:18:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2301952. Throughput: 0: 260.8. Samples: 565508. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:15,291][40219] Avg episode reward: [(0, '18.716')] [2025-05-13 00:18:15,415][46896] Signal inference workers to resume experience collection... (550 times) [2025-05-13 00:18:15,416][46910] InferenceWorker_p0-w0: resuming experience collection (550 times) [2025-05-13 00:18:19,123][46896] Saving new best policy, reward=18.716! [2025-05-13 00:18:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2310144. Throughput: 0: 255.2. Samples: 566980. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:18:20,292][40219] Avg episode reward: [(0, '19.383')] [2025-05-13 00:18:24,089][46896] Saving new best policy, reward=19.383! [2025-05-13 00:18:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2314240. Throughput: 0: 252.2. Samples: 567828. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:18:25,291][40219] Avg episode reward: [(0, '19.743')] [2025-05-13 00:18:27,886][46896] Saving new best policy, reward=19.743! [2025-05-13 00:18:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2318336. Throughput: 0: 257.8. Samples: 569118. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:18:30,296][40219] Avg episode reward: [(0, '19.758')] [2025-05-13 00:18:35,241][46896] Saving new best policy, reward=19.758! [2025-05-13 00:18:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2326528. Throughput: 0: 251.0. Samples: 570874. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:35,291][40219] Avg episode reward: [(0, '20.198')] [2025-05-13 00:18:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2326528. Throughput: 0: 250.1. Samples: 571438. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:40,295][40219] Avg episode reward: [(0, '20.462')] [2025-05-13 00:18:40,304][46896] Saving new best policy, reward=20.198! [2025-05-13 00:18:43,879][46896] Saving new best policy, reward=20.462! [2025-05-13 00:18:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2334720. Throughput: 0: 250.8. Samples: 572904. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:45,291][40219] Avg episode reward: [(0, '20.122')] [2025-05-13 00:18:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2338816. Throughput: 0: 256.8. Samples: 574862. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:50,291][40219] Avg episode reward: [(0, '20.250')] [2025-05-13 00:18:51,794][46910] Updated weights for policy 0, policy_version 572 (0.0534) [2025-05-13 00:18:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2342912. Throughput: 0: 252.0. Samples: 575114. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:18:55,291][40219] Avg episode reward: [(0, '21.046')] [2025-05-13 00:18:59,833][46896] Saving new best policy, reward=21.046! [2025-05-13 00:19:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2351104. Throughput: 0: 255.3. Samples: 576998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:19:00,291][40219] Avg episode reward: [(0, '20.701')] [2025-05-13 00:19:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2351104. Throughput: 0: 245.8. Samples: 578040. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:19:05,291][40219] Avg episode reward: [(0, '21.050')] [2025-05-13 00:19:09,809][46896] Saving new best policy, reward=21.050! [2025-05-13 00:19:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2359296. Throughput: 0: 238.5. Samples: 578562. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:19:10,291][40219] Avg episode reward: [(0, '20.885')] [2025-05-13 00:19:15,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2363392. Throughput: 0: 246.2. Samples: 580196. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:15,299][40219] Avg episode reward: [(0, '21.501')] [2025-05-13 00:19:16,838][46896] Saving new best policy, reward=21.501! [2025-05-13 00:19:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2367488. Throughput: 0: 248.4. Samples: 582050. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:20,291][40219] Avg episode reward: [(0, '21.824')] [2025-05-13 00:19:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2371584. Throughput: 0: 243.6. Samples: 582398. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:25,296][40219] Avg episode reward: [(0, '22.472')] [2025-05-13 00:19:25,511][46896] Saving new best policy, reward=21.824! [2025-05-13 00:19:29,035][46896] Saving new best policy, reward=22.472! [2025-05-13 00:19:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2379776. Throughput: 0: 251.3. Samples: 584214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:30,296][40219] Avg episode reward: [(0, '23.064')] [2025-05-13 00:19:32,665][46896] Saving new best policy, reward=23.064! [2025-05-13 00:19:32,675][46910] Updated weights for policy 0, policy_version 582 (0.0053) [2025-05-13 00:19:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2383872. Throughput: 0: 240.3. Samples: 585674. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:35,297][40219] Avg episode reward: [(0, '22.922')] [2025-05-13 00:19:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2387968. Throughput: 0: 248.1. Samples: 586278. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:19:40,297][40219] Avg episode reward: [(0, '22.966')] [2025-05-13 00:19:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2396160. Throughput: 0: 250.2. Samples: 588258. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:19:45,291][40219] Avg episode reward: [(0, '22.848')] [2025-05-13 00:19:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2400256. Throughput: 0: 253.6. Samples: 589454. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:19:50,294][40219] Avg episode reward: [(0, '22.831')] [2025-05-13 00:19:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2404352. Throughput: 0: 262.8. Samples: 590390. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:19:55,291][40219] Avg episode reward: [(0, '22.577')] [2025-05-13 00:20:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2408448. Throughput: 0: 263.3. Samples: 592044. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:00,296][40219] Avg episode reward: [(0, '22.157')] [2025-05-13 00:20:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2412544. Throughput: 0: 253.0. Samples: 593434. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:05,291][40219] Avg episode reward: [(0, '21.605')] [2025-05-13 00:20:09,566][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000591_2420736.pth... [2025-05-13 00:20:09,647][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000532_2179072.pth [2025-05-13 00:20:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2420736. Throughput: 0: 261.1. Samples: 594148. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:10,295][40219] Avg episode reward: [(0, '21.282')] [2025-05-13 00:20:13,129][46910] Updated weights for policy 0, policy_version 592 (0.0927) [2025-05-13 00:20:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2424832. Throughput: 0: 255.8. Samples: 595726. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:15,291][40219] Avg episode reward: [(0, '21.683')] [2025-05-13 00:20:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2428928. Throughput: 0: 258.4. Samples: 597302. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:20,291][40219] Avg episode reward: [(0, '21.597')] [2025-05-13 00:20:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1013.6). Total num frames: 2437120. Throughput: 0: 258.0. Samples: 597890. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:25,294][40219] Avg episode reward: [(0, '21.975')] [2025-05-13 00:20:30,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2441216. Throughput: 0: 252.1. Samples: 599604. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:20:30,296][40219] Avg episode reward: [(0, '22.122')] [2025-05-13 00:20:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2445312. Throughput: 0: 255.4. Samples: 600946. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:20:35,291][40219] Avg episode reward: [(0, '22.062')] [2025-05-13 00:20:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2449408. Throughput: 0: 248.6. Samples: 601576. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:20:40,298][40219] Avg episode reward: [(0, '21.869')] [2025-05-13 00:20:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2457600. Throughput: 0: 257.8. Samples: 603646. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:20:45,298][40219] Avg episode reward: [(0, '22.446')] [2025-05-13 00:20:50,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2461696. Throughput: 0: 249.7. Samples: 604670. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:20:50,291][40219] Avg episode reward: [(0, '21.615')] [2025-05-13 00:20:53,182][46910] Updated weights for policy 0, policy_version 602 (0.0490) [2025-05-13 00:20:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2465792. Throughput: 0: 254.8. Samples: 605616. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:20:55,296][40219] Avg episode reward: [(0, '21.330')] [2025-05-13 00:21:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2469888. Throughput: 0: 260.5. Samples: 607450. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:21:00,291][40219] Avg episode reward: [(0, '21.079')] [2025-05-13 00:21:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2473984. Throughput: 0: 252.9. Samples: 608684. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:21:05,296][40219] Avg episode reward: [(0, '20.582')] [2025-05-13 00:21:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2482176. Throughput: 0: 255.0. Samples: 609366. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:21:10,291][40219] Avg episode reward: [(0, '20.961')] [2025-05-13 00:21:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2486272. Throughput: 0: 254.9. Samples: 611076. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:21:15,295][40219] Avg episode reward: [(0, '21.328')] [2025-05-13 00:21:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2490368. Throughput: 0: 256.0. Samples: 612468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:21:20,295][40219] Avg episode reward: [(0, '21.419')] [2025-05-13 00:21:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 2498560. Throughput: 0: 256.8. Samples: 613132. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:21:25,291][40219] Avg episode reward: [(0, '21.013')] [2025-05-13 00:21:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2502656. Throughput: 0: 252.2. Samples: 614994. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:21:30,297][40219] Avg episode reward: [(0, '21.621')] [2025-05-13 00:21:34,305][46910] Updated weights for policy 0, policy_version 612 (0.0960) [2025-05-13 00:21:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2506752. Throughput: 0: 254.3. Samples: 616112. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:21:35,291][40219] Avg episode reward: [(0, '22.211')] [2025-05-13 00:21:36,635][46896] Signal inference workers to stop experience collection... (600 times) [2025-05-13 00:21:36,674][46910] InferenceWorker_p0-w0: stopping experience collection (600 times) [2025-05-13 00:21:38,002][46896] Signal inference workers to resume experience collection... (600 times) [2025-05-13 00:21:38,003][46910] InferenceWorker_p0-w0: resuming experience collection (600 times) [2025-05-13 00:21:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2510848. Throughput: 0: 254.0. Samples: 617048. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:21:40,291][40219] Avg episode reward: [(0, '22.206')] [2025-05-13 00:21:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 2514944. Throughput: 0: 255.7. Samples: 618956. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:21:45,291][40219] Avg episode reward: [(0, '22.299')] [2025-05-13 00:21:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2519040. Throughput: 0: 253.1. Samples: 620072. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:21:50,293][40219] Avg episode reward: [(0, '22.440')] [2025-05-13 00:21:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2527232. Throughput: 0: 260.1. Samples: 621070. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:21:55,295][40219] Avg episode reward: [(0, '21.984')] [2025-05-13 00:22:00,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2531328. Throughput: 0: 256.3. Samples: 622610. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:22:00,291][40219] Avg episode reward: [(0, '21.480')] [2025-05-13 00:22:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2535424. Throughput: 0: 247.5. Samples: 623606. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:22:05,291][40219] Avg episode reward: [(0, '21.445')] [2025-05-13 00:22:07,576][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000620_2539520.pth... [2025-05-13 00:22:07,717][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000562_2301952.pth [2025-05-13 00:22:10,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2539520. Throughput: 0: 243.8. Samples: 624104. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:22:10,291][40219] Avg episode reward: [(0, '21.353')] [2025-05-13 00:22:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2543616. Throughput: 0: 245.2. Samples: 626026. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:22:15,291][40219] Avg episode reward: [(0, '20.960')] [2025-05-13 00:22:15,957][46910] Updated weights for policy 0, policy_version 622 (0.0656) [2025-05-13 00:22:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2551808. Throughput: 0: 245.2. Samples: 627148. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:22:20,296][40219] Avg episode reward: [(0, '20.626')] [2025-05-13 00:22:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 2555904. Throughput: 0: 249.1. Samples: 628256. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:22:25,291][40219] Avg episode reward: [(0, '20.090')] [2025-05-13 00:22:30,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2560000. Throughput: 0: 237.6. Samples: 629646. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:22:30,295][40219] Avg episode reward: [(0, '19.682')] [2025-05-13 00:22:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2564096. Throughput: 0: 247.8. Samples: 631224. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:22:35,291][40219] Avg episode reward: [(0, '20.671')] [2025-05-13 00:22:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2572288. Throughput: 0: 242.1. Samples: 631966. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:22:40,291][40219] Avg episode reward: [(0, '21.106')] [2025-05-13 00:22:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2576384. Throughput: 0: 240.3. Samples: 633422. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:22:45,294][40219] Avg episode reward: [(0, '20.943')] [2025-05-13 00:22:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2580480. Throughput: 0: 248.1. Samples: 634772. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:22:50,298][40219] Avg episode reward: [(0, '20.237')] [2025-05-13 00:22:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2584576. Throughput: 0: 258.0. Samples: 635714. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:22:55,298][40219] Avg episode reward: [(0, '19.576')] [2025-05-13 00:22:55,628][46910] Updated weights for policy 0, policy_version 632 (0.0972) [2025-05-13 00:23:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2588672. Throughput: 0: 252.0. Samples: 637368. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:00,291][40219] Avg episode reward: [(0, '19.284')] [2025-05-13 00:23:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2596864. Throughput: 0: 255.4. Samples: 638640. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:05,291][40219] Avg episode reward: [(0, '19.385')] [2025-05-13 00:23:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2600960. Throughput: 0: 252.1. Samples: 639602. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:10,293][40219] Avg episode reward: [(0, '18.719')] [2025-05-13 00:23:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2605056. Throughput: 0: 252.0. Samples: 640988. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:15,301][40219] Avg episode reward: [(0, '18.945')] [2025-05-13 00:23:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2613248. Throughput: 0: 254.3. Samples: 642666. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:20,298][40219] Avg episode reward: [(0, '19.023')] [2025-05-13 00:23:25,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2617344. Throughput: 0: 257.8. Samples: 643568. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:25,292][40219] Avg episode reward: [(0, '19.302')] [2025-05-13 00:23:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2621440. Throughput: 0: 251.1. Samples: 644720. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:30,299][40219] Avg episode reward: [(0, '19.234')] [2025-05-13 00:23:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2625536. Throughput: 0: 261.6. Samples: 646542. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:35,291][40219] Avg episode reward: [(0, '19.729')] [2025-05-13 00:23:36,530][46910] Updated weights for policy 0, policy_version 642 (0.1887) [2025-05-13 00:23:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2633728. Throughput: 0: 256.1. Samples: 647240. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:40,295][40219] Avg episode reward: [(0, '19.649')] [2025-05-13 00:23:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2637824. Throughput: 0: 251.7. Samples: 648694. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:45,295][40219] Avg episode reward: [(0, '19.331')] [2025-05-13 00:23:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2641920. Throughput: 0: 255.4. Samples: 650132. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:50,291][40219] Avg episode reward: [(0, '20.266')] [2025-05-13 00:23:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2646016. Throughput: 0: 250.4. Samples: 650870. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:23:55,295][40219] Avg episode reward: [(0, '20.888')] [2025-05-13 00:24:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2650112. Throughput: 0: 253.9. Samples: 652414. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:24:00,296][40219] Avg episode reward: [(0, '20.815')] [2025-05-13 00:24:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2658304. Throughput: 0: 252.9. Samples: 654046. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:24:05,291][40219] Avg episode reward: [(0, '20.656')] [2025-05-13 00:24:07,993][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000650_2662400.pth... [2025-05-13 00:24:08,089][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000591_2420736.pth [2025-05-13 00:24:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2662400. Throughput: 0: 253.6. Samples: 654978. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:24:10,292][40219] Avg episode reward: [(0, '20.423')] [2025-05-13 00:24:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2666496. Throughput: 0: 254.4. Samples: 656168. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:24:15,292][40219] Avg episode reward: [(0, '20.466')] [2025-05-13 00:24:16,741][46910] Updated weights for policy 0, policy_version 652 (0.0966) [2025-05-13 00:24:20,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 2674688. Throughput: 0: 256.1. Samples: 658066. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:24:20,296][40219] Avg episode reward: [(0, '20.212')] [2025-05-13 00:24:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2678784. Throughput: 0: 259.1. Samples: 658900. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:24:25,296][40219] Avg episode reward: [(0, '20.599')] [2025-05-13 00:24:30,290][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2682880. Throughput: 0: 253.6. Samples: 660108. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:24:30,293][40219] Avg episode reward: [(0, '20.837')] [2025-05-13 00:24:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2686976. Throughput: 0: 263.1. Samples: 661970. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:24:35,296][40219] Avg episode reward: [(0, '20.998')] [2025-05-13 00:24:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2691072. Throughput: 0: 263.3. Samples: 662720. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:24:40,295][40219] Avg episode reward: [(0, '20.958')] [2025-05-13 00:24:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2699264. Throughput: 0: 260.8. Samples: 664152. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:24:45,291][40219] Avg episode reward: [(0, '20.883')] [2025-05-13 00:24:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2703360. Throughput: 0: 258.9. Samples: 665696. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:24:50,295][40219] Avg episode reward: [(0, '20.402')] [2025-05-13 00:24:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2707456. Throughput: 0: 254.9. Samples: 666450. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:24:55,292][40219] Avg episode reward: [(0, '20.831')] [2025-05-13 00:24:57,233][46910] Updated weights for policy 0, policy_version 662 (0.0508) [2025-05-13 00:24:59,388][46896] Signal inference workers to stop experience collection... (650 times) [2025-05-13 00:24:59,428][46910] InferenceWorker_p0-w0: stopping experience collection (650 times) [2025-05-13 00:25:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2711552. Throughput: 0: 261.2. Samples: 667924. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:25:00,296][40219] Avg episode reward: [(0, '20.975')] [2025-05-13 00:25:00,735][46896] Signal inference workers to resume experience collection... (650 times) [2025-05-13 00:25:00,738][46910] InferenceWorker_p0-w0: resuming experience collection (650 times) [2025-05-13 00:25:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2719744. Throughput: 0: 253.6. Samples: 669478. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:25:05,294][40219] Avg episode reward: [(0, '20.529')] [2025-05-13 00:25:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2723840. Throughput: 0: 256.0. Samples: 670418. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:25:10,292][40219] Avg episode reward: [(0, '20.597')] [2025-05-13 00:25:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2727936. Throughput: 0: 251.9. Samples: 671442. Policy #0 lag: (min: 1.0, avg: 1.7, max: 2.0) [2025-05-13 00:25:15,291][40219] Avg episode reward: [(0, '20.589')] [2025-05-13 00:25:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 2732032. Throughput: 0: 241.5. Samples: 672836. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:20,291][40219] Avg episode reward: [(0, '20.631')] [2025-05-13 00:25:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2736128. Throughput: 0: 245.8. Samples: 673782. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:25,295][40219] Avg episode reward: [(0, '21.304')] [2025-05-13 00:25:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2740224. Throughput: 0: 244.6. Samples: 675160. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:30,297][40219] Avg episode reward: [(0, '21.017')] [2025-05-13 00:25:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2748416. Throughput: 0: 243.4. Samples: 676648. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:35,291][40219] Avg episode reward: [(0, '21.052')] [2025-05-13 00:25:38,589][46910] Updated weights for policy 0, policy_version 672 (0.1422) [2025-05-13 00:25:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2752512. Throughput: 0: 247.2. Samples: 677572. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:40,291][40219] Avg episode reward: [(0, '21.449')] [2025-05-13 00:25:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2756608. Throughput: 0: 241.4. Samples: 678786. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:45,291][40219] Avg episode reward: [(0, '23.109')] [2025-05-13 00:25:46,729][46896] Saving new best policy, reward=23.109! [2025-05-13 00:25:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2764800. Throughput: 0: 248.4. Samples: 680658. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:50,297][40219] Avg episode reward: [(0, '23.124')] [2025-05-13 00:25:55,114][46896] Saving new best policy, reward=23.124! [2025-05-13 00:25:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2768896. Throughput: 0: 244.8. Samples: 681434. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:25:55,291][40219] Avg episode reward: [(0, '22.985')] [2025-05-13 00:26:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2772992. Throughput: 0: 249.0. Samples: 682646. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:26:00,294][40219] Avg episode reward: [(0, '22.527')] [2025-05-13 00:26:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2777088. Throughput: 0: 260.5. Samples: 684560. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:26:05,292][40219] Avg episode reward: [(0, '22.707')] [2025-05-13 00:26:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2781184. Throughput: 0: 249.1. Samples: 684992. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:26:10,296][40219] Avg episode reward: [(0, '22.732')] [2025-05-13 00:26:11,577][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000680_2785280.pth... [2025-05-13 00:26:11,670][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000620_2539520.pth [2025-05-13 00:26:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2789376. Throughput: 0: 254.0. Samples: 686590. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:26:15,291][40219] Avg episode reward: [(0, '22.281')] [2025-05-13 00:26:18,790][46910] Updated weights for policy 0, policy_version 682 (0.0038) [2025-05-13 00:26:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2793472. Throughput: 0: 254.4. Samples: 688098. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:26:20,291][40219] Avg episode reward: [(0, '22.238')] [2025-05-13 00:26:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2797568. Throughput: 0: 248.8. Samples: 688770. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:25,291][40219] Avg episode reward: [(0, '22.807')] [2025-05-13 00:26:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2801664. Throughput: 0: 257.0. Samples: 690352. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:30,291][40219] Avg episode reward: [(0, '22.852')] [2025-05-13 00:26:35,292][40219] Fps is (10 sec: 1228.5, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2809856. Throughput: 0: 250.9. Samples: 691948. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:26:35,293][40219] Avg episode reward: [(0, '22.643')] [2025-05-13 00:26:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2813952. Throughput: 0: 253.0. Samples: 692818. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:40,291][40219] Avg episode reward: [(0, '22.808')] [2025-05-13 00:26:45,289][40219] Fps is (10 sec: 819.4, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2818048. Throughput: 0: 253.6. Samples: 694060. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:45,298][40219] Avg episode reward: [(0, '23.669')] [2025-05-13 00:26:50,282][46896] Saving new best policy, reward=23.669! [2025-05-13 00:26:50,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2826240. Throughput: 0: 250.6. Samples: 695838. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:50,293][40219] Avg episode reward: [(0, '23.451')] [2025-05-13 00:26:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2826240. Throughput: 0: 253.6. Samples: 696402. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:26:55,294][40219] Avg episode reward: [(0, '23.454')] [2025-05-13 00:26:58,910][46910] Updated weights for policy 0, policy_version 692 (0.0607) [2025-05-13 00:27:00,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2834432. Throughput: 0: 250.9. Samples: 697880. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:00,291][40219] Avg episode reward: [(0, '23.078')] [2025-05-13 00:27:05,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2838528. Throughput: 0: 257.4. Samples: 699680. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:05,291][40219] Avg episode reward: [(0, '22.934')] [2025-05-13 00:27:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2842624. Throughput: 0: 252.3. Samples: 700124. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:10,299][40219] Avg episode reward: [(0, '22.563')] [2025-05-13 00:27:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2850816. Throughput: 0: 257.2. Samples: 701928. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:27:15,291][40219] Avg episode reward: [(0, '23.009')] [2025-05-13 00:27:20,292][40219] Fps is (10 sec: 1228.5, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2854912. Throughput: 0: 248.7. Samples: 703138. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:20,293][40219] Avg episode reward: [(0, '21.835')] [2025-05-13 00:27:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2859008. Throughput: 0: 247.0. Samples: 703932. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:25,298][40219] Avg episode reward: [(0, '21.772')] [2025-05-13 00:27:30,289][40219] Fps is (10 sec: 819.4, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2863104. Throughput: 0: 244.8. Samples: 705076. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:30,296][40219] Avg episode reward: [(0, '21.638')] [2025-05-13 00:27:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 2867200. Throughput: 0: 244.8. Samples: 706854. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:35,291][40219] Avg episode reward: [(0, '21.977')] [2025-05-13 00:27:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2871296. Throughput: 0: 240.4. Samples: 707218. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:40,298][40219] Avg episode reward: [(0, '22.141')] [2025-05-13 00:27:41,186][46910] Updated weights for policy 0, policy_version 702 (0.1636) [2025-05-13 00:27:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2879488. Throughput: 0: 246.8. Samples: 708986. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:45,291][40219] Avg episode reward: [(0, '22.082')] [2025-05-13 00:27:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 1013.6). Total num frames: 2883584. Throughput: 0: 242.8. Samples: 710606. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:27:50,293][40219] Avg episode reward: [(0, '22.868')] [2025-05-13 00:27:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2887680. Throughput: 0: 242.1. Samples: 711020. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:27:55,291][40219] Avg episode reward: [(0, '23.098')] [2025-05-13 00:28:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2891776. Throughput: 0: 241.8. Samples: 712808. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:00,296][40219] Avg episode reward: [(0, '22.912')] [2025-05-13 00:28:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2899968. Throughput: 0: 247.9. Samples: 714294. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:05,296][40219] Avg episode reward: [(0, '23.505')] [2025-05-13 00:28:09,348][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000709_2904064.pth... [2025-05-13 00:28:09,440][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000650_2662400.pth [2025-05-13 00:28:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2904064. Throughput: 0: 246.1. Samples: 715008. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:10,291][40219] Avg episode reward: [(0, '23.980')] [2025-05-13 00:28:12,988][46896] Saving new best policy, reward=23.980! [2025-05-13 00:28:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2908160. Throughput: 0: 252.5. Samples: 716440. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:15,292][40219] Avg episode reward: [(0, '23.309')] [2025-05-13 00:28:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 2912256. Throughput: 0: 245.8. Samples: 717914. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:20,295][40219] Avg episode reward: [(0, '23.005')] [2025-05-13 00:28:22,857][46910] Updated weights for policy 0, policy_version 712 (0.0498) [2025-05-13 00:28:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2916352. Throughput: 0: 246.4. Samples: 718308. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:28:25,303][40219] Avg episode reward: [(0, '23.188')] [2025-05-13 00:28:25,867][46896] Signal inference workers to stop experience collection... (700 times) [2025-05-13 00:28:25,904][46910] InferenceWorker_p0-w0: stopping experience collection (700 times) [2025-05-13 00:28:27,042][46896] Signal inference workers to resume experience collection... (700 times) [2025-05-13 00:28:27,043][46910] InferenceWorker_p0-w0: resuming experience collection (700 times) [2025-05-13 00:28:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2920448. Throughput: 0: 244.2. Samples: 719974. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:28:30,296][40219] Avg episode reward: [(0, '23.005')] [2025-05-13 00:28:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2928640. Throughput: 0: 242.7. Samples: 721528. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:28:35,295][40219] Avg episode reward: [(0, '22.262')] [2025-05-13 00:28:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2932736. Throughput: 0: 251.9. Samples: 722354. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:28:40,292][40219] Avg episode reward: [(0, '22.434')] [2025-05-13 00:28:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2936832. Throughput: 0: 241.1. Samples: 723658. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:28:45,291][40219] Avg episode reward: [(0, '23.511')] [2025-05-13 00:28:50,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2940928. Throughput: 0: 248.9. Samples: 725494. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:28:50,291][40219] Avg episode reward: [(0, '22.966')] [2025-05-13 00:28:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2949120. Throughput: 0: 241.5. Samples: 725876. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:28:55,293][40219] Avg episode reward: [(0, '23.213')] [2025-05-13 00:29:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2953216. Throughput: 0: 245.1. Samples: 727468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:29:00,291][40219] Avg episode reward: [(0, '22.536')] [2025-05-13 00:29:02,613][46910] Updated weights for policy 0, policy_version 722 (0.1601) [2025-05-13 00:29:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2957312. Throughput: 0: 253.6. Samples: 729324. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:29:05,297][40219] Avg episode reward: [(0, '22.678')] [2025-05-13 00:29:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 2961408. Throughput: 0: 254.0. Samples: 729738. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:29:10,299][40219] Avg episode reward: [(0, '22.680')] [2025-05-13 00:29:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2969600. Throughput: 0: 256.8. Samples: 731528. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:29:15,291][40219] Avg episode reward: [(0, '23.003')] [2025-05-13 00:29:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2973696. Throughput: 0: 256.2. Samples: 733058. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:29:20,291][40219] Avg episode reward: [(0, '22.551')] [2025-05-13 00:29:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2977792. Throughput: 0: 250.0. Samples: 733606. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:29:25,291][40219] Avg episode reward: [(0, '22.273')] [2025-05-13 00:29:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1013.6). Total num frames: 2985984. Throughput: 0: 262.8. Samples: 735486. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:29:30,291][40219] Avg episode reward: [(0, '22.424')] [2025-05-13 00:29:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 2990080. Throughput: 0: 250.7. Samples: 736776. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:29:35,295][40219] Avg episode reward: [(0, '21.805')] [2025-05-13 00:29:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2994176. Throughput: 0: 258.5. Samples: 737510. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:29:40,291][40219] Avg episode reward: [(0, '22.620')] [2025-05-13 00:29:42,682][46910] Updated weights for policy 0, policy_version 732 (0.0491) [2025-05-13 00:29:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 2998272. Throughput: 0: 260.4. Samples: 739184. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:29:45,299][40219] Avg episode reward: [(0, '22.296')] [2025-05-13 00:29:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3002368. Throughput: 0: 254.7. Samples: 740786. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:29:50,291][40219] Avg episode reward: [(0, '21.962')] [2025-05-13 00:29:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3010560. Throughput: 0: 258.1. Samples: 741354. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:29:55,292][40219] Avg episode reward: [(0, '22.277')] [2025-05-13 00:30:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3014656. Throughput: 0: 254.9. Samples: 743000. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:00,296][40219] Avg episode reward: [(0, '22.022')] [2025-05-13 00:30:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3018752. Throughput: 0: 259.1. Samples: 744718. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:05,291][40219] Avg episode reward: [(0, '22.387')] [2025-05-13 00:30:07,041][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000738_3022848.pth... [2025-05-13 00:30:07,128][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000680_2785280.pth [2025-05-13 00:30:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3022848. Throughput: 0: 258.0. Samples: 745218. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:10,296][40219] Avg episode reward: [(0, '22.441')] [2025-05-13 00:30:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3031040. Throughput: 0: 256.4. Samples: 747026. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:30:15,295][40219] Avg episode reward: [(0, '21.730')] [2025-05-13 00:30:20,295][40219] Fps is (10 sec: 1228.1, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 3035136. Throughput: 0: 254.1. Samples: 748210. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:20,301][40219] Avg episode reward: [(0, '21.597')] [2025-05-13 00:30:23,064][46910] Updated weights for policy 0, policy_version 742 (0.1521) [2025-05-13 00:30:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3039232. Throughput: 0: 256.4. Samples: 749046. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:25,296][40219] Avg episode reward: [(0, '22.249')] [2025-05-13 00:30:30,290][40219] Fps is (10 sec: 1229.4, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3047424. Throughput: 0: 265.4. Samples: 751126. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:30,291][40219] Avg episode reward: [(0, '21.477')] [2025-05-13 00:30:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3051520. Throughput: 0: 252.5. Samples: 752150. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:35,291][40219] Avg episode reward: [(0, '21.929')] [2025-05-13 00:30:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3055616. Throughput: 0: 262.8. Samples: 753182. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:30:40,290][40219] Avg episode reward: [(0, '21.894')] [2025-05-13 00:30:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3059712. Throughput: 0: 262.0. Samples: 754790. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:30:45,291][40219] Avg episode reward: [(0, '21.681')] [2025-05-13 00:30:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3063808. Throughput: 0: 255.2. Samples: 756202. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:30:50,298][40219] Avg episode reward: [(0, '21.769')] [2025-05-13 00:30:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3072000. Throughput: 0: 265.0. Samples: 757144. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:30:55,291][40219] Avg episode reward: [(0, '21.936')] [2025-05-13 00:31:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3076096. Throughput: 0: 260.3. Samples: 758738. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:31:00,291][40219] Avg episode reward: [(0, '22.727')] [2025-05-13 00:31:01,549][46910] Updated weights for policy 0, policy_version 752 (0.1518) [2025-05-13 00:31:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3080192. Throughput: 0: 264.7. Samples: 760122. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:31:05,296][40219] Avg episode reward: [(0, '22.524')] [2025-05-13 00:31:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1013.6). Total num frames: 3088384. Throughput: 0: 266.5. Samples: 761040. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:31:10,291][40219] Avg episode reward: [(0, '23.220')] [2025-05-13 00:31:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3092480. Throughput: 0: 255.8. Samples: 762636. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:31:15,296][40219] Avg episode reward: [(0, '22.919')] [2025-05-13 00:31:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 3096576. Throughput: 0: 262.4. Samples: 763958. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:31:20,291][40219] Avg episode reward: [(0, '23.602')] [2025-05-13 00:31:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3104768. Throughput: 0: 261.3. Samples: 764940. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:25,294][40219] Avg episode reward: [(0, '23.726')] [2025-05-13 00:31:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3108864. Throughput: 0: 261.7. Samples: 766568. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:30,291][40219] Avg episode reward: [(0, '23.756')] [2025-05-13 00:31:35,289][40219] Fps is (10 sec: 409.6, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3108864. Throughput: 0: 252.4. Samples: 767558. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:35,291][40219] Avg episode reward: [(0, '23.680')] [2025-05-13 00:31:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3117056. Throughput: 0: 253.6. Samples: 768554. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:40,295][40219] Avg episode reward: [(0, '23.838')] [2025-05-13 00:31:42,523][46910] Updated weights for policy 0, policy_version 762 (0.1062) [2025-05-13 00:31:44,579][46896] Signal inference workers to stop experience collection... (750 times) [2025-05-13 00:31:44,612][46910] InferenceWorker_p0-w0: stopping experience collection (750 times) [2025-05-13 00:31:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3121152. Throughput: 0: 253.0. Samples: 770124. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:45,291][40219] Avg episode reward: [(0, '23.195')] [2025-05-13 00:31:46,503][46896] Signal inference workers to resume experience collection... (750 times) [2025-05-13 00:31:46,505][46910] InferenceWorker_p0-w0: resuming experience collection (750 times) [2025-05-13 00:31:50,295][40219] Fps is (10 sec: 818.7, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 3125248. Throughput: 0: 250.5. Samples: 771394. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:50,301][40219] Avg episode reward: [(0, '23.043')] [2025-05-13 00:31:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3133440. Throughput: 0: 254.0. Samples: 772470. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:31:55,291][40219] Avg episode reward: [(0, '23.111')] [2025-05-13 00:32:00,289][40219] Fps is (10 sec: 1229.5, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3137536. Throughput: 0: 250.7. Samples: 773918. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:00,291][40219] Avg episode reward: [(0, '23.565')] [2025-05-13 00:32:05,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3141632. Throughput: 0: 254.3. Samples: 775400. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2025-05-13 00:32:05,291][40219] Avg episode reward: [(0, '23.374')] [2025-05-13 00:32:06,518][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000768_3145728.pth... [2025-05-13 00:32:06,609][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000709_2904064.pth [2025-05-13 00:32:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3149824. Throughput: 0: 252.7. Samples: 776312. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:32:10,296][40219] Avg episode reward: [(0, '23.431')] [2025-05-13 00:32:15,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3153920. Throughput: 0: 250.5. Samples: 777840. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:32:15,293][40219] Avg episode reward: [(0, '22.755')] [2025-05-13 00:32:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3158016. Throughput: 0: 258.6. Samples: 779194. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:32:20,291][40219] Avg episode reward: [(0, '22.728')] [2025-05-13 00:32:22,114][46910] Updated weights for policy 0, policy_version 772 (0.1565) [2025-05-13 00:32:25,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3162112. Throughput: 0: 255.3. Samples: 780044. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:32:25,291][40219] Avg episode reward: [(0, '23.178')] [2025-05-13 00:32:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3170304. Throughput: 0: 261.7. Samples: 781902. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:32:30,294][40219] Avg episode reward: [(0, '23.649')] [2025-05-13 00:32:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3174400. Throughput: 0: 260.6. Samples: 783120. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:35,296][40219] Avg episode reward: [(0, '23.552')] [2025-05-13 00:32:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3178496. Throughput: 0: 256.2. Samples: 783998. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:40,292][40219] Avg episode reward: [(0, '23.199')] [2025-05-13 00:32:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3182592. Throughput: 0: 267.3. Samples: 785948. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:45,291][40219] Avg episode reward: [(0, '23.601')] [2025-05-13 00:32:50,293][40219] Fps is (10 sec: 1228.4, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3190784. Throughput: 0: 259.0. Samples: 787054. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:50,294][40219] Avg episode reward: [(0, '23.819')] [2025-05-13 00:32:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3194880. Throughput: 0: 261.9. Samples: 788096. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:32:55,291][40219] Avg episode reward: [(0, '23.390')] [2025-05-13 00:33:00,289][40219] Fps is (10 sec: 819.5, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3198976. Throughput: 0: 262.0. Samples: 789630. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:33:00,294][40219] Avg episode reward: [(0, '22.614')] [2025-05-13 00:33:01,776][46910] Updated weights for policy 0, policy_version 782 (0.1392) [2025-05-13 00:33:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3207168. Throughput: 0: 263.2. Samples: 791038. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:33:05,291][40219] Avg episode reward: [(0, '21.879')] [2025-05-13 00:33:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3211264. Throughput: 0: 266.2. Samples: 792022. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:33:10,291][40219] Avg episode reward: [(0, '22.379')] [2025-05-13 00:33:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3215360. Throughput: 0: 253.6. Samples: 793312. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:33:15,295][40219] Avg episode reward: [(0, '22.635')] [2025-05-13 00:33:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3219456. Throughput: 0: 265.0. Samples: 795044. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:33:20,297][40219] Avg episode reward: [(0, '22.728')] [2025-05-13 00:33:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3227648. Throughput: 0: 264.1. Samples: 795884. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:25,291][40219] Avg episode reward: [(0, '23.030')] [2025-05-13 00:33:30,295][40219] Fps is (10 sec: 1228.1, 60 sec: 1023.9, 300 sec: 1027.5). Total num frames: 3231744. Throughput: 0: 248.5. Samples: 797132. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:30,296][40219] Avg episode reward: [(0, '22.830')] [2025-05-13 00:33:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3235840. Throughput: 0: 265.5. Samples: 799000. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:35,295][40219] Avg episode reward: [(0, '23.331')] [2025-05-13 00:33:39,554][46910] Updated weights for policy 0, policy_version 792 (0.0034) [2025-05-13 00:33:40,289][40219] Fps is (10 sec: 1229.5, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3244032. Throughput: 0: 260.0. Samples: 799798. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:40,294][40219] Avg episode reward: [(0, '24.273')] [2025-05-13 00:33:44,242][46896] Saving new best policy, reward=24.273! [2025-05-13 00:33:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3248128. Throughput: 0: 256.4. Samples: 801168. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:45,291][40219] Avg episode reward: [(0, '23.865')] [2025-05-13 00:33:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1027.5). Total num frames: 3252224. Throughput: 0: 262.8. Samples: 802866. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:50,294][40219] Avg episode reward: [(0, '24.203')] [2025-05-13 00:33:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3260416. Throughput: 0: 253.5. Samples: 803430. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:33:55,291][40219] Avg episode reward: [(0, '24.149')] [2025-05-13 00:34:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3260416. Throughput: 0: 262.2. Samples: 805112. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:34:00,298][40219] Avg episode reward: [(0, '24.714')] [2025-05-13 00:34:03,785][46896] Saving new best policy, reward=24.714! [2025-05-13 00:34:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1041.4). Total num frames: 3268608. Throughput: 0: 256.7. Samples: 806596. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:34:05,293][40219] Avg episode reward: [(0, '24.537')] [2025-05-13 00:34:07,432][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000799_3272704.pth... [2025-05-13 00:34:07,527][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000738_3022848.pth [2025-05-13 00:34:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3272704. Throughput: 0: 255.4. Samples: 807378. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:34:10,291][40219] Avg episode reward: [(0, '24.267')] [2025-05-13 00:34:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3276800. Throughput: 0: 257.0. Samples: 808694. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:34:15,291][40219] Avg episode reward: [(0, '24.787')] [2025-05-13 00:34:16,303][46896] Saving new best policy, reward=24.787! [2025-05-13 00:34:20,263][46910] Updated weights for policy 0, policy_version 802 (0.1501) [2025-05-13 00:34:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3284992. Throughput: 0: 256.0. Samples: 810522. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:34:20,297][40219] Avg episode reward: [(0, '24.137')] [2025-05-13 00:34:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3289088. Throughput: 0: 261.9. Samples: 811582. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:34:25,296][40219] Avg episode reward: [(0, '24.152')] [2025-05-13 00:34:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1027.5). Total num frames: 3293184. Throughput: 0: 254.2. Samples: 812608. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:34:30,290][40219] Avg episode reward: [(0, '24.328')] [2025-05-13 00:34:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3297280. Throughput: 0: 260.5. Samples: 814588. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:34:35,296][40219] Avg episode reward: [(0, '24.037')] [2025-05-13 00:34:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1027.5). Total num frames: 3301376. Throughput: 0: 265.3. Samples: 815368. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:34:40,291][40219] Avg episode reward: [(0, '24.510')] [2025-05-13 00:34:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1027.5). Total num frames: 3305472. Throughput: 0: 249.2. Samples: 816328. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:34:45,295][40219] Avg episode reward: [(0, '24.334')] [2025-05-13 00:34:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3313664. Throughput: 0: 248.0. Samples: 817758. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:34:50,291][40219] Avg episode reward: [(0, '25.026')] [2025-05-13 00:34:53,236][46896] Saving new best policy, reward=25.026! [2025-05-13 00:34:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 1027.5). Total num frames: 3317760. Throughput: 0: 253.1. Samples: 818766. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:34:55,291][40219] Avg episode reward: [(0, '25.740')] [2025-05-13 00:34:57,690][46896] Saving new best policy, reward=25.740! [2025-05-13 00:35:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3321856. Throughput: 0: 252.4. Samples: 820050. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:35:00,291][40219] Avg episode reward: [(0, '26.033')] [2025-05-13 00:35:01,843][46896] Saving new best policy, reward=26.033! [2025-05-13 00:35:01,848][46910] Updated weights for policy 0, policy_version 812 (0.0958) [2025-05-13 00:35:03,995][46896] Signal inference workers to stop experience collection... (800 times) [2025-05-13 00:35:04,057][46910] InferenceWorker_p0-w0: stopping experience collection (800 times) [2025-05-13 00:35:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1027.5). Total num frames: 3325952. Throughput: 0: 252.1. Samples: 821868. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:35:05,302][40219] Avg episode reward: [(0, '25.703')] [2025-05-13 00:35:05,460][46896] Signal inference workers to resume experience collection... (800 times) [2025-05-13 00:35:05,462][46910] InferenceWorker_p0-w0: resuming experience collection (800 times) [2025-05-13 00:35:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3334144. Throughput: 0: 245.5. Samples: 822630. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:10,293][40219] Avg episode reward: [(0, '25.488')] [2025-05-13 00:35:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3338240. Throughput: 0: 250.4. Samples: 823874. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:15,291][40219] Avg episode reward: [(0, '25.911')] [2025-05-13 00:35:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1027.5). Total num frames: 3342336. Throughput: 0: 248.3. Samples: 825760. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:20,297][40219] Avg episode reward: [(0, '25.000')] [2025-05-13 00:35:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3350528. Throughput: 0: 248.0. Samples: 826528. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:25,291][40219] Avg episode reward: [(0, '25.181')] [2025-05-13 00:35:30,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3354624. Throughput: 0: 257.4. Samples: 827910. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:30,291][40219] Avg episode reward: [(0, '24.916')] [2025-05-13 00:35:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3358720. Throughput: 0: 261.7. Samples: 829536. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:35,291][40219] Avg episode reward: [(0, '24.969')] [2025-05-13 00:35:40,292][40219] Fps is (10 sec: 819.0, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3362816. Throughput: 0: 256.9. Samples: 830326. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:40,294][40219] Avg episode reward: [(0, '25.044')] [2025-05-13 00:35:41,169][46910] Updated weights for policy 0, policy_version 822 (0.0491) [2025-05-13 00:35:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3366912. Throughput: 0: 259.9. Samples: 831746. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:45,296][40219] Avg episode reward: [(0, '24.963')] [2025-05-13 00:35:50,289][40219] Fps is (10 sec: 1229.1, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3375104. Throughput: 0: 255.1. Samples: 833346. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:35:50,298][40219] Avg episode reward: [(0, '24.764')] [2025-05-13 00:35:55,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3379200. Throughput: 0: 256.0. Samples: 834152. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:35:55,294][40219] Avg episode reward: [(0, '24.465')] [2025-05-13 00:36:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3383296. Throughput: 0: 259.6. Samples: 835554. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:36:00,301][40219] Avg episode reward: [(0, '24.122')] [2025-05-13 00:36:05,289][40219] Fps is (10 sec: 1229.0, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3391488. Throughput: 0: 254.0. Samples: 837188. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:36:05,295][40219] Avg episode reward: [(0, '24.524')] [2025-05-13 00:36:08,620][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000829_3395584.pth... [2025-05-13 00:36:08,737][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000768_3145728.pth [2025-05-13 00:36:10,294][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3395584. Throughput: 0: 259.6. Samples: 838212. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:10,296][40219] Avg episode reward: [(0, '24.198')] [2025-05-13 00:36:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3399680. Throughput: 0: 251.6. Samples: 839234. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:15,291][40219] Avg episode reward: [(0, '24.138')] [2025-05-13 00:36:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3403776. Throughput: 0: 261.2. Samples: 841288. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:36:20,291][40219] Avg episode reward: [(0, '24.489')] [2025-05-13 00:36:20,649][46910] Updated weights for policy 0, policy_version 832 (0.0504) [2025-05-13 00:36:25,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3411968. Throughput: 0: 261.6. Samples: 842100. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:25,293][40219] Avg episode reward: [(0, '24.516')] [2025-05-13 00:36:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1041.4). Total num frames: 3416064. Throughput: 0: 257.2. Samples: 843320. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:30,293][40219] Avg episode reward: [(0, '24.594')] [2025-05-13 00:36:35,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3420160. Throughput: 0: 263.4. Samples: 845200. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:36:35,291][40219] Avg episode reward: [(0, '24.709')] [2025-05-13 00:36:40,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3424256. Throughput: 0: 258.9. Samples: 845800. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:36:40,291][40219] Avg episode reward: [(0, '24.859')] [2025-05-13 00:36:45,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3432448. Throughput: 0: 262.6. Samples: 847370. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:45,295][40219] Avg episode reward: [(0, '25.065')] [2025-05-13 00:36:50,291][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3436544. Throughput: 0: 261.6. Samples: 848962. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:50,295][40219] Avg episode reward: [(0, '25.322')] [2025-05-13 00:36:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3440640. Throughput: 0: 251.3. Samples: 849522. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:36:55,295][40219] Avg episode reward: [(0, '25.033')] [2025-05-13 00:37:00,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3444736. Throughput: 0: 266.0. Samples: 851204. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:37:00,294][40219] Avg episode reward: [(0, '25.887')] [2025-05-13 00:37:00,516][46910] Updated weights for policy 0, policy_version 842 (0.0035) [2025-05-13 00:37:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3452928. Throughput: 0: 250.9. Samples: 852580. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:37:05,292][40219] Avg episode reward: [(0, '25.600')] [2025-05-13 00:37:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3457024. Throughput: 0: 253.1. Samples: 853490. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:37:10,291][40219] Avg episode reward: [(0, '25.615')] [2025-05-13 00:37:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3461120. Throughput: 0: 256.7. Samples: 854872. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:37:15,294][40219] Avg episode reward: [(0, '25.130')] [2025-05-13 00:37:20,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1092.3, 300 sec: 1041.4). Total num frames: 3469312. Throughput: 0: 253.5. Samples: 856606. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:20,291][40219] Avg episode reward: [(0, '25.413')] [2025-05-13 00:37:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3473408. Throughput: 0: 255.6. Samples: 857300. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:25,296][40219] Avg episode reward: [(0, '25.205')] [2025-05-13 00:37:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3477504. Throughput: 0: 249.8. Samples: 858612. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:30,291][40219] Avg episode reward: [(0, '25.387')] [2025-05-13 00:37:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3481600. Throughput: 0: 260.3. Samples: 860674. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:35,291][40219] Avg episode reward: [(0, '25.874')] [2025-05-13 00:37:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3485696. Throughput: 0: 253.7. Samples: 860938. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:40,299][40219] Avg episode reward: [(0, '25.861')] [2025-05-13 00:37:40,897][46910] Updated weights for policy 0, policy_version 852 (0.1400) [2025-05-13 00:37:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3493888. Throughput: 0: 257.4. Samples: 862786. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:45,291][40219] Avg episode reward: [(0, '26.294')] [2025-05-13 00:37:48,160][46896] Saving new best policy, reward=26.294! [2025-05-13 00:37:50,293][40219] Fps is (10 sec: 1228.3, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3497984. Throughput: 0: 256.6. Samples: 864126. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:37:50,295][40219] Avg episode reward: [(0, '25.988')] [2025-05-13 00:37:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3502080. Throughput: 0: 251.2. Samples: 864794. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:37:55,293][40219] Avg episode reward: [(0, '25.860')] [2025-05-13 00:38:00,289][40219] Fps is (10 sec: 819.5, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3506176. Throughput: 0: 244.5. Samples: 865874. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:38:00,291][40219] Avg episode reward: [(0, '26.470')] [2025-05-13 00:38:01,892][46896] Saving new best policy, reward=26.470! [2025-05-13 00:38:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3510272. Throughput: 0: 249.7. Samples: 867844. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:05,304][40219] Avg episode reward: [(0, '27.288')] [2025-05-13 00:38:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3514368. Throughput: 0: 247.4. Samples: 868434. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:10,291][40219] Avg episode reward: [(0, '27.352')] [2025-05-13 00:38:10,588][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000859_3518464.pth... [2025-05-13 00:38:10,682][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000799_3272704.pth [2025-05-13 00:38:10,698][46896] Saving new best policy, reward=27.288! [2025-05-13 00:38:14,294][46896] Saving new best policy, reward=27.352! [2025-05-13 00:38:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3522560. Throughput: 0: 250.7. Samples: 869894. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:15,291][40219] Avg episode reward: [(0, '26.926')] [2025-05-13 00:38:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3526656. Throughput: 0: 244.2. Samples: 871664. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:20,299][40219] Avg episode reward: [(0, '26.935')] [2025-05-13 00:38:22,042][46910] Updated weights for policy 0, policy_version 862 (0.0498) [2025-05-13 00:38:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3530752. Throughput: 0: 248.0. Samples: 872096. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:25,300][40219] Avg episode reward: [(0, '27.096')] [2025-05-13 00:38:25,381][46896] Signal inference workers to stop experience collection... (850 times) [2025-05-13 00:38:25,424][46910] InferenceWorker_p0-w0: stopping experience collection (850 times) [2025-05-13 00:38:26,501][46896] Signal inference workers to resume experience collection... (850 times) [2025-05-13 00:38:26,502][46910] InferenceWorker_p0-w0: resuming experience collection (850 times) [2025-05-13 00:38:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3538944. Throughput: 0: 245.0. Samples: 873812. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:38:30,298][40219] Avg episode reward: [(0, '26.822')] [2025-05-13 00:38:35,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3543040. Throughput: 0: 253.3. Samples: 875522. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:38:35,292][40219] Avg episode reward: [(0, '25.332')] [2025-05-13 00:38:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3547136. Throughput: 0: 251.5. Samples: 876112. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:38:40,291][40219] Avg episode reward: [(0, '26.399')] [2025-05-13 00:38:45,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3551232. Throughput: 0: 264.3. Samples: 877766. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2025-05-13 00:38:45,292][40219] Avg episode reward: [(0, '26.470')] [2025-05-13 00:38:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 3559424. Throughput: 0: 252.3. Samples: 879196. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:38:50,291][40219] Avg episode reward: [(0, '26.281')] [2025-05-13 00:38:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3563520. Throughput: 0: 256.4. Samples: 879972. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:38:55,295][40219] Avg episode reward: [(0, '25.529')] [2025-05-13 00:39:00,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3567616. Throughput: 0: 257.1. Samples: 881464. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:00,298][40219] Avg episode reward: [(0, '26.243')] [2025-05-13 00:39:01,405][46910] Updated weights for policy 0, policy_version 872 (0.1369) [2025-05-13 00:39:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3571712. Throughput: 0: 256.8. Samples: 883218. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:05,294][40219] Avg episode reward: [(0, '26.401')] [2025-05-13 00:39:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1092.3, 300 sec: 1027.5). Total num frames: 3579904. Throughput: 0: 258.0. Samples: 883708. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:10,291][40219] Avg episode reward: [(0, '26.000')] [2025-05-13 00:39:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3584000. Throughput: 0: 256.1. Samples: 885336. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:39:15,298][40219] Avg episode reward: [(0, '26.432')] [2025-05-13 00:39:20,298][40219] Fps is (10 sec: 818.5, 60 sec: 1023.9, 300 sec: 1013.6). Total num frames: 3588096. Throughput: 0: 255.6. Samples: 887028. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:39:20,299][40219] Avg episode reward: [(0, '25.692')] [2025-05-13 00:39:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3592192. Throughput: 0: 251.2. Samples: 887414. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:25,295][40219] Avg episode reward: [(0, '25.779')] [2025-05-13 00:39:30,289][40219] Fps is (10 sec: 1229.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3600384. Throughput: 0: 258.9. Samples: 889416. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:39:30,299][40219] Avg episode reward: [(0, '25.423')] [2025-05-13 00:39:35,290][40219] Fps is (10 sec: 1228.7, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3604480. Throughput: 0: 255.2. Samples: 890680. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:39:35,292][40219] Avg episode reward: [(0, '24.875')] [2025-05-13 00:39:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3608576. Throughput: 0: 254.6. Samples: 891428. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:40,291][40219] Avg episode reward: [(0, '23.459')] [2025-05-13 00:39:42,128][46910] Updated weights for policy 0, policy_version 882 (0.0947) [2025-05-13 00:39:45,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3612672. Throughput: 0: 256.9. Samples: 893024. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:39:45,296][40219] Avg episode reward: [(0, '23.633')] [2025-05-13 00:39:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3620864. Throughput: 0: 248.6. Samples: 894404. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:39:50,292][40219] Avg episode reward: [(0, '23.643')] [2025-05-13 00:39:55,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3624960. Throughput: 0: 256.2. Samples: 895236. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:39:55,299][40219] Avg episode reward: [(0, '23.938')] [2025-05-13 00:40:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3629056. Throughput: 0: 253.4. Samples: 896740. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:00,299][40219] Avg episode reward: [(0, '24.172')] [2025-05-13 00:40:05,292][40219] Fps is (10 sec: 819.0, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3633152. Throughput: 0: 251.0. Samples: 898322. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:05,293][40219] Avg episode reward: [(0, '23.642')] [2025-05-13 00:40:09,746][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000889_3641344.pth... [2025-05-13 00:40:09,833][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000829_3395584.pth [2025-05-13 00:40:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3641344. Throughput: 0: 259.0. Samples: 899070. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:10,291][40219] Avg episode reward: [(0, '23.679')] [2025-05-13 00:40:15,289][40219] Fps is (10 sec: 1229.1, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3645440. Throughput: 0: 247.2. Samples: 900540. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:15,291][40219] Avg episode reward: [(0, '24.057')] [2025-05-13 00:40:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.1, 300 sec: 1013.6). Total num frames: 3649536. Throughput: 0: 251.8. Samples: 902010. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:20,291][40219] Avg episode reward: [(0, '23.330')] [2025-05-13 00:40:22,666][46910] Updated weights for policy 0, policy_version 892 (0.2001) [2025-05-13 00:40:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3653632. Throughput: 0: 249.6. Samples: 902658. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:25,299][40219] Avg episode reward: [(0, '23.312')] [2025-05-13 00:40:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3661824. Throughput: 0: 258.2. Samples: 904642. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:30,296][40219] Avg episode reward: [(0, '23.172')] [2025-05-13 00:40:35,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3665920. Throughput: 0: 252.3. Samples: 905756. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:40:35,291][40219] Avg episode reward: [(0, '22.856')] [2025-05-13 00:40:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3670016. Throughput: 0: 254.4. Samples: 906684. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:40:40,299][40219] Avg episode reward: [(0, '22.708')] [2025-05-13 00:40:45,291][40219] Fps is (10 sec: 1228.6, 60 sec: 1092.2, 300 sec: 1027.5). Total num frames: 3678208. Throughput: 0: 264.7. Samples: 908650. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:40:45,293][40219] Avg episode reward: [(0, '22.710')] [2025-05-13 00:40:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3682304. Throughput: 0: 252.0. Samples: 909662. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:50,291][40219] Avg episode reward: [(0, '23.290')] [2025-05-13 00:40:55,289][40219] Fps is (10 sec: 819.3, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3686400. Throughput: 0: 257.3. Samples: 910650. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:40:55,291][40219] Avg episode reward: [(0, '23.894')] [2025-05-13 00:41:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3690496. Throughput: 0: 262.1. Samples: 912336. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:41:00,291][40219] Avg episode reward: [(0, '24.249')] [2025-05-13 00:41:01,155][46910] Updated weights for policy 0, policy_version 902 (0.0491) [2025-05-13 00:41:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3694592. Throughput: 0: 256.8. Samples: 913566. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:41:05,296][40219] Avg episode reward: [(0, '24.148')] [2025-05-13 00:41:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3698688. Throughput: 0: 250.4. Samples: 913928. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:41:10,298][40219] Avg episode reward: [(0, '25.639')] [2025-05-13 00:41:15,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3706880. Throughput: 0: 249.6. Samples: 915872. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:41:15,294][40219] Avg episode reward: [(0, '25.153')] [2025-05-13 00:41:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3710976. Throughput: 0: 248.0. Samples: 916914. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:41:20,291][40219] Avg episode reward: [(0, '25.124')] [2025-05-13 00:41:25,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3715072. Throughput: 0: 250.0. Samples: 917934. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:41:25,291][40219] Avg episode reward: [(0, '25.409')] [2025-05-13 00:41:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3719168. Throughput: 0: 243.1. Samples: 919588. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:41:30,295][40219] Avg episode reward: [(0, '26.458')] [2025-05-13 00:41:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3723264. Throughput: 0: 251.5. Samples: 920978. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:41:35,293][40219] Avg episode reward: [(0, '26.623')] [2025-05-13 00:41:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3731456. Throughput: 0: 251.1. Samples: 921950. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:41:40,295][40219] Avg episode reward: [(0, '26.650')] [2025-05-13 00:41:42,513][46910] Updated weights for policy 0, policy_version 912 (0.0926) [2025-05-13 00:41:45,292][40219] Fps is (10 sec: 1228.5, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3735552. Throughput: 0: 247.1. Samples: 923454. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2025-05-13 00:41:45,293][40219] Avg episode reward: [(0, '25.792')] [2025-05-13 00:41:45,623][46896] Signal inference workers to stop experience collection... (900 times) [2025-05-13 00:41:45,725][46910] InferenceWorker_p0-w0: stopping experience collection (900 times) [2025-05-13 00:41:47,451][46896] Signal inference workers to resume experience collection... (900 times) [2025-05-13 00:41:47,452][46910] InferenceWorker_p0-w0: resuming experience collection (900 times) [2025-05-13 00:41:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3739648. Throughput: 0: 248.1. Samples: 924732. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:41:50,295][40219] Avg episode reward: [(0, '25.877')] [2025-05-13 00:41:55,289][40219] Fps is (10 sec: 1229.1, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3747840. Throughput: 0: 255.8. Samples: 925438. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:41:55,292][40219] Avg episode reward: [(0, '26.317')] [2025-05-13 00:42:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3751936. Throughput: 0: 249.2. Samples: 927086. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:42:00,291][40219] Avg episode reward: [(0, '25.973')] [2025-05-13 00:42:05,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3756032. Throughput: 0: 254.5. Samples: 928368. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:42:05,291][40219] Avg episode reward: [(0, '25.780')] [2025-05-13 00:42:07,116][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000918_3760128.pth... [2025-05-13 00:42:07,212][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000859_3518464.pth [2025-05-13 00:42:10,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3760128. Throughput: 0: 251.8. Samples: 929266. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:42:10,299][40219] Avg episode reward: [(0, '25.604')] [2025-05-13 00:42:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3764224. Throughput: 0: 257.2. Samples: 931164. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:15,291][40219] Avg episode reward: [(0, '25.665')] [2025-05-13 00:42:20,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3772416. Throughput: 0: 250.4. Samples: 932244. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:20,297][40219] Avg episode reward: [(0, '25.486')] [2025-05-13 00:42:23,670][46910] Updated weights for policy 0, policy_version 922 (0.3082) [2025-05-13 00:42:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3776512. Throughput: 0: 251.1. Samples: 933248. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:25,295][40219] Avg episode reward: [(0, '25.360')] [2025-05-13 00:42:30,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3780608. Throughput: 0: 245.8. Samples: 934514. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:42:30,293][40219] Avg episode reward: [(0, '25.508')] [2025-05-13 00:42:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3784704. Throughput: 0: 255.4. Samples: 936226. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:42:35,291][40219] Avg episode reward: [(0, '25.138')] [2025-05-13 00:42:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3792896. Throughput: 0: 252.6. Samples: 936804. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:40,295][40219] Avg episode reward: [(0, '24.926')] [2025-05-13 00:42:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3796992. Throughput: 0: 248.8. Samples: 938284. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:45,295][40219] Avg episode reward: [(0, '25.028')] [2025-05-13 00:42:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3801088. Throughput: 0: 254.8. Samples: 939834. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:50,291][40219] Avg episode reward: [(0, '24.850')] [2025-05-13 00:42:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3805184. Throughput: 0: 252.5. Samples: 940628. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:42:55,299][40219] Avg episode reward: [(0, '23.880')] [2025-05-13 00:43:00,290][40219] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3809280. Throughput: 0: 241.7. Samples: 942040. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:00,294][40219] Avg episode reward: [(0, '23.783')] [2025-05-13 00:43:04,625][46910] Updated weights for policy 0, policy_version 932 (0.1659) [2025-05-13 00:43:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1027.5). Total num frames: 3817472. Throughput: 0: 249.5. Samples: 943472. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:05,291][40219] Avg episode reward: [(0, '23.361')] [2025-05-13 00:43:10,289][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3821568. Throughput: 0: 248.9. Samples: 944450. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:10,291][40219] Avg episode reward: [(0, '23.395')] [2025-05-13 00:43:15,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3825664. Throughput: 0: 249.2. Samples: 945728. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:15,292][40219] Avg episode reward: [(0, '22.244')] [2025-05-13 00:43:20,290][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3829760. Throughput: 0: 251.2. Samples: 947528. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:20,298][40219] Avg episode reward: [(0, '22.751')] [2025-05-13 00:43:25,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3837952. Throughput: 0: 255.7. Samples: 948310. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:25,297][40219] Avg episode reward: [(0, '22.862')] [2025-05-13 00:43:30,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3842048. Throughput: 0: 251.4. Samples: 949598. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:30,293][40219] Avg episode reward: [(0, '23.697')] [2025-05-13 00:43:35,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3846144. Throughput: 0: 255.3. Samples: 951322. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:35,295][40219] Avg episode reward: [(0, '23.335')] [2025-05-13 00:43:40,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 1013.6). Total num frames: 3850240. Throughput: 0: 255.2. Samples: 952114. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2025-05-13 00:43:40,295][40219] Avg episode reward: [(0, '23.006')] [2025-05-13 00:43:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3854336. Throughput: 0: 255.2. Samples: 953522. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:45,291][40219] Avg episode reward: [(0, '23.403')] [2025-05-13 00:43:45,443][46910] Updated weights for policy 0, policy_version 942 (0.0490) [2025-05-13 00:43:50,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3862528. Throughput: 0: 257.4. Samples: 955054. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:50,295][40219] Avg episode reward: [(0, '23.194')] [2025-05-13 00:43:55,290][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3866624. Throughput: 0: 254.5. Samples: 955904. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2025-05-13 00:43:55,291][40219] Avg episode reward: [(0, '22.947')] [2025-05-13 00:44:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3870720. Throughput: 0: 258.4. Samples: 957354. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:00,299][40219] Avg episode reward: [(0, '23.051')] [2025-05-13 00:44:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3878912. Throughput: 0: 254.7. Samples: 958990. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:05,291][40219] Avg episode reward: [(0, '24.050')] [2025-05-13 00:44:08,315][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000948_3883008.pth... [2025-05-13 00:44:08,412][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000889_3641344.pth [2025-05-13 00:44:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3883008. Throughput: 0: 260.0. Samples: 960010. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:10,291][40219] Avg episode reward: [(0, '24.533')] [2025-05-13 00:44:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3887104. Throughput: 0: 254.5. Samples: 961050. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:44:15,291][40219] Avg episode reward: [(0, '23.798')] [2025-05-13 00:44:20,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3891200. Throughput: 0: 261.3. Samples: 963082. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:44:20,292][40219] Avg episode reward: [(0, '23.005')] [2025-05-13 00:44:25,291][40219] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3895296. Throughput: 0: 253.6. Samples: 963528. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:25,292][40219] Avg episode reward: [(0, '23.292')] [2025-05-13 00:44:26,205][46910] Updated weights for policy 0, policy_version 952 (0.0495) [2025-05-13 00:44:30,291][40219] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3899392. Throughput: 0: 247.7. Samples: 964670. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:30,299][40219] Avg episode reward: [(0, '23.756')] [2025-05-13 00:44:35,289][40219] Fps is (10 sec: 1229.0, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3907584. Throughput: 0: 246.5. Samples: 966146. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:44:35,291][40219] Avg episode reward: [(0, '23.484')] [2025-05-13 00:44:40,290][40219] Fps is (10 sec: 1228.9, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3911680. Throughput: 0: 249.6. Samples: 967138. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:44:40,292][40219] Avg episode reward: [(0, '23.769')] [2025-05-13 00:44:45,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3915776. Throughput: 0: 242.7. Samples: 968274. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:44:45,291][40219] Avg episode reward: [(0, '23.953')] [2025-05-13 00:44:50,289][40219] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3919872. Throughput: 0: 247.9. Samples: 970144. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:50,297][40219] Avg episode reward: [(0, '24.364')] [2025-05-13 00:44:55,293][40219] Fps is (10 sec: 1228.4, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3928064. Throughput: 0: 237.1. Samples: 970680. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:44:55,294][40219] Avg episode reward: [(0, '24.019')] [2025-05-13 00:45:00,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3932160. Throughput: 0: 249.8. Samples: 972292. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:00,292][40219] Avg episode reward: [(0, '24.505')] [2025-05-13 00:45:05,289][40219] Fps is (10 sec: 819.5, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3936256. Throughput: 0: 237.9. Samples: 973786. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:05,291][40219] Avg episode reward: [(0, '24.262')] [2025-05-13 00:45:07,128][46910] Updated weights for policy 0, policy_version 962 (0.1422) [2025-05-13 00:45:09,270][46896] Signal inference workers to stop experience collection... (950 times) [2025-05-13 00:45:09,337][46910] InferenceWorker_p0-w0: stopping experience collection (950 times) [2025-05-13 00:45:10,291][40219] Fps is (10 sec: 819.0, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3940352. Throughput: 0: 244.6. Samples: 974536. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:10,293][40219] Avg episode reward: [(0, '23.938')] [2025-05-13 00:45:11,151][46896] Signal inference workers to resume experience collection... (950 times) [2025-05-13 00:45:11,151][46910] InferenceWorker_p0-w0: resuming experience collection (950 times) [2025-05-13 00:45:15,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3944448. Throughput: 0: 251.6. Samples: 975990. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:15,291][40219] Avg episode reward: [(0, '24.301')] [2025-05-13 00:45:20,289][40219] Fps is (10 sec: 1229.0, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3952640. Throughput: 0: 251.0. Samples: 977440. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:45:20,298][40219] Avg episode reward: [(0, '24.054')] [2025-05-13 00:45:25,292][40219] Fps is (10 sec: 1228.5, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3956736. Throughput: 0: 252.5. Samples: 978502. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:45:25,293][40219] Avg episode reward: [(0, '24.553')] [2025-05-13 00:45:30,290][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3960832. Throughput: 0: 250.8. Samples: 979558. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:30,291][40219] Avg episode reward: [(0, '25.312')] [2025-05-13 00:45:35,289][40219] Fps is (10 sec: 819.4, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3964928. Throughput: 0: 252.8. Samples: 981520. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2025-05-13 00:45:35,296][40219] Avg episode reward: [(0, '25.915')] [2025-05-13 00:45:40,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3973120. Throughput: 0: 259.8. Samples: 982372. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2025-05-13 00:45:40,291][40219] Avg episode reward: [(0, '25.690')] [2025-05-13 00:45:45,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3977216. Throughput: 0: 250.4. Samples: 983562. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:45:45,291][40219] Avg episode reward: [(0, '26.104')] [2025-05-13 00:45:48,554][46910] Updated weights for policy 0, policy_version 972 (0.1869) [2025-05-13 00:45:50,289][40219] Fps is (10 sec: 819.2, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 3981312. Throughput: 0: 250.7. Samples: 985066. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:45:50,291][40219] Avg episode reward: [(0, '26.000')] [2025-05-13 00:45:55,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 999.7). Total num frames: 3985408. Throughput: 0: 254.7. Samples: 985998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:45:55,291][40219] Avg episode reward: [(0, '25.762')] [2025-05-13 00:46:00,289][40219] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 999.7). Total num frames: 3989504. Throughput: 0: 249.6. Samples: 987222. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:46:00,298][40219] Avg episode reward: [(0, '25.422')] [2025-05-13 00:46:05,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 1013.6). Total num frames: 3997696. Throughput: 0: 256.8. Samples: 988996. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:46:05,291][40219] Avg episode reward: [(0, '26.193')] [2025-05-13 00:46:07,493][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000977_4001792.pth... [2025-05-13 00:46:07,589][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000918_3760128.pth [2025-05-13 00:46:10,289][40219] Fps is (10 sec: 1228.8, 60 sec: 1024.0, 300 sec: 999.7). Total num frames: 4001792. Throughput: 0: 249.3. Samples: 989718. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2025-05-13 00:46:10,291][40219] Avg episode reward: [(0, '26.094')] [2025-05-13 00:46:12,772][46896] Stopping Batcher_0... [2025-05-13 00:46:12,774][46896] Loop batcher_evt_loop terminating... [2025-05-13 00:46:12,775][40219] Component Batcher_0 stopped! [2025-05-13 00:46:12,893][46910] Weights refcount: 2 0 [2025-05-13 00:46:12,909][40219] Component InferenceWorker_p0-w0 stopped! [2025-05-13 00:46:12,919][46910] Stopping InferenceWorker_p0-w0... [2025-05-13 00:46:12,919][46910] Loop inference_proc0-0_evt_loop terminating... [2025-05-13 00:46:13,257][40219] Component RolloutWorker_w7 stopped! [2025-05-13 00:46:13,261][46917] Stopping RolloutWorker_w7... [2025-05-13 00:46:13,282][40219] Component RolloutWorker_w5 stopped! [2025-05-13 00:46:13,289][46917] Loop rollout_proc7_evt_loop terminating... [2025-05-13 00:46:13,283][46915] Stopping RolloutWorker_w5... [2025-05-13 00:46:13,294][46915] Loop rollout_proc5_evt_loop terminating... [2025-05-13 00:46:13,295][40219] Component RolloutWorker_w3 stopped! [2025-05-13 00:46:13,311][40219] Component RolloutWorker_w1 stopped! [2025-05-13 00:46:13,305][46913] Stopping RolloutWorker_w3... [2025-05-13 00:46:13,316][46909] Stopping RolloutWorker_w1... [2025-05-13 00:46:13,319][46913] Loop rollout_proc3_evt_loop terminating... [2025-05-13 00:46:13,322][46909] Loop rollout_proc1_evt_loop terminating... [2025-05-13 00:46:13,509][40219] Component RolloutWorker_w4 stopped! [2025-05-13 00:46:13,514][40219] Component RolloutWorker_w6 stopped! [2025-05-13 00:46:13,515][46914] Stopping RolloutWorker_w6... [2025-05-13 00:46:13,523][46914] Loop rollout_proc6_evt_loop terminating... [2025-05-13 00:46:13,509][46916] Stopping RolloutWorker_w4... [2025-05-13 00:46:13,536][40219] Component RolloutWorker_w0 stopped! [2025-05-13 00:46:13,530][46916] Loop rollout_proc4_evt_loop terminating... [2025-05-13 00:46:13,536][46912] Stopping RolloutWorker_w0... [2025-05-13 00:46:13,563][46912] Loop rollout_proc0_evt_loop terminating... [2025-05-13 00:46:13,566][40219] Component RolloutWorker_w2 stopped! [2025-05-13 00:46:13,567][46911] Stopping RolloutWorker_w2... [2025-05-13 00:46:13,595][46911] Loop rollout_proc2_evt_loop terminating... [2025-05-13 00:46:17,965][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2025-05-13 00:46:18,061][46896] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000948_3883008.pth [2025-05-13 00:46:18,077][46896] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2025-05-13 00:46:18,223][46896] Stopping LearnerWorker_p0... [2025-05-13 00:46:18,224][46896] Loop learner_proc0_evt_loop terminating... [2025-05-13 00:46:18,223][40219] Component LearnerWorker_p0 stopped! [2025-05-13 00:46:18,226][40219] Waiting for process learner_proc0 to stop... [2025-05-13 00:46:19,441][40219] Waiting for process inference_proc0-0 to join... [2025-05-13 00:46:19,443][40219] Waiting for process rollout_proc0 to join... [2025-05-13 00:46:19,985][40219] Waiting for process rollout_proc1 to join... [2025-05-13 00:46:19,986][40219] Waiting for process rollout_proc2 to join... [2025-05-13 00:46:19,988][40219] Waiting for process rollout_proc3 to join... [2025-05-13 00:46:19,991][40219] Waiting for process rollout_proc4 to join... [2025-05-13 00:46:19,992][40219] Waiting for process rollout_proc5 to join... [2025-05-13 00:46:19,996][40219] Waiting for process rollout_proc6 to join... [2025-05-13 00:46:19,997][40219] Waiting for process rollout_proc7 to join... [2025-05-13 00:46:19,999][40219] Batcher 0 profile tree view: batching: 11.8818, releasing_batches: 0.2984 [2025-05-13 00:46:20,000][40219] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 46.8878 update_model: 111.3880 weight_update: 0.1993 one_step: 0.0279 handle_policy_step: 2443.7901 deserialize: 71.7293, stack: 10.2786, obs_to_device_normalize: 453.2122, forward: 1773.7187, send_messages: 56.0023 prepare_outputs: 25.1632 to_cpu: 2.5198 [2025-05-13 00:46:20,002][40219] Learner 0 profile tree view: misc: 0.0056, prepare_batch: 1101.4090 train: 2805.3554 epoch_init: 0.0059, minibatch_init: 0.0240, losses_postprocess: 0.1312, kl_divergence: 0.4107, after_optimizer: 2.2424 calculate_losses: 1435.0971 losses_init: 0.0042, forward_head: 1309.3427, bptt_initial: 3.6056, tail: 2.4824, advantages_returns: 0.1550, losses: 1.1305 bptt: 118.0433 bptt_forward_core: 117.2696 update: 1366.8667 clip: 3.3976 [2025-05-13 00:46:20,003][40219] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.3317, enqueue_policy_requests: 37.1535, env_step: 1404.7475, overhead: 38.3532, complete_rollouts: 16.6352 save_policy_outputs: 19.5956 split_output_tensors: 9.7645 [2025-05-13 00:46:20,004][40219] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.4371, enqueue_policy_requests: 37.7348, env_step: 1390.9844, overhead: 41.1963, complete_rollouts: 12.1543 save_policy_outputs: 20.0792 split_output_tensors: 9.6006 [2025-05-13 00:46:20,005][40219] Loop Runner_EvtLoop terminating... [2025-05-13 00:46:20,006][40219] Runner profile tree view: main_loop: 3971.9463 [2025-05-13 00:46:20,007][40219] Collected {0: 4009984}, FPS: 997.2 [2025-05-13 00:46:20,049][40219] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-05-13 00:46:20,050][40219] Overriding arg 'num_workers' with value 1 passed from command line [2025-05-13 00:46:20,052][40219] Adding new argument 'no_render'=True that is not in the saved config file! [2025-05-13 00:46:20,053][40219] Adding new argument 'save_video'=True that is not in the saved config file! [2025-05-13 00:46:20,054][40219] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-05-13 00:46:20,055][40219] Adding new argument 'video_name'=None that is not in the saved config file! [2025-05-13 00:46:20,056][40219] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-05-13 00:46:20,057][40219] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-05-13 00:46:20,058][40219] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-05-13 00:46:20,060][40219] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-05-13 00:46:20,061][40219] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-05-13 00:46:20,062][40219] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-05-13 00:46:20,063][40219] Adding new argument 'train_script'=None that is not in the saved config file! [2025-05-13 00:46:20,064][40219] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-05-13 00:46:20,065][40219] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-05-13 00:46:20,125][40219] RunningMeanStd input shape: (3, 72, 128) [2025-05-13 00:46:20,129][40219] RunningMeanStd input shape: (1,) [2025-05-13 00:46:20,162][40219] ConvEncoder: input_channels=3 [2025-05-13 00:46:20,219][40219] Conv encoder output size: 512 [2025-05-13 00:46:20,220][40219] Policy head output size: 512 [2025-05-13 00:46:20,241][40219] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2025-05-13 00:46:20,787][40219] Num frames 100... [2025-05-13 00:46:20,982][40219] Num frames 200... [2025-05-13 00:46:21,234][40219] Num frames 300... [2025-05-13 00:46:21,478][40219] Num frames 400... [2025-05-13 00:46:21,689][40219] Num frames 500... [2025-05-13 00:46:21,891][40219] Num frames 600... [2025-05-13 00:46:22,064][40219] Avg episode rewards: #0: 14.550, true rewards: #0: 6.550 [2025-05-13 00:46:22,065][40219] Avg episode reward: 14.550, avg true_objective: 6.550 [2025-05-13 00:46:22,161][40219] Num frames 700... [2025-05-13 00:46:22,366][40219] Num frames 800... [2025-05-13 00:46:22,588][40219] Num frames 900... [2025-05-13 00:46:22,904][40219] Num frames 1000... [2025-05-13 00:46:23,204][40219] Num frames 1100... [2025-05-13 00:46:23,489][40219] Num frames 1200... [2025-05-13 00:46:23,766][40219] Num frames 1300... [2025-05-13 00:46:24,065][40219] Num frames 1400... [2025-05-13 00:46:24,349][40219] Num frames 1500... [2025-05-13 00:46:24,638][40219] Num frames 1600... [2025-05-13 00:46:24,936][40219] Num frames 1700... [2025-05-13 00:46:25,137][40219] Num frames 1800... [2025-05-13 00:46:25,332][40219] Num frames 1900... [2025-05-13 00:46:25,533][40219] Num frames 2000... [2025-05-13 00:46:25,717][40219] Avg episode rewards: #0: 27.315, true rewards: #0: 10.315 [2025-05-13 00:46:25,718][40219] Avg episode reward: 27.315, avg true_objective: 10.315 [2025-05-13 00:46:25,795][40219] Num frames 2100... [2025-05-13 00:46:25,997][40219] Num frames 2200... [2025-05-13 00:46:26,201][40219] Num frames 2300... [2025-05-13 00:46:26,411][40219] Num frames 2400... [2025-05-13 00:46:26,614][40219] Num frames 2500... [2025-05-13 00:46:26,815][40219] Num frames 2600... [2025-05-13 00:46:27,019][40219] Num frames 2700... [2025-05-13 00:46:27,214][40219] Num frames 2800... [2025-05-13 00:46:27,401][40219] Num frames 2900... [2025-05-13 00:46:27,594][40219] Num frames 3000... [2025-05-13 00:46:27,789][40219] Num frames 3100... [2025-05-13 00:46:27,989][40219] Num frames 3200... [2025-05-13 00:46:28,182][40219] Num frames 3300... [2025-05-13 00:46:28,260][40219] Avg episode rewards: #0: 28.367, true rewards: #0: 11.033 [2025-05-13 00:46:28,261][40219] Avg episode reward: 28.367, avg true_objective: 11.033 [2025-05-13 00:46:28,434][40219] Num frames 3400... [2025-05-13 00:46:28,628][40219] Num frames 3500... [2025-05-13 00:46:28,827][40219] Num frames 3600... [2025-05-13 00:46:29,030][40219] Num frames 3700... [2025-05-13 00:46:29,225][40219] Num frames 3800... [2025-05-13 00:46:29,419][40219] Num frames 3900... [2025-05-13 00:46:29,610][40219] Num frames 4000... [2025-05-13 00:46:29,807][40219] Num frames 4100... [2025-05-13 00:46:29,886][40219] Avg episode rewards: #0: 25.025, true rewards: #0: 10.275 [2025-05-13 00:46:29,888][40219] Avg episode reward: 25.025, avg true_objective: 10.275 [2025-05-13 00:46:30,066][40219] Num frames 4200... [2025-05-13 00:46:30,265][40219] Num frames 4300... [2025-05-13 00:46:30,458][40219] Num frames 4400... [2025-05-13 00:46:30,647][40219] Num frames 4500... [2025-05-13 00:46:30,842][40219] Num frames 4600... [2025-05-13 00:46:31,033][40219] Num frames 4700... [2025-05-13 00:46:31,245][40219] Num frames 4800... [2025-05-13 00:46:31,447][40219] Num frames 4900... [2025-05-13 00:46:31,636][40219] Num frames 5000... [2025-05-13 00:46:31,826][40219] Num frames 5100... [2025-05-13 00:46:32,015][40219] Num frames 5200... [2025-05-13 00:46:32,218][40219] Num frames 5300... [2025-05-13 00:46:32,404][40219] Num frames 5400... [2025-05-13 00:46:32,504][40219] Avg episode rewards: #0: 25.844, true rewards: #0: 10.844 [2025-05-13 00:46:32,505][40219] Avg episode reward: 25.844, avg true_objective: 10.844 [2025-05-13 00:46:32,654][40219] Num frames 5500... [2025-05-13 00:46:32,852][40219] Num frames 5600... [2025-05-13 00:46:33,047][40219] Num frames 5700... [2025-05-13 00:46:33,255][40219] Num frames 5800... [2025-05-13 00:46:33,451][40219] Num frames 5900... [2025-05-13 00:46:33,643][40219] Num frames 6000... [2025-05-13 00:46:33,838][40219] Num frames 6100... [2025-05-13 00:46:34,038][40219] Num frames 6200... [2025-05-13 00:46:34,252][40219] Num frames 6300... [2025-05-13 00:46:34,468][40219] Avg episode rewards: #0: 25.135, true rewards: #0: 10.635 [2025-05-13 00:46:34,469][40219] Avg episode reward: 25.135, avg true_objective: 10.635 [2025-05-13 00:46:34,508][40219] Num frames 6400... [2025-05-13 00:46:34,708][40219] Num frames 6500... [2025-05-13 00:46:34,903][40219] Num frames 6600... [2025-05-13 00:46:35,167][40219] Num frames 6700... [2025-05-13 00:46:35,447][40219] Num frames 6800... [2025-05-13 00:46:35,703][40219] Num frames 6900... [2025-05-13 00:46:35,964][40219] Num frames 7000... [2025-05-13 00:46:36,220][40219] Num frames 7100... [2025-05-13 00:46:36,492][40219] Num frames 7200... [2025-05-13 00:46:36,763][40219] Num frames 7300... [2025-05-13 00:46:37,042][40219] Num frames 7400... [2025-05-13 00:46:37,187][40219] Avg episode rewards: #0: 24.903, true rewards: #0: 10.617 [2025-05-13 00:46:37,189][40219] Avg episode reward: 24.903, avg true_objective: 10.617 [2025-05-13 00:46:37,326][40219] Num frames 7500... [2025-05-13 00:46:37,524][40219] Num frames 7600... [2025-05-13 00:46:37,715][40219] Num frames 7700... [2025-05-13 00:46:37,904][40219] Num frames 7800... [2025-05-13 00:46:38,107][40219] Num frames 7900... [2025-05-13 00:46:38,297][40219] Num frames 8000... [2025-05-13 00:46:38,498][40219] Num frames 8100... [2025-05-13 00:46:38,687][40219] Avg episode rewards: #0: 23.710, true rewards: #0: 10.210 [2025-05-13 00:46:38,688][40219] Avg episode reward: 23.710, avg true_objective: 10.210 [2025-05-13 00:46:38,753][40219] Num frames 8200... [2025-05-13 00:46:38,944][40219] Num frames 8300... [2025-05-13 00:46:39,130][40219] Num frames 8400... [2025-05-13 00:46:39,312][40219] Num frames 8500... [2025-05-13 00:46:39,509][40219] Num frames 8600... [2025-05-13 00:46:39,694][40219] Num frames 8700... [2025-05-13 00:46:39,835][40219] Avg episode rewards: #0: 22.160, true rewards: #0: 9.716 [2025-05-13 00:46:39,836][40219] Avg episode reward: 22.160, avg true_objective: 9.716 [2025-05-13 00:46:39,949][40219] Num frames 8800... [2025-05-13 00:46:40,136][40219] Num frames 8900... [2025-05-13 00:46:40,325][40219] Num frames 9000... [2025-05-13 00:46:40,520][40219] Num frames 9100... [2025-05-13 00:46:40,745][40219] Avg episode rewards: #0: 20.792, true rewards: #0: 9.192 [2025-05-13 00:46:40,746][40219] Avg episode reward: 20.792, avg true_objective: 9.192 [2025-05-13 00:47:40,143][40219] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-05-13 00:49:28,401][40219] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-05-13 00:49:28,402][40219] Overriding arg 'num_workers' with value 1 passed from command line [2025-05-13 00:49:28,405][40219] Adding new argument 'no_render'=True that is not in the saved config file! [2025-05-13 00:49:28,406][40219] Adding new argument 'save_video'=True that is not in the saved config file! [2025-05-13 00:49:28,407][40219] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-05-13 00:49:28,409][40219] Adding new argument 'video_name'=None that is not in the saved config file! [2025-05-13 00:49:28,410][40219] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-05-13 00:49:28,411][40219] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-05-13 00:49:28,413][40219] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-05-13 00:49:28,414][40219] Adding new argument 'hf_repository'='DTambu/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-05-13 00:49:28,416][40219] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-05-13 00:49:28,417][40219] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-05-13 00:49:28,418][40219] Adding new argument 'train_script'=None that is not in the saved config file! [2025-05-13 00:49:28,419][40219] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-05-13 00:49:28,421][40219] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-05-13 00:49:28,451][40219] RunningMeanStd input shape: (3, 72, 128) [2025-05-13 00:49:28,452][40219] RunningMeanStd input shape: (1,) [2025-05-13 00:49:28,471][40219] ConvEncoder: input_channels=3 [2025-05-13 00:49:28,543][40219] Conv encoder output size: 512 [2025-05-13 00:49:28,545][40219] Policy head output size: 512 [2025-05-13 00:49:28,570][40219] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2025-05-13 00:49:29,337][40219] Num frames 100... [2025-05-13 00:49:29,591][40219] Num frames 200... [2025-05-13 00:49:29,829][40219] Num frames 300... [2025-05-13 00:49:30,059][40219] Num frames 400... [2025-05-13 00:49:30,300][40219] Num frames 500... [2025-05-13 00:49:30,387][40219] Avg episode rewards: #0: 7.120, true rewards: #0: 5.120 [2025-05-13 00:49:30,388][40219] Avg episode reward: 7.120, avg true_objective: 5.120 [2025-05-13 00:49:30,615][40219] Num frames 600... [2025-05-13 00:49:30,872][40219] Num frames 700... [2025-05-13 00:49:31,117][40219] Num frames 800... [2025-05-13 00:49:31,284][40219] Num frames 900... [2025-05-13 00:49:31,459][40219] Num frames 1000... [2025-05-13 00:49:31,630][40219] Num frames 1100... [2025-05-13 00:49:31,815][40219] Num frames 1200... [2025-05-13 00:49:31,984][40219] Num frames 1300... [2025-05-13 00:49:32,161][40219] Num frames 1400... [2025-05-13 00:49:32,334][40219] Num frames 1500... [2025-05-13 00:49:32,511][40219] Num frames 1600... [2025-05-13 00:49:32,694][40219] Num frames 1700... [2025-05-13 00:49:32,878][40219] Num frames 1800... [2025-05-13 00:49:33,050][40219] Num frames 1900... [2025-05-13 00:49:33,222][40219] Num frames 2000... [2025-05-13 00:49:33,396][40219] Num frames 2100... [2025-05-13 00:49:33,567][40219] Num frames 2200... [2025-05-13 00:49:33,759][40219] Num frames 2300... [2025-05-13 00:49:33,931][40219] Num frames 2400... [2025-05-13 00:49:34,115][40219] Avg episode rewards: #0: 32.859, true rewards: #0: 12.360 [2025-05-13 00:49:34,116][40219] Avg episode reward: 32.859, avg true_objective: 12.360 [2025-05-13 00:49:34,168][40219] Num frames 2500... [2025-05-13 00:49:34,340][40219] Num frames 2600... [2025-05-13 00:49:34,513][40219] Num frames 2700... [2025-05-13 00:49:34,692][40219] Num frames 2800... [2025-05-13 00:49:34,879][40219] Num frames 2900... [2025-05-13 00:49:35,051][40219] Num frames 3000... [2025-05-13 00:49:35,262][40219] Num frames 3100... [2025-05-13 00:49:35,552][40219] Num frames 3200... [2025-05-13 00:49:35,900][40219] Num frames 3300... [2025-05-13 00:49:36,097][40219] Num frames 3400... [2025-05-13 00:49:36,264][40219] Avg episode rewards: #0: 29.213, true rewards: #0: 11.547 [2025-05-13 00:49:36,265][40219] Avg episode reward: 29.213, avg true_objective: 11.547 [2025-05-13 00:49:36,336][40219] Num frames 3500... [2025-05-13 00:49:36,511][40219] Num frames 3600... [2025-05-13 00:49:36,693][40219] Num frames 3700... [2025-05-13 00:49:36,880][40219] Num frames 3800... [2025-05-13 00:49:37,059][40219] Num frames 3900... [2025-05-13 00:49:37,229][40219] Num frames 4000... [2025-05-13 00:49:37,422][40219] Avg episode rewards: #0: 25.445, true rewards: #0: 10.195 [2025-05-13 00:49:37,423][40219] Avg episode reward: 25.445, avg true_objective: 10.195 [2025-05-13 00:49:37,465][40219] Num frames 4100... [2025-05-13 00:49:37,644][40219] Num frames 4200... [2025-05-13 00:49:37,816][40219] Num frames 4300... [2025-05-13 00:49:38,010][40219] Num frames 4400... [2025-05-13 00:49:38,194][40219] Num frames 4500... [2025-05-13 00:49:38,366][40219] Num frames 4600... [2025-05-13 00:49:38,538][40219] Num frames 4700... [2025-05-13 00:49:38,712][40219] Num frames 4800... [2025-05-13 00:49:38,885][40219] Num frames 4900... [2025-05-13 00:49:39,067][40219] Num frames 5000... [2025-05-13 00:49:39,239][40219] Num frames 5100... [2025-05-13 00:49:39,452][40219] Avg episode rewards: #0: 26.172, true rewards: #0: 10.372 [2025-05-13 00:49:39,453][40219] Avg episode reward: 26.172, avg true_objective: 10.372 [2025-05-13 00:49:39,480][40219] Num frames 5200... [2025-05-13 00:49:39,656][40219] Num frames 5300... [2025-05-13 00:49:39,832][40219] Num frames 5400... [2025-05-13 00:49:40,014][40219] Num frames 5500... [2025-05-13 00:49:40,190][40219] Num frames 5600... [2025-05-13 00:49:40,362][40219] Num frames 5700... [2025-05-13 00:49:40,535][40219] Num frames 5800... [2025-05-13 00:49:40,708][40219] Num frames 5900... [2025-05-13 00:49:40,880][40219] Num frames 6000... [2025-05-13 00:49:41,067][40219] Num frames 6100... [2025-05-13 00:49:41,318][40219] Num frames 6200... [2025-05-13 00:49:41,574][40219] Num frames 6300... [2025-05-13 00:49:41,822][40219] Num frames 6400... [2025-05-13 00:49:42,060][40219] Num frames 6500... [2025-05-13 00:49:42,307][40219] Num frames 6600... [2025-05-13 00:49:42,551][40219] Num frames 6700... [2025-05-13 00:49:42,801][40219] Num frames 6800... [2025-05-13 00:49:43,056][40219] Num frames 6900... [2025-05-13 00:49:43,315][40219] Num frames 7000... [2025-05-13 00:49:43,571][40219] Num frames 7100... [2025-05-13 00:49:43,767][40219] Num frames 7200... [2025-05-13 00:49:43,973][40219] Avg episode rewards: #0: 31.643, true rewards: #0: 12.143 [2025-05-13 00:49:43,975][40219] Avg episode reward: 31.643, avg true_objective: 12.143 [2025-05-13 00:49:44,001][40219] Num frames 7300... [2025-05-13 00:49:44,170][40219] Num frames 7400... [2025-05-13 00:49:44,356][40219] Num frames 7500... [2025-05-13 00:49:44,527][40219] Num frames 7600... [2025-05-13 00:49:44,698][40219] Num frames 7700... [2025-05-13 00:49:44,953][40219] Num frames 7800... [2025-05-13 00:49:45,163][40219] Num frames 7900... [2025-05-13 00:49:45,346][40219] Num frames 8000... [2025-05-13 00:49:45,529][40219] Num frames 8100... [2025-05-13 00:49:45,705][40219] Num frames 8200... [2025-05-13 00:49:45,826][40219] Avg episode rewards: #0: 30.190, true rewards: #0: 11.761 [2025-05-13 00:49:45,828][40219] Avg episode reward: 30.190, avg true_objective: 11.761 [2025-05-13 00:49:45,944][40219] Num frames 8300... [2025-05-13 00:49:46,128][40219] Num frames 8400... [2025-05-13 00:49:46,309][40219] Num frames 8500... [2025-05-13 00:49:46,483][40219] Num frames 8600... [2025-05-13 00:49:46,662][40219] Num frames 8700... [2025-05-13 00:49:46,800][40219] Avg episode rewards: #0: 27.806, true rewards: #0: 10.931 [2025-05-13 00:49:46,802][40219] Avg episode reward: 27.806, avg true_objective: 10.931 [2025-05-13 00:49:46,898][40219] Num frames 8800... [2025-05-13 00:49:47,074][40219] Num frames 8900... [2025-05-13 00:49:47,251][40219] Num frames 9000... [2025-05-13 00:49:47,438][40219] Num frames 9100... [2025-05-13 00:49:47,615][40219] Num frames 9200... [2025-05-13 00:49:47,789][40219] Num frames 9300... [2025-05-13 00:49:47,961][40219] Num frames 9400... [2025-05-13 00:49:48,140][40219] Num frames 9500... [2025-05-13 00:49:48,318][40219] Num frames 9600... [2025-05-13 00:49:48,504][40219] Num frames 9700... [2025-05-13 00:49:48,680][40219] Num frames 9800... [2025-05-13 00:49:48,891][40219] Avg episode rewards: #0: 28.094, true rewards: #0: 10.983 [2025-05-13 00:49:48,892][40219] Avg episode reward: 28.094, avg true_objective: 10.983 [2025-05-13 00:49:48,920][40219] Num frames 9900... [2025-05-13 00:49:49,099][40219] Num frames 10000... [2025-05-13 00:49:49,312][40219] Num frames 10100... [2025-05-13 00:49:49,502][40219] Num frames 10200... [2025-05-13 00:49:49,685][40219] Num frames 10300... [2025-05-13 00:49:49,859][40219] Num frames 10400... [2025-05-13 00:49:50,032][40219] Num frames 10500... [2025-05-13 00:49:50,203][40219] Num frames 10600... [2025-05-13 00:49:50,384][40219] Num frames 10700... [2025-05-13 00:49:50,589][40219] Num frames 10800... [2025-05-13 00:49:50,820][40219] Avg episode rewards: #0: 27.799, true rewards: #0: 10.899 [2025-05-13 00:49:50,822][40219] Avg episode reward: 27.799, avg true_objective: 10.899 [2025-05-13 00:49:50,826][40219] Num frames 10900... [2025-05-13 00:51:03,461][40219] Replay video saved to /content/train_dir/default_experiment/replay.mp4!