running_log.txt

[INFO|2025-04-28 22:00:30] configuration_utils.py:696 >> loading configuration file config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json

[INFO|2025-04-28 22:00:30] configuration_utils.py:768 >> Model config LlamaConfig {
  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
  "architectures": [
    "LlamaForCausalLM"
  ],
  "attention_bias": false,
  "attention_dropout": 0.0,
  "bos_token_id": 128000,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 131072,
  "mlp_bias": false,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-05,
  "rope_scaling": {
    "factor": 8.0,
    "high_freq_factor": 4.0,
    "low_freq_factor": 1.0,
    "original_max_position_embeddings": 8192,
    "rope_type": "llama3"
  },
  "rope_theta": 500000.0,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.48.2",
  "use_cache": true,
  "vocab_size": 128256
}


[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file tokenizer.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json

[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file tokenizer.model from cache at None

[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file added_tokens.json from cache at None

[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file special_tokens_map.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json

[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file tokenizer_config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json

[INFO|2025-04-28 22:00:30] tokenization_utils_base.py:2034 >> loading file chat_template.jinja from cache at None

[INFO|2025-04-28 22:00:31] tokenization_utils_base.py:2304 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[INFO|2025-04-28 22:00:32] configuration_utils.py:696 >> loading configuration file config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json

[INFO|2025-04-28 22:00:32] configuration_utils.py:768 >> Model config LlamaConfig {
  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
  "architectures": [
    "LlamaForCausalLM"
  ],
  "attention_bias": false,
  "attention_dropout": 0.0,
  "bos_token_id": 128000,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 131072,
  "mlp_bias": false,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-05,
  "rope_scaling": {
    "factor": 8.0,
    "high_freq_factor": 4.0,
    "low_freq_factor": 1.0,
    "original_max_position_embeddings": 8192,
    "rope_type": "llama3"
  },
  "rope_theta": 500000.0,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.48.2",
  "use_cache": true,
  "vocab_size": 128256
}


[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file tokenizer.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer.json

[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file tokenizer.model from cache at None

[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file added_tokens.json from cache at None

[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file special_tokens_map.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/special_tokens_map.json

[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file tokenizer_config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/tokenizer_config.json

[INFO|2025-04-28 22:00:34] tokenization_utils_base.py:2034 >> loading file chat_template.jinja from cache at None

[INFO|2025-04-28 22:00:35] tokenization_utils_base.py:2304 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[INFO|2025-04-28 22:00:35] logging.py:157 >> Add pad token: <|eot_id|>

[INFO|2025-04-28 22:00:35] logging.py:157 >> Add <|eot_id|>,<|eom_id|> to stop words.

[INFO|2025-04-28 22:00:35] logging.py:157 >> Loading dataset Codes_query_filtered_330k_ns.json...

[INFO|2025-04-28 22:01:31] configuration_utils.py:696 >> loading configuration file config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/config.json

[INFO|2025-04-28 22:01:31] configuration_utils.py:768 >> Model config LlamaConfig {
  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
  "architectures": [
    "LlamaForCausalLM"
  ],
  "attention_bias": false,
  "attention_dropout": 0.0,
  "bos_token_id": 128000,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 131072,
  "mlp_bias": false,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-05,
  "rope_scaling": {
    "factor": 8.0,
    "high_freq_factor": 4.0,
    "low_freq_factor": 1.0,
    "original_max_position_embeddings": 8192,
    "rope_type": "llama3"
  },
  "rope_theta": 500000.0,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.48.2",
  "use_cache": true,
  "vocab_size": 128256
}


[WARNING|2025-04-28 22:01:31] logging.py:162 >> Input length is smaller than max length. Consider increase input length.

[INFO|2025-04-28 22:01:31] logging.py:157 >> Using llama3 scaling strategy and setting scaling factor to 1.0.

[INFO|2025-04-28 22:01:31] logging.py:157 >> Using block diagonal attention for sequence packing without cross-attention.

[INFO|2025-04-28 22:01:32] logging.py:157 >> Liger kernel has been applied to the model.

[INFO|2025-04-28 22:01:32] modeling_utils.py:3904 >> loading weights file model.safetensors from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/model.safetensors.index.json

[INFO|2025-04-28 22:01:32] modeling_utils.py:1582 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.

[INFO|2025-04-28 22:01:32] configuration_utils.py:1140 >> Generate config GenerationConfig {
  "bos_token_id": 128000,
  "eos_token_id": [
    128001,
    128008,
    128009
  ]
}


[INFO|2025-04-28 22:01:45] modeling_utils.py:4888 >> All model checkpoint weights were used when initializing LlamaForCausalLM.


[INFO|2025-04-28 22:01:45] modeling_utils.py:4896 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at meta-llama/Llama-3.1-8B-Instruct.
If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.

[INFO|2025-04-28 22:01:45] configuration_utils.py:1095 >> loading configuration file generation_config.json from cache at /home/kiho/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659/generation_config.json

[INFO|2025-04-28 22:01:45] configuration_utils.py:1140 >> Generate config GenerationConfig {
  "bos_token_id": 128000,
  "do_sample": true,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "temperature": 0.6,
  "top_p": 0.9
}


[INFO|2025-04-28 22:01:45] logging.py:157 >> Gradient checkpointing enabled.

[INFO|2025-04-28 22:01:45] logging.py:157 >> Using torch SDPA for faster training and inference.

[INFO|2025-04-28 22:01:45] logging.py:157 >> Upcasting trainable params to float32.

[INFO|2025-04-28 22:01:45] logging.py:157 >> Fine-tuning method: Freeze

[INFO|2025-04-28 22:01:45] logging.py:157 >> Set trainable layers: .15.,.31.

[INFO|2025-04-28 22:01:45] logging.py:157 >> trainable params: 436,224,000 || all params: 8,030,261,248 || trainable%: 5.4323

[INFO|2025-04-28 22:01:45] trainer.py:741 >> Using auto half precision backend

[INFO|2025-04-28 22:01:46] logging.py:157 >> Found linear modules: q_proj,k_proj,o_proj,up_proj,gate_proj,v_proj,down_proj

[INFO|2025-04-28 22:01:46] logging.py:157 >> Using APOLLO optimizer with args: {'rank': 256, 'proj': 'random', 'proj_type': 'std', 'update_proj_gap': 200, 'scale': 1, 'scale_type': 'channel', 'scale_front': False}.

[INFO|2025-04-28 22:01:46] trainer.py:2369 >> ***** Running training *****

[INFO|2025-04-28 22:01:46] trainer.py:2370 >>   Num examples = 52,860

[INFO|2025-04-28 22:01:46] trainer.py:2371 >>   Num Epochs = 1

[INFO|2025-04-28 22:01:46] trainer.py:2372 >>   Instantaneous batch size per device = 16

[INFO|2025-04-28 22:01:46] trainer.py:2375 >>   Total train batch size (w. parallel, distributed & accumulation) = 512

[INFO|2025-04-28 22:01:46] trainer.py:2376 >>   Gradient Accumulation steps = 8

[INFO|2025-04-28 22:01:46] trainer.py:2377 >>   Total optimization steps = 103

[INFO|2025-04-28 22:01:46] trainer.py:2378 >>   Number of trainable parameters = 436,224,000

[INFO|2025-04-28 22:04:48] logging.py:157 >> {'loss': 1.2027, 'learning_rate': 4.9988e-05, 'epoch': 0.01, 'throughput': 11584.80}

[INFO|2025-04-28 22:07:41] logging.py:157 >> {'loss': 1.0647, 'learning_rate': 4.9953e-05, 'epoch': 0.02, 'throughput': 11850.11}

[INFO|2025-04-28 22:10:33] logging.py:157 >> {'loss': 1.0243, 'learning_rate': 4.9895e-05, 'epoch': 0.03, 'throughput': 11957.01}

[INFO|2025-04-28 22:13:25] logging.py:157 >> {'loss': 0.9974, 'learning_rate': 4.9814e-05, 'epoch': 0.04, 'throughput': 12012.50}

[INFO|2025-04-28 22:16:17] logging.py:157 >> {'loss': 0.9874, 'learning_rate': 4.9710e-05, 'epoch': 0.05, 'throughput': 12050.19}

[INFO|2025-04-28 22:19:09] logging.py:157 >> {'loss': 0.9478, 'learning_rate': 4.9583e-05, 'epoch': 0.06, 'throughput': 12071.84}

[INFO|2025-04-28 22:22:01] logging.py:157 >> {'loss': 0.9761, 'learning_rate': 4.9432e-05, 'epoch': 0.07, 'throughput': 12087.55}

[INFO|2025-04-28 22:24:54] logging.py:157 >> {'loss': 0.9698, 'learning_rate': 4.9259e-05, 'epoch': 0.08, 'throughput': 12096.64}

[INFO|2025-04-28 22:27:46] logging.py:157 >> {'loss': 0.9804, 'learning_rate': 4.9064e-05, 'epoch': 0.09, 'throughput': 12106.38}

[INFO|2025-04-28 22:30:38] logging.py:157 >> {'loss': 0.9562, 'learning_rate': 4.8846e-05, 'epoch': 0.10, 'throughput': 12115.60}

[INFO|2025-04-28 22:33:30] logging.py:157 >> {'loss': 0.9649, 'learning_rate': 4.8606e-05, 'epoch': 0.11, 'throughput': 12121.46}

[INFO|2025-04-28 22:36:23] logging.py:157 >> {'loss': 0.9300, 'learning_rate': 4.8344e-05, 'epoch': 0.12, 'throughput': 12123.79}

[INFO|2025-04-28 22:39:16] logging.py:157 >> {'loss': 0.9356, 'learning_rate': 4.8060e-05, 'epoch': 0.13, 'throughput': 12124.38}

[INFO|2025-04-28 22:42:08] logging.py:157 >> {'loss': 0.9279, 'learning_rate': 4.7755e-05, 'epoch': 0.14, 'throughput': 12125.60}

[INFO|2025-04-28 22:45:00] logging.py:157 >> {'loss': 0.9311, 'learning_rate': 4.7429e-05, 'epoch': 0.15, 'throughput': 12130.58}

[INFO|2025-04-28 22:47:52] logging.py:157 >> {'loss': 0.9311, 'learning_rate': 4.7082e-05, 'epoch': 0.15, 'throughput': 12134.17}

[INFO|2025-04-28 22:50:45] logging.py:157 >> {'loss': 0.9381, 'learning_rate': 4.6714e-05, 'epoch': 0.16, 'throughput': 12136.31}

[INFO|2025-04-28 22:53:37] logging.py:157 >> {'loss': 0.9354, 'learning_rate': 4.6326e-05, 'epoch': 0.17, 'throughput': 12138.18}

[INFO|2025-04-28 22:56:29] logging.py:157 >> {'loss': 0.9574, 'learning_rate': 4.5918e-05, 'epoch': 0.18, 'throughput': 12139.11}

[INFO|2025-04-28 22:59:22] logging.py:157 >> {'loss': 0.9124, 'learning_rate': 4.5491e-05, 'epoch': 0.19, 'throughput': 12140.73}

[INFO|2025-04-28 23:02:14] logging.py:157 >> {'loss': 0.9500, 'learning_rate': 4.5045e-05, 'epoch': 0.20, 'throughput': 12142.69}

[INFO|2025-04-28 23:05:06] logging.py:157 >> {'loss': 0.9179, 'learning_rate': 4.4580e-05, 'epoch': 0.21, 'throughput': 12143.43}

[INFO|2025-04-28 23:07:59] logging.py:157 >> {'loss': 0.9414, 'learning_rate': 4.4097e-05, 'epoch': 0.22, 'throughput': 12144.94}

[INFO|2025-04-28 23:10:50] logging.py:157 >> {'loss': 0.9348, 'learning_rate': 4.3596e-05, 'epoch': 0.23, 'throughput': 12148.22}

[INFO|2025-04-28 23:13:43] logging.py:157 >> {'loss': 0.9260, 'learning_rate': 4.3077e-05, 'epoch': 0.24, 'throughput': 12148.04}

[INFO|2025-04-28 23:16:35] logging.py:157 >> {'loss': 0.9323, 'learning_rate': 4.2542e-05, 'epoch': 0.25, 'throughput': 12150.35}

[INFO|2025-04-28 23:19:28] logging.py:157 >> {'loss': 0.9218, 'learning_rate': 4.1991e-05, 'epoch': 0.26, 'throughput': 12149.24}

[INFO|2025-04-28 23:22:20] logging.py:157 >> {'loss': 0.9319, 'learning_rate': 4.1424e-05, 'epoch': 0.27, 'throughput': 12149.20}

[INFO|2025-04-28 23:25:12] logging.py:157 >> {'loss': 0.9202, 'learning_rate': 4.0841e-05, 'epoch': 0.28, 'throughput': 12150.25}

[INFO|2025-04-28 23:28:04] logging.py:157 >> {'loss': 0.9141, 'learning_rate': 4.0244e-05, 'epoch': 0.29, 'throughput': 12152.71}

[INFO|2025-04-28 23:30:56] logging.py:157 >> {'loss': 0.9145, 'learning_rate': 3.9633e-05, 'epoch': 0.30, 'throughput': 12153.01}

[INFO|2025-04-28 23:33:47] logging.py:157 >> {'loss': 0.8975, 'learning_rate': 3.9008e-05, 'epoch': 0.31, 'throughput': 12156.93}

[INFO|2025-04-28 23:36:39] logging.py:157 >> {'loss': 0.9614, 'learning_rate': 3.8370e-05, 'epoch': 0.32, 'throughput': 12159.26}

[INFO|2025-04-28 23:39:30] logging.py:157 >> {'loss': 0.9327, 'learning_rate': 3.7719e-05, 'epoch': 0.33, 'throughput': 12161.37}

[INFO|2025-04-28 23:42:21] logging.py:157 >> {'loss': 0.9153, 'learning_rate': 3.7057e-05, 'epoch': 0.34, 'throughput': 12164.47}

[INFO|2025-04-28 23:45:12] logging.py:157 >> {'loss': 0.8977, 'learning_rate': 3.6384e-05, 'epoch': 0.35, 'throughput': 12166.53}

[INFO|2025-04-28 23:48:04] logging.py:157 >> {'loss': 0.9147, 'learning_rate': 3.5700e-05, 'epoch': 0.36, 'throughput': 12168.64}

[INFO|2025-04-28 23:50:55] logging.py:157 >> {'loss': 0.9264, 'learning_rate': 3.5006e-05, 'epoch': 0.37, 'throughput': 12170.33}

[INFO|2025-04-28 23:53:46] logging.py:157 >> {'loss': 0.9041, 'learning_rate': 3.4302e-05, 'epoch': 0.38, 'throughput': 12173.21}

[INFO|2025-04-28 23:56:37] logging.py:157 >> {'loss': 0.9322, 'learning_rate': 3.3590e-05, 'epoch': 0.39, 'throughput': 12175.56}

[INFO|2025-04-28 23:59:28] logging.py:157 >> {'loss': 0.9257, 'learning_rate': 3.2870e-05, 'epoch': 0.40, 'throughput': 12176.34}

[INFO|2025-04-29 00:02:19] logging.py:157 >> {'loss': 0.8952, 'learning_rate': 3.2143e-05, 'epoch': 0.41, 'throughput': 12178.42}

[INFO|2025-04-29 00:05:11] logging.py:157 >> {'loss': 0.9072, 'learning_rate': 3.1409e-05, 'epoch': 0.42, 'throughput': 12179.30}

[INFO|2025-04-29 00:08:02] logging.py:157 >> {'loss': 0.9166, 'learning_rate': 3.0669e-05, 'epoch': 0.43, 'throughput': 12181.27}

[INFO|2025-04-29 00:10:53] logging.py:157 >> {'loss': 0.8924, 'learning_rate': 2.9924e-05, 'epoch': 0.44, 'throughput': 12182.88}

[INFO|2025-04-29 00:13:45] logging.py:157 >> {'loss': 0.8871, 'learning_rate': 2.9174e-05, 'epoch': 0.45, 'throughput': 12184.12}

[INFO|2025-04-29 00:16:36] logging.py:157 >> {'loss': 0.9263, 'learning_rate': 2.8421e-05, 'epoch': 0.46, 'throughput': 12185.10}

[INFO|2025-04-29 00:19:27] logging.py:157 >> {'loss': 0.9158, 'learning_rate': 2.7664e-05, 'epoch': 0.46, 'throughput': 12186.29}

[INFO|2025-04-29 00:22:19] logging.py:157 >> {'loss': 0.9034, 'learning_rate': 2.6904e-05, 'epoch': 0.47, 'throughput': 12187.37}

[INFO|2025-04-29 00:25:10] logging.py:157 >> {'loss': 0.9044, 'learning_rate': 2.6143e-05, 'epoch': 0.48, 'throughput': 12187.76}

[INFO|2025-04-29 00:28:02] logging.py:157 >> {'loss': 0.9173, 'learning_rate': 2.5381e-05, 'epoch': 0.49, 'throughput': 12188.76}

[INFO|2025-04-29 00:30:53] logging.py:157 >> {'loss': 0.9349, 'learning_rate': 2.4619e-05, 'epoch': 0.50, 'throughput': 12190.37}

[INFO|2025-04-29 00:33:44] logging.py:157 >> {'loss': 0.8996, 'learning_rate': 2.3857e-05, 'epoch': 0.51, 'throughput': 12191.00}

[INFO|2025-04-29 00:36:35] logging.py:157 >> {'loss': 0.9018, 'learning_rate': 2.3096e-05, 'epoch': 0.52, 'throughput': 12192.15}

[INFO|2025-04-29 00:39:27] logging.py:157 >> {'loss': 0.8797, 'learning_rate': 2.2336e-05, 'epoch': 0.53, 'throughput': 12192.73}

[INFO|2025-04-29 00:42:19] logging.py:157 >> {'loss': 0.9075, 'learning_rate': 2.1579e-05, 'epoch': 0.54, 'throughput': 12193.09}

[INFO|2025-04-29 00:45:10] logging.py:157 >> {'loss': 0.9068, 'learning_rate': 2.0826e-05, 'epoch': 0.55, 'throughput': 12194.01}

[INFO|2025-04-29 00:48:02] logging.py:157 >> {'loss': 0.9154, 'learning_rate': 2.0076e-05, 'epoch': 0.56, 'throughput': 12194.06}

[INFO|2025-04-29 00:50:54] logging.py:157 >> {'loss': 0.8895, 'learning_rate': 1.9331e-05, 'epoch': 0.57, 'throughput': 12194.36}

[INFO|2025-04-29 00:53:45] logging.py:157 >> {'loss': 0.8910, 'learning_rate': 1.8591e-05, 'epoch': 0.58, 'throughput': 12194.93}

[INFO|2025-04-29 00:56:37] logging.py:157 >> {'loss': 0.8928, 'learning_rate': 1.7857e-05, 'epoch': 0.59, 'throughput': 12195.08}

[INFO|2025-04-29 00:59:29] logging.py:157 >> {'loss': 0.8918, 'learning_rate': 1.7130e-05, 'epoch': 0.60, 'throughput': 12195.46}

[INFO|2025-04-29 01:02:20] logging.py:157 >> {'loss': 0.8807, 'learning_rate': 1.6410e-05, 'epoch': 0.61, 'throughput': 12195.62}

[INFO|2025-04-29 01:05:12] logging.py:157 >> {'loss': 0.8934, 'learning_rate': 1.5698e-05, 'epoch': 0.62, 'throughput': 12196.25}

[INFO|2025-04-29 01:08:03] logging.py:157 >> {'loss': 0.8809, 'learning_rate': 1.4994e-05, 'epoch': 0.63, 'throughput': 12196.80}

[INFO|2025-04-29 01:10:55] logging.py:157 >> {'loss': 0.8856, 'learning_rate': 1.4300e-05, 'epoch': 0.64, 'throughput': 12197.24}

[INFO|2025-04-29 01:13:46] logging.py:157 >> {'loss': 0.9117, 'learning_rate': 1.3616e-05, 'epoch': 0.65, 'throughput': 12197.76}

[INFO|2025-04-29 01:16:38] logging.py:157 >> {'loss': 0.8993, 'learning_rate': 1.2943e-05, 'epoch': 0.66, 'throughput': 12198.14}

[INFO|2025-04-29 01:19:29] logging.py:157 >> {'loss': 0.9215, 'learning_rate': 1.2281e-05, 'epoch': 0.67, 'throughput': 12198.43}

[INFO|2025-04-29 01:22:21] logging.py:157 >> {'loss': 0.8916, 'learning_rate': 1.1630e-05, 'epoch': 0.68, 'throughput': 12199.00}

[INFO|2025-04-29 01:25:12] logging.py:157 >> {'loss': 0.8913, 'learning_rate': 1.0992e-05, 'epoch': 0.69, 'throughput': 12199.79}

[INFO|2025-04-29 01:28:03] logging.py:157 >> {'loss': 0.8973, 'learning_rate': 1.0367e-05, 'epoch': 0.70, 'throughput': 12200.41}

[INFO|2025-04-29 01:30:54] logging.py:157 >> {'loss': 0.9136, 'learning_rate': 9.7558e-06, 'epoch': 0.71, 'throughput': 12201.41}

[INFO|2025-04-29 01:33:45] logging.py:157 >> {'loss': 0.9217, 'learning_rate': 9.1586e-06, 'epoch': 0.72, 'throughput': 12201.98}

[INFO|2025-04-29 01:36:37] logging.py:157 >> {'loss': 0.8926, 'learning_rate': 8.5762e-06, 'epoch': 0.73, 'throughput': 12202.13}

[INFO|2025-04-29 01:39:28] logging.py:157 >> {'loss': 0.8828, 'learning_rate': 8.0090e-06, 'epoch': 0.74, 'throughput': 12202.67}

[INFO|2025-04-29 01:42:20] logging.py:157 >> {'loss': 0.8944, 'learning_rate': 7.4576e-06, 'epoch': 0.75, 'throughput': 12202.71}

[INFO|2025-04-29 01:45:12] logging.py:157 >> {'loss': 0.9085, 'learning_rate': 6.9226e-06, 'epoch': 0.76, 'throughput': 12203.07}

[INFO|2025-04-29 01:48:04] logging.py:157 >> {'loss': 0.8926, 'learning_rate': 6.4044e-06, 'epoch': 0.77, 'throughput': 12202.71}

[INFO|2025-04-29 01:50:55] logging.py:157 >> {'loss': 0.9022, 'learning_rate': 5.9035e-06, 'epoch': 0.77, 'throughput': 12203.40}

[INFO|2025-04-29 01:53:46] logging.py:157 >> {'loss': 0.9081, 'learning_rate': 5.4203e-06, 'epoch': 0.78, 'throughput': 12203.86}

[INFO|2025-04-29 01:56:38] logging.py:157 >> {'loss': 0.8964, 'learning_rate': 4.9554e-06, 'epoch': 0.79, 'throughput': 12203.74}

[INFO|2025-04-29 01:59:30] logging.py:157 >> {'loss': 0.8872, 'learning_rate': 4.5091e-06, 'epoch': 0.80, 'throughput': 12204.10}

[INFO|2025-04-29 02:02:21] logging.py:157 >> {'loss': 0.9052, 'learning_rate': 4.0818e-06, 'epoch': 0.81, 'throughput': 12204.57}

[INFO|2025-04-29 02:05:13] logging.py:157 >> {'loss': 0.8887, 'learning_rate': 3.6740e-06, 'epoch': 0.82, 'throughput': 12204.65}

[INFO|2025-04-29 02:08:04] logging.py:157 >> {'loss': 0.9228, 'learning_rate': 3.2861e-06, 'epoch': 0.83, 'throughput': 12205.11}

[INFO|2025-04-29 02:10:55] logging.py:157 >> {'loss': 0.9070, 'learning_rate': 2.9184e-06, 'epoch': 0.84, 'throughput': 12205.63}

[INFO|2025-04-29 02:13:46] logging.py:157 >> {'loss': 0.8912, 'learning_rate': 2.5712e-06, 'epoch': 0.85, 'throughput': 12206.01}

[INFO|2025-04-29 02:16:38] logging.py:157 >> {'loss': 0.9100, 'learning_rate': 2.2448e-06, 'epoch': 0.86, 'throughput': 12206.13}

[INFO|2025-04-29 02:19:30] logging.py:157 >> {'loss': 0.8817, 'learning_rate': 1.9397e-06, 'epoch': 0.87, 'throughput': 12206.42}

[INFO|2025-04-29 02:22:21] logging.py:157 >> {'loss': 0.8795, 'learning_rate': 1.6559e-06, 'epoch': 0.88, 'throughput': 12206.66}

[INFO|2025-04-29 02:25:13] logging.py:157 >> {'loss': 0.8945, 'learning_rate': 1.3939e-06, 'epoch': 0.89, 'throughput': 12206.32}

[INFO|2025-04-29 02:28:05] logging.py:157 >> {'loss': 0.8941, 'learning_rate': 1.1539e-06, 'epoch': 0.90, 'throughput': 12206.23}

[INFO|2025-04-29 02:30:57] logging.py:157 >> {'loss': 0.8771, 'learning_rate': 9.3603e-07, 'epoch': 0.91, 'throughput': 12206.25}

[INFO|2025-04-29 02:33:48] logging.py:157 >> {'loss': 0.8779, 'learning_rate': 7.4056e-07, 'epoch': 0.92, 'throughput': 12206.55}

[INFO|2025-04-29 02:36:40] logging.py:157 >> {'loss': 0.8978, 'learning_rate': 5.6765e-07, 'epoch': 0.93, 'throughput': 12207.07}

[INFO|2025-04-29 02:39:31] logging.py:157 >> {'loss': 0.8858, 'learning_rate': 4.1747e-07, 'epoch': 0.94, 'throughput': 12207.47}

[INFO|2025-04-29 02:42:22] logging.py:157 >> {'loss': 0.8983, 'learning_rate': 2.9016e-07, 'epoch': 0.95, 'throughput': 12207.74}

[INFO|2025-04-29 02:45:13] logging.py:157 >> {'loss': 0.9030, 'learning_rate': 1.8583e-07, 'epoch': 0.96, 'throughput': 12208.21}

[INFO|2025-04-29 02:48:05] logging.py:157 >> {'loss': 0.8660, 'learning_rate': 1.0459e-07, 'epoch': 0.97, 'throughput': 12208.23}

[INFO|2025-04-29 02:50:57] logging.py:157 >> {'loss': 0.9024, 'learning_rate': 4.6501e-08, 'epoch': 0.98, 'throughput': 12208.43}

[INFO|2025-04-29 02:53:48] logging.py:157 >> {'loss': 0.8841, 'learning_rate': 1.1628e-08, 'epoch': 0.99, 'throughput': 12208.72}

[INFO|2025-04-29 02:56:40] logging.py:157 >> {'loss': 0.8893, 'learning_rate': 0.0000e+00, 'epoch': 1.00, 'throughput': 12208.75}

[INFO|2025-04-29 02:56:40] trainer.py:3910 >> Saving model checkpoint to saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103

[INFO|2025-04-29 02:56:40] configuration_utils.py:420 >> Configuration saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103/config.json

[INFO|2025-04-29 02:56:40] configuration_utils.py:909 >> Configuration saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103/generation_config.json

[INFO|2025-04-29 02:57:06] modeling_utils.py:2996 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 4 checkpoint shards. You can find where each parameters has been saved in the index located at saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103/model.safetensors.index.json.

[INFO|2025-04-29 02:57:06] tokenization_utils_base.py:2491 >> tokenizer config file saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103/tokenizer_config.json

[INFO|2025-04-29 02:57:06] tokenization_utils_base.py:2500 >> Special tokens file saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/checkpoint-103/special_tokens_map.json

[INFO|2025-04-29 02:57:06] trainer.py:2643 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)



[INFO|2025-04-29 02:57:06] trainer.py:3910 >> Saving model checkpoint to saves/Llama-3.1-8B-Instruct/freeze/llama_nsx

[INFO|2025-04-29 02:57:06] configuration_utils.py:420 >> Configuration saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/config.json

[INFO|2025-04-29 02:57:06] configuration_utils.py:909 >> Configuration saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/generation_config.json

[INFO|2025-04-29 02:57:36] modeling_utils.py:2996 >> The model is bigger than the maximum size per checkpoint (5GB) and is going to be split in 4 checkpoint shards. You can find where each parameters has been saved in the index located at saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/model.safetensors.index.json.

[INFO|2025-04-29 02:57:36] tokenization_utils_base.py:2491 >> tokenizer config file saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/tokenizer_config.json

[INFO|2025-04-29 02:57:36] tokenization_utils_base.py:2500 >> Special tokens file saved in saves/Llama-3.1-8B-Instruct/freeze/llama_nsx/special_tokens_map.json

[WARNING|2025-04-29 02:57:36] logging.py:162 >> No metric eval_loss to plot.

[WARNING|2025-04-29 02:57:36] logging.py:162 >> No metric eval_accuracy to plot.

[INFO|2025-04-29 02:57:36] modelcard.py:449 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}