End of training

README.md

@@ -1,12 +1,16 @@
 ---
 base_model: sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat
 tags:
+- alignment-handbook
+- trl
+- sft
+- generated_from_trainer
 - trl
 - sft
 - alignment-handbook
 - generated_from_trainer
 datasets:
-- generator
+- HuggingFaceH4/ultrachat_200k
 model-index:
 - name: sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat
   results: []
@@ -17,7 +21,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat
 
-This model is a fine-tuned version of [sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat](https://huggingface.co/sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat) on the generator dataset.
+This model is a fine-tuned version of [sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat](https://huggingface.co/sanchit-gandhi/distil-zephyr-1.5b-ssft-ultrachat) on the HuggingFaceH4/ultrachat_200k dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.1555
 

config.json

@@ -21,6 +21,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.40.1",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 32000
 }

wandb/debug-internal.log

@@ -107,3 +107,14 @@
 2024-04-26 14:33:54,805 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
 2024-04-26 14:33:55,947 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
 2024-04-26 14:33:58,809 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:00,550 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: stop_status
+2024-04-26 14:34:00,550 DEBUG   SenderThread:906599 [sender.py:send_request():406] send_request: stop_status
+2024-04-26 14:34:00,552 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: internal_messages
+2024-04-26 14:34:00,811 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:01,641 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:02,813 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:06,644 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:08,821 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:11,761 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:12,893 DEBUG   SystemMonitor:906599 [system_monitor.py:_start():172] Starting system metrics aggregation loop
+2024-04-26 14:34:12,907 DEBUG   SenderThread:906599 [sender.py:send():379] send: stats

wandb/run-20240426_143312-xista79n/files/output.log

@@ -50,3 +50,20 @@ Training completed. Do not forget to share your model on huggingface.co/models =
 [INFO|tokenization_utils_base.py:2488] 2024-04-26 14:33:57,470 >> tokenizer config file saved in ./tokenizer_config.json
 [INFO|tokenization_utils_base.py:2497] 2024-04-26 14:33:57,472 >> Special tokens file saved in ./special_tokens_map.json
 [INFO|modelcard.py:450] 2024-04-26 14:33:57,519 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'generator', 'type': 'generator', 'config': 'default', 'split': 'train', 'args': 'default'}}
+events.out.tfevents.1714141991.ip-26-0-165-24.906132.0: 100%|██████████| 5.06k/5.06k [00:00<00:00, 39.6kB/s]
+training_args.bin: 100%|██████████| 4.98k/4.98k [00:00<00:00, 37.2kB/s]| 0.00/5.06k [00:00<?, ?B/s]
+events.out.tfevents.1714142026.ip-26-0-165-24.906132.1: 100%|██████████| 359/359 [00:00<00:00, 2.35kB/s]
+Upload 3 LFS files: 100%|██████████| 3/3 [00:00<00:00,  6.66it/s]      | 0.00/359 [00:00<?, ?B/s]
+2024-04-26 14:34:06 - INFO - __main__ - Model saved to ./
+2024-04-26 14:34:06 - INFO - __main__ - Pushing to hub...
+[INFO|modelcard.py:450] 2024-04-26 14:34:06,743 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'HuggingFaceH4/ultrachat_200k', 'type': 'HuggingFaceH4/ultrachat_200k', 'config': 'default', 'split': 'train', 'args': 'default'}}
+[INFO|configuration_utils.py:471] 2024-04-26 14:34:06,747 >> Configuration saved in ./config.json
+[INFO|trainer.py:3305] 2024-04-26 14:34:06,748 >> Saving model checkpoint to ./
+[INFO|configuration_utils.py:471] 2024-04-26 14:34:06,750 >> Configuration saved in ./config.json
+[INFO|configuration_utils.py:697] 2024-04-26 14:34:06,752 >> Configuration saved in ./generation_config.json
+[INFO|modeling_utils.py:2590] 2024-04-26 14:34:12,138 >> Model weights saved in ./model.safetensors
+[INFO|tokenization_utils_base.py:2488] 2024-04-26 14:34:12,140 >> tokenizer config file saved in ./tokenizer_config.json
+[INFO|tokenization_utils_base.py:2497] 2024-04-26 14:34:12,142 >> Special tokens file saved in ./special_tokens_map.json
+[INFO|modelcard.py:450] 2024-04-26 14:34:12,190 >> Dropping the following result as it does not have all the necessary fields:

wandb/run-20240426_143312-xista79n/logs/debug-internal.log

@@ -107,3 +107,14 @@
 2024-04-26 14:33:54,805 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
 2024-04-26 14:33:55,947 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
 2024-04-26 14:33:58,809 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:00,550 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: stop_status
+2024-04-26 14:34:00,550 DEBUG   SenderThread:906599 [sender.py:send_request():406] send_request: stop_status
+2024-04-26 14:34:00,552 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: internal_messages
+2024-04-26 14:34:00,811 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:01,641 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:02,813 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:06,644 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:08,821 INFO    Thread-12 :906599 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat-200k/wandb/run-20240426_143312-xista79n/files/output.log
+2024-04-26 14:34:11,761 DEBUG   HandlerThread:906599 [handler.py:handle_request():146] handle_request: status_report
+2024-04-26 14:34:12,893 DEBUG   SystemMonitor:906599 [system_monitor.py:_start():172] Starting system metrics aggregation loop
+2024-04-26 14:34:12,907 DEBUG   SenderThread:906599 [sender.py:send():379] send: stats