Update unsloth_SmolLM2 notebook to refine model training parameters and output formatting. Adjust dropout rates for better performance, enhance layer configurations, and increase training steps for improved stability. Revise output messages for clarity and update execution counts to reflect changes in training duration.

notebooks/unsloth_SmolLM2-135M-Instruct-bnb-4bit_xingyaoww_code-act.ipynb

@@ -207,7 +207,7 @@
           "name": "stderr",
           "output_type": "stream",
           "text": [
-            "Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.1.\n",
+            "Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.05.\n",
             "Unsloth will patch all other layers, except LoRA matrices, causing a performance hit.\n",
             "Unsloth 2025.4.3 patched 30 layers with 0 QKV layers, 0 O layers and 0 MLP layers.\n"
           ]
@@ -216,17 +216,16 @@
       "source": [
         "model = FastLanguageModel.get_peft_model(\n",
         "    model,\n",
-        "    r=64,  # Increased from 16\n",
+        "    r=64,  # Keep at 64 as it's working well\n",
         "    target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
-        "                      \"gate_proj\", \"up_proj\", \"down_proj\",],\n",
-        "    lora_alpha=128,  # 2*r\n",
-        "    lora_dropout=0.1,  # Added dropout\n",
-        "    bias = \"none\",    # Supports any, but = \"none\" is optimized\n",
-        "    # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n",
-        "    use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n",
+        "                     \"gate_proj\", \"up_proj\", \"down_proj\",],\n",
+        "    lora_alpha=128,  # Keep at 2*r as it's working well\n",
+        "    lora_dropout=0.05,  # Reduced from 0.1 to 0.05 for better balance\n",
+        "    bias = \"none\",    # Keep as is for optimization\n",
+        "    use_gradient_checkpointing = \"unsloth\",\n",
         "    random_state = 3407,\n",
-        "    use_rslora = False,  # We support rank stabilized LoRA\n",
-        "    loftq_config = None, # And LoftQ\n",
+        "    use_rslora = True,  # Enable rank stabilized LoRA for better stability\n",
+        "    loftq_config = None,\n",
         ")"
       ]
     },
@@ -650,22 +649,25 @@
         "    dataset_text_field = \"text\",\n",
         "    max_seq_length = max_seq_length,\n",
         "    dataset_num_proc = 2,\n",
-        "    packing = False, # Can make training 5x faster for short sequences.\n",
+        "    packing = False,\n",
         "    args = TrainingArguments(\n",
         "        per_device_train_batch_size = 2,\n",
-        "        gradient_accumulation_steps=8,  # Increased from 4\n",
-        "        warmup_steps=50,\n",
-        "        max_steps = 60,\n",
-        "        learning_rate=5e-5,  # Reduced from 2e-4\n",
+        "        gradient_accumulation_steps=16,  # Increased from 8 for better stability\n",
+        "        warmup_steps=100,  # Increased from 50 for more gradual warmup\n",
+        "        max_steps = 120,  # Increased from 60 to allow more training\n",
+        "        learning_rate=5e-5,  # Keep at 5e-5 as it's working well\n",
         "        fp16 = not is_bfloat16_supported(),\n",
         "        bf16 = is_bfloat16_supported(),\n",
         "        logging_steps = 1,\n",
         "        optim = \"adamw_8bit\",\n",
         "        weight_decay = 0.01,\n",
-        "        lr_scheduler_type=\"cosine\",  # Changed to cosine schedule\n",
+        "        lr_scheduler_type=\"cosine_with_restarts\",  # Changed to cosine with restarts for better optimization\n",
         "        seed = 3407,\n",
         "        output_dir = \"outputs\",\n",
-        "        gradient_checkpointing=True,  # Added gradient checkpointing\n",
+        "        gradient_checkpointing=True,\n",
+        "        save_strategy=\"steps\",  # Add checkpointing\n",
+        "        save_steps=30,  # Save every 30 steps\n",
+        "        save_total_limit=2,  # Keep last 2 checkpoints\n",
         "    ),\n",
         ")"
       ]
@@ -717,9 +719,9 @@
           "output_type": "stream",
           "text": [
             "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1\n",
-            "   \\\\   /|    Num examples = 7,139 | Num Epochs = 1 | Total steps = 60\n",
-            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 8\n",
-            "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 8 x 1) = 16\n",
+            "   \\\\   /|    Num examples = 7,139 | Num Epochs = 1 | Total steps = 120\n",
+            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 16\n",
+            "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 16 x 1) = 32\n",
             " \"-____-\"     Trainable parameters = 19,537,920/4,000,000,000 (0.49% trained)\n",
             "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.\n",
             "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mmjschock\u001b[0m to \u001b[32mhttps://api.wandb.ai\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
@@ -740,7 +742,7 @@
         {
           "data": {
             "text/html": [
-              "Run data is saved locally in <code>/home/mjschock/Projects/hf-agents-course/agents/Final_Assignment_Template/notebooks/wandb/run-20250430_141858-0ph0gbln</code>"
+              "Run data is saved locally in <code>/home/mjschock/Projects/hf-agents-course/agents/Final_Assignment_Template/notebooks/wandb/run-20250430_204355-efwwlovd</code>"
             ],
             "text/plain": [
               "<IPython.core.display.HTML object>"
@@ -752,7 +754,7 @@
         {
           "data": {
             "text/html": [
-              "Syncing run <strong><a href='https://wandb.ai/mjschock/huggingface/runs/0ph0gbln' target=\"_blank\">outputs</a></strong> to <a href='https://wandb.ai/mjschock/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+              "Syncing run <strong><a href='https://wandb.ai/mjschock/huggingface/runs/efwwlovd' target=\"_blank\">outputs</a></strong> to <a href='https://wandb.ai/mjschock/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
             ],
             "text/plain": [
               "<IPython.core.display.HTML object>"
@@ -776,7 +778,7 @@
         {
           "data": {
             "text/html": [
-              " View run at <a href='https://wandb.ai/mjschock/huggingface/runs/0ph0gbln' target=\"_blank\">https://wandb.ai/mjschock/huggingface/runs/0ph0gbln</a>"
+              " View run at <a href='https://wandb.ai/mjschock/huggingface/runs/efwwlovd' target=\"_blank\">https://wandb.ai/mjschock/huggingface/runs/efwwlovd</a>"
             ],
             "text/plain": [
               "<IPython.core.display.HTML object>"
@@ -798,8 +800,8 @@
               "\n",
               "    <div>\n",
               "      \n",
-              "      <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
-              "      [60/60 3:06:36, Epoch 0/1]\n",
+              "      <progress value='120' max='120' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+              "      [120/120 12:38:52, Epoch 0/1]\n",
               "    </div>\n",
               "    <table border=\"1\" class=\"dataframe\">\n",
               "  <thead>\n",
@@ -811,243 +813,483 @@
               "  <tbody>\n",
               "    <tr>\n",
               "      <td>1</td>\n",
-              "      <td>1.661700</td>\n",
+              "      <td>1.676700</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>2</td>\n",
-              "      <td>1.693100</td>\n",
+              "      <td>1.804600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>3</td>\n",
-              "      <td>1.747500</td>\n",
+              "      <td>1.718600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>4</td>\n",
-              "      <td>1.862600</td>\n",
+              "      <td>1.732300</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>5</td>\n",
-              "      <td>1.704500</td>\n",
+              "      <td>1.931200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>6</td>\n",
-              "      <td>1.727900</td>\n",
+              "      <td>1.780200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>7</td>\n",
-              "      <td>1.869500</td>\n",
+              "      <td>1.745600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>8</td>\n",
-              "      <td>1.570300</td>\n",
+              "      <td>1.753100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>9</td>\n",
-              "      <td>1.910300</td>\n",
+              "      <td>1.708400</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>10</td>\n",
-              "      <td>1.944300</td>\n",
+              "      <td>1.801600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>11</td>\n",
-              "      <td>1.830000</td>\n",
+              "      <td>1.658200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>12</td>\n",
-              "      <td>1.715700</td>\n",
+              "      <td>1.699800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>13</td>\n",
-              "      <td>1.647800</td>\n",
+              "      <td>1.612700</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>14</td>\n",
-              "      <td>1.842400</td>\n",
+              "      <td>1.674300</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>15</td>\n",
-              "      <td>1.741600</td>\n",
+              "      <td>1.728900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>16</td>\n",
-              "      <td>1.753800</td>\n",
+              "      <td>1.562500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>17</td>\n",
-              "      <td>1.703500</td>\n",
+              "      <td>1.623100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>18</td>\n",
-              "      <td>1.700800</td>\n",
+              "      <td>1.631200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>19</td>\n",
-              "      <td>1.955300</td>\n",
+              "      <td>1.491700</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>20</td>\n",
-              "      <td>1.645000</td>\n",
+              "      <td>1.740600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>21</td>\n",
-              "      <td>1.663200</td>\n",
+              "      <td>1.562900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>22</td>\n",
-              "      <td>1.637100</td>\n",
+              "      <td>1.419600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>23</td>\n",
-              "      <td>1.586000</td>\n",
+              "      <td>1.416500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>24</td>\n",
-              "      <td>1.813300</td>\n",
+              "      <td>1.392800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>25</td>\n",
-              "      <td>1.541500</td>\n",
+              "      <td>1.504100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>26</td>\n",
-              "      <td>1.658000</td>\n",
+              "      <td>1.401300</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>27</td>\n",
-              "      <td>1.666100</td>\n",
+              "      <td>1.289800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>28</td>\n",
-              "      <td>1.659300</td>\n",
+              "      <td>1.308900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>29</td>\n",
-              "      <td>1.639000</td>\n",
+              "      <td>1.227500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>30</td>\n",
-              "      <td>1.797200</td>\n",
+              "      <td>1.207500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>31</td>\n",
-              "      <td>1.498100</td>\n",
+              "      <td>1.213000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>32</td>\n",
-              "      <td>1.596800</td>\n",
+              "      <td>1.197700</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>33</td>\n",
-              "      <td>1.608000</td>\n",
+              "      <td>1.295900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>34</td>\n",
-              "      <td>1.608700</td>\n",
+              "      <td>1.233800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>35</td>\n",
-              "      <td>1.724600</td>\n",
+              "      <td>1.173900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>36</td>\n",
-              "      <td>1.498700</td>\n",
+              "      <td>0.930500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>37</td>\n",
-              "      <td>1.453600</td>\n",
+              "      <td>0.904000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>38</td>\n",
-              "      <td>1.493500</td>\n",
+              "      <td>1.203600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>39</td>\n",
-              "      <td>1.711300</td>\n",
+              "      <td>1.074700</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>40</td>\n",
-              "      <td>1.723000</td>\n",
+              "      <td>1.069200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>41</td>\n",
-              "      <td>1.440200</td>\n",
+              "      <td>1.098100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>42</td>\n",
-              "      <td>1.628000</td>\n",
+              "      <td>1.039800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>43</td>\n",
-              "      <td>1.435800</td>\n",
+              "      <td>0.931900</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>44</td>\n",
-              "      <td>1.348700</td>\n",
+              "      <td>0.959200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>45</td>\n",
-              "      <td>1.340700</td>\n",
+              "      <td>1.007500</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>46</td>\n",
-              "      <td>1.428200</td>\n",
+              "      <td>1.028000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>47</td>\n",
-              "      <td>1.257900</td>\n",
+              "      <td>1.118600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>48</td>\n",
-              "      <td>1.462600</td>\n",
+              "      <td>1.013400</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>49</td>\n",
-              "      <td>1.520300</td>\n",
+              "      <td>0.916100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>50</td>\n",
-              "      <td>1.403800</td>\n",
+              "      <td>0.968300</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>51</td>\n",
-              "      <td>1.472400</td>\n",
+              "      <td>0.885600</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>52</td>\n",
-              "      <td>1.245900</td>\n",
+              "      <td>0.954300</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>53</td>\n",
-              "      <td>1.113200</td>\n",
+              "      <td>0.945000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>54</td>\n",
-              "      <td>1.338300</td>\n",
+              "      <td>0.984000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>55</td>\n",
-              "      <td>1.375600</td>\n",
+              "      <td>0.887000</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>56</td>\n",
-              "      <td>1.158800</td>\n",
+              "      <td>0.871100</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>57</td>\n",
-              "      <td>1.208400</td>\n",
+              "      <td>0.909800</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>58</td>\n",
-              "      <td>1.160900</td>\n",
+              "      <td>0.801200</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>59</td>\n",
-              "      <td>1.203100</td>\n",
+              "      <td>0.842400</td>\n",
               "    </tr>\n",
               "    <tr>\n",
               "      <td>60</td>\n",
-              "      <td>1.167000</td>\n",
+              "      <td>0.827100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>61</td>\n",
+              "      <td>0.858800</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>62</td>\n",
+              "      <td>0.987800</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>63</td>\n",
+              "      <td>0.831200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>64</td>\n",
+              "      <td>0.786000</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>65</td>\n",
+              "      <td>0.711400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>66</td>\n",
+              "      <td>0.836100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>67</td>\n",
+              "      <td>0.862600</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>68</td>\n",
+              "      <td>0.917900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>69</td>\n",
+              "      <td>0.910800</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>70</td>\n",
+              "      <td>0.617600</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>71</td>\n",
+              "      <td>0.499200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>72</td>\n",
+              "      <td>0.924200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>73</td>\n",
+              "      <td>0.892500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>74</td>\n",
+              "      <td>0.748600</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>75</td>\n",
+              "      <td>1.002700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>76</td>\n",
+              "      <td>0.751900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>77</td>\n",
+              "      <td>0.854900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>78</td>\n",
+              "      <td>0.642900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>79</td>\n",
+              "      <td>0.670400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>80</td>\n",
+              "      <td>0.656400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>81</td>\n",
+              "      <td>0.820400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>82</td>\n",
+              "      <td>0.891800</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>83</td>\n",
+              "      <td>0.723100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>84</td>\n",
+              "      <td>0.812400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>85</td>\n",
+              "      <td>1.006500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>86</td>\n",
+              "      <td>0.568000</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>87</td>\n",
+              "      <td>0.704300</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>88</td>\n",
+              "      <td>0.623900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>89</td>\n",
+              "      <td>0.889100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>90</td>\n",
+              "      <td>0.559900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>91</td>\n",
+              "      <td>0.910700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>92</td>\n",
+              "      <td>0.800300</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>93</td>\n",
+              "      <td>0.830000</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>94</td>\n",
+              "      <td>0.669400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>95</td>\n",
+              "      <td>0.626700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>96</td>\n",
+              "      <td>0.817700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>97</td>\n",
+              "      <td>0.786500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>98</td>\n",
+              "      <td>0.687900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>99</td>\n",
+              "      <td>0.687300</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>100</td>\n",
+              "      <td>0.954200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>101</td>\n",
+              "      <td>0.717600</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>102</td>\n",
+              "      <td>0.629100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>103</td>\n",
+              "      <td>0.718600</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>104</td>\n",
+              "      <td>0.662000</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>105</td>\n",
+              "      <td>0.615200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>106</td>\n",
+              "      <td>0.675300</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>107</td>\n",
+              "      <td>0.859100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>108</td>\n",
+              "      <td>0.832700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>109</td>\n",
+              "      <td>0.777900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>110</td>\n",
+              "      <td>0.589500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>111</td>\n",
+              "      <td>0.730100</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>112</td>\n",
+              "      <td>0.715500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>113</td>\n",
+              "      <td>0.750200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>114</td>\n",
+              "      <td>0.675900</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>115</td>\n",
+              "      <td>0.635700</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>116</td>\n",
+              "      <td>0.650500</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>117</td>\n",
+              "      <td>0.696400</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>118</td>\n",
+              "      <td>0.871200</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>119</td>\n",
+              "      <td>0.737000</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <td>120</td>\n",
+              "      <td>0.576800</td>\n",
               "    </tr>\n",
               "  </tbody>\n",
               "</table><p>"
@@ -1080,8 +1322,8 @@
           "name": "stdout",
           "output_type": "stream",
           "text": [
-            "11430.7159 seconds used for training.\n",
-            "190.51 minutes used for training.\n",
+            "45963.0635 seconds used for training.\n",
+            "766.05 minutes used for training.\n",
             "Peak reserved memory = 2.342 GB.\n",
             "Peak reserved memory for training = 2.117 GB.\n",
             "Peak reserved memory % of max memory = 59.442 %.\n",
@@ -1135,7 +1377,7 @@
         {
           "data": {
             "text/plain": [
-              "['<|im_start|>user\\nContinue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,<|im_end|>\\n<|im_start|>assistant\\nThe Fibonacci sequence is a series of numbers where each number is the sum of the two preceding ones, starting from 1 and 1. To find the first few Fibonacci numbers, we can use a simple iterative approach.\\n\\nWe begin by assuming the first Fibonacci number is 0. Then, we']"
+              "['<|im_start|>user\\nContinue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,<|im_end|>\\n<|im_start|>assistant\\nThe Fibonacci sequence is a series of numbers where each number is the sum of the previous two numbers, starting from 0 and 1. To find the Fibonacci sequence, we can start with the first two terms, 1 and 1, and then add the last two terms, 1 and 2']"
             ]
           },
           "execution_count": 12,
@@ -1196,9 +1438,18 @@
             "<|im_start|>user\n",
             "Continue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,<|im_end|>\n",
             "<|im_start|>assistant\n",
-            "The Fibonacci sequence is a series of numbers where each number is the sum of the two preceding ones, starting from 1 and 1. To find the first few Fibonacci numbers, we can use a simple iterative approach.\n",
+            "To find the nth Fibonacci number, we can start with the first few terms of the Fibonacci sequence: 1, 1, 2, 3, 5, 8, 13, 21, 34, and so on.\n",
+            "\n",
+            "The Fibonacci sequence is defined recursively as:\n",
             "\n",
-            "We begin by assuming the first Fibonacci number is 0. Then, we multiply the current Fibonacci number by 1 and add it to the previous one. This results in a new Fibonacci number of 1 + 0 = 1. Next, we multiply the current Fibonacci number by 2 and add it to the previous one. This results in a new Fibonacci number of\n"
+            "F(0) = 0\n",
+            "F(1) = 1\n",
+            "F(2) = 1\n",
+            "F(3) = 1\n",
+            "F(4) = 2\n",
+            "F(5) = 3\n",
+            "F(6) = 5\n",
+            "F\n"
           ]
         }
       ],
@@ -1290,9 +1541,9 @@
             "<|im_start|>user\n",
             "What is a famous tall tower in Paris?<|im_end|>\n",
             "<|im_start|>assistant\n",
-            "The famous Tall Tower in Paris is the Arc de Triomphe, a monumental arch built in 1789 by Napoleon Bonaparte to commemorate the 18th birthday of his wife, Marie Antoinette. It is a symbol of the French monarchy and a symbol of the French Empire. The Arc de Triomphe is 140 meters tall and 150 meters wide, with a height of 150 meters. It is located in the Latin Quarter of Paris, a historic neighborhood that has been home to the French monarchy since the 18th century.\n",
+            "The famous tall tower in Paris is the Arc de Triomphe, a 13th-century French landmark that was built in 1882. It is a 130-meter-tall Gothic spire that serves as a symbol of Paris and is considered one of the most recognizable landmarks in the world. The tower is located in the heart of the city, and its height is considered one of the most impressive in the world.\n",
             "\n",
-            "The Arc de Triomphe\n"
+            "The Arc de Triomphe is a tribute to the French monarchy and is a symbol of the city's history and culture. It was built in 1882\n"
           ]
         }
       ],