{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OZuBIVRs18lP"
   },
   "source": [
    "Inspired by: https://huggingface.co/blog/fine-tune-vit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rwb1Sdi0QASD"
   },
   "source": [
    "# Fine-Tuning Vision Transformers for Image Classification\n",
    "\n",
    "Just as transformers-based models have revolutionized NLP, we're now seeing an explosion of papers applying them to all sorts of other domains. One of the most revolutionary of these was the Vision Transformer (ViT), which was introduced in [June 2021](https://arxiv.org/abs/2010.11929) by a team of researchers at Google Brain.\n",
    "\n",
    "This paper explored how you can tokenize images, just as you would tokenize sentences, so that they can be passed to transformer models for training. Its quite a simple concept, really...\n",
    "\n",
    "1. Split an image into a grid of sub-image patches\n",
    "1. Embed each patch with a linear projection\n",
    "1. Each embedded patch becomes a token, and the resulting sequence of embedded patches is the sequence you pass to the model.\n",
    "\n",
    "![vit_figure.png](https://raw.githubusercontent.com/google-research/vision_transformer/main/vit_figure.png)\n",
    "\n",
    "\n",
    "It turns out that once you've done the above, you can pre-train and finetune transformers just as you're used to with NLP tasks. Pretty sweet 😎."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "! pip install datasets transformers evaluate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "dataset_name = \"jonathan-roberts1/Satellite-Images-of-Hurricane-Damage\"\n",
    "\n",
    "def get_ds():\n",
    "  ds = load_dataset(dataset_name)\n",
    "  ds = ds[\"train\"].train_test_split(test_size=0.5)\n",
    "  ds[\"train\"][\"label\"].count(1), ds[\"test\"][\"label\"].count(0)\n",
    "  ds_ = ds[\"test\"].train_test_split(test_size=0.5)\n",
    "  ds[\"validation\"] = ds_[\"train\"]\n",
    "  ds[\"test\"] = ds_[\"test\"]\n",
    "  return ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = get_ds()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image = ds['train'][400]['image']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ooXr_55XICXM"
   },
   "source": [
    "## Loading ViT Feature Extractor\n",
    "\n",
    "Now that we know what our images look like and have a better understanding of the problem we're trying to solve, let's see how we can prepare these images for our model.\n",
    "\n",
    "When ViT models are trained, specific transformations are applied to images being fed into them. Use the wrong transformations on your image and the model won't be able to understand what it's seeing! 🖼 ➡️ 🔢\n",
    "\n",
    "To make sure we apply the correct transformations, we will use a [`ViTFeatureExtractor`](https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=classlabel#datasets.ClassLabel.int2str) initialized with a configuration that was saved along with the pretrained model we plan to use. In our case, we'll be using the [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) model, so lets load its feature extractor from the 🤗 Hub."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 103,
     "referenced_widgets": [
      "57012c0224244c3080261b8d0ab34ce8",
      "2e7f50f2e01049978d4094f2353cdd0c",
      "0c5cb87527db42e8a992da9bb0976fab",
      "196ae5aee81345238928fc5a06f2faa8",
      "9758e18c65b44506aa8208504bc2cca6",
      "a2a454d1aa654002a52967f00343aa7c",
      "c73ebc46be0f49089a5cfb00e883fef8",
      "62695a863fb74564b636c1b891a24b8f",
      "4e0989c80f1f4494a60d3bb8f9569621",
      "8f62ab0b2f7442d581afe3909344c9b0",
      "d774a2342d3b438b99a00132e92c30a1"
     ]
    },
    "executionInfo": {
     "elapsed": 14251,
     "status": "ok",
     "timestamp": 1734165764141,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "Ct6zPRixIUoI",
    "outputId": "feb48e50-3c67-467e-81d9-77c3937a2af3"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "57012c0224244c3080261b8d0ab34ce8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "preprocessor_config.json:   0%|          | 0.00/160 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/dist-packages/transformers/models/vit/feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from transformers import ViTFeatureExtractor\n",
    "\n",
    "model_name_or_path = 'google/vit-base-patch16-224-in21k'\n",
    "feature_extractor = ViTFeatureExtractor.from_pretrained(model_name_or_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EbGIOc_FIbU7"
   },
   "source": [
    "If we print a feature extractor, we can see its configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 7,
     "status": "ok",
     "timestamp": 1734165764142,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "ea22sEWLIg4e",
    "outputId": "fb0fbea4-d61e-4f09-cea6-55d43ae7785c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ViTFeatureExtractor {\n",
       "  \"do_normalize\": true,\n",
       "  \"do_rescale\": true,\n",
       "  \"do_resize\": true,\n",
       "  \"image_mean\": [\n",
       "    0.5,\n",
       "    0.5,\n",
       "    0.5\n",
       "  ],\n",
       "  \"image_processor_type\": \"ViTFeatureExtractor\",\n",
       "  \"image_std\": [\n",
       "    0.5,\n",
       "    0.5,\n",
       "    0.5\n",
       "  ],\n",
       "  \"resample\": 2,\n",
       "  \"rescale_factor\": 0.00392156862745098,\n",
       "  \"size\": {\n",
       "    \"height\": 224,\n",
       "    \"width\": 224\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_extractor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DmhC6aSKIics"
   },
   "source": [
    "To process an image, simply pass it to the feature extractor's call function. This will return a dict containing `pixel values`, which is the numeric representation of your image that we'll pass to the model.\n",
    "\n",
    "We get a numpy array by default, but if we add the `return_tensors='pt'` argument, we'll get back `torch` tensors instead.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 251,
     "status": "ok",
     "timestamp": 1734165764389,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "3lne4VrXJRIe",
    "outputId": "4d1dc59e-8480-45cd-c011-7f126ef54d6d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'pixel_values': tensor([[[[-0.4118, -0.4039, -0.3882,  ..., -0.1137, -0.1137, -0.1137],\n",
       "          [-0.4353, -0.4275, -0.4118,  ..., -0.1294, -0.1373, -0.1373],\n",
       "          [-0.4745, -0.4667, -0.4431,  ..., -0.1608, -0.1686, -0.1686],\n",
       "          ...,\n",
       "          [-0.6078, -0.6000, -0.5765,  ..., -0.5294, -0.5373, -0.5451],\n",
       "          [-0.6000, -0.6000, -0.5843,  ..., -0.5216, -0.5294, -0.5373],\n",
       "          [-0.6000, -0.6000, -0.5922,  ..., -0.5216, -0.5294, -0.5373]],\n",
       "\n",
       "         [[-0.3255, -0.3176, -0.3020,  ..., -0.0824, -0.0824, -0.0824],\n",
       "          [-0.3490, -0.3412, -0.3255,  ..., -0.0980, -0.1059, -0.1059],\n",
       "          [-0.3882, -0.3804, -0.3569,  ..., -0.1294, -0.1373, -0.1373],\n",
       "          ...,\n",
       "          [-0.6000, -0.5922, -0.5686,  ..., -0.4118, -0.4196, -0.4275],\n",
       "          [-0.5922, -0.5922, -0.5765,  ..., -0.4039, -0.4118, -0.4196],\n",
       "          [-0.5922, -0.5922, -0.5843,  ..., -0.4039, -0.4118, -0.4196]],\n",
       "\n",
       "         [[-0.4510, -0.4431, -0.4275,  ..., -0.1922, -0.1922, -0.1922],\n",
       "          [-0.4745, -0.4667, -0.4510,  ..., -0.2078, -0.2157, -0.2157],\n",
       "          [-0.5137, -0.5059, -0.4824,  ..., -0.2392, -0.2471, -0.2471],\n",
       "          ...,\n",
       "          [-0.7412, -0.7333, -0.7098,  ..., -0.6392, -0.6471, -0.6549],\n",
       "          [-0.7333, -0.7333, -0.7176,  ..., -0.6314, -0.6392, -0.6471],\n",
       "          [-0.7333, -0.7333, -0.7255,  ..., -0.6314, -0.6392, -0.6471]]]])}"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_extractor(image, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ujbbcaIPJiAW"
   },
   "source": [
    "## Processing the Dataset\n",
    "\n",
    "Now that we know how to read in images and transform them into inputs, let's write a function that will put those two things together to process a single example from the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "executionInfo": {
     "elapsed": 6,
     "status": "ok",
     "timestamp": 1734165764389,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "0U48pAEuMLQh"
   },
   "outputs": [],
   "source": [
    "def process_example(example):\n",
    "    inputs = feature_extractor(example['image'], return_tensors='pt')\n",
    "    inputs['label'] = example['label']\n",
    "    return inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 6,
     "status": "ok",
     "timestamp": 1734165764389,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "WmIilnQ-MbhG",
    "outputId": "0555b530-3ef0-4563-db49-2f9c4aad474b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'pixel_values': tensor([[[[-0.3569, -0.3490, -0.3412,  ..., -0.5294, -0.5137, -0.5059],\n",
       "          [-0.3412, -0.3333, -0.3255,  ..., -0.5294, -0.5216, -0.5137],\n",
       "          [-0.3176, -0.3098, -0.3020,  ..., -0.5373, -0.5294, -0.5216],\n",
       "          ...,\n",
       "          [-0.5686, -0.5765, -0.5843,  ..., -0.4353, -0.4431, -0.4510],\n",
       "          [-0.5686, -0.5686, -0.5765,  ..., -0.3882, -0.3882, -0.3961],\n",
       "          [-0.5686, -0.5686, -0.5686,  ..., -0.3569, -0.3569, -0.3569]],\n",
       "\n",
       "         [[-0.3961, -0.3882, -0.3804,  ..., -0.4196, -0.4039, -0.3961],\n",
       "          [-0.3804, -0.3725, -0.3647,  ..., -0.4196, -0.4118, -0.4039],\n",
       "          [-0.3569, -0.3490, -0.3412,  ..., -0.4275, -0.4196, -0.4118],\n",
       "          ...,\n",
       "          [-0.3882, -0.3961, -0.4039,  ..., -0.3490, -0.3569, -0.3647],\n",
       "          [-0.3882, -0.3882, -0.3961,  ..., -0.3020, -0.3020, -0.3098],\n",
       "          [-0.3882, -0.3882, -0.3882,  ..., -0.2706, -0.2706, -0.2706]],\n",
       "\n",
       "         [[-0.5686, -0.5608, -0.5529,  ..., -0.6235, -0.6078, -0.6000],\n",
       "          [-0.5529, -0.5451, -0.5373,  ..., -0.6235, -0.6157, -0.6078],\n",
       "          [-0.5294, -0.5216, -0.5137,  ..., -0.6314, -0.6235, -0.6157],\n",
       "          ...,\n",
       "          [-0.6157, -0.6235, -0.6314,  ..., -0.4824, -0.4902, -0.4902],\n",
       "          [-0.6157, -0.6157, -0.6235,  ..., -0.4275, -0.4353, -0.4353],\n",
       "          [-0.6157, -0.6157, -0.6157,  ..., -0.3961, -0.3961, -0.3961]]]]), 'label': 0}"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "process_example(ds['train'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fusnj3EHMk5g"
   },
   "source": [
    "While we could call `ds.map` and apply this to every example at once, this can be very slow, especially if you use a larger dataset. Instead, we'll apply a ***transform*** to the dataset. Transforms are only applied to examples as you index them.\n",
    "\n",
    "First, though, we'll need to update our last function to accept a batch of data, as that's what `ds.with_transform` expects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "executionInfo": {
     "elapsed": 8,
     "status": "ok",
     "timestamp": 1734165773522,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "Z_sF61AoM3X1"
   },
   "outputs": [],
   "source": [
    "def transform(example_batch):\n",
    "    # Take a list of PIL images and turn them to pixel values\n",
    "    inputs = feature_extractor([x for x in example_batch['image']], return_tensors='pt')\n",
    "\n",
    "    # Don't forget to include the labels!\n",
    "    inputs['label'] = example_batch['label']\n",
    "    return inputs\n",
    "\n",
    "prepared_ds = ds.with_transform(transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p31_fIQ3N5ej"
   },
   "source": [
    "We can directly apply this to our dataset using `ds.with_transform(transform)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "executionInfo": {
     "elapsed": 7,
     "status": "ok",
     "timestamp": 1734165773522,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "VlCsAUJxOlZy"
   },
   "outputs": [],
   "source": [
    "prepared_ds = ds.with_transform(transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_Xng7C3pOq9Q"
   },
   "source": [
    "Now, whenever we get an example from the dataset, our transform will be\n",
    "applied in real time (on both samples and slices, as shown below)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 7,
     "status": "ok",
     "timestamp": 1734165773522,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "SZEwL06H9IQr",
    "outputId": "841f9c46-dabb-4d0b-de7a-e7b94ab21016"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'pixel_values': tensor([[[[-0.0275, -0.0118,  0.0118,  ...,  0.1216,  0.0980,  0.0824],\n",
       "          [-0.0196, -0.0039,  0.0196,  ...,  0.1137,  0.0980,  0.0902],\n",
       "          [-0.0039,  0.0118,  0.0431,  ...,  0.0980,  0.0980,  0.0980],\n",
       "          ...,\n",
       "          [-0.1765, -0.1059,  0.0196,  ...,  0.0275,  0.0118,  0.0039],\n",
       "          [-0.2000, -0.1686, -0.1059,  ...,  0.0275,  0.0353,  0.0353],\n",
       "          [-0.2157, -0.2078, -0.1843,  ...,  0.0275,  0.0510,  0.0588]],\n",
       "\n",
       "         [[-0.0431, -0.0275, -0.0039,  ...,  0.1137,  0.0902,  0.0745],\n",
       "          [-0.0353, -0.0196,  0.0039,  ...,  0.1059,  0.0902,  0.0824],\n",
       "          [-0.0196, -0.0039,  0.0275,  ...,  0.0902,  0.0902,  0.0902],\n",
       "          ...,\n",
       "          [-0.1843, -0.1137,  0.0118,  ...,  0.0588,  0.0431,  0.0353],\n",
       "          [-0.2078, -0.1765, -0.1137,  ...,  0.0588,  0.0667,  0.0667],\n",
       "          [-0.2235, -0.2157, -0.1922,  ...,  0.0588,  0.0824,  0.0902]],\n",
       "\n",
       "         [[-0.2235, -0.2078, -0.1843,  ..., -0.1294, -0.1529, -0.1686],\n",
       "          [-0.2157, -0.2000, -0.1765,  ..., -0.1373, -0.1529, -0.1608],\n",
       "          [-0.2000, -0.1843, -0.1529,  ..., -0.1451, -0.1529, -0.1529],\n",
       "          ...,\n",
       "          [-0.3255, -0.2549, -0.1294,  ..., -0.0510, -0.0745, -0.0745],\n",
       "          [-0.3490, -0.3176, -0.2549,  ..., -0.0510, -0.0431, -0.0431],\n",
       "          [-0.3647, -0.3569, -0.3333,  ..., -0.0510, -0.0275, -0.0196]]],\n",
       "\n",
       "\n",
       "        [[[-0.4431, -0.4275, -0.4039,  ...,  0.1373,  0.1843,  0.2157],\n",
       "          [-0.4353, -0.4275, -0.4039,  ...,  0.1216,  0.1686,  0.2000],\n",
       "          [-0.4275, -0.4196, -0.3961,  ...,  0.0980,  0.1451,  0.1765],\n",
       "          ...,\n",
       "          [-0.5529, -0.5529, -0.5451,  ..., -0.6078, -0.5843, -0.5765],\n",
       "          [-0.5843, -0.5843, -0.5765,  ..., -0.6000, -0.5686, -0.5529],\n",
       "          [-0.6078, -0.6078, -0.6000,  ..., -0.6000, -0.5608, -0.5373]],\n",
       "\n",
       "         [[-0.3647, -0.3490, -0.3255,  ...,  0.0980,  0.1373,  0.1686],\n",
       "          [-0.3569, -0.3490, -0.3255,  ...,  0.0824,  0.1216,  0.1529],\n",
       "          [-0.3490, -0.3412, -0.3176,  ...,  0.0588,  0.0980,  0.1294],\n",
       "          ...,\n",
       "          [-0.4667, -0.4667, -0.4588,  ..., -0.5373, -0.5137, -0.5059],\n",
       "          [-0.4980, -0.4980, -0.4902,  ..., -0.5294, -0.4980, -0.4824],\n",
       "          [-0.5216, -0.5216, -0.5137,  ..., -0.5294, -0.4902, -0.4667]],\n",
       "\n",
       "         [[-0.6314, -0.6157, -0.5922,  ..., -0.0745, -0.0353, -0.0039],\n",
       "          [-0.6235, -0.6157, -0.5922,  ..., -0.0902, -0.0510, -0.0196],\n",
       "          [-0.6157, -0.6078, -0.5843,  ..., -0.1137, -0.0745, -0.0431],\n",
       "          ...,\n",
       "          [-0.7333, -0.7333, -0.7255,  ..., -0.6863, -0.6627, -0.6549],\n",
       "          [-0.7647, -0.7647, -0.7569,  ..., -0.6784, -0.6471, -0.6314],\n",
       "          [-0.7882, -0.7882, -0.7804,  ..., -0.6784, -0.6392, -0.6157]]]]), 'label': [1, 1]}"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prepared_ds['train'][0:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4ngOi7XiTFpB"
   },
   "source": [
    "# Training and Evaluation\n",
    "\n",
    "The data is processed and we are ready to start setting up the training pipeline. We will make use of 🤗's Trainer, but that'll require us to do a few things first:\n",
    "\n",
    "- Define a collate function.\n",
    "\n",
    "- Define an evaluation metric. During training, the model should be evaluated on its prediction accuracy. We should define a compute_metrics function accordingly.\n",
    "\n",
    "- Load a pretrained checkpoint. We need to load a pretrained checkpoint and configure it correctly for training.\n",
    "\n",
    "- Define the training configuration.\n",
    "\n",
    "After having fine-tuned the model, we will correctly evaluate it on the evaluation data and verify that it has indeed learned to correctly classify our images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 17,
     "referenced_widgets": [
      "80c2e52a73ba4d229efa479a04004c96",
      "cf3b95476d37406280d277739dc9d478",
      "2c1c56f0e3154c61b0da3884923dc13d",
      "181a76ecbad14b25a66d4a4a593271c2",
      "fd1b883f9a4b4947896d43470feecee6",
      "4bc343824e404ac0b33d33da97777abf",
      "c47eaf6d663f41e98de253b70454f11f",
      "837609239b4143979482857cc43710d8",
      "7ed6568e7a7b4cf1b7b895ff485a75ac",
      "282a26f0ae3b4c2e9f5bb369933e9b4a",
      "0a8f2fb303744d8c8253050b99b5f885",
      "20ffdb1a2d8644e6aa817dcd29375a11",
      "b475a8691ee54bc1b59f0aa441dd1db5",
      "5ea9aa98624e4ccfa7f7582ff666fcf5",
      "5455b872cbad468c936cf712d7734f0d",
      "a731040efce144308aae0bfd7cd4ad24",
      "f5a35dba3ff340fdb0a12b2d5741d7ad",
      "e4b82a180f7b4387a75cdbdedea814b7",
      "2485c0cb825340de8e12cb408cdda7c4",
      "e5135f975329493594fa5a406636a632"
     ]
    },
    "executionInfo": {
     "elapsed": 219,
     "status": "ok",
     "timestamp": 1734165773735,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "omHT-thePyhn",
    "outputId": "b86d4f65-dbd7-4e94-afdb-648ec0409a0f"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "80c2e52a73ba4d229efa479a04004c96",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "\n",
    "# log into huggingface to upload the model to your account\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PRjeQr_XXI0y"
   },
   "source": [
    "### Define our data collator\n",
    "\n",
    "Batches are coming in as lists of dicts, so we just unpack + stack those into batch tensors.\n",
    "\n",
    "We return a batch `dict` from our `collate_fn` so we can simply `**unpack` the inputs to our model later. ✨"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "executionInfo": {
     "elapsed": 315,
     "status": "ok",
     "timestamp": 1734165942005,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "aWOAPU__XLCX"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "\n",
    "def collate_fn(batch):\n",
    "    return {\n",
    "        'pixel_values': torch.stack([x['pixel_values'] for x in batch]),\n",
    "        'labels': torch.tensor([x['label'] for x in batch])\n",
    "    }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zSzB2DEzElnx"
   },
   "source": [
    "### Define an evaluation metric\n",
    "\n",
    "Here, we load the [accuracy](https://huggingface.co/metrics/accuracy) metric from `datasets`, and then write a function that takes in a model prediction + computes the accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "executionInfo": {
     "elapsed": 703,
     "status": "ok",
     "timestamp": 1734165942705,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "9mT-g1j2U5xd"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import evaluate\n",
    "\n",
    "metric = evaluate.load(\"accuracy\")\n",
    "def compute_metrics(p):\n",
    "    return metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8GBNg3LZVDTE"
   },
   "source": [
    "Now we can load our pretrained model. We'll add `num_labels` on init to make sure the model creates a classification head with the right number of units. We'll also include the `id2label` and `label2id` mappings so we have human readable labels in the 🤗 hub widget if we choose to `push_to_hub`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 495,
     "status": "ok",
     "timestamp": 1734166858427,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "O1ikGEXYVDPv",
    "outputId": "9440871d-01ac-496a-bb48-3e50e644990a"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of ViTForImageClassification were not initialized from the model checkpoint at google/vit-base-patch16-224-in21k and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "from transformers import ViTForImageClassification\n",
    "\n",
    "labels = ds['train'].features['label'].names\n",
    "\n",
    "model = ViTForImageClassification.from_pretrained(\n",
    "    model_name_or_path,\n",
    "    num_labels=len(labels),\n",
    "    id2label={str(i): c for i, c in enumerate(labels)},\n",
    "    label2id={c: str(i) for i, c in enumerate(labels)}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "teXliS6h3LY-"
   },
   "outputs": [],
   "source": [
    "model = ViTForImageClassification.from_pretrained(\n",
    "    model_name_or_path,\n",
    "    num_labels=len(labels),\n",
    "    id2label={str(i): c for i, c in enumerate(labels)},\n",
    "    label2id={c: str(i) for i, c in enumerate(labels)}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1146,
     "status": "ok",
     "timestamp": 1734166897426,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "0ZfYWtNvZjoO",
    "outputId": "74768da9-d43c-41a1-b5e7-07d0f6e13eaf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "undamaged buildings\n"
     ]
    }
   ],
   "source": [
    "inputs = feature_extractor(image, return_tensors=\"pt\")\n",
    "\n",
    "with torch.no_grad():\n",
    "    logits = model(**inputs).logits\n",
    "\n",
    "# model predicts one of the 1000 ImageNet classes\n",
    "predicted_label = logits.argmax(-1).item()\n",
    "print(model.config.id2label[str(predicted_label)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BeWeU33lVDNR"
   },
   "source": [
    "We're almost ready to train! The last thing we'll do before that is set up the training configuration by defining [`TrainingArguments`](https://huggingface.co/docs/transformers/v4.16.2/en/main_classes/trainer#transformers.TrainingArguments).\n",
    "\n",
    "Most of these are pretty self-explanatory, but one that is quite important here is `remove_unused_columns=False`. This one will drop any features not used by the model's call function. By default it's `True` because usually its ideal to drop unused feature columns, as it makes it easier to unpack inputs into the model's call function. But, in our case, we need the unused features ('image' in particular) in order to create 'pixel_values'.\n",
    "\n",
    "What I'm trying to say is that you'll have a bad time if you forget to set `remove_unused_columns=False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1734165943982,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "t6g_73VHVDKK",
    "outputId": "a1173728-28fe-453c-d7a9-342ecb24487e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/dist-packages/transformers/training_args.py:1568: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from transformers import TrainingArguments\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "  output_dir=\"hurricane_model\",\n",
    "  per_device_train_batch_size=16,\n",
    "  evaluation_strategy=\"steps\",\n",
    "  num_train_epochs=4,\n",
    "  fp16=True,\n",
    "  save_steps=100,\n",
    "  eval_steps=100,\n",
    "  logging_steps=10,\n",
    "  learning_rate=2e-4,\n",
    "  save_total_limit=2,\n",
    "  remove_unused_columns=False,\n",
    "  push_to_hub=True,\n",
    "  report_to='tensorboard',\n",
    "  load_best_model_at_end=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KMJE-CkSWTMG"
   },
   "source": [
    "Now, all instances can be passed to Trainer and we are ready to start training!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 334,
     "status": "ok",
     "timestamp": 1734165944313,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "mwaYQIFMVDHW",
    "outputId": "d6abcc12-c4b0-4c81-d7eb-915bc06b6c7b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-88-966ff6ab886d>:3: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.\n",
      "  trainer = Trainer(\n"
     ]
    }
   ],
   "source": [
    "from transformers import Trainer\n",
    "\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    data_collator=collate_fn,\n",
    "    compute_metrics=compute_metrics,\n",
    "    train_dataset=prepared_ds[\"train\"],\n",
    "    eval_dataset=prepared_ds[\"validation\"],\n",
    "    processing_class=feature_extractor,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 139
    },
    "executionInfo": {
     "elapsed": 13285,
     "status": "ok",
     "timestamp": 1734165957596,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "16ph-T_-O0K9",
    "outputId": "7b0cdb59-22dc-48ff-e909-3692e3babd05"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='313' max='313' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [313/313 00:12]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.6793570518493652,\n",
       " 'eval_model_preparation_time': 0.0051,\n",
       " 'eval_accuracy': 0.5884,\n",
       " 'eval_runtime': 13.2034,\n",
       " 'eval_samples_per_second': 189.346,\n",
       " 'eval_steps_per_second': 23.706}"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics = trainer.evaluate(prepared_ds['validation'])\n",
    "metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 586
    },
    "executionInfo": {
     "elapsed": 553681,
     "status": "ok",
     "timestamp": 1734166511274,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "3rHP8IEEVDBE",
    "outputId": "00e319ce-ce78-4945-c5dd-63793d9afcc4"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1252' max='1252' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1252/1252 08:52, Epoch 4/4]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Model Preparation Time</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.111800</td>\n",
       "      <td>0.148582</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.947600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>0.111200</td>\n",
       "      <td>0.070119</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.975200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>300</td>\n",
       "      <td>0.069400</td>\n",
       "      <td>0.060849</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.980800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400</td>\n",
       "      <td>0.004800</td>\n",
       "      <td>0.091668</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.974400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>500</td>\n",
       "      <td>0.036000</td>\n",
       "      <td>0.055198</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.983600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>600</td>\n",
       "      <td>0.059400</td>\n",
       "      <td>0.054691</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.980800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>700</td>\n",
       "      <td>0.011500</td>\n",
       "      <td>0.062730</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.984400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>0.001600</td>\n",
       "      <td>0.029573</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.993600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>900</td>\n",
       "      <td>0.004000</td>\n",
       "      <td>0.032514</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.991600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>0.000900</td>\n",
       "      <td>0.022371</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.994800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1100</td>\n",
       "      <td>0.000800</td>\n",
       "      <td>0.027039</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.993600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1200</td>\n",
       "      <td>0.000800</td>\n",
       "      <td>0.025595</td>\n",
       "      <td>0.005100</td>\n",
       "      <td>0.994000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='626' max='313' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [313/313 00:46]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** train metrics *****\n",
      "  epoch                    =          4.0\n",
      "  total_flos               = 1443400785GF\n",
      "  train_loss               =       0.0559\n",
      "  train_runtime            =   0:08:53.40\n",
      "  train_samples_per_second =       37.495\n",
      "  train_steps_per_second   =        2.347\n"
     ]
    }
   ],
   "source": [
    "train_results = trainer.train()\n",
    "trainer.save_model()\n",
    "trainer.log_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_state()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 173
    },
    "executionInfo": {
     "elapsed": 12599,
     "status": "ok",
     "timestamp": 1734166523862,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "7fFsGHbLhxYK",
    "outputId": "30de7f28-a932-4db8-81e1-95a8ca2cdc11"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='313' max='313' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [313/313 00:12]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** eval metrics *****\n",
      "  epoch                       =        4.0\n",
      "  eval_accuracy               =     0.9948\n",
      "  eval_loss                   =     0.0224\n",
      "  eval_model_preparation_time =     0.0051\n",
      "  eval_runtime                = 0:00:12.58\n",
      "  eval_samples_per_second     =    198.586\n",
      "  eval_steps_per_second       =     24.863\n"
     ]
    }
   ],
   "source": [
    "metrics = trainer.evaluate(prepared_ds['validation'])\n",
    "trainer.log_metrics(\"eval\", metrics)\n",
    "trainer.save_metrics(\"eval\", metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66,
     "referenced_widgets": [
      "cb04c67c4d50461dbf625a439fd8283f",
      "548e021a559143b9b3a41131993fe698",
      "fb0214369c114b1fae488079132f46be",
      "c8f825a4391447978e207ef8f76fe2f3",
      "59458860f54242348a3347de31706002",
      "dd95023402a24d21b6b27e5b51fbdeec",
      "d0c0c766e076439e8139223b64a40b2f",
      "9240d1738d084608900222e34e3f3575",
      "2e412985ada445cbb779f1fe387504c5",
      "6a873aff8d5e4884a9429549324a6385",
      "fff2f2759be54bc78741f6ab8134bea3"
     ]
    },
    "executionInfo": {
     "elapsed": 4455,
     "status": "ok",
     "timestamp": 1734166528309,
     "user": {
      "displayName": "Till Wenke",
      "userId": "10971785981473027459"
     },
     "user_tz": -60
    },
    "id": "dePRjHqsZM6M",
    "outputId": "eaae1569-5a8a-4bdd-c04b-a8d8caa3552d"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb04c67c4d50461dbf625a439fd8283f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "events.out.tfevents.1734166523.592abeefaa6c.3097.1:   0%|          | 0.00/477 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pushed to hub\n"
     ]
    }
   ],
   "source": [
    "kwargs = {\n",
    "    \"finetuned_from\": model.config._name_or_path,\n",
    "    \"tasks\": \"image-classification\",\n",
    "    \"dataset\": dataset_name,\n",
    "    \"tags\": ['image-classification'],\n",
    "}\n",
    "\n",
    "if training_args.push_to_hub:\n",
    "    trainer.push_to_hub('🍻 cheers', **kwargs)\n",
    "    print(\"pushed to hub\")\n",
    "else:\n",
    "    trainer.create_model_card(**kwargs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K5BUyoAn1clH"
   },
   "source": [
    "# Done :)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": [
    {
     "file_id": "https://github.com/nateraw/huggingface-hub-examples/blob/main/vit_image_classification_explained.ipynb",
     "timestamp": 1734108695560
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0a8f2fb303744d8c8253050b99b5f885": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0c5cb87527db42e8a992da9bb0976fab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_62695a863fb74564b636c1b891a24b8f",
      "max": 160,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4e0989c80f1f4494a60d3bb8f9569621",
      "value": 160
     }
    },
    "0f461135c8784481a98b22a3fe25986c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "181a76ecbad14b25a66d4a4a593271c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "CheckboxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "CheckboxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "CheckboxView",
      "description": "Add token as git credential?",
      "description_tooltip": null,
      "disabled": false,
      "indent": true,
      "layout": "IPY_MODEL_20ffdb1a2d8644e6aa817dcd29375a11",
      "style": "IPY_MODEL_b475a8691ee54bc1b59f0aa441dd1db5",
      "value": false
     }
    },
    "196ae5aee81345238928fc5a06f2faa8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8f62ab0b2f7442d581afe3909344c9b0",
      "placeholder": "​",
      "style": "IPY_MODEL_d774a2342d3b438b99a00132e92c30a1",
      "value": " 160/160 [00:00&lt;00:00, 5.17kB/s]"
     }
    },
    "1a3549ee41634d2ca4fbda9d879d13d2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_385b67f7f17a4c8b85ae9df52506e630",
       "IPY_MODEL_91c48fb5a2f748059137f0abe3e6caa1",
       "IPY_MODEL_7e682a1f7e2247a6ae2241f7950b51ee"
      ],
      "layout": "IPY_MODEL_e23aefea5d5542fc8278831b58f673f1"
     }
    },
    "1ad39a49abf14f2c8fe30c781725a61b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "20ffdb1a2d8644e6aa817dcd29375a11": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2485c0cb825340de8e12cb408cdda7c4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "282a26f0ae3b4c2e9f5bb369933e9b4a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c1c56f0e3154c61b0da3884923dc13d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "PasswordModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "PasswordModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "PasswordView",
      "continuous_update": true,
      "description": "Token:",
      "description_tooltip": null,
      "disabled": false,
      "layout": "IPY_MODEL_282a26f0ae3b4c2e9f5bb369933e9b4a",
      "placeholder": "​",
      "style": "IPY_MODEL_0a8f2fb303744d8c8253050b99b5f885",
      "value": ""
     }
    },
    "2e412985ada445cbb779f1fe387504c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2e7f50f2e01049978d4094f2353cdd0c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a2a454d1aa654002a52967f00343aa7c",
      "placeholder": "​",
      "style": "IPY_MODEL_c73ebc46be0f49089a5cfb00e883fef8",
      "value": "preprocessor_config.json: 100%"
     }
    },
    "3333fff5d71e49c5baae7af931bd07f2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "385b67f7f17a4c8b85ae9df52506e630": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8c355d921bd0429b9768d78415490d33",
      "placeholder": "​",
      "style": "IPY_MODEL_e2ca441cda7e4073b37f4b6367068a28",
      "value": "Filter: 100%"
     }
    },
    "4bc343824e404ac0b33d33da97777abf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a731040efce144308aae0bfd7cd4ad24",
      "placeholder": "​",
      "style": "IPY_MODEL_f5a35dba3ff340fdb0a12b2d5741d7ad",
      "value": "\n<b>Pro Tip:</b> If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks. </center>"
     }
    },
    "4e0989c80f1f4494a60d3bb8f9569621": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "507cd160589147b692433f410b18bf36": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dfab16bd3067421dad0855b527692b45",
      "placeholder": "​",
      "style": "IPY_MODEL_ac824fa996834adf9de197db65356416",
      "value": " 5000/5000 [00:01&lt;00:00, 2967.16 examples/s]"
     }
    },
    "5455b872cbad468c936cf712d7734f0d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ButtonStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ButtonStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "button_color": null,
      "font_weight": ""
     }
    },
    "548e021a559143b9b3a41131993fe698": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dd95023402a24d21b6b27e5b51fbdeec",
      "placeholder": "​",
      "style": "IPY_MODEL_d0c0c766e076439e8139223b64a40b2f",
      "value": "events.out.tfevents.1734166523.592abeefaa6c.3097.1: 100%"
     }
    },
    "57012c0224244c3080261b8d0ab34ce8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2e7f50f2e01049978d4094f2353cdd0c",
       "IPY_MODEL_0c5cb87527db42e8a992da9bb0976fab",
       "IPY_MODEL_196ae5aee81345238928fc5a06f2faa8"
      ],
      "layout": "IPY_MODEL_9758e18c65b44506aa8208504bc2cca6"
     }
    },
    "59458860f54242348a3347de31706002": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5ea9aa98624e4ccfa7f7582ff666fcf5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "62695a863fb74564b636c1b891a24b8f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a873aff8d5e4884a9429549324a6385": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e682a1f7e2247a6ae2241f7950b51ee": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3333fff5d71e49c5baae7af931bd07f2",
      "placeholder": "​",
      "style": "IPY_MODEL_86482cb0bba244c89a5da218a6eb7281",
      "value": " 5000/5000 [00:04&lt;00:00, 1038.02 examples/s]"
     }
    },
    "7ed6568e7a7b4cf1b7b895ff485a75ac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "80c2e52a73ba4d229efa479a04004c96": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "VBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "VBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "VBoxView",
      "box_style": "",
      "children": [],
      "layout": "IPY_MODEL_c47eaf6d663f41e98de253b70454f11f"
     }
    },
    "837609239b4143979482857cc43710d8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "86482cb0bba244c89a5da218a6eb7281": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "87dcfa8595f240c38db8fe0f57e255fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "8c355d921bd0429b9768d78415490d33": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d3f0fe34a564788b2bbe2e6e4def77f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8f62ab0b2f7442d581afe3909344c9b0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91c48fb5a2f748059137f0abe3e6caa1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8d3f0fe34a564788b2bbe2e6e4def77f",
      "max": 5000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1ad39a49abf14f2c8fe30c781725a61b",
      "value": 5000
     }
    },
    "9240d1738d084608900222e34e3f3575": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9758e18c65b44506aa8208504bc2cca6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a122a5328304452e909478ea07c851d2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a2a454d1aa654002a52967f00343aa7c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a731040efce144308aae0bfd7cd4ad24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ac824fa996834adf9de197db65356416": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ae3ce1bae7f84edea0a27b833b1e2d1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c438bd23e6f54b3baa8c31969f1f8564",
       "IPY_MODEL_df9adcf07dae4718b86c8d0a1dedcc8f",
       "IPY_MODEL_507cd160589147b692433f410b18bf36"
      ],
      "layout": "IPY_MODEL_e5aa1f64bb2b4eadab34dd23996fafe7"
     }
    },
    "b475a8691ee54bc1b59f0aa441dd1db5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c438bd23e6f54b3baa8c31969f1f8564": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a122a5328304452e909478ea07c851d2",
      "placeholder": "​",
      "style": "IPY_MODEL_0f461135c8784481a98b22a3fe25986c",
      "value": "Filter: 100%"
     }
    },
    "c47eaf6d663f41e98de253b70454f11f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": "center",
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": "flex",
      "flex": null,
      "flex_flow": "column",
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "50%"
     }
    },
    "c73ebc46be0f49089a5cfb00e883fef8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c8f825a4391447978e207ef8f76fe2f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6a873aff8d5e4884a9429549324a6385",
      "placeholder": "​",
      "style": "IPY_MODEL_fff2f2759be54bc78741f6ab8134bea3",
      "value": " 477/477 [00:00&lt;00:00, 1.29kB/s]"
     }
    },
    "cb04c67c4d50461dbf625a439fd8283f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_548e021a559143b9b3a41131993fe698",
       "IPY_MODEL_fb0214369c114b1fae488079132f46be",
       "IPY_MODEL_c8f825a4391447978e207ef8f76fe2f3"
      ],
      "layout": "IPY_MODEL_59458860f54242348a3347de31706002"
     }
    },
    "cf3b95476d37406280d277739dc9d478": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_837609239b4143979482857cc43710d8",
      "placeholder": "​",
      "style": "IPY_MODEL_7ed6568e7a7b4cf1b7b895ff485a75ac",
      "value": "<center> <img\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.svg\nalt='Hugging Face'> <br> Copy a token from <a\nhref=\"https://huggingface.co/settings/tokens\" target=\"_blank\">your Hugging Face\ntokens page</a> and paste it below. <br> Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file. </center>"
     }
    },
    "d0c0c766e076439e8139223b64a40b2f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d774a2342d3b438b99a00132e92c30a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dd95023402a24d21b6b27e5b51fbdeec": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "df9adcf07dae4718b86c8d0a1dedcc8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eafd32a8f7bf456d97e67e6546cb18bb",
      "max": 5000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_87dcfa8595f240c38db8fe0f57e255fa",
      "value": 5000
     }
    },
    "dfab16bd3067421dad0855b527692b45": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e23aefea5d5542fc8278831b58f673f1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e2ca441cda7e4073b37f4b6367068a28": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e4b82a180f7b4387a75cdbdedea814b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2485c0cb825340de8e12cb408cdda7c4",
      "placeholder": "​",
      "style": "IPY_MODEL_e5135f975329493594fa5a406636a632",
      "value": "Connecting..."
     }
    },
    "e5135f975329493594fa5a406636a632": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e5aa1f64bb2b4eadab34dd23996fafe7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eafd32a8f7bf456d97e67e6546cb18bb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f5a35dba3ff340fdb0a12b2d5741d7ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fb0214369c114b1fae488079132f46be": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9240d1738d084608900222e34e3f3575",
      "max": 477,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2e412985ada445cbb779f1fe387504c5",
      "value": 477
     }
    },
    "fd1b883f9a4b4947896d43470feecee6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ButtonModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ButtonModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ButtonView",
      "button_style": "",
      "description": "Login",
      "disabled": false,
      "icon": "",
      "layout": "IPY_MODEL_5ea9aa98624e4ccfa7f7582ff666fcf5",
      "style": "IPY_MODEL_5455b872cbad468c936cf712d7734f0d",
      "tooltip": ""
     }
    },
    "fff2f2759be54bc78741f6ab8134bea3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}