Lune_(2).ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeaneigsi/Humain-/blob/main/Lune_(2).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IqM-T1RTzY6C"
      },
      "source": [
        "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://github.com/unslothai/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
        "  <a href=\"https://discord.gg/u54VK8m8tk\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
        "  <a href=\"https://ko-fi.com/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Kofi button.png\" width=\"145\"></a></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
        "</div>\n",
        "\n",
        "To install Unsloth on your own computer, follow the installation instructions on our Github page [here](https://github.com/unslothai/unsloth#installation-instructions---conda).\n",
        "\n",
        "You will learn how to do [data prep](#Data), how to [train](#Train), how to [run the model](#Inference), & [how to save it](#Save) (eg for Llama.cpp).\n",
        "\n",
        "**[NEW] Unsloth supports Qwen2 0.5b, 1.5b. 7b, and the largest 72b!**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2eSvM9zX_2d3"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "# Installs Unsloth, Xformers (Flash Attention) and all other packages!\n",
        "# !pip install \"unsloth @ git+https://github.com/unslothai/unsloth.git\"\n",
        "!pip install \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git@June-2024\"\n",
        "!pip install --no-deps xformers \"trl<0.9.0\" peft accelerate bitsandbytes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r2v_X2fA0Df5"
      },
      "source": [
        "* We support Llama, Mistral, Phi-3, Gemma, Yi, DeepSeek, Qwen, TinyLlama, Vicuna, Open Hermes etc\n",
        "* We support 16bit LoRA or 4bit QLoRA. Both 2x faster.\n",
        "* `max_seq_length` can be set to anything, since we do automatic RoPE Scaling via [kaiokendev's](https://kaiokendev.github.io/til) method.\n",
        "* With [PR 26037](https://github.com/huggingface/transformers/pull/26037), we support downloading 4bit models **4x faster**! [Our repo](https://huggingface.co/unsloth) has Llama, Mistral 4bit models.\n",
        "* [**NEW**] We make Phi-3 Medium / Mini **2x faster**! See our [Phi-3 Medium notebook](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing)\n",
        "* [**NEW**] We make Llama-3 8b, 70b **2x faster**! See our [Llama-3 8b notebook](https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 488,
          "referenced_widgets": [
            "90959101a5704914ab0124283cd6148f",
            "63c548abf6714edfbb238e1489d89271",
            "908dfb6d6cf74986ab7e3136bb50786e",
            "bbf9174351b047db96676d6149ab4e56",
            "51644fc5c4de4f508757cf747547d650",
            "a025d355cfad459793de1370917311f3",
            "6aeaa8ba15fc4553a78f971aaf5c9b6f",
            "6fef4f30afcf4f9ba8a18eb9f3851ba6",
            "01d204c64942434190bceaf709f5f931",
            "99b0415ead364e87b903709767727d1e",
            "1d19743955844705919a5a0d3fd697b3",
            "bbac5270c6a94ea7b303d6ae94094a8f",
            "3260b7232dec49e5bbb603e3d5eabe6e",
            "8d0d0f8d01a6421bbb868103200ff85a",
            "1c5206dfea7443379635ce0f462f5a7b",
            "e4c0da26192a47ec9afc61be919406e7",
            "a9bbc5894b3d424496c976e86b21e935",
            "a7e05c492b7c4ba4be12f5ec6a4c05d0",
            "e7e20df38ba34de69a86412b32c4a1dd",
            "b003af4f20364eb1a535d4f9c71f74b7",
            "c1fee2a44f2d40cbab06bf4232c357a8",
            "c9fba9eb5fac40a286a3c77a01cd95a1",
            "030f9e33f584417186dbec2771b4d1b3",
            "28b7406dd9d14401961dd85684398e9c",
            "5cf4435e5bf24817991572defb5405aa",
            "a4f17ec7c2bc45eb8f0da08f498685b0",
            "4031b7b5c97545dea5a5158b3d8b5372",
            "02ed31374bf64bf4be92252f0a1778dc",
            "7650642c89c24210807d8f47952b454b",
            "2a3c4acb149e49d59339fd1261c2a5e2",
            "c9c26af3974f4fd38b24da608278feeb",
            "1ca9fef0577a45948ae75f90be59a70e",
            "4808ffbbf36840c9a4b4a5f299fecb87",
            "50e3d52bb7d945dbb5c4613d839c0546",
            "075c339cc40d41139f5c3ddfe769b6b1",
            "87dd0e57fb724a6ca4fa5c1e280cf99b",
            "ad8a89357f94402ca570c9c8e63a7e52",
            "3d85bb349f784df797fe5bf127f48f6a",
            "28ac7bb29bf94dedb0f9c9ea034a961e",
            "fe10dcb7bd7f479784353c4cbc701878",
            "a32cb7bd6a124800b2c0756ac49e9098",
            "54f2803db94b412db0215ab460cf5796",
            "1ead678d6c8c4edf9f220a4c8ec5e638",
            "970be236ebb345e7a1e51f6b71c67402",
            "3894df9a3a9949d6bcb628d53ffbea6d",
            "ef81a04741274d23bb5f05d968b0adb0",
            "0ffeef7b0a2a4457b3b44e825f2bcf05",
            "6ed1604e47ca4dcdb465a5839bab83e8",
            "b951792c8d2e4083935f3b3c3ba46c7c",
            "d31b9cce122e445d850e082be42f6025",
            "de3846b1e677437586e96d02196a91d3",
            "03ed126b633c479483dfc3eb835b0405",
            "44d87d579cab4d51965ee47cdb48b171",
            "45ccd999b65a4b3683a3631811d225b7",
            "665f561628db42869cee4eb7fbbe23cb",
            "7276b9ac4012476c9526e647f1ecca0f",
            "9e13417fe7f34aeda74bbbf960015162",
            "798d13daf6034191a8b5ee0cc5120311",
            "c490f30af0814201bd6ac10cac782343",
            "503f98f7d9c34b0e949737d7562f6d0a",
            "4c52d7d1f44c493ebe49ddfbb2233f07",
            "5b563f417adf4eae907838ac617b61cb",
            "e1805e4085cc4e59a6d470990ddbf327",
            "695afe6ce1474018ab72d546bcecfa95",
            "cd916fdc68cb46b9944a2ed9b77c9092",
            "7c46d51adc9f4444983f6201320a0bf3",
            "6b55332f538a49b78423cfe7a848d330",
            "51dee8d0c6dd48aa875f4fc4eb07bc0f",
            "07511b292577461da25e13729a23590c",
            "8dd1f70feca14569844a11e5a5769aa9",
            "ffb6897d76114aa3ac91fd4ece67a8e7",
            "2617fc4a85184b1dbbf4033c5484ad64",
            "aa55a840ad0b42739f00148f0a6d03ca",
            "6b86ee15fc1b4a31b5d3cb3d5ffba0ce",
            "1445d964582a412ea62331322128a90c",
            "06b6c5a916294ba1923b6059dd8392ff",
            "45b959fa51904484bd9f1fb4dccf9a02",
            "5abeb01d19d047b0adc34aeacdd7e009",
            "69910647d9ff45318131af41fdad6949",
            "f6f1fa54e754444baffd11bcc360a24b",
            "6059958a5ead476cabda576c30d98a4e",
            "0d1f94b5f3ce4f15afd17a6313a1f420",
            "43d449028abc4b919448dc7413ec6590",
            "dd8ed98d098346d884ce815e04b204c5",
            "189035e02ce54bd395ea11991d6a487b",
            "f2b193b6a6ef4afb94b7b2f984aa6d70",
            "e0151151ff5145739f7e680c9c3bc0bc",
            "597d5cd5e96745988a5f86cb9479e57d",
            "0c975c8b2030416a8cd687175c40612d",
            "8c9a2f08a4df45efa2312ed0e9a4cfb1",
            "a7a7d8fcf7eb4df2bcb1b585cb145c42",
            "9c34686459a64ff2bce9140f2abe9f6c",
            "534c1b2ffc7843758adb4cb62e9c8131",
            "b232bbd78e4a49868051b0673ef76f60",
            "b06914a13bff410f80683416780e4e95",
            "e56e205549c54fce8f5ba1df6f2e00b4",
            "214bb87d7d25411abed64de2352216fa",
            "b9456b4e62744896903cbd0ba9478772",
            "04ce495c6fcb4b90abcb5b551775b42d",
            "ae2e57d1272b4e9a98e3a22b503978e3"
          ]
        },
        "id": "QmUBVEnvCDJv",
        "outputId": "3da078b9-86a0-47bb-cf98-e26dfe43229f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n",
            "==((====))==  Unsloth: Fast Qwen2 patching release 2024.5\n",
            "   \\\\   /|    GPU: NVIDIA L4. Max memory: 22.168 GB. Platform = Linux.\n",
            "O^O/ \\_/ \\    Pytorch: 2.1.0+cu121. CUDA = 8.9. CUDA Toolkit = 12.1.\n",
            "\\        /    Bfloat16 = TRUE. Xformers = 0.0.22.post7. FA = False.\n",
            " \"-____-\"     Free Apache license: http://github.com/unslothai/unsloth\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ae2e57d1272b4e9a98e3a22b503978e3",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
            "unsloth/Qwen2-7B does not have a padding token! Will use pad_token = <|PAD_TOKEN|>.\n"
          ]
        }
      ],
      "source": [
        "from unsloth import FastLanguageModel\n",
        "import torch\n",
        "max_seq_length = 4000 # Choose any! We auto support RoPE Scaling internally!\n",
        "dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\n",
        "load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.\n",
        "\n",
        "# 4bit pre quantized models we support for 4x faster downloading + no OOMs.\n",
        "fourbit_models = [\n",
        "    \"unsloth/mistral-7b-v0.3-bnb-4bit\",      # New Mistral v3 2x faster!\n",
        "    \"unsloth/llama-3-8b-bnb-4bit\",           # Llama-3 15 trillion tokens model 2x faster!\n",
        "    \"unsloth/llama-3-8b-Instruct-bnb-4bit\",\n",
        "    \"unsloth/llama-3-70b-bnb-4bit\",\n",
        "    \"unsloth/Phi-3-mini-4k-instruct\",        # Phi-3 2x faster!\n",
        "    \"unsloth/Phi-3-medium-4k-instruct\",\n",
        "    \"unsloth/Qwen2-0.5b-bnb-4bit\",           # Qwen2 2x faster!\n",
        "    \"unsloth/Qwen2-1.5b-bnb-4bit\",\n",
        "    \"unsloth/Qwen2-7b-bnb-4bit\",\n",
        "    \"unsloth/Qwen2-72b-bnb-4bit\",\n",
        "    \"unsloth/gemma-7b-bnb-4bit\",             # Gemma 2.2x faster!\n",
        "] # Try more models at https://huggingface.co/unsloth!\n",
        "\n",
        "model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "    model_name = \"unsloth/Qwen2-7B\", # Reminder we support ANY Hugging Face model!\n",
        "    max_seq_length = max_seq_length,\n",
        "    dtype = dtype,\n",
        "    load_in_4bit = load_in_4bit,\n",
        "    # token = \"hf_...\", # use one if using gated models like meta-llama/Llama-2-7b-hf\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SXd9bTZd1aaL"
      },
      "source": [
        "We now add LoRA adapters so we only need to update 1 to 10% of all parameters!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6bZsfBuZDeCL",
        "outputId": "c705cbf6-d8f4-4afa-c0bb-9ad039f05fd7"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Not an error, but Unsloth cannot patch Attention layers with our manual autograd engine since either LoRA adapters\n",
            "are not enabled or a bias term (like in Qwen) is used.\n",
            "Unsloth 2024.5 patched 28 layers with 0 QKV layers, 28 O layers and 28 MLP layers.\n"
          ]
        }
      ],
      "source": [
        "model = FastLanguageModel.get_peft_model(\n",
        "    model,\n",
        "    r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n",
        "    target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
        "                      \"gate_proj\", \"up_proj\", \"down_proj\",],\n",
        "    lora_alpha = 16,\n",
        "    lora_dropout = 0, # Supports any, but = 0 is optimized\n",
        "    bias = \"none\",    # Supports any, but = \"none\" is optimized\n",
        "    # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n",
        "    use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n",
        "    random_state = 3407,\n",
        "    use_rslora = False,  # We support rank stabilized LoRA\n",
        "    loftq_config = None, # And LoftQ\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vITh0KVJ10qX"
      },
      "source": [
        "<a name=\"Data\"></a>\n",
        "### Data Prep\n",
        "We now use the Alpaca dataset from [yahma](https://huggingface.co/datasets/yahma/alpaca-cleaned), which is a filtered version of 52K of the original [Alpaca dataset](https://crfm.stanford.edu/2023/03/13/alpaca.html). You can replace this code section with your own data prep.\n",
        "\n",
        "**[NOTE]** To train only on completions (ignoring the user's input) read TRL's docs [here](https://huggingface.co/docs/trl/sft_trainer#train-on-completions-only).\n",
        "\n",
        "**[NOTE]** Remember to add the **EOS_TOKEN** to the tokenized output!! Otherwise you'll get infinite generations!\n",
        "\n",
        "If you want to use the `llama-3` template for ShareGPT datasets, try our conversational [notebook](https://colab.research.google.com/drive/1XamvWYinY6FOSX9GLvnqSjjsNflxdhNc?usp=sharing).\n",
        "\n",
        "For text completions like novel writing, try this [notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LjY75GoYUCB8"
      },
      "outputs": [],
      "source": [
        "alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
        "\n",
        "### Instruction:\n",
        "{}\n",
        "\n",
        "### Input:\n",
        "{}\n",
        "\n",
        "### Response:\n",
        "{}\"\"\"\n",
        "\n",
        "EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN\n",
        "def formatting_prompts_func(examples):\n",
        "    instructions = examples[\"instruction\"]\n",
        "    inputs       = examples[\"input\"]\n",
        "    outputs      = examples[\"output\"]\n",
        "    texts = []\n",
        "    for instruction, input, output in zip(instructions, inputs, outputs):\n",
        "        # Must add EOS_TOKEN, otherwise your generation will go on forever!\n",
        "        text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n",
        "        texts.append(text)\n",
        "    return { \"text\" : texts, }\n",
        "pass\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cp0syoELTOk2"
      },
      "outputs": [],
      "source": [
        "from datasets import load_dataset\n",
        "dataset = load_dataset(\"PleIAs/Post-OCR-Correction\", \"english\",split = \"train\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FHLYkx6CPlf8",
        "outputId": "6e56e06c-abe7-449a-b1c7-88011c11824b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['index_id', 'id', 'date', 'edition', 'page', 'file_name', 'word_count', 'text', 'corrected_text', '__index_level_0__'],\n",
              "    num_rows: 31323\n",
              "})"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "usenH47SSeOJ"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "# Sélectionner aléatoirement 100 indices\n",
        "random_indices = random.sample(range(len(dataset)), 200)\n",
        "\n",
        "# Extraire les 100 lignes aléatoires\n",
        "sampled_dataset = dataset.select(random_indices)\n",
        "\n",
        "# Garder uniquement les colonnes 'text' et 'corrected_text'\n",
        "sampled_dataset = sampled_dataset.remove_columns([col for col in sampled_dataset.column_names if col not in ['text', 'corrected_text']])\n",
        "\n",
        "# Renommer les colonnes\n",
        "sampled_dataset = sampled_dataset.rename_column('text', 'input')\n",
        "sampled_dataset = sampled_dataset.rename_column('corrected_text', 'output')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NuQlFaJLgJ0H",
        "outputId": "8829d1a4-6abb-41e3-d3bc-22f8f7ac6a0b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[1862,\n",
              " 13739,\n",
              " 4348,\n",
              " 10843,\n",
              " 7042,\n",
              " 13213,\n",
              " 9986,\n",
              " 19242,\n",
              " 27740,\n",
              " 11671,\n",
              " 8007,\n",
              " 25532,\n",
              " 27747,\n",
              " 15592,\n",
              " 23190,\n",
              " 16870,\n",
              " 13340,\n",
              " 22035,\n",
              " 8574,\n",
              " 28018,\n",
              " 8941,\n",
              " 7306,\n",
              " 3559,\n",
              " 9944,\n",
              " 27872,\n",
              " 6309,\n",
              " 10418,\n",
              " 18774,\n",
              " 24376,\n",
              " 3680,\n",
              " 3879,\n",
              " 19387,\n",
              " 21110,\n",
              " 7288,\n",
              " 10923,\n",
              " 920,\n",
              " 3654,\n",
              " 9098,\n",
              " 30888,\n",
              " 27184,\n",
              " 18348,\n",
              " 1313,\n",
              " 797,\n",
              " 24086,\n",
              " 5306,\n",
              " 26292,\n",
              " 27513,\n",
              " 18504,\n",
              " 7764,\n",
              " 16018,\n",
              " 18148,\n",
              " 28112,\n",
              " 9644,\n",
              " 17514,\n",
              " 29528,\n",
              " 4479,\n",
              " 6998,\n",
              " 16304,\n",
              " 19411,\n",
              " 23824,\n",
              " 19039,\n",
              " 18580,\n",
              " 15284,\n",
              " 21982,\n",
              " 453,\n",
              " 12549,\n",
              " 15535,\n",
              " 8218,\n",
              " 21346,\n",
              " 24126,\n",
              " 11009,\n",
              " 2695,\n",
              " 7964,\n",
              " 13601,\n",
              " 3922,\n",
              " 30743,\n",
              " 28390,\n",
              " 27360,\n",
              " 30365,\n",
              " 180,\n",
              " 2702,\n",
              " 29056,\n",
              " 21641,\n",
              " 1080,\n",
              " 9824,\n",
              " 5216,\n",
              " 5697,\n",
              " 20106,\n",
              " 7685,\n",
              " 25088,\n",
              " 8719,\n",
              " 16661,\n",
              " 892,\n",
              " 29829,\n",
              " 29798,\n",
              " 3213,\n",
              " 18565,\n",
              " 25016,\n",
              " 30165,\n",
              " 107,\n",
              " 244,\n",
              " 2536,\n",
              " 15435,\n",
              " 5422,\n",
              " 6591,\n",
              " 11186,\n",
              " 13376,\n",
              " 4756,\n",
              " 19585,\n",
              " 26785,\n",
              " 19295,\n",
              " 5861,\n",
              " 24573,\n",
              " 11683,\n",
              " 2528,\n",
              " 23973,\n",
              " 27577,\n",
              " 3619,\n",
              " 3962,\n",
              " 12887,\n",
              " 30853,\n",
              " 26531,\n",
              " 16095,\n",
              " 2296,\n",
              " 83,\n",
              " 3561,\n",
              " 12948,\n",
              " 16098,\n",
              " 26370,\n",
              " 7245,\n",
              " 3560,\n",
              " 9554,\n",
              " 7812,\n",
              " 22182,\n",
              " 28721,\n",
              " 24146,\n",
              " 21315,\n",
              " 15177,\n",
              " 12543,\n",
              " 24589,\n",
              " 7668,\n",
              " 10292,\n",
              " 23179,\n",
              " 2150,\n",
              " 15880,\n",
              " 5961,\n",
              " 25139,\n",
              " 27115,\n",
              " 23586,\n",
              " 1780,\n",
              " 30823,\n",
              " 17872,\n",
              " 12952,\n",
              " 12285,\n",
              " 7776,\n",
              " 10868,\n",
              " 6165,\n",
              " 15412,\n",
              " 27224,\n",
              " 3902,\n",
              " 31285,\n",
              " 16262,\n",
              " 30831,\n",
              " 22804,\n",
              " 28917,\n",
              " 23112,\n",
              " 12731,\n",
              " 13037,\n",
              " 22877,\n",
              " 26543,\n",
              " 23335,\n",
              " 27154,\n",
              " 17809,\n",
              " 1898,\n",
              " 10202,\n",
              " 23,\n",
              " 8005,\n",
              " 31227,\n",
              " 23023,\n",
              " 23939,\n",
              " 8466,\n",
              " 14992,\n",
              " 15399,\n",
              " 22470,\n",
              " 27015,\n",
              " 19784,\n",
              " 17005,\n",
              " 16826,\n",
              " 14850,\n",
              " 8502,\n",
              " 8953,\n",
              " 12833,\n",
              " 9836,\n",
              " 3620,\n",
              " 2188,\n",
              " 27998,\n",
              " 30817,\n",
              " 9326,\n",
              " 17523,\n",
              " 3733]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "random_indices"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bJmCxvt_Xx1S"
      },
      "outputs": [],
      "source": [
        "instruction_text = \"The text given have gramatical and synthaxe error. Rewrite it and make it understadable. \"\n",
        "sampled_dataset = sampled_dataset.add_column('instruction', [instruction_text] * len(sampled_dataset))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J6-VNFfeSoyZ",
        "outputId": "e22c11ee-5a88-4404-b6be-f96bafc2698b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['input', 'output', 'instruction'],\n",
              "    num_rows: 200\n",
              "})"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sampled_dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eNX9RrvSZ_gE"
      },
      "outputs": [],
      "source": [
        "from datasets import Dataset\n",
        "def split_into_chunks(text, max_length, overlap):\n",
        "    chunks = []\n",
        "    start = 0\n",
        "    while start < len(text):\n",
        "        end = min(start + max_length, len(text))\n",
        "        chunk = text[start:end]\n",
        "        chunks.append(chunk)\n",
        "        if end == len(text):\n",
        "            break\n",
        "        start += max_length - overlap\n",
        "    return chunks\n",
        "\n",
        "def create_chunks(examples):\n",
        "    max_length = 512\n",
        "    overlap = 100\n",
        "\n",
        "    input_chunks = []\n",
        "    output_chunks = []\n",
        "    instructions = []\n",
        "\n",
        "    for input_text, output_text, instruction in zip(examples['input'], examples['output'], examples['instruction']):\n",
        "        input_text_chunks = split_into_chunks(input_text, max_length, overlap)\n",
        "        output_text_chunks = split_into_chunks(output_text, max_length, overlap)\n",
        "\n",
        "        # Assurez-vous que les longueurs des morceaux correspondent\n",
        "        min_len = min(len(input_text_chunks), len(output_text_chunks))\n",
        "        input_chunks.extend(input_text_chunks[:min_len])\n",
        "        output_chunks.extend(output_text_chunks[:min_len])\n",
        "        instructions.extend([instruction] * min_len)\n",
        "\n",
        "    return {'input': input_chunks, 'output': output_chunks, 'instruction': instructions}\n",
        "\n",
        "# Convertir votre dataset en format Dataset de Hugging Face\n",
        "# dataset = Dataset.from_dict(dataset)\n",
        "\n",
        "# Appliquer la fonction de découpage\n",
        "dataset_new = sampled_dataset.map(create_chunks, batched=True)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zxSKJwTBeFVy",
        "outputId": "5f4effe8-ef0e-43f4-9934-d2765a25670c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['input', 'output', 'instruction'],\n",
              "    num_rows: 200\n",
              "})"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sampled_dataset # basic"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sj94U7Y6d7Wt",
        "outputId": "45edc2ba-a826-4fcb-a33c-c776914c953d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['input', 'output', 'instruction'],\n",
              "    num_rows: 22545\n",
              "})"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_new #after refactoring"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V1jSzr8GTFMw"
      },
      "outputs": [],
      "source": [
        "rep=formatting_prompts_func(dataset_new)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LoDlr-qcPlTu"
      },
      "outputs": [],
      "source": [
        "dataset = dataset_new.map(formatting_prompts_func, batched = True,)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BG1KVbpJjMmP",
        "outputId": "b93c061c-0d71-4ac7-f29b-7631d3cf5b15"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['input', 'output', 'instruction', 'text'],\n",
              "    num_rows: 22545\n",
              "})"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wnSDXxyRQ8rn",
        "outputId": "d490d742-0207-4de4-bcc9-16aeb82f5feb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fri Jun 14 12:17:25 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  NVIDIA L4                      Off | 00000000:35:00.0 Off |                    0 |\n",
            "| N/A   66C    P0              31W /  72W |   7481MiB / 23034MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
            "To disable this warning, you can either:\n",
            "\t- Avoid using `tokenizers` before the fork if possible\n",
            "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "idAEIeSQ3xdS"
      },
      "source": [
        "<a name=\"Train\"></a>\n",
        "### Train the model\n",
        "Now let's use Huggingface TRL's `SFTTrainer`! More docs here: [TRL SFT docs](https://huggingface.co/docs/trl/sft_trainer). We do 60 steps to speed things up, but you can set `num_train_epochs=1` for a full run, and turn off `max_steps=None`. We also support TRL's `DPOTrainer`!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IfAyGVImyU0S"
      },
      "outputs": [],
      "source": [
        "max_seq_length = 4000"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "id": "znQD2SiNHvAg",
        "outputId": "2160dc94-fdd9-4ed9-c403-28d1363e4c53"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mlompojeanolivier\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "env: WANDB_PROJECT=qwen\n"
          ]
        }
      ],
      "source": [
        "# !pip install wandb -Uq\n",
        "# Log in to your W&B account\n",
        "import wandb\n",
        "wandb.login()\n",
        "\n",
        "%env WANDB_PROJECT=qwen"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "95_Nn-89DhsL",
        "outputId": "6fd5035b-9536-4e2a-9ea8-ee6c55a7bf79"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "max_steps is given, it will override any value given in num_train_epochs\n"
          ]
        }
      ],
      "source": [
        "from trl import SFTTrainer\n",
        "from transformers import TrainingArguments\n",
        "from unsloth import is_bfloat16_supported\n",
        "\n",
        "trainer = SFTTrainer(\n",
        "    model = model,\n",
        "    tokenizer = tokenizer,\n",
        "    train_dataset = dataset,\n",
        "    dataset_text_field = \"text\",\n",
        "    max_seq_length = max_seq_length,\n",
        "    dataset_num_proc = 2,\n",
        "    args = TrainingArguments(\n",
        "        report_to = 'wandb',\n",
        "        save_steps = 150,\n",
        "        per_device_train_batch_size = 2,\n",
        "        gradient_accumulation_steps = 4,\n",
        "        save_strategy = \"steps\",\n",
        "        save_total_limit = 2,\n",
        "        logging_steps = 1,\n",
        "        warmup_steps = 5,\n",
        "        max_steps =800,\n",
        "        learning_rate = 2e-4,\n",
        "        fp16 = not is_bfloat16_supported(),\n",
        "        bf16 = is_bfloat16_supported(),\n",
        "        optim = \"adamw_8bit\",\n",
        "        weight_decay = 0.01,\n",
        "        lr_scheduler_type = \"linear\",\n",
        "        seed = 3407,\n",
        "        output_dir = \"Outputs\",\n",
        "        run_name = 'qwen',\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "2ejIt2xSNKKp",
        "outputId": "c0e3848b-caf7-4919-ca31-3d8ebae39544"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "GPU = NVIDIA L4. Max memory = 22.168 GB.\n",
            "7.092 GB of memory reserved.\n"
          ]
        }
      ],
      "source": [
        "#@title Show current memory stats\n",
        "gpu_stats = torch.cuda.get_device_properties(0)\n",
        "start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n",
        "max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)\n",
        "print(f\"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.\")\n",
        "print(f\"{start_gpu_memory} GB of memory reserved.\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "igivd7J-QBYw",
        "outputId": "3d7adabc-6fd7-4c11-ba03-3aed7182784c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'12.1'"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import torch; torch.version.cuda"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 640
        },
        "id": "yqxqAZ7KJ4oL",
        "outputId": "e0acb99a-05bf-4be4-e5ff-11ec86529e88"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs = 1\n",
            "   \\\\   /|    Num examples = 22,545 | Num Epochs = 1\n",
            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient Accumulation steps = 4\n",
            "\\        /    Total batch size = 8 | Total steps = 800\n",
            " \"-____-\"     Number of trainable parameters = 40,370,176\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "Changes to your `wandb` environment variables will be ignored because your `wandb` session has already started. For more information on how to modify your settings with `wandb.init()` arguments, please refer to <a href='https://wandb.me/wandb-init' target=\"_blank\">the W&B docs</a>."
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Tracking run with wandb version 0.17.1"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Run data is saved locally in <code>/teamspace/studios/this_studio/wandb/run-20240614_121727-t1fd1zef</code>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "Syncing run <strong><a href='https://wandb.ai/lompojeanolivier/qwen/runs/t1fd1zef' target=\"_blank\">qwen</a></strong> to <a href='https://wandb.ai/lompojeanolivier/qwen' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              " View project at <a href='https://wandb.ai/lompojeanolivier/qwen' target=\"_blank\">https://wandb.ai/lompojeanolivier/qwen</a>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              " View run at <a href='https://wandb.ai/lompojeanolivier/qwen/runs/t1fd1zef' target=\"_blank\">https://wandb.ai/lompojeanolivier/qwen/runs/t1fd1zef</a>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='800' max='800' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [800/800 58:32, Epoch 0/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>3.326100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>3.851500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>3.765700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>3.622300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>4.317000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>3.494500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>3.447800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>3.659800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>3.871800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>3.314900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>3.593700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>3.452900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>3.247000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>2.825000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>3.287000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>3.318800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>17</td>\n",
              "      <td>3.319200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>18</td>\n",
              "      <td>3.426300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>19</td>\n",
              "      <td>3.486000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>20</td>\n",
              "      <td>3.379100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>21</td>\n",
              "      <td>2.831700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>22</td>\n",
              "      <td>3.293900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>23</td>\n",
              "      <td>2.748400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>24</td>\n",
              "      <td>3.220100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>25</td>\n",
              "      <td>3.239700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>26</td>\n",
              "      <td>3.437900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>27</td>\n",
              "      <td>2.962100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>28</td>\n",
              "      <td>2.814000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>29</td>\n",
              "      <td>3.468300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>30</td>\n",
              "      <td>3.216300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>31</td>\n",
              "      <td>3.216600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>32</td>\n",
              "      <td>3.610900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>33</td>\n",
              "      <td>2.772800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>34</td>\n",
              "      <td>3.072700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>35</td>\n",
              "      <td>3.210500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>36</td>\n",
              "      <td>2.552100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>37</td>\n",
              "      <td>3.483700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>38</td>\n",
              "      <td>3.352500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>39</td>\n",
              "      <td>3.044600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>40</td>\n",
              "      <td>3.531200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>41</td>\n",
              "      <td>3.284000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>42</td>\n",
              "      <td>3.110000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>43</td>\n",
              "      <td>3.122700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>44</td>\n",
              "      <td>3.337800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>45</td>\n",
              "      <td>3.335800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>46</td>\n",
              "      <td>3.162100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>47</td>\n",
              "      <td>3.664900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>48</td>\n",
              "      <td>3.107700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>49</td>\n",
              "      <td>2.820400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>50</td>\n",
              "      <td>2.879100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>51</td>\n",
              "      <td>2.932900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>52</td>\n",
              "      <td>3.715000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>53</td>\n",
              "      <td>2.956900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>54</td>\n",
              "      <td>3.082500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>55</td>\n",
              "      <td>2.959000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>56</td>\n",
              "      <td>3.221900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>57</td>\n",
              "      <td>3.068900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>58</td>\n",
              "      <td>3.425300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>59</td>\n",
              "      <td>3.156400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>60</td>\n",
              "      <td>3.190600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>61</td>\n",
              "      <td>3.300000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>62</td>\n",
              "      <td>3.245500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>63</td>\n",
              "      <td>3.256700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>64</td>\n",
              "      <td>3.073600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>65</td>\n",
              "      <td>2.862800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>66</td>\n",
              "      <td>3.025400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>67</td>\n",
              "      <td>3.184900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>68</td>\n",
              "      <td>3.115200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>69</td>\n",
              "      <td>2.828600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>70</td>\n",
              "      <td>2.649800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>71</td>\n",
              "      <td>2.775600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>72</td>\n",
              "      <td>2.958200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>73</td>\n",
              "      <td>3.083700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>74</td>\n",
              "      <td>2.964100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>75</td>\n",
              "      <td>3.393500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>76</td>\n",
              "      <td>2.990100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>77</td>\n",
              "      <td>3.086600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>78</td>\n",
              "      <td>3.198900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>79</td>\n",
              "      <td>2.774000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>80</td>\n",
              "      <td>3.075100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>81</td>\n",
              "      <td>3.045600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>82</td>\n",
              "      <td>3.173600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>83</td>\n",
              "      <td>2.871100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>84</td>\n",
              "      <td>3.110100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>85</td>\n",
              "      <td>3.185200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>86</td>\n",
              "      <td>2.474300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>87</td>\n",
              "      <td>2.693300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>88</td>\n",
              "      <td>3.285200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>89</td>\n",
              "      <td>3.021500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>90</td>\n",
              "      <td>3.248500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>91</td>\n",
              "      <td>3.132300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>92</td>\n",
              "      <td>3.076500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>93</td>\n",
              "      <td>3.132200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>94</td>\n",
              "      <td>3.569700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>95</td>\n",
              "      <td>2.776200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>96</td>\n",
              "      <td>3.025500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>97</td>\n",
              "      <td>3.143000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>98</td>\n",
              "      <td>3.143000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>99</td>\n",
              "      <td>3.231300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>2.443000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>101</td>\n",
              "      <td>3.047200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>102</td>\n",
              "      <td>3.045400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>103</td>\n",
              "      <td>2.701300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>104</td>\n",
              "      <td>3.092200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>105</td>\n",
              "      <td>2.635600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>106</td>\n",
              "      <td>3.287400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>107</td>\n",
              "      <td>3.238100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>108</td>\n",
              "      <td>2.959700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>109</td>\n",
              "      <td>3.189900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>110</td>\n",
              "      <td>2.900300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>111</td>\n",
              "      <td>3.246400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>112</td>\n",
              "      <td>3.362400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>113</td>\n",
              "      <td>3.342100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>114</td>\n",
              "      <td>2.963500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>115</td>\n",
              "      <td>2.761800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>116</td>\n",
              "      <td>3.271400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>117</td>\n",
              "      <td>3.054300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>118</td>\n",
              "      <td>2.684100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>119</td>\n",
              "      <td>2.858900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>120</td>\n",
              "      <td>2.809400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>121</td>\n",
              "      <td>2.896900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>122</td>\n",
              "      <td>3.324200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>123</td>\n",
              "      <td>3.006400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>124</td>\n",
              "      <td>2.881700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>125</td>\n",
              "      <td>3.330300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>126</td>\n",
              "      <td>3.480600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>127</td>\n",
              "      <td>3.224900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>128</td>\n",
              "      <td>3.106900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>129</td>\n",
              "      <td>3.197700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>130</td>\n",
              "      <td>3.300500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>131</td>\n",
              "      <td>2.456100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>132</td>\n",
              "      <td>3.066200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>133</td>\n",
              "      <td>2.912900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>134</td>\n",
              "      <td>2.810800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>135</td>\n",
              "      <td>3.172600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>136</td>\n",
              "      <td>3.734200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>137</td>\n",
              "      <td>3.043300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>138</td>\n",
              "      <td>2.954700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>139</td>\n",
              "      <td>2.915300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>140</td>\n",
              "      <td>3.004700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>141</td>\n",
              "      <td>2.648500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>142</td>\n",
              "      <td>3.401700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>143</td>\n",
              "      <td>3.179500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>144</td>\n",
              "      <td>3.181200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>145</td>\n",
              "      <td>3.109400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>146</td>\n",
              "      <td>3.162100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>147</td>\n",
              "      <td>3.059100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>148</td>\n",
              "      <td>3.024700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>149</td>\n",
              "      <td>3.029100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>150</td>\n",
              "      <td>3.581200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>151</td>\n",
              "      <td>3.078600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>152</td>\n",
              "      <td>3.051500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>153</td>\n",
              "      <td>3.108900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>154</td>\n",
              "      <td>2.933800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>155</td>\n",
              "      <td>2.738100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>156</td>\n",
              "      <td>3.032300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>157</td>\n",
              "      <td>3.509500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>158</td>\n",
              "      <td>2.766200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>159</td>\n",
              "      <td>3.133600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>160</td>\n",
              "      <td>2.978500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>161</td>\n",
              "      <td>2.725800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>162</td>\n",
              "      <td>3.290200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>163</td>\n",
              "      <td>2.869700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>164</td>\n",
              "      <td>3.346800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>165</td>\n",
              "      <td>2.580400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>166</td>\n",
              "      <td>2.805200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>167</td>\n",
              "      <td>2.920200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>168</td>\n",
              "      <td>3.061200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>169</td>\n",
              "      <td>3.380300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>170</td>\n",
              "      <td>3.179400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>171</td>\n",
              "      <td>3.103100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>172</td>\n",
              "      <td>2.837500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>173</td>\n",
              "      <td>3.457600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>174</td>\n",
              "      <td>3.435900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>175</td>\n",
              "      <td>2.966900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>176</td>\n",
              "      <td>3.248700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>177</td>\n",
              "      <td>3.213900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>178</td>\n",
              "      <td>3.168000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>179</td>\n",
              "      <td>3.121100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>180</td>\n",
              "      <td>2.957800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>181</td>\n",
              "      <td>3.008800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>182</td>\n",
              "      <td>3.203400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>183</td>\n",
              "      <td>2.898600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>184</td>\n",
              "      <td>3.351400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>185</td>\n",
              "      <td>3.440500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>186</td>\n",
              "      <td>3.230500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>187</td>\n",
              "      <td>3.292300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>188</td>\n",
              "      <td>3.235700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>189</td>\n",
              "      <td>2.868400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>190</td>\n",
              "      <td>2.897900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>191</td>\n",
              "      <td>3.385100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>192</td>\n",
              "      <td>2.979000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>193</td>\n",
              "      <td>3.444000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>194</td>\n",
              "      <td>2.534200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>195</td>\n",
              "      <td>2.717000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>196</td>\n",
              "      <td>2.901300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>197</td>\n",
              "      <td>2.763800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>198</td>\n",
              "      <td>3.676700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>199</td>\n",
              "      <td>2.884300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>200</td>\n",
              "      <td>2.538500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>201</td>\n",
              "      <td>2.512300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>202</td>\n",
              "      <td>2.888600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>203</td>\n",
              "      <td>3.399200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>204</td>\n",
              "      <td>3.323900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>205</td>\n",
              "      <td>3.027600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>206</td>\n",
              "      <td>3.124700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>207</td>\n",
              "      <td>3.000600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>208</td>\n",
              "      <td>2.724500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>209</td>\n",
              "      <td>3.521500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>210</td>\n",
              "      <td>3.089000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>211</td>\n",
              "      <td>2.746300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>212</td>\n",
              "      <td>3.257800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>213</td>\n",
              "      <td>3.393000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>214</td>\n",
              "      <td>2.688900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>215</td>\n",
              "      <td>3.467000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>216</td>\n",
              "      <td>2.923500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>217</td>\n",
              "      <td>2.749700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>218</td>\n",
              "      <td>2.738200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>219</td>\n",
              "      <td>3.209400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>220</td>\n",
              "      <td>2.966600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>221</td>\n",
              "      <td>2.753400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>222</td>\n",
              "      <td>2.889000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>223</td>\n",
              "      <td>3.124800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>224</td>\n",
              "      <td>2.672800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>225</td>\n",
              "      <td>3.265900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>226</td>\n",
              "      <td>2.781100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>227</td>\n",
              "      <td>2.762400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>228</td>\n",
              "      <td>2.829400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>229</td>\n",
              "      <td>2.884200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>230</td>\n",
              "      <td>2.604400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>231</td>\n",
              "      <td>3.129700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>232</td>\n",
              "      <td>2.867300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>233</td>\n",
              "      <td>3.013700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>234</td>\n",
              "      <td>2.881200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>235</td>\n",
              "      <td>2.982200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>236</td>\n",
              "      <td>3.206500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>237</td>\n",
              "      <td>2.876800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>238</td>\n",
              "      <td>3.074600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>239</td>\n",
              "      <td>2.693000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>240</td>\n",
              "      <td>3.050800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>241</td>\n",
              "      <td>3.217100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>242</td>\n",
              "      <td>3.035300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>243</td>\n",
              "      <td>2.986300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>244</td>\n",
              "      <td>3.216600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>245</td>\n",
              "      <td>3.276100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>246</td>\n",
              "      <td>3.102800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>247</td>\n",
              "      <td>2.653500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>248</td>\n",
              "      <td>3.210300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>249</td>\n",
              "      <td>2.944700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>250</td>\n",
              "      <td>3.004000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>251</td>\n",
              "      <td>3.151200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>252</td>\n",
              "      <td>2.752200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>253</td>\n",
              "      <td>3.141700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>254</td>\n",
              "      <td>3.191800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>255</td>\n",
              "      <td>3.520900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>256</td>\n",
              "      <td>3.254800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>257</td>\n",
              "      <td>2.807100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>258</td>\n",
              "      <td>3.309300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>259</td>\n",
              "      <td>3.079100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>260</td>\n",
              "      <td>3.367900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>261</td>\n",
              "      <td>2.931500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>262</td>\n",
              "      <td>2.726400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>263</td>\n",
              "      <td>2.823100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>264</td>\n",
              "      <td>3.446600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>265</td>\n",
              "      <td>3.007400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>266</td>\n",
              "      <td>3.032200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>267</td>\n",
              "      <td>3.070300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>268</td>\n",
              "      <td>3.091000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>269</td>\n",
              "      <td>2.818500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>270</td>\n",
              "      <td>2.781200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>271</td>\n",
              "      <td>3.298200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>272</td>\n",
              "      <td>2.803900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>273</td>\n",
              "      <td>3.003400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>274</td>\n",
              "      <td>2.612700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>275</td>\n",
              "      <td>2.709500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>276</td>\n",
              "      <td>3.045700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>277</td>\n",
              "      <td>2.992000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>278</td>\n",
              "      <td>2.632400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>279</td>\n",
              "      <td>3.407500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>280</td>\n",
              "      <td>2.932200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>281</td>\n",
              "      <td>2.758300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>282</td>\n",
              "      <td>3.429900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>283</td>\n",
              "      <td>3.381800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>284</td>\n",
              "      <td>3.330300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>285</td>\n",
              "      <td>3.124100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>286</td>\n",
              "      <td>2.807200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>287</td>\n",
              "      <td>3.292800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>288</td>\n",
              "      <td>2.859000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>289</td>\n",
              "      <td>3.453700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>290</td>\n",
              "      <td>2.784800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>291</td>\n",
              "      <td>2.767300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>292</td>\n",
              "      <td>2.818900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>293</td>\n",
              "      <td>2.387900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>294</td>\n",
              "      <td>2.971800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>295</td>\n",
              "      <td>2.814700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>296</td>\n",
              "      <td>2.904700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>297</td>\n",
              "      <td>3.005600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>298</td>\n",
              "      <td>2.983400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>299</td>\n",
              "      <td>2.813500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>300</td>\n",
              "      <td>3.248500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>301</td>\n",
              "      <td>3.182000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>302</td>\n",
              "      <td>2.627800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>303</td>\n",
              "      <td>2.901300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>304</td>\n",
              "      <td>2.678000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>305</td>\n",
              "      <td>3.058300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>306</td>\n",
              "      <td>2.865500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>307</td>\n",
              "      <td>2.663100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>308</td>\n",
              "      <td>3.354900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>309</td>\n",
              "      <td>3.106000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>310</td>\n",
              "      <td>3.259900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>311</td>\n",
              "      <td>3.055400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>312</td>\n",
              "      <td>2.876200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>313</td>\n",
              "      <td>3.101200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>314</td>\n",
              "      <td>2.753000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>315</td>\n",
              "      <td>2.973500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>316</td>\n",
              "      <td>3.051300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>317</td>\n",
              "      <td>3.355700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>318</td>\n",
              "      <td>3.705800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>319</td>\n",
              "      <td>2.940700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>320</td>\n",
              "      <td>3.226100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>321</td>\n",
              "      <td>2.652300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>322</td>\n",
              "      <td>2.969700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>323</td>\n",
              "      <td>3.060000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>324</td>\n",
              "      <td>2.853300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>325</td>\n",
              "      <td>2.930400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>326</td>\n",
              "      <td>2.938000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>327</td>\n",
              "      <td>2.797800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>328</td>\n",
              "      <td>2.901500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>329</td>\n",
              "      <td>2.858500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>330</td>\n",
              "      <td>2.781400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>331</td>\n",
              "      <td>2.883000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>332</td>\n",
              "      <td>3.154900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>333</td>\n",
              "      <td>3.157300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>334</td>\n",
              "      <td>2.279400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>335</td>\n",
              "      <td>2.962000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>336</td>\n",
              "      <td>3.432600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>337</td>\n",
              "      <td>2.719200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>338</td>\n",
              "      <td>2.959600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>339</td>\n",
              "      <td>2.971600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>340</td>\n",
              "      <td>2.951500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>341</td>\n",
              "      <td>3.084500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>342</td>\n",
              "      <td>3.363300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>343</td>\n",
              "      <td>2.769000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>344</td>\n",
              "      <td>3.363400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>345</td>\n",
              "      <td>3.263200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>346</td>\n",
              "      <td>2.734000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>347</td>\n",
              "      <td>2.230100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348</td>\n",
              "      <td>2.731000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>349</td>\n",
              "      <td>3.435300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>350</td>\n",
              "      <td>3.065200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>351</td>\n",
              "      <td>2.888800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>352</td>\n",
              "      <td>3.269200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353</td>\n",
              "      <td>2.672000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354</td>\n",
              "      <td>3.111400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>355</td>\n",
              "      <td>2.930400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>356</td>\n",
              "      <td>2.504400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>357</td>\n",
              "      <td>2.804900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>358</td>\n",
              "      <td>2.780300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>359</td>\n",
              "      <td>3.300900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>360</td>\n",
              "      <td>2.924300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>361</td>\n",
              "      <td>3.467000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>362</td>\n",
              "      <td>3.004300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>363</td>\n",
              "      <td>3.069900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>364</td>\n",
              "      <td>3.567400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>365</td>\n",
              "      <td>3.336200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>366</td>\n",
              "      <td>2.927200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>367</td>\n",
              "      <td>2.870800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>368</td>\n",
              "      <td>3.130400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>369</td>\n",
              "      <td>3.202000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>370</td>\n",
              "      <td>2.432200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>371</td>\n",
              "      <td>2.652200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>372</td>\n",
              "      <td>2.425000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>373</td>\n",
              "      <td>3.032700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>374</td>\n",
              "      <td>3.437100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>375</td>\n",
              "      <td>2.990800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>376</td>\n",
              "      <td>2.616000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>377</td>\n",
              "      <td>3.022600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>378</td>\n",
              "      <td>2.908500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>379</td>\n",
              "      <td>3.347700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>380</td>\n",
              "      <td>2.813100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>381</td>\n",
              "      <td>2.940200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>382</td>\n",
              "      <td>3.034200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>383</td>\n",
              "      <td>3.098600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>384</td>\n",
              "      <td>3.469500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>385</td>\n",
              "      <td>3.068300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>386</td>\n",
              "      <td>3.057300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>387</td>\n",
              "      <td>2.697800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>388</td>\n",
              "      <td>2.855300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>389</td>\n",
              "      <td>2.845400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>390</td>\n",
              "      <td>3.410600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>391</td>\n",
              "      <td>2.570300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>392</td>\n",
              "      <td>2.634200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>393</td>\n",
              "      <td>3.218700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>394</td>\n",
              "      <td>2.768900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>395</td>\n",
              "      <td>3.225200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>396</td>\n",
              "      <td>2.873700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>397</td>\n",
              "      <td>2.697100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>398</td>\n",
              "      <td>2.618600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>399</td>\n",
              "      <td>3.083200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>400</td>\n",
              "      <td>3.037100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>401</td>\n",
              "      <td>2.731800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>402</td>\n",
              "      <td>3.263200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>403</td>\n",
              "      <td>2.732300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>404</td>\n",
              "      <td>3.094800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>405</td>\n",
              "      <td>3.003200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>406</td>\n",
              "      <td>2.856800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>407</td>\n",
              "      <td>3.462000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>408</td>\n",
              "      <td>2.800000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>409</td>\n",
              "      <td>2.837600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>410</td>\n",
              "      <td>3.527500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>411</td>\n",
              "      <td>3.087900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>412</td>\n",
              "      <td>2.518900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>413</td>\n",
              "      <td>3.039000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>414</td>\n",
              "      <td>3.397000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>415</td>\n",
              "      <td>2.617400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>416</td>\n",
              "      <td>2.913400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>417</td>\n",
              "      <td>3.186500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>418</td>\n",
              "      <td>3.140500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>419</td>\n",
              "      <td>2.864800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>420</td>\n",
              "      <td>3.246500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>421</td>\n",
              "      <td>3.120500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>422</td>\n",
              "      <td>2.835000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>423</td>\n",
              "      <td>3.080100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>424</td>\n",
              "      <td>3.019400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>425</td>\n",
              "      <td>2.937500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>426</td>\n",
              "      <td>2.827900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>427</td>\n",
              "      <td>3.101000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>428</td>\n",
              "      <td>2.857500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>429</td>\n",
              "      <td>2.964000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>430</td>\n",
              "      <td>2.889800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>431</td>\n",
              "      <td>2.843500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>432</td>\n",
              "      <td>2.555300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>433</td>\n",
              "      <td>3.186900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>434</td>\n",
              "      <td>3.141100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>435</td>\n",
              "      <td>3.179400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>436</td>\n",
              "      <td>2.990700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>437</td>\n",
              "      <td>3.314400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>438</td>\n",
              "      <td>3.249300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>439</td>\n",
              "      <td>3.057000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>440</td>\n",
              "      <td>3.404200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>441</td>\n",
              "      <td>2.800900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>442</td>\n",
              "      <td>3.227000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>443</td>\n",
              "      <td>3.057700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>444</td>\n",
              "      <td>3.234800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>445</td>\n",
              "      <td>3.148700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>446</td>\n",
              "      <td>3.154700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>447</td>\n",
              "      <td>3.483700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>448</td>\n",
              "      <td>2.526100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>449</td>\n",
              "      <td>2.844600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>450</td>\n",
              "      <td>2.945100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>451</td>\n",
              "      <td>3.444100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>452</td>\n",
              "      <td>2.794400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>453</td>\n",
              "      <td>3.040500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>454</td>\n",
              "      <td>3.396000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>455</td>\n",
              "      <td>2.831800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>456</td>\n",
              "      <td>2.829700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>457</td>\n",
              "      <td>2.840200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>458</td>\n",
              "      <td>2.718600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>459</td>\n",
              "      <td>3.187600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>460</td>\n",
              "      <td>2.758700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>461</td>\n",
              "      <td>2.860700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>462</td>\n",
              "      <td>2.853100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>463</td>\n",
              "      <td>2.913200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>464</td>\n",
              "      <td>3.065700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>465</td>\n",
              "      <td>2.800400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>466</td>\n",
              "      <td>3.429500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>467</td>\n",
              "      <td>3.181700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>468</td>\n",
              "      <td>3.512500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>469</td>\n",
              "      <td>2.776300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>470</td>\n",
              "      <td>3.185900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>471</td>\n",
              "      <td>2.836900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>472</td>\n",
              "      <td>3.270300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>473</td>\n",
              "      <td>3.095400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>474</td>\n",
              "      <td>3.619000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>475</td>\n",
              "      <td>3.080200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>476</td>\n",
              "      <td>3.635500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>477</td>\n",
              "      <td>2.795300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>478</td>\n",
              "      <td>3.003400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>479</td>\n",
              "      <td>3.249600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>480</td>\n",
              "      <td>3.339200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>481</td>\n",
              "      <td>3.109200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>482</td>\n",
              "      <td>2.963800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>483</td>\n",
              "      <td>3.066100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>484</td>\n",
              "      <td>2.663500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>485</td>\n",
              "      <td>2.916600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>486</td>\n",
              "      <td>2.656100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>487</td>\n",
              "      <td>3.078700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>488</td>\n",
              "      <td>3.022900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>489</td>\n",
              "      <td>3.212100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>490</td>\n",
              "      <td>2.852200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>491</td>\n",
              "      <td>3.287800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>492</td>\n",
              "      <td>3.173700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>493</td>\n",
              "      <td>2.932200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>494</td>\n",
              "      <td>3.144800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>495</td>\n",
              "      <td>2.585100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>496</td>\n",
              "      <td>3.241600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>497</td>\n",
              "      <td>2.968800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>498</td>\n",
              "      <td>3.090100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>499</td>\n",
              "      <td>3.076300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>2.604600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>501</td>\n",
              "      <td>3.557900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>502</td>\n",
              "      <td>3.183200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>503</td>\n",
              "      <td>3.189700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>504</td>\n",
              "      <td>2.666800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>505</td>\n",
              "      <td>3.136500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>506</td>\n",
              "      <td>3.259900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>507</td>\n",
              "      <td>3.260700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>508</td>\n",
              "      <td>3.122100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>509</td>\n",
              "      <td>3.173700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>510</td>\n",
              "      <td>2.677600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>511</td>\n",
              "      <td>2.903000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>512</td>\n",
              "      <td>3.244500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>513</td>\n",
              "      <td>2.890600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>514</td>\n",
              "      <td>2.936000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>515</td>\n",
              "      <td>2.755500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>516</td>\n",
              "      <td>2.775500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>517</td>\n",
              "      <td>2.613200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>518</td>\n",
              "      <td>3.144500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>519</td>\n",
              "      <td>3.321700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>520</td>\n",
              "      <td>2.994100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>521</td>\n",
              "      <td>2.774600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>522</td>\n",
              "      <td>2.973700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>523</td>\n",
              "      <td>3.312900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>524</td>\n",
              "      <td>3.164700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>525</td>\n",
              "      <td>2.629200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>526</td>\n",
              "      <td>3.154900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>527</td>\n",
              "      <td>3.339600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>528</td>\n",
              "      <td>2.885200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>529</td>\n",
              "      <td>2.809400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>530</td>\n",
              "      <td>3.155800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>531</td>\n",
              "      <td>2.812900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>532</td>\n",
              "      <td>3.082500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>533</td>\n",
              "      <td>2.653400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>534</td>\n",
              "      <td>2.755500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>535</td>\n",
              "      <td>3.164300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>536</td>\n",
              "      <td>3.142900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>537</td>\n",
              "      <td>2.919100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>538</td>\n",
              "      <td>3.036700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>539</td>\n",
              "      <td>2.825100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>540</td>\n",
              "      <td>2.583700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>541</td>\n",
              "      <td>2.936700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>542</td>\n",
              "      <td>2.988800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>543</td>\n",
              "      <td>2.679200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>544</td>\n",
              "      <td>3.146300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>545</td>\n",
              "      <td>3.135100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>546</td>\n",
              "      <td>3.344300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>547</td>\n",
              "      <td>3.146300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>548</td>\n",
              "      <td>3.003100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>549</td>\n",
              "      <td>3.158200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>550</td>\n",
              "      <td>2.820400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>551</td>\n",
              "      <td>2.806100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>552</td>\n",
              "      <td>3.119500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>553</td>\n",
              "      <td>2.659800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>554</td>\n",
              "      <td>3.438600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>555</td>\n",
              "      <td>3.135700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>556</td>\n",
              "      <td>3.085900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>557</td>\n",
              "      <td>3.223200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>558</td>\n",
              "      <td>2.851500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>559</td>\n",
              "      <td>3.046300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>560</td>\n",
              "      <td>2.754700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>561</td>\n",
              "      <td>3.325100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>562</td>\n",
              "      <td>3.093500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>563</td>\n",
              "      <td>3.402400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>564</td>\n",
              "      <td>2.654600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>565</td>\n",
              "      <td>3.043200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>566</td>\n",
              "      <td>3.125200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>567</td>\n",
              "      <td>3.095100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>568</td>\n",
              "      <td>2.813100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>569</td>\n",
              "      <td>3.117500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>570</td>\n",
              "      <td>3.292500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>571</td>\n",
              "      <td>3.328900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>572</td>\n",
              "      <td>2.827200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>573</td>\n",
              "      <td>2.732400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>574</td>\n",
              "      <td>3.100100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>575</td>\n",
              "      <td>2.969400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>576</td>\n",
              "      <td>2.800300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>577</td>\n",
              "      <td>2.849400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>578</td>\n",
              "      <td>2.946400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>579</td>\n",
              "      <td>2.795900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>580</td>\n",
              "      <td>2.713300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>581</td>\n",
              "      <td>2.919300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>582</td>\n",
              "      <td>2.805400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>583</td>\n",
              "      <td>2.640600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>584</td>\n",
              "      <td>2.832800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>585</td>\n",
              "      <td>3.257400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>586</td>\n",
              "      <td>3.078400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>587</td>\n",
              "      <td>3.567600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>588</td>\n",
              "      <td>3.193800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>589</td>\n",
              "      <td>2.933900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>590</td>\n",
              "      <td>2.959900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>591</td>\n",
              "      <td>3.123900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>592</td>\n",
              "      <td>2.974400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>593</td>\n",
              "      <td>2.846500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>594</td>\n",
              "      <td>2.814900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>595</td>\n",
              "      <td>2.876200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>596</td>\n",
              "      <td>3.014000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>597</td>\n",
              "      <td>3.281300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>598</td>\n",
              "      <td>2.679500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>599</td>\n",
              "      <td>3.074600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>600</td>\n",
              "      <td>2.526400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>601</td>\n",
              "      <td>3.124300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>602</td>\n",
              "      <td>3.150900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>603</td>\n",
              "      <td>3.021200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>604</td>\n",
              "      <td>3.038500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>605</td>\n",
              "      <td>3.171500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>606</td>\n",
              "      <td>2.961900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>607</td>\n",
              "      <td>3.272800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>608</td>\n",
              "      <td>3.151200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>609</td>\n",
              "      <td>2.677000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>610</td>\n",
              "      <td>2.830800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>611</td>\n",
              "      <td>3.095300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>612</td>\n",
              "      <td>2.780000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>613</td>\n",
              "      <td>2.998800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>614</td>\n",
              "      <td>3.131800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>615</td>\n",
              "      <td>2.848100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>616</td>\n",
              "      <td>3.051700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>617</td>\n",
              "      <td>3.237800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>618</td>\n",
              "      <td>3.180300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>619</td>\n",
              "      <td>2.986700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>620</td>\n",
              "      <td>2.992000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>621</td>\n",
              "      <td>3.079600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>622</td>\n",
              "      <td>3.598400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>623</td>\n",
              "      <td>3.225000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>624</td>\n",
              "      <td>2.999200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>625</td>\n",
              "      <td>2.945200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>626</td>\n",
              "      <td>3.318800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>627</td>\n",
              "      <td>3.125600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>628</td>\n",
              "      <td>2.950200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>629</td>\n",
              "      <td>3.155500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>630</td>\n",
              "      <td>3.090500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>631</td>\n",
              "      <td>3.464900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>632</td>\n",
              "      <td>3.224100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>633</td>\n",
              "      <td>2.911200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>634</td>\n",
              "      <td>2.673500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>635</td>\n",
              "      <td>2.820700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>636</td>\n",
              "      <td>3.161800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>637</td>\n",
              "      <td>3.268300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>638</td>\n",
              "      <td>3.443700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>639</td>\n",
              "      <td>3.183700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>640</td>\n",
              "      <td>2.622500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>641</td>\n",
              "      <td>2.898800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>642</td>\n",
              "      <td>3.002100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>643</td>\n",
              "      <td>2.711300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>644</td>\n",
              "      <td>2.848800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>645</td>\n",
              "      <td>2.628600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>646</td>\n",
              "      <td>2.968900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>647</td>\n",
              "      <td>2.862400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>648</td>\n",
              "      <td>2.509600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>649</td>\n",
              "      <td>2.923500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>650</td>\n",
              "      <td>2.537600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>651</td>\n",
              "      <td>2.506600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>652</td>\n",
              "      <td>2.876000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>653</td>\n",
              "      <td>2.913700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>654</td>\n",
              "      <td>2.758200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>655</td>\n",
              "      <td>2.850600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>656</td>\n",
              "      <td>3.322500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>657</td>\n",
              "      <td>2.842200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>658</td>\n",
              "      <td>2.890600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>659</td>\n",
              "      <td>2.821900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>660</td>\n",
              "      <td>3.161400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>661</td>\n",
              "      <td>2.607200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>662</td>\n",
              "      <td>3.190000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>663</td>\n",
              "      <td>2.618300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>664</td>\n",
              "      <td>2.593900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>665</td>\n",
              "      <td>3.219300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>666</td>\n",
              "      <td>2.877400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>667</td>\n",
              "      <td>2.936400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>668</td>\n",
              "      <td>3.035100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>669</td>\n",
              "      <td>2.975800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>670</td>\n",
              "      <td>3.277600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>671</td>\n",
              "      <td>3.301000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>672</td>\n",
              "      <td>2.953100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>673</td>\n",
              "      <td>3.367900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>674</td>\n",
              "      <td>2.380100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>675</td>\n",
              "      <td>3.014900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>676</td>\n",
              "      <td>3.150900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>677</td>\n",
              "      <td>3.011000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>678</td>\n",
              "      <td>3.321400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>679</td>\n",
              "      <td>3.021600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>680</td>\n",
              "      <td>2.772800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>681</td>\n",
              "      <td>3.135400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>682</td>\n",
              "      <td>3.614000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>683</td>\n",
              "      <td>3.242700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>684</td>\n",
              "      <td>2.682500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>685</td>\n",
              "      <td>2.780800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>686</td>\n",
              "      <td>2.761000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>687</td>\n",
              "      <td>2.780000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>688</td>\n",
              "      <td>3.024000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>689</td>\n",
              "      <td>2.804400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>690</td>\n",
              "      <td>2.484400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>691</td>\n",
              "      <td>2.861300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>692</td>\n",
              "      <td>2.977100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>693</td>\n",
              "      <td>2.879100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>694</td>\n",
              "      <td>3.227100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>695</td>\n",
              "      <td>2.699800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>696</td>\n",
              "      <td>3.003500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>697</td>\n",
              "      <td>3.072700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>698</td>\n",
              "      <td>3.277800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>699</td>\n",
              "      <td>3.144300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>700</td>\n",
              "      <td>2.533600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>701</td>\n",
              "      <td>3.049300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>702</td>\n",
              "      <td>2.718400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>703</td>\n",
              "      <td>3.040600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>704</td>\n",
              "      <td>3.001700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>705</td>\n",
              "      <td>2.897400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>706</td>\n",
              "      <td>3.347900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>707</td>\n",
              "      <td>2.888900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>708</td>\n",
              "      <td>2.940400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>709</td>\n",
              "      <td>3.073900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>710</td>\n",
              "      <td>2.778700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>711</td>\n",
              "      <td>3.099400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>712</td>\n",
              "      <td>3.050500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>713</td>\n",
              "      <td>2.999900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>714</td>\n",
              "      <td>2.482200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>715</td>\n",
              "      <td>3.052300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>716</td>\n",
              "      <td>3.046500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>717</td>\n",
              "      <td>3.088000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>718</td>\n",
              "      <td>2.757600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>719</td>\n",
              "      <td>3.046000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>720</td>\n",
              "      <td>3.310800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>721</td>\n",
              "      <td>2.648800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>722</td>\n",
              "      <td>2.757200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>723</td>\n",
              "      <td>2.908000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>724</td>\n",
              "      <td>3.174100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>725</td>\n",
              "      <td>3.390600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>726</td>\n",
              "      <td>2.747800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>727</td>\n",
              "      <td>2.824800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>728</td>\n",
              "      <td>3.279200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>729</td>\n",
              "      <td>2.605100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>730</td>\n",
              "      <td>3.014000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>731</td>\n",
              "      <td>2.991600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>732</td>\n",
              "      <td>3.046800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>733</td>\n",
              "      <td>3.223800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>734</td>\n",
              "      <td>3.127100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>735</td>\n",
              "      <td>2.359200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>736</td>\n",
              "      <td>3.265200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>737</td>\n",
              "      <td>2.664700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>738</td>\n",
              "      <td>2.676800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>739</td>\n",
              "      <td>2.773900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>740</td>\n",
              "      <td>2.509300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>741</td>\n",
              "      <td>2.786500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>742</td>\n",
              "      <td>3.319200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>743</td>\n",
              "      <td>3.162900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>744</td>\n",
              "      <td>2.414000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>745</td>\n",
              "      <td>2.994700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>746</td>\n",
              "      <td>2.832500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>747</td>\n",
              "      <td>2.605000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>748</td>\n",
              "      <td>3.632800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>749</td>\n",
              "      <td>3.019900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>750</td>\n",
              "      <td>2.653300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>751</td>\n",
              "      <td>2.561400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>752</td>\n",
              "      <td>2.641500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>753</td>\n",
              "      <td>2.828900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>754</td>\n",
              "      <td>3.028500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>755</td>\n",
              "      <td>2.700700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>756</td>\n",
              "      <td>3.524700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>757</td>\n",
              "      <td>3.103200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>758</td>\n",
              "      <td>3.091000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>759</td>\n",
              "      <td>3.168600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>760</td>\n",
              "      <td>2.979500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>761</td>\n",
              "      <td>2.416100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>762</td>\n",
              "      <td>2.771600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>763</td>\n",
              "      <td>2.849200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>764</td>\n",
              "      <td>3.016500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>765</td>\n",
              "      <td>3.132000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>766</td>\n",
              "      <td>2.845900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>767</td>\n",
              "      <td>2.831800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>768</td>\n",
              "      <td>2.545000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>769</td>\n",
              "      <td>3.201500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>770</td>\n",
              "      <td>2.781900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>771</td>\n",
              "      <td>2.739600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>772</td>\n",
              "      <td>3.212200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>773</td>\n",
              "      <td>3.128600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>774</td>\n",
              "      <td>2.804800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>775</td>\n",
              "      <td>2.703000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>776</td>\n",
              "      <td>3.240800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>777</td>\n",
              "      <td>3.149500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>778</td>\n",
              "      <td>2.921100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>779</td>\n",
              "      <td>2.439900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>780</td>\n",
              "      <td>2.754700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>781</td>\n",
              "      <td>3.272000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>782</td>\n",
              "      <td>3.190100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>783</td>\n",
              "      <td>3.021800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>784</td>\n",
              "      <td>2.826800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>785</td>\n",
              "      <td>2.249000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>786</td>\n",
              "      <td>2.658200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>787</td>\n",
              "      <td>2.853200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>788</td>\n",
              "      <td>2.603600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>789</td>\n",
              "      <td>3.232500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>790</td>\n",
              "      <td>2.889000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>791</td>\n",
              "      <td>3.198200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>792</td>\n",
              "      <td>2.898100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>793</td>\n",
              "      <td>3.257700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>794</td>\n",
              "      <td>2.753600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>795</td>\n",
              "      <td>2.818000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>796</td>\n",
              "      <td>2.764900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>797</td>\n",
              "      <td>2.559500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>798</td>\n",
              "      <td>3.015700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>799</td>\n",
              "      <td>3.041100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>800</td>\n",
              "      <td>2.851800</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "trainer_stats = trainer.train()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OGlCocyEzNf2"
      },
      "outputs": [],
      "source": [
        "wandb.finish()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "pCqnaKmlO1U9",
        "outputId": "334f36ca-e00b-44c1-fead-1e7e36b32c82"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "3519.3926 seconds used for training.\n",
            "58.66 minutes used for training.\n",
            "Peak reserved memory = 9.055 GB.\n",
            "Peak reserved memory for training = 1.963 GB.\n",
            "Peak reserved memory % of max memory = 40.847 %.\n",
            "Peak reserved memory for training % of max memory = 8.855 %.\n"
          ]
        }
      ],
      "source": [
        "#@title Show final memory and time stats\n",
        "used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n",
        "used_memory_for_lora = round(used_memory - start_gpu_memory, 3)\n",
        "used_percentage = round(used_memory         /max_memory*100, 3)\n",
        "lora_percentage = round(used_memory_for_lora/max_memory*100, 3)\n",
        "print(f\"{trainer_stats.metrics['train_runtime']} seconds used for training.\")\n",
        "print(f\"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.\")\n",
        "print(f\"Peak reserved memory = {used_memory} GB.\")\n",
        "print(f\"Peak reserved memory for training = {used_memory_for_lora} GB.\")\n",
        "print(f\"Peak reserved memory % of max memory = {used_percentage} %.\")\n",
        "print(f\"Peak reserved memory for training % of max memory = {lora_percentage} %.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ekOmTR1hSNcr"
      },
      "source": [
        "<a name=\"Inference\"></a>\n",
        "### Inference\n",
        "Let's run the model! You can change the instruction and input - leave the output blank!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "kR3gIAX-SM2q",
        "outputId": "4bb7ebbf-99c3-40e8-c25c-88ef6951ef7c"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "['Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\nThe text given have gramatical and synthaxe error. Rewrite it and make it understadable. \\n\\n### Input:\\nTHHJ C M A 14 A1 HAM p 0 _ _ THE OMAHA DAILY BEE , TUEBPAY , JUNE\\n\\n### Response:\\ne, and the\\nother two were in the rear. The\\ncar was thrown off the track and\\nturned over, and the occupants were\\nthrown out. The car was badly\\ndamaged, but the occupants were\\nnot injured. The train was\\ndelayed about an hour.\\nThe Omaha & Council Bluffs\\nrailroad company has been incorporated\\nunder the laws of the state of Iowa.\\nThe Omaha & Council Bluffs railroad\\ncompany has been incorporated under the laws of the state of Iowa.\\nThe Omaha & Council Bluffs railroad\\ncompany has been incorporated under the laws of the state of Iowa.\\nThe Omaha &<|endoftext|>']"
            ]
          },
          "execution_count": 32,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# alpaca_prompt = Copied from above\n",
        "FastLanguageModel.for_training(model) # Working on fixing 2x faster inference!\n",
        "tokenizer.pad_token_id=tokenizer.eos_token_id\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"The text given have gramatical and synthaxe error. Rewrite it and make it understadable. \", # instruction\n",
        "        \"THHJ C M A 14 A1 HAM p 0 _ _ THE OMAHA DAILY BEE , TUEBPAY , JUNE\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "outputs = model.generate(**inputs, max_new_tokens = 128,use_cache = True)\n",
        "tokenizer.batch_decode(outputs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CrSvZObor0lY"
      },
      "source": [
        " You can also use a `TextStreamer` for continuous inference - so you can see the generation token by token, instead of waiting the whole time!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e2pEuRb1r2Vg",
        "outputId": "0bcab74a-274a-412f-d171-ed44f80aa99f"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
            "\n",
            "### Instruction:\n",
            "Continue the fibonnaci sequence.\n",
            "\n",
            "### Input:\n",
            "1, 1, 2, 3, 5, 8\n",
            "\n",
            "### Response:\n",
            ", 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765, 10946, 17711, 28657, 46368, 75025, 1213\n"
          ]
        }
      ],
      "source": [
        "# alpaca_prompt = Copied from above\n",
        "FastLanguageModel.for_training(model) # Working on fixing 2x faster inference!\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"Continue the fibonnaci sequence.\", # instruction\n",
        "        \"1, 1, 2, 3, 5, 8\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "text_streamer = TextStreamer(tokenizer)\n",
        "_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "svUaoU7xfMRt",
        "outputId": "86f1f130-6d72-4253-90b9-1eb0e8a4d03e"
      },
      "outputs": [
        {
          "ename": "SyntaxError",
          "evalue": "invalid decimal literal (123660628.py, line 1)",
          "output_type": "error",
          "traceback": [
            "\u001b[0;36m  Cell \u001b[0;32mIn[28], line 1\u001b[0;36m\u001b[0m\n\u001b[0;31m    82/82=63.85k,+=502385\u001b[0m\n\u001b[0m              ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid decimal literal\n"
          ]
        }
      ],
      "source": [
        "82/82=63.85k,+=502385"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uMuVrWbjAzhc"
      },
      "source": [
        "<a name=\"Save\"></a>\n",
        "### Saving, loading finetuned models\n",
        "To save the final model as LoRA adapters, either use Huggingface's `push_to_hub` for an online save or `save_pretrained` for a local save.\n",
        "\n",
        "**[NOTE]** This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "upcOlWe7A1vc",
        "outputId": "7ad9dc9d-f9aa-4c85-8ddb-8f38825eeca4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "('lora_model/tokenizer_config.json',\n",
              " 'lora_model/special_tokens_map.json',\n",
              " 'lora_model/vocab.json',\n",
              " 'lora_model/merges.txt',\n",
              " 'lora_model/added_tokens.json',\n",
              " 'lora_model/tokenizer.json')"
            ]
          },
          "execution_count": 34,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.save_pretrained(\"lora_model\") # Local saving\n",
        "tokenizer.save_pretrained(\"lora_model\")\n",
        "# model.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving\n",
        "# tokenizer.push_to_hub(\"your_name/lora_model\", token = \"...\") # Online saving"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AEEcJ4qfC7Lp"
      },
      "source": [
        "Now if you want to load the LoRA adapters we just saved for inference, set `False` to `True`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MKX_XKs_BNZR",
        "outputId": "e6c13876-4fba-49c6-c959-00c384cfdbf2"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:151643 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
            "\n",
            "### Instruction:\n",
            "The text given have gramatical and synthaxe error. Rewrite it and make it understadable. \n",
            "\n",
            "### Input:\n",
            "THHJ C M A 14 A1 HAM p 0 _ _ THE OMAHA DAILY BEE , TUEBPAY , JUNE\n",
            "\n",
            "### Response:\n",
            "e, and the\n",
            "other two were in the rear. The\n",
            "car was thrown off the track and\n",
            "turned over, and the occupants were\n",
            "thrown out. The car was badly\n",
            "damaged, but the occupants were\n",
            "not injured. The train was\n",
            "delayed about an hour.\n",
            "The Omaha & Council Bl\n"
          ]
        }
      ],
      "source": [
        "if False:\n",
        "    from unsloth import FastLanguageModel\n",
        "    model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "        model_name = \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n",
        "        max_seq_length = max_seq_length,\n",
        "        dtype = dtype,\n",
        "        load_in_4bit = load_in_4bit,\n",
        "    )\n",
        "    FastLanguageModel.for_training(model) # Working on fixing 2x faster inference!\n",
        "\n",
        "# alpaca_prompt = You MUST copy from above!\n",
        "FastLanguageModel.for_training(model) # Working on fixing 2x faster inference!\n",
        "inputs = tokenizer(\n",
        "[\n",
        "    alpaca_prompt.format(\n",
        "        \"The text given have gramatical and synthaxe error. Rewrite it and make it understadable. \", # instruction\n",
        "        \"THHJ C M A 14 A1 HAM p 0 _ _ THE OMAHA DAILY BEE , TUEBPAY , JUNE\", # input\n",
        "        \"\", # output - leave this blank for generation!\n",
        "    )\n",
        "], return_tensors = \"pt\").to(\"cuda\")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "text_streamer = TextStreamer(tokenizer)\n",
        "_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 64)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QQMjaNrjsU5_"
      },
      "source": [
        "You can also use Hugging Face's `AutoModelForPeftCausalLM`. Only use this if you do not have `unsloth` installed. It can be hopelessly slow, since `4bit` model downloading is not supported, and Unsloth's **inference is 2x faster**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yFfaXG0WsQuE"
      },
      "outputs": [],
      "source": [
        "if False:\n",
        "    # I highly do NOT suggest - use Unsloth if possible\n",
        "    from peft import AutoPeftModelForCausalLM\n",
        "    from transformers import AutoTokenizer\n",
        "    model = AutoPeftModelForCausalLM.from_pretrained(\n",
        "        \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n",
        "        load_in_4bit = load_in_4bit,\n",
        "    )\n",
        "    tokenizer = AutoTokenizer.from_pretrained(\"lora_model\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f422JgM9sdVT"
      },
      "source": [
        "### Saving to float16 for VLLM\n",
        "\n",
        "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `merged_4bit` for int4. We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iHjt_SMYsd3P"
      },
      "outputs": [],
      "source": [
        "# Merge to 16bit\n",
        "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_16bit\",)\n",
        "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_16bit\", token = \"\")\n",
        "\n",
        "# Merge to 4bit\n",
        "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_4bit\",)\n",
        "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_4bit\", token = \"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\")\n",
        "\n",
        "# Just LoRA adapters\n",
        "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"lora\",)\n",
        "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"lora\", token = \"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\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCv4vXHd61i7"
      },
      "source": [
        "### GGUF / llama.cpp Conversion\n",
        "To save to `GGUF` / `llama.cpp`, we support it natively now! We clone `llama.cpp` and we default save it to `q8_0`. We allow all methods like `q4_k_m`. Use `save_pretrained_gguf` for local saving and `push_to_hub_gguf` for uploading to HF.\n",
        "\n",
        "Some supported quant methods (full list on our [Wiki page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n",
        "* `q8_0` - Fast conversion. High resource use, but generally acceptable.\n",
        "* `q4_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K.\n",
        "* `q5_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FqfebeAdT073"
      },
      "outputs": [],
      "source": [
        "# Save to 8bit Q8_0\n",
        "if False: model.save_pretrained_gguf(\"model\", tokenizer,)\n",
        "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, token = \"\")\n",
        "\n",
        "# Save to 16bit GGUF\n",
        "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"f16\")\n",
        "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"f16\", token = \"\")\n",
        "\n",
        "# Save to q4_k_m GGUF\n",
        "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"q4_k_m\")\n",
        "if False: model.push_to_hub_gguf(\"hf/model\", tokenizer, quantization_method = \"q4_k_m\", token = \"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bDp0zNpwe6U_"
      },
      "source": [
        "Now, use the `model-unsloth.gguf` file or `model-unsloth-Q4_K_M.gguf` file in `llama.cpp` or a UI based system like `GPT4All`. You can install GPT4All by going [here](https://gpt4all.io/index.html)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zt9CHJqO6p30"
      },
      "source": [
        "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/u54VK8m8tk) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
        "\n",
        "Some other links:\n",
        "1. Zephyr DPO 2x faster [free Colab](https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing)\n",
        "2. Llama 7b 2x faster [free Colab](https://colab.research.google.com/drive/1lBzz5KeZJKXjvivbYvmGarix9Ao6Wxe5?usp=sharing)\n",
        "3. TinyLlama 4x faster full Alpaca 52K in 1 hour [free Colab](https://colab.research.google.com/drive/1AZghoNBQaMDgWJpi4RbffGM1h6raLUj9?usp=sharing)\n",
        "4. CodeLlama 34b 2x faster [A100 on Colab](https://colab.research.google.com/drive/1y7A0AxE3y8gdj4AVkl2aZX47Xu3P1wJT?usp=sharing)\n",
        "5. Mistral 7b [free Kaggle version](https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook)\n",
        "6. We also did a [blog](https://huggingface.co/blog/unsloth-trl) with 🤗 HuggingFace, and we're in the TRL [docs](https://huggingface.co/docs/trl/main/en/sft_trainer#accelerate-fine-tuning-2x-using-unsloth)!\n",
        "7. `ChatML` for ShareGPT datasets, [conversational notebook](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)\n",
        "8. Text completions like novel writing [notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing)\n",
        "9. [**NEW**] We make Phi-3 Medium / Mini **2x faster**! See our [Phi-3 Medium notebook](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing)\n",
        "10. [**NEW**] We make Llama-3 8b, 70b **2x faster**! See our [Llama-3 8b notebook](https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing)\n",
        "\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://github.com/unslothai/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
        "  <a href=\"https://discord.gg/u54VK8m8tk\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
        "  <a href=\"https://ko-fi.com/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Kofi button.png\" width=\"145\"></a></a> Support our work if you can! Thanks!\n",
        "</div>"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.10"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "01d204c64942434190bceaf709f5f931": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "02ed31374bf64bf4be92252f0a1778dc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "030f9e33f584417186dbec2771b4d1b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_28b7406dd9d14401961dd85684398e9c",
              "IPY_MODEL_5cf4435e5bf24817991572defb5405aa",
              "IPY_MODEL_a4f17ec7c2bc45eb8f0da08f498685b0"
            ],
            "layout": "IPY_MODEL_4031b7b5c97545dea5a5158b3d8b5372"
          }
        },
        "03ed126b633c479483dfc3eb835b0405": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "04ce495c6fcb4b90abcb5b551775b42d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "06b6c5a916294ba1923b6059dd8392ff": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "07511b292577461da25e13729a23590c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b86ee15fc1b4a31b5d3cb3d5ffba0ce",
            "max": 80,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1445d964582a412ea62331322128a90c",
            "value": 80
          }
        },
        "075c339cc40d41139f5c3ddfe769b6b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_28ac7bb29bf94dedb0f9c9ea034a961e",
            "placeholder": "​",
            "style": "IPY_MODEL_fe10dcb7bd7f479784353c4cbc701878",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "0c975c8b2030416a8cd687175c40612d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8c9a2f08a4df45efa2312ed0e9a4cfb1",
              "IPY_MODEL_a7a7d8fcf7eb4df2bcb1b585cb145c42",
              "IPY_MODEL_9c34686459a64ff2bce9140f2abe9f6c"
            ],
            "layout": "IPY_MODEL_534c1b2ffc7843758adb4cb62e9c8131"
          }
        },
        "0d1f94b5f3ce4f15afd17a6313a1f420": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0ffeef7b0a2a4457b3b44e825f2bcf05": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_03ed126b633c479483dfc3eb835b0405",
            "max": 2776833,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_44d87d579cab4d51965ee47cdb48b171",
            "value": 2776833
          }
        },
        "1445d964582a412ea62331322128a90c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "189035e02ce54bd395ea11991d6a487b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1c5206dfea7443379635ce0f462f5a7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c1fee2a44f2d40cbab06bf4232c357a8",
            "placeholder": "​",
            "style": "IPY_MODEL_c9fba9eb5fac40a286a3c77a01cd95a1",
            "value": " 5.55G/5.55G [00:27&lt;00:00, 91.5MB/s]"
          }
        },
        "1ca9fef0577a45948ae75f90be59a70e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1d19743955844705919a5a0d3fd697b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1ead678d6c8c4edf9f220a4c8ec5e638": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "214bb87d7d25411abed64de2352216fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2617fc4a85184b1dbbf4033c5484ad64": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "28ac7bb29bf94dedb0f9c9ea034a961e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "28b7406dd9d14401961dd85684398e9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_02ed31374bf64bf4be92252f0a1778dc",
            "placeholder": "​",
            "style": "IPY_MODEL_7650642c89c24210807d8f47952b454b",
            "value": "generation_config.json: 100%"
          }
        },
        "2a3c4acb149e49d59339fd1261c2a5e2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3260b7232dec49e5bbb603e3d5eabe6e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a9bbc5894b3d424496c976e86b21e935",
            "placeholder": "​",
            "style": "IPY_MODEL_a7e05c492b7c4ba4be12f5ec6a4c05d0",
            "value": "model.safetensors: 100%"
          }
        },
        "3894df9a3a9949d6bcb628d53ffbea6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ef81a04741274d23bb5f05d968b0adb0",
              "IPY_MODEL_0ffeef7b0a2a4457b3b44e825f2bcf05",
              "IPY_MODEL_6ed1604e47ca4dcdb465a5839bab83e8"
            ],
            "layout": "IPY_MODEL_b951792c8d2e4083935f3b3c3ba46c7c"
          }
        },
        "3d85bb349f784df797fe5bf127f48f6a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4031b7b5c97545dea5a5158b3d8b5372": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "43d449028abc4b919448dc7413ec6590": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "44d87d579cab4d51965ee47cdb48b171": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "45b959fa51904484bd9f1fb4dccf9a02": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "45ccd999b65a4b3683a3631811d225b7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4808ffbbf36840c9a4b4a5f299fecb87": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4c52d7d1f44c493ebe49ddfbb2233f07": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "503f98f7d9c34b0e949737d7562f6d0a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50e3d52bb7d945dbb5c4613d839c0546": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_075c339cc40d41139f5c3ddfe769b6b1",
              "IPY_MODEL_87dd0e57fb724a6ca4fa5c1e280cf99b",
              "IPY_MODEL_ad8a89357f94402ca570c9c8e63a7e52"
            ],
            "layout": "IPY_MODEL_3d85bb349f784df797fe5bf127f48f6a"
          }
        },
        "51644fc5c4de4f508757cf747547d650": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "51dee8d0c6dd48aa875f4fc4eb07bc0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2617fc4a85184b1dbbf4033c5484ad64",
            "placeholder": "​",
            "style": "IPY_MODEL_aa55a840ad0b42739f00148f0a6d03ca",
            "value": "added_tokens.json: 100%"
          }
        },
        "534c1b2ffc7843758adb4cb62e9c8131": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "54f2803db94b412db0215ab460cf5796": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "597d5cd5e96745988a5f86cb9479e57d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5abeb01d19d047b0adc34aeacdd7e009": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_69910647d9ff45318131af41fdad6949",
              "IPY_MODEL_f6f1fa54e754444baffd11bcc360a24b",
              "IPY_MODEL_6059958a5ead476cabda576c30d98a4e"
            ],
            "layout": "IPY_MODEL_0d1f94b5f3ce4f15afd17a6313a1f420"
          }
        },
        "5b563f417adf4eae907838ac617b61cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5cf4435e5bf24817991572defb5405aa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2a3c4acb149e49d59339fd1261c2a5e2",
            "max": 117,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c9c26af3974f4fd38b24da608278feeb",
            "value": 117
          }
        },
        "6059958a5ead476cabda576c30d98a4e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e0151151ff5145739f7e680c9c3bc0bc",
            "placeholder": "​",
            "style": "IPY_MODEL_597d5cd5e96745988a5f86cb9479e57d",
            "value": " 370/370 [00:00&lt;00:00, 26.8kB/s]"
          }
        },
        "63c548abf6714edfbb238e1489d89271": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a025d355cfad459793de1370917311f3",
            "placeholder": "​",
            "style": "IPY_MODEL_6aeaa8ba15fc4553a78f971aaf5c9b6f",
            "value": "config.json: 100%"
          }
        },
        "665f561628db42869cee4eb7fbbe23cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "695afe6ce1474018ab72d546bcecfa95": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "69910647d9ff45318131af41fdad6949": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_43d449028abc4b919448dc7413ec6590",
            "placeholder": "​",
            "style": "IPY_MODEL_dd8ed98d098346d884ce815e04b204c5",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "6aeaa8ba15fc4553a78f971aaf5c9b6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6b55332f538a49b78423cfe7a848d330": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_51dee8d0c6dd48aa875f4fc4eb07bc0f",
              "IPY_MODEL_07511b292577461da25e13729a23590c",
              "IPY_MODEL_8dd1f70feca14569844a11e5a5769aa9"
            ],
            "layout": "IPY_MODEL_ffb6897d76114aa3ac91fd4ece67a8e7"
          }
        },
        "6b86ee15fc1b4a31b5d3cb3d5ffba0ce": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6ed1604e47ca4dcdb465a5839bab83e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_45ccd999b65a4b3683a3631811d225b7",
            "placeholder": "​",
            "style": "IPY_MODEL_665f561628db42869cee4eb7fbbe23cb",
            "value": " 2.78M/2.78M [00:00&lt;00:00, 5.72MB/s]"
          }
        },
        "6fef4f30afcf4f9ba8a18eb9f3851ba6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7276b9ac4012476c9526e647f1ecca0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9e13417fe7f34aeda74bbbf960015162",
              "IPY_MODEL_798d13daf6034191a8b5ee0cc5120311",
              "IPY_MODEL_c490f30af0814201bd6ac10cac782343"
            ],
            "layout": "IPY_MODEL_503f98f7d9c34b0e949737d7562f6d0a"
          }
        },
        "7650642c89c24210807d8f47952b454b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "798d13daf6034191a8b5ee0cc5120311": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e1805e4085cc4e59a6d470990ddbf327",
            "max": 1671853,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_695afe6ce1474018ab72d546bcecfa95",
            "value": 1671853
          }
        },
        "7c46d51adc9f4444983f6201320a0bf3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "87dd0e57fb724a6ca4fa5c1e280cf99b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a32cb7bd6a124800b2c0756ac49e9098",
            "max": 1327,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_54f2803db94b412db0215ab460cf5796",
            "value": 1327
          }
        },
        "8c9a2f08a4df45efa2312ed0e9a4cfb1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b232bbd78e4a49868051b0673ef76f60",
            "placeholder": "​",
            "style": "IPY_MODEL_b06914a13bff410f80683416780e4e95",
            "value": "tokenizer.json: 100%"
          }
        },
        "8d0d0f8d01a6421bbb868103200ff85a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e7e20df38ba34de69a86412b32c4a1dd",
            "max": 5547254651,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b003af4f20364eb1a535d4f9c71f74b7",
            "value": 5547254651
          }
        },
        "8dd1f70feca14569844a11e5a5769aa9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_06b6c5a916294ba1923b6059dd8392ff",
            "placeholder": "​",
            "style": "IPY_MODEL_45b959fa51904484bd9f1fb4dccf9a02",
            "value": " 80.0/80.0 [00:00&lt;00:00, 5.40kB/s]"
          }
        },
        "908dfb6d6cf74986ab7e3136bb50786e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6fef4f30afcf4f9ba8a18eb9f3851ba6",
            "max": 1181,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_01d204c64942434190bceaf709f5f931",
            "value": 1181
          }
        },
        "90959101a5704914ab0124283cd6148f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_63c548abf6714edfbb238e1489d89271",
              "IPY_MODEL_908dfb6d6cf74986ab7e3136bb50786e",
              "IPY_MODEL_bbf9174351b047db96676d6149ab4e56"
            ],
            "layout": "IPY_MODEL_51644fc5c4de4f508757cf747547d650"
          }
        },
        "970be236ebb345e7a1e51f6b71c67402": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "99b0415ead364e87b903709767727d1e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9c34686459a64ff2bce9140f2abe9f6c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b9456b4e62744896903cbd0ba9478772",
            "placeholder": "​",
            "style": "IPY_MODEL_04ce495c6fcb4b90abcb5b551775b42d",
            "value": " 7.03M/7.03M [00:00&lt;00:00, 13.7MB/s]"
          }
        },
        "9e13417fe7f34aeda74bbbf960015162": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4c52d7d1f44c493ebe49ddfbb2233f07",
            "placeholder": "​",
            "style": "IPY_MODEL_5b563f417adf4eae907838ac617b61cb",
            "value": "merges.txt: 100%"
          }
        },
        "a025d355cfad459793de1370917311f3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a32cb7bd6a124800b2c0756ac49e9098": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4f17ec7c2bc45eb8f0da08f498685b0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ca9fef0577a45948ae75f90be59a70e",
            "placeholder": "​",
            "style": "IPY_MODEL_4808ffbbf36840c9a4b4a5f299fecb87",
            "value": " 117/117 [00:00&lt;00:00, 9.51kB/s]"
          }
        },
        "a7a7d8fcf7eb4df2bcb1b585cb145c42": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e56e205549c54fce8f5ba1df6f2e00b4",
            "max": 7028043,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_214bb87d7d25411abed64de2352216fa",
            "value": 7028043
          }
        },
        "a7e05c492b7c4ba4be12f5ec6a4c05d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a9bbc5894b3d424496c976e86b21e935": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa55a840ad0b42739f00148f0a6d03ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ad8a89357f94402ca570c9c8e63a7e52": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ead678d6c8c4edf9f220a4c8ec5e638",
            "placeholder": "​",
            "style": "IPY_MODEL_970be236ebb345e7a1e51f6b71c67402",
            "value": " 1.33k/1.33k [00:00&lt;00:00, 91.4kB/s]"
          }
        },
        "b003af4f20364eb1a535d4f9c71f74b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b06914a13bff410f80683416780e4e95": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b232bbd78e4a49868051b0673ef76f60": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9456b4e62744896903cbd0ba9478772": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b951792c8d2e4083935f3b3c3ba46c7c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bbac5270c6a94ea7b303d6ae94094a8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3260b7232dec49e5bbb603e3d5eabe6e",
              "IPY_MODEL_8d0d0f8d01a6421bbb868103200ff85a",
              "IPY_MODEL_1c5206dfea7443379635ce0f462f5a7b"
            ],
            "layout": "IPY_MODEL_e4c0da26192a47ec9afc61be919406e7"
          }
        },
        "bbf9174351b047db96676d6149ab4e56": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_99b0415ead364e87b903709767727d1e",
            "placeholder": "​",
            "style": "IPY_MODEL_1d19743955844705919a5a0d3fd697b3",
            "value": " 1.18k/1.18k [00:00&lt;00:00, 50.8kB/s]"
          }
        },
        "c1fee2a44f2d40cbab06bf4232c357a8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c490f30af0814201bd6ac10cac782343": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cd916fdc68cb46b9944a2ed9b77c9092",
            "placeholder": "​",
            "style": "IPY_MODEL_7c46d51adc9f4444983f6201320a0bf3",
            "value": " 1.67M/1.67M [00:00&lt;00:00, 4.11MB/s]"
          }
        },
        "c9c26af3974f4fd38b24da608278feeb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c9fba9eb5fac40a286a3c77a01cd95a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cd916fdc68cb46b9944a2ed9b77c9092": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d31b9cce122e445d850e082be42f6025": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd8ed98d098346d884ce815e04b204c5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "de3846b1e677437586e96d02196a91d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e0151151ff5145739f7e680c9c3bc0bc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e1805e4085cc4e59a6d470990ddbf327": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e4c0da26192a47ec9afc61be919406e7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e56e205549c54fce8f5ba1df6f2e00b4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e7e20df38ba34de69a86412b32c4a1dd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ef81a04741274d23bb5f05d968b0adb0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d31b9cce122e445d850e082be42f6025",
            "placeholder": "​",
            "style": "IPY_MODEL_de3846b1e677437586e96d02196a91d3",
            "value": "vocab.json: 100%"
          }
        },
        "f2b193b6a6ef4afb94b7b2f984aa6d70": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f6f1fa54e754444baffd11bcc360a24b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_189035e02ce54bd395ea11991d6a487b",
            "max": 370,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f2b193b6a6ef4afb94b7b2f984aa6d70",
            "value": 370
          }
        },
        "fe10dcb7bd7f479784353c4cbc701878": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ffb6897d76114aa3ac91fd4ece67a8e7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}