SimCSE_vs_fast_sentence_embeddings.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2x59HFpU5sRM"
      },
      "source": [
        "# Sim-CSE vs Fast Sentence Embeddings \n",
        "\n",
        "Running the STS benchmark for the [SimCSE](https://github.com/princeton-nlp/SimCSE) model and the [Fast Sentence Embeddings ](https://github.com/oborchers/Fast_Sentence_Embeddings)model.\n",
        "\n",
        "The benchmark is the [STS benchmark](http://ixa2.si.ehu.eus/stswiki/index.php/STSbenchmark) and scripts from the benchmark to run Pearson correlation, have been used here.\n",
        "\n",
        "Files have been downloaded from the STS benchmark site. We only take the sts-dev file to run the benchmark.\n",
        "\n",
        "**SimCSE** scores around **65.98** while **FSE** scores around **45.5** for *glove-wiki-gigaword-50* and around **63.135** for *paranmt-300* and using the  *Averages* model.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "qhp4mQijLXXO",
        "outputId": "f900c796-dfc3-45c4-9e1b-1fdf7d4ce7b5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting simcse\n",
            "  Downloading simcse-0.4.tar.gz (18 kB)\n",
            "Collecting transformers\n",
            "  Downloading transformers-4.21.2-py3-none-any.whl (4.7 MB)\n",
            "\u001b[K     |████████████████████████████████| 4.7 MB 53.4 MB/s \n",
            "\u001b[?25hCollecting datasets\n",
            "  Downloading datasets-2.4.0-py3-none-any.whl (365 kB)\n",
            "\u001b[K     |████████████████████████████████| 365 kB 68.0 MB/s \n",
            "\u001b[?25hCollecting accelerate\n",
            "  Downloading accelerate-0.12.0-py3-none-any.whl (143 kB)\n",
            "\u001b[K     |████████████████████████████████| 143 kB 71.8 MB/s \n",
            "\u001b[?25hCollecting nvidia-ml-py3\n",
            "  Downloading nvidia-ml-py3-7.352.0.tar.gz (19 kB)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from simcse) (4.64.0)\n",
            "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from simcse) (1.0.2)\n",
            "Collecting scipy<1.6,>=1.5.4\n",
            "  Downloading scipy-1.5.4-cp37-cp37m-manylinux1_x86_64.whl (25.9 MB)\n",
            "\u001b[K     |████████████████████████████████| 25.9 MB 78.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from simcse) (1.12.1+cu113)\n",
            "Collecting numpy<1.20,>=1.19.5\n",
            "  Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8 MB)\n",
            "\u001b[K     |████████████████████████████████| 14.8 MB 57.2 MB/s \n",
            "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from simcse) (57.4.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Collecting tokenizers!=0.11.3,<0.13,>=0.11.1\n",
            "  Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB)\n",
            "\u001b[K     |████████████████████████████████| 6.6 MB 54.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (6.0)\n",
            "Collecting huggingface-hub<1.0,>=0.1.0\n",
            "  Downloading huggingface_hub-0.9.1-py3-none-any.whl (120 kB)\n",
            "\u001b[K     |████████████████████████████████| 120 kB 69.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.8.0)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.12.0)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2022.6.2)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.1.1)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
            "Requirement already satisfied: dill<0.3.6 in /usr/local/lib/python3.7/dist-packages (from datasets) (0.3.5.1)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from datasets) (1.3.5)\n",
            "Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets) (6.0.1)\n",
            "Collecting responses<0.19\n",
            "  Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n",
            "Collecting xxhash\n",
            "  Downloading xxhash-3.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)\n",
            "\u001b[K     |████████████████████████████████| 212 kB 74.9 MB/s \n",
            "\u001b[?25hRequirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from datasets) (3.8.1)\n",
            "Collecting multiprocess\n",
            "  Downloading multiprocess-0.70.13-py37-none-any.whl (115 kB)\n",
            "\u001b[K     |████████████████████████████████| 115 kB 71.3 MB/s \n",
            "\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.7/dist-packages (from datasets) (2022.7.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2022.6.15)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)\n",
            "Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1\n",
            "  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)\n",
            "\u001b[K     |████████████████████████████████| 127 kB 60.1 MB/s \n",
            "\u001b[?25hRequirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from accelerate) (5.4.8)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.8.1)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (4.0.2)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (6.0.2)\n",
            "Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (2.1.1)\n",
            "Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (0.13.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.2.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (22.1.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.8.1)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->datasets) (2022.2.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->simcse) (1.1.0)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->simcse) (3.1.0)\n",
            "Building wheels for collected packages: simcse, nvidia-ml-py3\n",
            "  Building wheel for simcse (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for simcse: filename=simcse-0.4-py3-none-any.whl size=15033 sha256=14fb418aafc72b4fb57157f53209c43940aab777836953cac0dfc33861dc7393\n",
            "  Stored in directory: /root/.cache/pip/wheels/29/b0/58/448de751ccbe2a0e75c054e34416e7a5a1aead371d5d163450\n",
            "  Building wheel for nvidia-ml-py3 (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for nvidia-ml-py3: filename=nvidia_ml_py3-7.352.0-py3-none-any.whl size=19190 sha256=7c846188ff6a1a1a960f25bf940c20c95e9f381d3f0317047424f225ffe492cf\n",
            "  Stored in directory: /root/.cache/pip/wheels/df/99/da/c34f202dc8fd1dffd35e0ecf1a7d7f8374ca05fbcbaf974b83\n",
            "Successfully built simcse nvidia-ml-py3\n",
            "Installing collected packages: urllib3, numpy, tokenizers, scipy, huggingface-hub, xxhash, transformers, responses, multiprocess, simcse, nvidia-ml-py3, datasets, accelerate\n",
            "  Attempting uninstall: urllib3\n",
            "    Found existing installation: urllib3 1.24.3\n",
            "    Uninstalling urllib3-1.24.3:\n",
            "      Successfully uninstalled urllib3-1.24.3\n",
            "  Attempting uninstall: numpy\n",
            "    Found existing installation: numpy 1.21.6\n",
            "    Uninstalling numpy-1.21.6:\n",
            "      Successfully uninstalled numpy-1.21.6\n",
            "  Attempting uninstall: scipy\n",
            "    Found existing installation: scipy 1.7.3\n",
            "    Uninstalling scipy-1.7.3:\n",
            "      Successfully uninstalled scipy-1.7.3\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "xarray-einstats 0.2.2 requires numpy>=1.21, but you have numpy 1.19.5 which is incompatible.\n",
            "tensorflow 2.8.2+zzzcolab20220719082949 requires numpy>=1.20, but you have numpy 1.19.5 which is incompatible.\n",
            "cmdstanpy 1.0.7 requires numpy>=1.21, but you have numpy 1.19.5 which is incompatible.\u001b[0m\n",
            "Successfully installed accelerate-0.12.0 datasets-2.4.0 huggingface-hub-0.9.1 multiprocess-0.70.13 numpy-1.19.5 nvidia-ml-py3-7.352.0 responses-0.18.0 scipy-1.5.4 simcse-0.4 tokenizers-0.12.1 transformers-4.21.2 urllib3-1.25.11 xxhash-3.0.0\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "numpy"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install simcse transformers datasets accelerate nvidia-ml-py3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "SD4jIXnpM1R-"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import torch\n",
        "import csv\n",
        "\n",
        "\n",
        "\n",
        "df = pd.read_csv(\"sts-dev-new.csv\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "Jy3mCPP2LtsI"
      },
      "outputs": [],
      "source": [
        "from pynvml import *\n",
        "\n",
        "\n",
        "def print_gpu_utilization():\n",
        "    nvmlInit()\n",
        "    handle = nvmlDeviceGetHandleByIndex(0)\n",
        "    info = nvmlDeviceGetMemoryInfo(handle)\n",
        "    print(f\"GPU memory occupied: {info.used//1024**2} MB.\")\n",
        "\n",
        "\n",
        "def print_summary(result):\n",
        "    print(f\"Time: {result.metrics['train_runtime']:.2f}\")\n",
        "    print(f\"Samples/second: {result.metrics['train_samples_per_second']:.2f}\")\n",
        "    print_gpu_utilization()\n",
        "\n",
        "def cosine_similarity(n1):\n",
        "  with torch.no_grad():\n",
        "    cos = torch.nn.CosineSimilarity(dim=-1)\n",
        "    return cos(*n1) * 5\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317,
          "referenced_widgets": [
            "d8006ec1ce484b2db878efa2de803346",
            "4334fd8a1161434ca7b8e43c92b93995",
            "1cce355f121c43b98882524738eb9ce9",
            "a0f4fc1727d140f79e19cdec457fd395",
            "ac61e88d126149d7aec525043a473e1c",
            "003dd16d6f8d462db3041b5d8357bc36",
            "fd34ccc53bea4031bb0324b88cc55333",
            "8e1efd4eb87b44e8bd7f9b47783b44e1",
            "1dab331710b549488aab2b48bf0df477",
            "09e427231287419c90ca7ea2763dc9b6",
            "276bdee57f53446cae940b9ea1841d5a",
            "4d59af8a27de4ef6ad0b72fb0e9fb667",
            "334210826f2d447aa2a307802dcb740b",
            "74a4ab12824a4fa5b4d6d1a75fa32f23",
            "7e85cac165ab4a0db95208bea46aa87a",
            "41b122010cdf4b2fb77a0a210ea336ec",
            "13ce8aafcb764c1a9e62e24775744a13",
            "1fc63bf478694de6951563c7e896f9be",
            "2ca9725c94b94774aaf0bdb229b2a5e7",
            "08504531b202485aaa18ea068312dd51",
            "b362abe5cfa7435a8cad51e76c33971d",
            "13b855ba59694ce994c12a5007f85fa2",
            "be9ca588b16b49efa7fa20254df146c9",
            "37fe97af735a4d48a90f13a83bc9285d",
            "36e5b462fa7c414da7048f03b78f8075",
            "b0f81c177e8f46b3b0063659d0e6e603",
            "d355ebce9ef44c5ea20704f8a386ca6f",
            "5bbac811c4624e0ca9c6add3d67d1adb",
            "4d5a76e4166d4c368277951bf503e5f6",
            "e93b47091cc040a19baa7234400737b8",
            "116933b0cb8a42caafd96830cc607756",
            "f92dbe50635141ffa7810f07ecbede0c",
            "328e5c80e0bb4e7d91bcf52fea7c4eaf",
            "055510d446f14a289c774f73daa4c998",
            "1a74639222c245338c5e98589442d223",
            "720aeb8ba97a4914961199e615e7c31d",
            "5c3d340902844f90be1d4a81eb14642c",
            "c639e828361b4554b9b62991f9136c54",
            "e494279972be47de898169df3d55b748",
            "94488ec867944b2daa02b863fb217bfb",
            "fcab23c459e846f9af532cc51f018fff",
            "8b9927f8e01a416d9a24a10dd4cd6a0d",
            "6db521a8a1a845268e7d53769d13ec0c",
            "c0b70f8fa29d48d5a067dce22e7455fd",
            "9474bf1b7f2e41d6b7f02265ab211508",
            "a637c28d7f3449f7b90821702173dba1",
            "0606c62286ed49f39e031bc729358f12",
            "d13809843a1d498b987b8523971d0b0f",
            "8d448dac7e664c5790cc5bcbb5c1b72e",
            "567eef54b8264b8c9f581715f04a2b5f",
            "d2098375206f41bfada8829e15930be9",
            "1cb973e5fc3e41f2a517f6feca8b3059",
            "dc17fb3ce0504681af7bb884ca0b8987",
            "0b0621a877dc47818c0ac64b04ee4514",
            "fd06b534c85e4c02aea2400660b12e2f"
          ]
        },
        "id": "redwJyMRLvvh",
        "outputId": "b13253ab-c817-4a62-ee5d-786165b40d52"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d8006ec1ce484b2db878efa2de803346",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading tokenizer_config.json:   0%|          | 0.00/252 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4d59af8a27de4ef6ad0b72fb0e9fb667",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading config.json:   0%|          | 0.00/689 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "be9ca588b16b49efa7fa20254df146c9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading vocab.txt:   0%|          | 0.00/226k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "055510d446f14a289c774f73daa4c998",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading special_tokens_map.json:   0%|          | 0.00/112 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9474bf1b7f2e41d6b7f02265ab211508",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading pytorch_model.bin:   0%|          | 0.00/418M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import torch\n",
        "from transformers import AutoModel, AutoTokenizer\n",
        "\n",
        "\n",
        "# Import our models. The package will take care of downloading the models automatically\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"princeton-nlp/sup-simcse-bert-base-uncased\")\n",
        "model = AutoModel.from_pretrained(\"princeton-nlp/sup-simcse-bert-base-uncased\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "irbgHc5HLx3b",
        "outputId": "c070dd02-7d40-4cd6-eaf9-8bdad6bcaee7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "cuda:0\n",
            "GPU memory occupied: 1080 MB.\n"
          ]
        }
      ],
      "source": [
        "device = \"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n",
        "torch.cuda.empty_cache()\n",
        "print(device)\n",
        "model = model.to(device)\n",
        "print_gpu_utilization()\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 80,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KDd-dmmVOy_J",
        "outputId": "d9ce5bba-8137-484d-d569-8b679e0f74cc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1470\n",
            "1470\n",
            "err:  [764, 1024]\n"
          ]
        }
      ],
      "source": [
        "# preprocess dev set\n",
        "\n",
        "sent = df[[\"Sent1\", \"Sent2\"]].values\n",
        "score = df[\"Score\"]\n",
        "print(len(sent))\n",
        "nsent = []\n",
        "scores = []\n",
        "err = []\n",
        "for i in range(len(sent)):\n",
        "  try:\n",
        "    t = sent[i]\n",
        "    s1 = t[0].encode(\"utf-8\").decode()\n",
        "    s2 = t[1].encode(\"utf-8\").decode()\n",
        "  except Exception as e:\n",
        "    err.append(i)\n",
        "    nsent.append([\"SS\", \"SS\"])\n",
        "    scores.append(5.0)\n",
        "  else:\n",
        "    nsent.append([s1, s2])\n",
        "    scores.append(score[i])\n",
        "\n",
        "print(len(nsent))\n",
        "print(\"err: \", err)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 81,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jSB7FoRsUctr",
        "outputId": "b4a6ca57-ae86-4a92-f070-1f5c4fbfbc47"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[['A man with a hard hat is dancing.', 'A man wearing a hard hat is dancing.'],\n",
              " ['A young child is riding a horse.', 'A child is riding a horse.'],\n",
              " ['A man is feeding a mouse to a snake.',\n",
              "  'The man is feeding a mouse to the snake.'],\n",
              " ['A woman is playing the guitar.', 'A man is playing guitar.'],\n",
              " ['A woman is playing the flute.', 'A man is playing a flute.'],\n",
              " ['A woman is cutting an onion.', 'A man is cutting onions.'],\n",
              " ['A man is erasing a chalk board.', 'The man is erasing the chalk board.'],\n",
              " ['A woman is carrying a boy.', 'A woman is carrying her baby.'],\n",
              " ['Three men are playing guitars.', 'Three men are on stage playing guitars.'],\n",
              " ['A woman peels a potato.', 'A woman is peeling a potato.'],\n",
              " ['People are playing cricket.', 'Men are playing cricket.'],\n",
              " ['A man is playing a guitar.', 'A man is playing a flute.'],\n",
              " ['The cougar is chasing the bear.', 'A cougar is chasing a bear.'],\n",
              " ['The man cut down a tree with an axe.',\n",
              "  'A man chops down a tree with an axe.'],\n",
              " ['The man is playing the guitar.', 'A man is playing a guitar.'],\n",
              " ['A man is finding something.', 'A woman is slicing something.'],\n",
              " ['The girl sang into a microphone.', 'The lady sang into the microphone.'],\n",
              " ['A man is climbing a rope.', 'A man climbs a rope.'],\n",
              " ['Kittens are eating food.', 'Kittens are eating from dishes.'],\n",
              " ['A man is speaking.', 'A man is spitting.'],\n",
              " ['A woman is cooking eggs.', 'A woman is cooking something.'],\n",
              " ['A man is playing a guitar.', 'A man is playing a trumpet.'],\n",
              " ['An animal is chewing on something.',\n",
              "  'An animal is chewing on a key chain.'],\n",
              " ['A man is playing a flute.', 'A man is playing guitar.'],\n",
              " ['A man is playing soccer.', 'A man is playing flute.']]"
            ]
          },
          "execution_count": 81,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "nsent[:25]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 82,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pYsjKDrZO5Hn",
        "outputId": "b4952451-8ef9-4747-c09b-210c19450099"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0\n",
            "20\n",
            "40\n",
            "60\n",
            "80\n",
            "100\n",
            "120\n",
            "140\n",
            "160\n",
            "180\n",
            "200\n",
            "220\n",
            "240\n",
            "260\n",
            "280\n",
            "300\n",
            "320\n",
            "340\n",
            "360\n",
            "380\n",
            "400\n",
            "420\n",
            "440\n",
            "460\n",
            "480\n",
            "500\n",
            "520\n",
            "540\n",
            "560\n",
            "580\n",
            "600\n",
            "620\n",
            "640\n",
            "660\n",
            "680\n",
            "700\n",
            "720\n",
            "740\n",
            "760\n",
            "780\n",
            "800\n",
            "820\n",
            "840\n",
            "860\n",
            "880\n",
            "900\n",
            "920\n",
            "940\n",
            "960\n",
            "980\n",
            "1000\n",
            "1020\n",
            "1040\n",
            "1060\n",
            "1080\n",
            "1100\n",
            "1120\n",
            "1140\n",
            "1160\n",
            "1180\n",
            "1200\n",
            "1220\n",
            "1240\n",
            "1260\n",
            "1280\n",
            "1300\n",
            "1320\n",
            "1340\n",
            "1360\n",
            "1380\n",
            "1400\n",
            "1420\n",
            "1440\n",
            "1460\n"
          ]
        }
      ],
      "source": [
        "sc = []\n",
        "true_sc = []\n",
        "for i in range(len(nsent)):\n",
        "  test = nsent[i]\n",
        "  inputs = tokenizer(test, padding=True, truncation=True, return_tensors=\"pt\").to(device)\n",
        "  with torch.no_grad():\n",
        "      embeddings = model(**inputs, output_hidden_states=True, return_dict=True).pooler_output\n",
        "      embds = embeddings\n",
        "      del inputs\n",
        "      torch.cuda.empty_cache()\n",
        "  # embds = embds.cpu()\n",
        "  res = cosine_similarity(embds)\n",
        "  sc.append(res.cpu())\n",
        "  if (i%250 == 0):\n",
        "    print(i)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sNZCeg73PiGN",
        "outputId": "46da8a1d-bf8f-4990-b809-25ddb1feb4c8"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array(['The grass family is one of the most widely distributed and abundant groups of plants on Earth.\\tAs noted on the Wiki page, grass seed was imported to the new world to improve pasturage for livestock.\\nmain-forums\\tanswers-forums\\t2015\\t0782\\t1.20\\thttp://en.wikipedia.org/wiki/History_of_Delhi#Early_history shows that the city was called Delhi (or Dilli) at least since the 12/13th century.\\tI am not exactly sure but I remember the media (in India) addressing Delhi as HASTHINA\".',\n",
              "       nan], dtype=object)"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import gc\n",
        "inputs = None\n",
        "model = None \n",
        "gc.collect()\n",
        "\n",
        "with torch.no_grad():\n",
        "    torch.cuda.empty_cache()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "u7kdBBTzSWMw"
      },
      "outputs": [],
      "source": [
        "sc= [i.cpu() for i in sc]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 89,
      "metadata": {
        "id": "8UzO3Me7Sb3G"
      },
      "outputs": [],
      "source": [
        "with open(\"res.txt\", \"w\") as rfile:\n",
        "  for i in sc:\n",
        "    rfile.write(str(i.item())+\"\\n\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 73,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4qM5_qaWSksM",
        "outputId": "61903e46-a081-41ec-b88b-571753ff012c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "3445.731333333332"
            ]
          },
          "execution_count": 73,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sum(score)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 90,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T5-G0VJjSssy",
        "outputId": "e83c87db-1568-441d-d0c9-996813c4c2e8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Pearson: 0.65988\n"
          ]
        }
      ],
      "source": [
        "!perl correlation.pl sts-dev.csv res.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-q8Z8dgWYvNc"
      },
      "source": [
        "### FSE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 100,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T_cxCpy6XTEa",
        "outputId": "a9fcd772-c322-4a98-d3b5-905f9241f921"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 100,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# !pip install -U gensim fse\n",
        "# !pip install nltk\n",
        "import nltk\n",
        "nltk.download('punkt')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 146,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 229,
          "referenced_widgets": [
            "6cc9a9c6f06d4b18b1b7f7e49cd3415a",
            "7fbb1cfe230544448f7eda3e92574373",
            "20724fe595014b50956b20edc2796e82",
            "a2a1003a84ea4c2aac79f4b909e071b1",
            "769aab99065240c8a6a4bd65b6d9c17f",
            "15d8611be2504abe82b2f127f50243b5",
            "35640a4f096b4d3289157979a56c14ae",
            "6b0c1f8486e647bb80ac3b9b9bc1e453",
            "420c5adf8fdd42858aeb5d7baa24c600",
            "260d20d9c62d4365b7b5852d798a8fe9",
            "c8ab752034fd423b92a814171647a577",
            "823947b3ab524cec992c0e504c279dea",
            "c34cc9ae1d3d44e08c7b7cced0845d89",
            "11df1f5a82774061ba0eb515f25e2506",
            "a0d6654d739e413ab5dad65e325412d7",
            "66b1d8c3d1ae426fb2e302529e5a696b",
            "3317c3066dd049d4b286acbf2cc23f75",
            "dda9fa4615cc4c0c86ff1f8f8c7aa2b6",
            "f9830dfe20574ea5ad71dd18335c625f",
            "e8f1b2422df9457986f1f1c1a01329dd",
            "1817e141ef1b41778e56d9ff39d8099f",
            "0b7db0af91dd428f9f103c499e7c7953",
            "ed8995012b8342829991156406f97898",
            "50774ee5e96a45a1bc188227ef6ed9a9",
            "446a973acca84f16a76d5dc6cf16062f",
            "be57c23214644b9b854443c78eb8bfa6",
            "f2dddfa584274ee69b0681b896104c46",
            "3c50628623cc47039ada0b6b32d77717",
            "2a92d364647b483e8515c593410c0550",
            "df3f7f04bdbe4f9f8e61e8b9d5e280b9",
            "aac6a03ce30545cd92e684f29ea89c61",
            "96ff0a3c81e94c1792e7a183a506b339",
            "178038682e2942568f69e21fea3030d2",
            "525f2014d82641799d13fd1c9e27991d",
            "0062ce6289af4927ac483959550cff3d",
            "885696668fc5460c8956c7da863be929",
            "edad53b846fd4a67aeafe2b55910e29e",
            "3d8c6d73026342dd8ed3abc68b95d804",
            "1535b2eae132474a85c48b39675d4785",
            "e7903433d3544f918a166a130b1e582d",
            "037afda0efac4953a0eeb8e7828c92b1",
            "d3ae4937d3a843d294c38d7529035a94",
            "6744c1358205465c888b9fa75353bfe0",
            "5220e601079f44839322b07a6d229aa7"
          ]
        },
        "id": "j8rA7XnZYyWp",
        "outputId": "5389350f-bdfa-4d88-dd6e-48f39c487085"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6cc9a9c6f06d4b18b1b7f7e49cd3415a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/1.26k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "823947b3ab524cec992c0e504c279dea",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/278 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ed8995012b8342829991156406f97898",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/3.84M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "525f2014d82641799d13fd1c9e27991d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading:   0%|          | 0.00/92.7M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from fse import Vectors, Average, uSIF, IndexedList, CSplitIndexedList\n",
        "\n",
        "from nltk import word_tokenize\n",
        "\n",
        "# vecs = Vectors.from_pretrained(\"glove-wiki-gigaword-50\")\n",
        "vecs = Vectors.from_pretrained(\"paranmt-300\")\n",
        "model = uSIF(vecs)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 106,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-urmGvmDZRhB",
        "outputId": "fcd46200-7a07-4bab-f33f-5c2b4d275ba9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "execution_count": 106,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from fse.models.average import FAST_VERSION\n",
        "FAST_VERSION #1 means available"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 147,
      "metadata": {
        "id": "SzMYyYIoZoNP"
      },
      "outputs": [],
      "source": [
        "# tokenizer from https://github.com/oborchers/Fast_Sentence_Embeddings/blob/master/notebooks/STS-Benchmarks.ipynb\n",
        "import re\n",
        "\n",
        "not_punc = re.compile('.*[A-Za-z0-9].*')\n",
        "\n",
        "def prep_token(token):\n",
        "    t = token.lower().strip(\"';.:()\").strip('\"')\n",
        "    t = 'not' if t == \"n't\" else t\n",
        "    return re.split(r'[-]', t)\n",
        "\n",
        "def prep_sentence(sentence):\n",
        "    tokens = []\n",
        "    for token in word_tokenize(sentence):\n",
        "        if not_punc.match(token):\n",
        "            tokens = tokens + prep_token(token)\n",
        "    return tokens\n",
        "\n",
        "sent_a = [i[0] for i in nsent]\n",
        "sent_b = [i[1] for i in nsent]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 148,
      "metadata": {
        "id": "gk81-Wl4a3bo"
      },
      "outputs": [],
      "source": [
        "sentences = CSplitIndexedList(sent_a, sent_b, custom_split=prep_sentence)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 149,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZhuU39x3a7eO",
        "outputId": "93f53da0-3fda-4da7-8613-bb224501fdde"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[['a', 'man', 'with', 'a', 'hard', 'hat', 'is', 'dancing'],\n",
              " ['a', 'man', 'wearing', 'a', 'hard', 'hat', 'is', 'dancing']]"
            ]
          },
          "execution_count": 149,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sentence[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 150,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gb3z87iFa8dT",
        "outputId": "0f27eaca-0a0a-4e27-f42b-e63a58abcfbd"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(2940, 34468)"
            ]
          },
          "execution_count": 150,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.train(sentences)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 151,
      "metadata": {
        "id": "vBpFSIITbt3O"
      },
      "outputs": [],
      "source": [
        "def compute_similarities(task_length, model):\n",
        "    sims = []\n",
        "    for i, j in zip(range(task_length), range(task_length, 2*task_length)):\n",
        "        sims.append(model.sv.similarity(i,j) * 5)\n",
        "    return sims"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 152,
      "metadata": {
        "id": "h0Nvto36el-E"
      },
      "outputs": [],
      "source": [
        "fts_sc = compute_similarities(len(nsent), model)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 142,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6MlsV8L2e8_Z",
        "outputId": "097fb907-f581-4cdb-903d-03558edabe59"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[4.720543026924133,\n",
              " 4.820749461650848,\n",
              " 4.654754996299744,\n",
              " 4.795777797698975,\n",
              " 4.762759506702423,\n",
              " 4.236574172973633,\n",
              " 4.066173434257507,\n",
              " 4.4780027866363525,\n",
              " 4.752212464809418,\n",
              " 4.6947914361953735,\n",
              " 4.6185302734375,\n",
              " 4.734078049659729,\n",
              " 4.12793755531311,\n",
              " 4.489364326000214,\n",
              " 4.398241937160492,\n",
              " 4.079786539077759,\n",
              " 4.249792993068695,\n",
              " 4.649677574634552,\n",
              " 4.585241377353668,\n",
              " 3.5085314512252808,\n",
              " 4.087885022163391,\n",
              " 4.803565442562103,\n",
              " 3.883202075958252,\n",
              " 4.6032145619392395,\n",
              " 3.8448941707611084,\n",
              " 4.721610844135284,\n",
              " 4.304442405700684,\n",
              " 3.80185604095459,\n",
              " 2.2768478095531464,\n",
              " 3.928616940975189,\n",
              " 4.421308636665344,\n",
              " 4.022016525268555,\n",
              " 4.401788711547852,\n",
              " 4.765162169933319,\n",
              " 4.7049203515052795,\n",
              " 4.581669867038727,\n",
              " 4.761216044425964,\n",
              " 4.763786196708679,\n",
              " 4.263761937618256,\n",
              " 4.055345952510834,\n",
              " 4.375719428062439,\n",
              " 4.067807197570801,\n",
              " 3.5285937786102295,\n",
              " 4.1559600830078125,\n",
              " 4.6062517166137695,\n",
              " 2.255125790834427,\n",
              " 3.096795976161957,\n",
              " 3.005911707878113,\n",
              " 4.218945205211639,\n",
              " 4.440076053142548,\n",
              " 1.4814242720603943,\n",
              " 4.624915421009064,\n",
              " 4.188668727874756,\n",
              " 4.239260256290436,\n",
              " 3.551119863986969,\n",
              " 3.929903209209442,\n",
              " 2.5030621886253357,\n",
              " 2.554067373275757,\n",
              " 4.474776983261108,\n",
              " 1.8317988514900208,\n",
              " 3.7829360365867615,\n",
              " 4.113384485244751,\n",
              " 2.7389737963676453,\n",
              " 4.230766296386719,\n",
              " 4.032308757305145,\n",
              " 3.435298204421997,\n",
              " 4.262484312057495,\n",
              " 4.569188952445984,\n",
              " 4.034511148929596,\n",
              " 4.735104143619537,\n",
              " 4.1486793756484985,\n",
              " 3.671606481075287,\n",
              " 3.6306336522102356,\n",
              " 2.407453954219818,\n",
              " 0.5877748876810074,\n",
              " 4.012525379657745,\n",
              " 3.366749882698059,\n",
              " 3.7259849905967712,\n",
              " 2.8554773330688477,\n",
              " 4.06485378742218,\n",
              " 4.043145179748535,\n",
              " 2.2220221161842346,\n",
              " 2.4068298935890198,\n",
              " 2.2518005967140198,\n",
              " 4.6144163608551025,\n",
              " 2.346523106098175,\n",
              " 3.7499243021011353,\n",
              " 3.6507707834243774,\n",
              " 3.6278486251831055,\n",
              " 3.5927814245224,\n",
              " 4.450433254241943,\n",
              " 3.361470103263855,\n",
              " 3.094499111175537,\n",
              " 2.660113275051117,\n",
              " 4.054781794548035,\n",
              " 3.7531498074531555,\n",
              " 4.2729562520980835,\n",
              " 2.04140841960907,\n",
              " 3.0096709728240967,\n",
              " 3.7262478470802307,\n",
              " 2.015080153942108,\n",
              " 2.7031245827674866,\n",
              " 4.406978785991669,\n",
              " 4.407861232757568,\n",
              " 2.2229306399822235,\n",
              " 3.577660918235779,\n",
              " 4.437828958034515,\n",
              " 3.408457040786743,\n",
              " 3.1445178389549255,\n",
              " 1.819554716348648,\n",
              " 3.522329032421112,\n",
              " 3.0138537287712097,\n",
              " 0.5622320994734764,\n",
              " 2.947462797164917,\n",
              " 2.4481047689914703,\n",
              " 1.286385953426361,\n",
              " 2.1034930646419525,\n",
              " 4.161680340766907,\n",
              " 1.408531814813614,\n",
              " 2.3311105370521545,\n",
              " 2.9427912831306458,\n",
              " 2.3646382987499237,\n",
              " 3.4072181582450867,\n",
              " 2.2688642144203186,\n",
              " 3.7980878353118896,\n",
              " 3.515862822532654,\n",
              " 2.139926105737686,\n",
              " 2.210743874311447,\n",
              " 3.0237984657287598,\n",
              " 2.5979864597320557,\n",
              " 1.338513046503067,\n",
              " 4.27348256111145,\n",
              " 4.740723669528961,\n",
              " 4.688567817211151,\n",
              " 4.881656169891357,\n",
              " 4.881656169891357,\n",
              " 4.7121623158454895,\n",
              " 4.031150937080383,\n",
              " 4.584527313709259,\n",
              " 4.733282029628754,\n",
              " 4.545150995254517,\n",
              " 4.6886953711509705,\n",
              " 4.637944400310516,\n",
              " 4.1911086440086365,\n",
              " 4.679660201072693,\n",
              " 4.659781754016876,\n",
              " 3.547259569168091,\n",
              " 2.761199176311493,\n",
              " 4.6032145619392395,\n",
              " 4.6958911418914795,\n",
              " 4.613575339317322,\n",
              " 4.551112055778503,\n",
              " 4.728600978851318,\n",
              " 3.1919223070144653,\n",
              " 3.7219244241714478,\n",
              " 4.357436299324036,\n",
              " 3.729265034198761,\n",
              " 3.729265034198761,\n",
              " 4.206719696521759,\n",
              " 4.698147475719452,\n",
              " 4.657581448554993,\n",
              " 4.312481582164764,\n",
              " 4.026252627372742,\n",
              " 4.0579599142074585,\n",
              " 4.042930901050568,\n",
              " 4.207916557788849,\n",
              " 4.040825963020325,\n",
              " 3.6289173364639282,\n",
              " 3.3545663952827454,\n",
              " 4.387195408344269,\n",
              " 4.594704508781433,\n",
              " 2.399968057870865,\n",
              " 3.8276848196983337,\n",
              " 1.8921037018299103,\n",
              " 4.36998575925827,\n",
              " 4.615810215473175,\n",
              " 3.4198403358459473,\n",
              " 4.3525779247283936,\n",
              " 3.9159852266311646,\n",
              " 4.274643659591675,\n",
              " 2.9331865906715393,\n",
              " 4.341682493686676,\n",
              " 4.433382153511047,\n",
              " 3.863028585910797,\n",
              " 3.494730591773987,\n",
              " 4.61669385433197,\n",
              " 3.01482230424881,\n",
              " 4.061462879180908,\n",
              " 3.069612979888916,\n",
              " 2.713754177093506,\n",
              " 4.100247919559479,\n",
              " 4.443104565143585,\n",
              " 4.064220488071442,\n",
              " 2.74150550365448,\n",
              " 3.8502871990203857,\n",
              " 3.4520360827445984,\n",
              " 2.478218823671341,\n",
              " 4.377394914627075,\n",
              " 4.476075172424316,\n",
              " 0.2002435177564621,\n",
              " 2.6818665862083435,\n",
              " 1.9714169204235077,\n",
              " 2.7665019035339355,\n",
              " 3.67249995470047,\n",
              " 3.1658369302749634,\n",
              " 3.292464017868042,\n",
              " 3.058609366416931,\n",
              " 4.213269948959351,\n",
              " 3.4764140844345093,\n",
              " 3.649716079235077,\n",
              " 2.9614347219467163,\n",
              " 3.179197311401367,\n",
              " 3.9032793045043945,\n",
              " 3.165501058101654,\n",
              " 2.7912944555282593,\n",
              " 1.7778924107551575,\n",
              " 3.2697197794914246,\n",
              " 1.7557966709136963,\n",
              " 4.366301000118256,\n",
              " 4.637693166732788,\n",
              " 3.5218286514282227,\n",
              " 3.0162397027015686,\n",
              " 3.943428695201874,\n",
              " 3.282923698425293,\n",
              " 4.10629540681839,\n",
              " 4.03443306684494,\n",
              " 0.46491589397192,\n",
              " 2.6446008682250977,\n",
              " 2.0010527968406677,\n",
              " 2.2560985386371613,\n",
              " 2.392999976873398,\n",
              " 2.9169800877571106,\n",
              " 1.585434377193451,\n",
              " 0.9529103338718414,\n",
              " 2.3103827238082886,\n",
              " 2.3116326332092285,\n",
              " 4.307521879673004,\n",
              " 3.696162700653076,\n",
              " 4.839329421520233,\n",
              " 3.195732831954956,\n",
              " 2.148483842611313,\n",
              " 2.5747698545455933,\n",
              " 2.6030808687210083,\n",
              " 1.6246099770069122,\n",
              " 2.2129859030246735,\n",
              " 0.6639736890792847,\n",
              " 1.9020141661167145,\n",
              " 3.451525568962097,\n",
              " 4.3797144293785095,\n",
              " 2.4335475265979767,\n",
              " 4.122931063175201,\n",
              " 2.6889050006866455,\n",
              " 3.1623026728630066,\n",
              " 3.3941173553466797,\n",
              " 3.2751134037971497,\n",
              " 3.97316575050354,\n",
              " 2.058517634868622,\n",
              " 3.721971809864044,\n",
              " 4.432563483715057,\n",
              " 4.129107594490051,\n",
              " 3.409512937068939,\n",
              " 3.511139750480652,\n",
              " 4.297917187213898,\n",
              " 3.9309704303741455,\n",
              " 4.715441167354584,\n",
              " 4.5082515478134155,\n",
              " 2.938927710056305,\n",
              " 3.728276789188385,\n",
              " 3.568176031112671,\n",
              " 4.057987332344055,\n",
              " 4.5012447237968445,\n",
              " 4.544374942779541,\n",
              " 3.8745829463005066,\n",
              " 3.869767487049103,\n",
              " 3.79117488861084,\n",
              " 3.5744118690490723,\n",
              " 0.6188254430890083,\n",
              " 4.345328211784363,\n",
              " 4.057393372058868,\n",
              " 3.5996398329734802,\n",
              " 3.7282416224479675,\n",
              " 4.541890025138855,\n",
              " 4.625455141067505,\n",
              " 4.5367878675460815,\n",
              " 2.795715630054474,\n",
              " 3.1496411561965942,\n",
              " 3.0873647332191467,\n",
              " 2.266547381877899,\n",
              " 1.7512661218643188,\n",
              " 4.472947418689728,\n",
              " 4.4101482629776,\n",
              " 1.6674606502056122,\n",
              " 3.8184458017349243,\n",
              " 3.745308518409729,\n",
              " 4.537948668003082,\n",
              " 3.9438053965568542,\n",
              " 3.9099767804145813,\n",
              " 4.47338342666626,\n",
              " 3.3360305428504944,\n",
              " 4.196004867553711,\n",
              " 4.226686656475067,\n",
              " 3.503151535987854,\n",
              " 4.6373289823532104,\n",
              " 4.140846133232117,\n",
              " 4.667772054672241,\n",
              " 4.455043971538544,\n",
              " 4.7880858182907104,\n",
              " 4.579034447669983,\n",
              " 1.0732949525117874,\n",
              " 3.2226788997650146,\n",
              " 2.741331458091736,\n",
              " 3.3282169699668884,\n",
              " 3.585503399372101,\n",
              " 3.6438381671905518,\n",
              " 3.8582760095596313,\n",
              " 2.4998190999031067,\n",
              " 4.0175265073776245,\n",
              " 3.5141202807426453,\n",
              " 4.3544575572013855,\n",
              " 1.5313303470611572,\n",
              " 4.512774348258972,\n",
              " 4.159890711307526,\n",
              " 3.637474477291107,\n",
              " 3.733516037464142,\n",
              " 3.5643231868743896,\n",
              " 4.141541123390198,\n",
              " 4.603604376316071,\n",
              " 4.430871903896332,\n",
              " 4.537800550460815,\n",
              " 2.8954631090164185,\n",
              " 4.327340126037598,\n",
              " 2.911112606525421,\n",
              " 3.314658999443054,\n",
              " 2.3826633393764496,\n",
              " 3.401365876197815,\n",
              " 3.77948135137558,\n",
              " 3.7526604533195496,\n",
              " 2.8712430596351624,\n",
              " 4.1591522097587585,\n",
              " 3.9718839526176453,\n",
              " 3.458862602710724,\n",
              " 4.230733811855316,\n",
              " 3.892958164215088,\n",
              " 4.561621844768524,\n",
              " 3.9687159657478333,\n",
              " 4.4101205468177795,\n",
              " 4.429334104061127,\n",
              " 4.0771326422691345,\n",
              " 4.049904048442841,\n",
              " 3.8801786303520203,\n",
              " 4.5860931277275085,\n",
              " 4.413844645023346,\n",
              " 4.322392642498016,\n",
              " 4.4731685519218445,\n",
              " 4.48846697807312,\n",
              " 3.7879061698913574,\n",
              " 4.699122309684753,\n",
              " 4.6989428997039795,\n",
              " 4.792045056819916,\n",
              " 3.044736385345459,\n",
              " 1.7373557388782501,\n",
              " 4.705310761928558,\n",
              " 3.5552912950515747,\n",
              " 2.697489857673645,\n",
              " 4.1198840737342834,\n",
              " 4.677714109420776,\n",
              " 3.6526843905448914,\n",
              " 4.717147052288055,\n",
              " 4.565531015396118,\n",
              " 4.717718660831451,\n",
              " 3.5223224759101868,\n",
              " 1.9048233330249786,\n",
              " 4.4496893882751465,\n",
              " 4.049040973186493,\n",
              " 4.73944753408432,\n",
              " 4.607807099819183,\n",
              " 4.11998450756073,\n",
              " 3.7942880392074585,\n",
              " 4.793049395084381,\n",
              " 3.0238178372383118,\n",
              " 1.7653805017471313,\n",
              " 3.8042157888412476,\n",
              " 3.70214581489563,\n",
              " 2.5142377614974976,\n",
              " 4.066020846366882,\n",
              " 4.3505024909973145,\n",
              " 3.8431701064109802,\n",
              " 4.205771386623383,\n",
              " 3.100808560848236,\n",
              " 4.364709854125977,\n",
              " 4.460908472537994,\n",
              " 4.47115033864975,\n",
              " 3.1764233112335205,\n",
              " 4.005785882472992,\n",
              " 3.909715414047241,\n",
              " 4.634480774402618,\n",
              " 4.32835191488266,\n",
              " 4.668208360671997,\n",
              " 4.749873578548431,\n",
              " 4.4155871868133545,\n",
              " 3.487611711025238,\n",
              " 3.5354310274124146,\n",
              " 3.596808910369873,\n",
              " 3.102651536464691,\n",
              " 3.9282968640327454,\n",
              " 4.385804533958435,\n",
              " 2.345361113548279,\n",
              " 1.7121629416942596,\n",
              " 3.4423118829727173,\n",
              " 4.462471902370453,\n",
              " 4.631572663784027,\n",
              " 3.68528813123703,\n",
              " 1.6885989904403687,\n",
              " 2.872985601425171,\n",
              " 1.830509901046753,\n",
              " 4.345065951347351,\n",
              " 4.662007689476013,\n",
              " 3.5758838057518005,\n",
              " 2.833094298839569,\n",
              " 4.03936892747879,\n",
              " 3.2776042819023132,\n",
              " 4.311838150024414,\n",
              " 2.5323787331581116,\n",
              " 4.540489912033081,\n",
              " 4.091868698596954,\n",
              " 4.62704211473465,\n",
              " 4.3373435735702515,\n",
              " 4.185535311698914,\n",
              " 3.224215805530548,\n",
              " 3.9643436670303345,\n",
              " 3.59706848859787,\n",
              " 4.451766312122345,\n",
              " 3.3409318327903748,\n",
              " 4.327402710914612,\n",
              " 4.120607674121857,\n",
              " 2.615782916545868,\n",
              " 2.249208837747574,\n",
              " 1.2208965420722961,\n",
              " 4.1801518201828,\n",
              " 4.691340923309326,\n",
              " 4.12116676568985,\n",
              " 4.003743827342987,\n",
              " 3.1383299827575684,\n",
              " 2.8006601333618164,\n",
              " 4.476615488529205,\n",
              " 4.842179119586945,\n",
              " 2.9535990953445435,\n",
              " 3.7758594751358032,\n",
              " 4.150403141975403,\n",
              " 4.160149395465851,\n",
              " 4.294267892837524,\n",
              " 2.8562575578689575,\n",
              " 4.6641868352890015,\n",
              " 3.06258887052536,\n",
              " 4.1467925906181335,\n",
              " 4.08280223608017,\n",
              " 3.044424057006836,\n",
              " 2.453792095184326,\n",
              " 4.023530185222626,\n",
              " 3.41412216424942,\n",
              " 4.7807833552360535,\n",
              " 2.9598894715309143,\n",
              " 2.7129241824150085,\n",
              " 2.9059362411499023,\n",
              " 2.2875240445137024,\n",
              " 4.225322008132935,\n",
              " 2.934371531009674,\n",
              " 3.2039418816566467,\n",
              " 3.4916338324546814,\n",
              " 3.8721436262130737,\n",
              " 4.192797243595123,\n",
              " 3.2711657881736755,\n",
              " 4.539093971252441,\n",
              " 2.7008330821990967,\n",
              " 3.0469727516174316,\n",
              " 4.475569725036621,\n",
              " 4.4464221596717834,\n",
              " 2.527363896369934,\n",
              " 3.8718149065971375,\n",
              " 4.549687206745148,\n",
              " 4.461563527584076,\n",
              " 0.7824499905109406,\n",
              " 3.164282739162445,\n",
              " 3.7250661849975586,\n",
              " 4.86922025680542,\n",
              " 3.1378236413002014,\n",
              " 3.0585667490959167,\n",
              " 2.9303762316703796,\n",
              " 3.9831677079200745,\n",
              " 3.5372337698936462,\n",
              " 4.188322126865387,\n",
              " 1.3637037575244904,\n",
              " 4.582883715629578,\n",
              " 3.115057051181793,\n",
              " 4.160678386688232,\n",
              " 3.294193148612976,\n",
              " 4.199561178684235,\n",
              " 3.9582473039627075,\n",
              " 4.510564506053925,\n",
              " 4.249399900436401,\n",
              " 4.208906888961792,\n",
              " 4.047233462333679,\n",
              " 4.047180414199829,\n",
              " 4.329943954944611,\n",
              " 4.822031259536743,\n",
              " 4.057513773441315,\n",
              " 4.322914183139801,\n",
              " 4.8916321992874146,\n",
              " 4.156195223331451,\n",
              " 4.557371735572815,\n",
              " 3.7787699699401855,\n",
              " 3.7163829803466797,\n",
              " 4.71849650144577,\n",
              " 4.876241981983185,\n",
              " 3.7180203199386597,\n",
              " 4.753452837467194,\n",
              " 3.9208614826202393,\n",
              " 4.680369794368744,\n",
              " 4.668127596378326,\n",
              " 4.774352014064789,\n",
              " 4.479229152202606,\n",
              " 4.3767619132995605,\n",
              " 4.503757357597351,\n",
              " 4.099760055541992,\n",
              " 4.572286009788513,\n",
              " 4.870579838752747,\n",
              " 4.634434878826141,\n",
              " 3.934548497200012,\n",
              " 4.516833424568176,\n",
              " 4.107195138931274,\n",
              " 4.53766793012619,\n",
              " 4.343020021915436,\n",
              " 3.8113686442375183,\n",
              " 4.751004278659821,\n",
              " 4.913299381732941,\n",
              " 4.720024466514587,\n",
              " 4.481312930583954,\n",
              " 4.92453008890152,\n",
              " 3.7985873222351074,\n",
              " 4.602884650230408,\n",
              " 4.259000718593597,\n",
              " 4.054345488548279,\n",
              " 4.569036364555359,\n",
              " 4.715689718723297,\n",
              " 4.3435123562812805,\n",
              " 4.2952147126197815,\n",
              " 4.3528348207473755,\n",
              " 4.766850173473358,\n",
              " 4.712001383304596,\n",
              " 3.1873399019241333,\n",
              " 4.743062853813171,\n",
              " 4.606428146362305,\n",
              " 3.644893765449524,\n",
              " 4.794362187385559,\n",
              " 4.146180748939514,\n",
              " 4.302423596382141,\n",
              " 4.826061725616455,\n",
              " 4.0248361229896545,\n",
              " 3.375280201435089,\n",
              " 4.604608714580536,\n",
              " 4.4021934270858765,\n",
              " 4.122347831726074,\n",
              " 4.676816165447235,\n",
              " 4.607552886009216,\n",
              " 4.214023649692535,\n",
              " 4.367305934429169,\n",
              " 4.441133439540863,\n",
              " 4.3191418051719666,\n",
              " 4.616020917892456,\n",
              " 4.359330534934998,\n",
              " 4.389229714870453,\n",
              " 4.212645590305328,\n",
              " 4.657376706600189,\n",
              " 4.217576086521149,\n",
              " 4.55386757850647,\n",
              " 4.238177537918091,\n",
              " 4.340062737464905,\n",
              " 4.899324476718903,\n",
              " 4.458394348621368,\n",
              " 4.321160018444061,\n",
              " 4.398529231548309,\n",
              " 4.452177584171295,\n",
              " 0.6512559950351715,\n",
              " 4.80143815279007,\n",
              " 5.0,\n",
              " 4.144712388515472,\n",
              " 4.6557557582855225,\n",
              " 4.546382129192352,\n",
              " 5.0,\n",
              " 4.160954058170319,\n",
              " 4.2452472448349,\n",
              " 4.3839070200920105,\n",
              " 4.345744550228119,\n",
              " 3.835080564022064,\n",
              " 4.3327730894088745,\n",
              " 4.428716003894806,\n",
              " 4.635534286499023,\n",
              " 4.415026307106018,\n",
              " 3.9362403750419617,\n",
              " 4.344860017299652,\n",
              " 4.337042570114136,\n",
              " 4.628168046474457,\n",
              " 4.380664229393005,\n",
              " 4.228696823120117,\n",
              " 4.262453019618988,\n",
              " 4.563879668712616,\n",
              " 4.528308808803558,\n",
              " 4.431212246417999,\n",
              " 4.921921789646149,\n",
              " 4.693760275840759,\n",
              " 4.364651143550873,\n",
              " 4.7493332624435425,\n",
              " 4.245128035545349,\n",
              " 4.656349420547485,\n",
              " 4.391404092311859,\n",
              " 4.378721117973328,\n",
              " 4.42870557308197,\n",
              " 4.623194336891174,\n",
              " 4.507259428501129,\n",
              " 4.388078153133392,\n",
              " 4.579244256019592,\n",
              " 4.19077455997467,\n",
              " 4.719108045101166,\n",
              " 4.766819477081299,\n",
              " 4.559450745582581,\n",
              " 4.450108110904694,\n",
              " 2.600739896297455,\n",
              " 2.8204670548439026,\n",
              " 3.2950255274772644,\n",
              " 4.344841539859772,\n",
              " 4.498291909694672,\n",
              " 2.704594135284424,\n",
              " 3.0365681648254395,\n",
              " 2.0280595123767853,\n",
              " 4.5654162764549255,\n",
              " 1.8216568231582642,\n",
              " 3.503705859184265,\n",
              " 2.83547043800354,\n",
              " 2.6608356833457947,\n",
              " 2.3332011699676514,\n",
              " 0.9890647232532501,\n",
              " 2.246149927377701,\n",
              " 3.0427956581115723,\n",
              " 3.3261194825172424,\n",
              " 3.6615651845932007,\n",
              " 3.9710313081741333,\n",
              " 3.8504263758659363,\n",
              " 3.4404966235160828,\n",
              " 3.8133347034454346,\n",
              " 3.1920471787452698,\n",
              " 2.666311264038086,\n",
              " 1.8672536313533783,\n",
              " 3.1376001238822937,\n",
              " 3.2037803530693054,\n",
              " 4.0923914313316345,\n",
              " 3.492574989795685,\n",
              " 0.8173461258411407,\n",
              " 3.093162178993225,\n",
              " 1.0658848285675049,\n",
              " 2.659357190132141,\n",
              " 4.018399119377136,\n",
              " 3.775262236595154,\n",
              " 3.721301257610321,\n",
              " 3.891119658946991,\n",
              " 4.140504002571106,\n",
              " 2.8105252981185913,\n",
              " 2.4734045565128326,\n",
              " 2.205165922641754,\n",
              " 3.322599232196808,\n",
              " 3.1096550822257996,\n",
              " 2.4499966204166412,\n",
              " 3.1375136971473694,\n",
              " 3.0998027324676514,\n",
              " 3.3097901940345764,\n",
              " 2.612300217151642,\n",
              " 2.913479506969452,\n",
              " 0.7188236713409424,\n",
              " 4.614906907081604,\n",
              " 3.81788969039917,\n",
              " 3.2063812017440796,\n",
              " 3.10810387134552,\n",
              " 1.678319126367569,\n",
              " 2.8964444994926453,\n",
              " 2.883295714855194,\n",
              " 3.9339950680732727,\n",
              " 3.95279198884964,\n",
              " 4.1872528195381165,\n",
              " 2.9374590516090393,\n",
              " 3.900018334388733,\n",
              " 1.6167861223220825,\n",
              " 1.1067647486925125,\n",
              " 3.805103302001953,\n",
              " 1.8938106298446655,\n",
              " 3.283390700817108,\n",
              " 3.099016845226288,\n",
              " 3.8274070620536804,\n",
              " 3.6498138308525085,\n",
              " 4.445691704750061,\n",
              " 3.460102081298828,\n",
              " 3.615274131298065,\n",
              " 2.659682035446167,\n",
              " 0.9281670302152634,\n",
              " 3.144741654396057,\n",
              " 4.116080403327942,\n",
              " 4.131189286708832,\n",
              " 3.6807388067245483,\n",
              " 1.8712878227233887,\n",
              " 3.5528242588043213,\n",
              " 2.240844815969467,\n",
              " 3.47412109375,\n",
              " 2.9188355803489685,\n",
              " 3.4632816910743713,\n",
              " 3.1080687046051025,\n",
              " 1.7828576266765594,\n",
              " 1.6732852160930634,\n",
              " 2.926674783229828,\n",
              " 1.5427567064762115,\n",
              " 4.058626890182495,\n",
              " 4.533917009830475,\n",
              " 1.1154738068580627,\n",
              " 3.8444405794143677,\n",
              " 2.9468220472335815,\n",
              " 2.7720388770103455,\n",
              " 3.294461965560913,\n",
              " 0.9936357289552689,\n",
              " 2.8113600611686707,\n",
              " 2.096095234155655,\n",
              " 1.0485803335905075,\n",
              " 3.6198535561561584,\n",
              " 2.7019044756889343,\n",
              " 3.189358413219452,\n",
              " 3.39504212141037,\n",
              " 3.228633403778076,\n",
              " 1.9921110570430756,\n",
              " 3.0360785126686096,\n",
              " 2.1272461116313934,\n",
              " 3.8026708364486694,\n",
              " 1.4403709769248962,\n",
              " 3.2754874229431152,\n",
              " 3.716801106929779,\n",
              " 3.8984575867652893,\n",
              " 2.154262810945511,\n",
              " 2.768770456314087,\n",
              " 2.367667406797409,\n",
              " 1.4162637293338776,\n",
              " 2.218901813030243,\n",
              " 2.547559440135956,\n",
              " 3.7164008617401123,\n",
              " 2.7855736017227173,\n",
              " 3.1117501854896545,\n",
              " 4.067858159542084,\n",
              " 3.6717548966407776,\n",
              " 3.965441584587097,\n",
              " 2.9655134677886963,\n",
              " 3.8366425037384033,\n",
              " 3.711637854576111,\n",
              " 1.3670076429843903,\n",
              " 3.0730926990509033,\n",
              " 2.3960550129413605,\n",
              " 3.480578660964966,\n",
              " 1.057203784584999,\n",
              " 2.3411664366722107,\n",
              " 4.16361391544342,\n",
              " 2.0075206458568573,\n",
              " 4.999999701976776,\n",
              " 3.013620972633362,\n",
              " 2.426706850528717,\n",
              " 3.5945823788642883,\n",
              " 2.723434567451477,\n",
              " 4.260136187076569,\n",
              " 4.103004336357117,\n",
              " 3.520383834838867,\n",
              " 1.5407073497772217,\n",
              " 2.882772386074066,\n",
              " 2.884596586227417,\n",
              " 2.6712679862976074,\n",
              " 2.7184706926345825,\n",
              " 2.9718273878097534,\n",
              " 2.827650010585785,\n",
              " 3.188946843147278,\n",
              " 3.087986409664154,\n",
              " 3.5227209329605103,\n",
              " 4.104776084423065,\n",
              " 3.576180338859558,\n",
              " 3.74251127243042,\n",
              " 2.155410349369049,\n",
              " 3.0486661195755005,\n",
              " 3.891335129737854,\n",
              " 2.7340131998062134,\n",
              " 0.021301882807165384,\n",
              " 2.6090896129608154,\n",
              " 2.321973592042923,\n",
              " 2.8409945964813232,\n",
              " 1.9555766880512238,\n",
              " 2.6986122131347656,\n",
              " 4.112841784954071,\n",
              " 1.704733520746231,\n",
              " 2.54949152469635,\n",
              " 1.5008977055549622,\n",
              " 4.263184368610382,\n",
              " 4.324500262737274,\n",
              " 2.9295197129249573,\n",
              " 4.003551006317139,\n",
              " 3.856935501098633,\n",
              " 4.481714367866516,\n",
              " 2.2525185346603394,\n",
              " 4.054607450962067,\n",
              " 2.2604119777679443,\n",
              " 2.17952162027359,\n",
              " 2.1745333075523376,\n",
              " 3.6764460802078247,\n",
              " 3.6414650082588196,\n",
              " 2.0163023471832275,\n",
              " 1.5964344143867493,\n",
              " 1.395668238401413,\n",
              " 3.1495946645736694,\n",
              " 3.2282868027687073,\n",
              " 3.9757639169692993,\n",
              " 2.4405817687511444,\n",
              " 3.075765371322632,\n",
              " 3.7321028113365173,\n",
              " 2.2771044075489044,\n",
              " 3.082476854324341,\n",
              " 3.704243004322052,\n",
              " 2.5980770587921143,\n",
              " 3.3594048023223877,\n",
              " 3.9027774333953857,\n",
              " 3.112604320049286,\n",
              " 4.547170698642731,\n",
              " 3.5442963242530823,\n",
              " 2.2565466165542603,\n",
              " 4.075592756271362,\n",
              " 2.27511927485466,\n",
              " 3.966302275657654,\n",
              " 0.6882684677839279,\n",
              " 2.953055202960968,\n",
              " 3.375379741191864,\n",
              " 2.6005107164382935,\n",
              " 3.0271396040916443,\n",
              " 2.001814991235733,\n",
              " 2.9880207777023315,\n",
              " 4.009273648262024,\n",
              " 2.3017702996730804,\n",
              " 3.1838440895080566,\n",
              " 1.3574430346488953,\n",
              " 1.0191267728805542,\n",
              " 3.969452977180481,\n",
              " 4.479048252105713,\n",
              " 4.213904440402985,\n",
              " 2.3738740384578705,\n",
              " 3.4077095985412598,\n",
              " 1.8504205346107483,\n",
              " 4.214026629924774,\n",
              " 2.5192004442214966,\n",
              " 3.3994999527931213,\n",
              " 3.588365316390991,\n",
              " 3.754330277442932,\n",
              " 2.68944650888443,\n",
              " 3.2518991827964783,\n",
              " 4.072572886943817,\n",
              " 3.6674094200134277,\n",
              " 3.4802937507629395,\n",
              " 2.9831984639167786,\n",
              " 3.3096563816070557,\n",
              " 3.9757630228996277,\n",
              " 3.5953983664512634,\n",
              " 3.7980204820632935,\n",
              " 2.769593596458435,\n",
              " 2.1151232719421387,\n",
              " 2.2163932025432587,\n",
              " 1.6270233690738678,\n",
              " 2.906199097633362,\n",
              " 1.9153916835784912,\n",
              " 2.4046005308628082,\n",
              " 3.009219467639923,\n",
              " 3.6143088340759277,\n",
              " 3.1474000215530396,\n",
              " 2.9279205203056335,\n",
              " 2.916838228702545,\n",
              " 1.2195315212011337,\n",
              " 3.5088008642196655,\n",
              " 4.590144753456116,\n",
              " 4.117300808429718,\n",
              " 4.133673012256622,\n",
              " 3.4661924839019775,\n",
              " 3.56322705745697,\n",
              " 4.276840686798096,\n",
              " 2.2153234481811523,\n",
              " 1.5669837594032288,\n",
              " 2.1410392224788666,\n",
              " 3.503601551055908,\n",
              " 3.868396580219269,\n",
              " 0.8156893402338028,\n",
              " 1.4286920428276062,\n",
              " 3.4119734168052673,\n",
              " 4.172632396221161,\n",
              " 3.0711838603019714,\n",
              " 2.940799593925476,\n",
              " 0.7976864278316498,\n",
              " 4.6276092529296875,\n",
              " 2.6558655500411987,\n",
              " 2.020626813173294,\n",
              " 3.470800220966339,\n",
              " 3.7644824385643005,\n",
              " 1.6650892794132233,\n",
              " 3.5114774107933044,\n",
              " 2.9304295778274536,\n",
              " 4.52534019947052,\n",
              " 2.9748255014419556,\n",
              " 2.120368182659149,\n",
              " 3.494727611541748,\n",
              " 3.621942698955536,\n",
              " 1.4071863889694214,\n",
              " 3.393285572528839,\n",
              " 2.7592846751213074,\n",
              " 2.203846722841263,\n",
              " 3.1432032585144043,\n",
              " 3.5064077377319336,\n",
              " 1.651589572429657,\n",
              " 3.23684960603714,\n",
              " 1.6984884440898895,\n",
              " 2.385641932487488,\n",
              " 3.393166959285736,\n",
              " 2.800038158893585,\n",
              " 3.7521621584892273,\n",
              " 3.821699321269989,\n",
              " 1.519245058298111,\n",
              " 3.958643078804016,\n",
              " 2.1474945545196533,\n",
              " 1.8023207783699036,\n",
              " 0.4192368686199188,\n",
              " 1.4689724147319794,\n",
              " 1.773819476366043,\n",
              " 1.8906404078006744,\n",
              " 2.0529738068580627,\n",
              " 4.086858332157135,\n",
              " 3.4157127141952515,\n",
              " 3.1967946887016296,\n",
              " 3.835928738117218,\n",
              " 3.274199366569519,\n",
              " 4.804133474826813,\n",
              " 3.3580631017684937,\n",
              " 3.5854527354240417,\n",
              " 4.239985644817352,\n",
              " 3.8790199160575867,\n",
              " 4.012579619884491,\n",
              " 4.560084939002991,\n",
              " 2.6452848315238953,\n",
              " 2.505858540534973,\n",
              " 3.9938893914222717,\n",
              " 3.6038413643836975,\n",
              " 2.909451127052307,\n",
              " 3.235260248184204,\n",
              " 2.3722411692142487,\n",
              " 2.9814469814300537,\n",
              " 2.1403132379055023,\n",
              " 3.639626204967499,\n",
              " 0.023908524308353662,\n",
              " 3.7101131677627563,\n",
              " 3.5454964637756348,\n",
              " 2.749270498752594,\n",
              " 0.6502685695886612,\n",
              " 3.3383044600486755,\n",
              " 2.497592866420746,\n",
              " 4.300389289855957,\n",
              " 3.498409688472748,\n",
              " 3.440922796726227,\n",
              " 2.9718372225761414,\n",
              " 3.2524585723876953,\n",
              " 2.575906813144684,\n",
              " 2.0661661028862,\n",
              " 1.6422243416309357,\n",
              " 2.9396450519561768,\n",
              " 3.0721405148506165,\n",
              " 1.2033072113990784,\n",
              " 1.1206121742725372,\n",
              " 2.9106467962265015,\n",
              " 2.0195867121219635,\n",
              " 1.8674126267433167,\n",
              " 2.61554092168808,\n",
              " 3.641922175884247,\n",
              " 3.9602985978126526,\n",
              " 4.380226135253906,\n",
              " 4.125816524028778,\n",
              " 3.2579749822616577,\n",
              " 3.3078527450561523,\n",
              " 2.5569599866867065,\n",
              " 4.225663244724274,\n",
              " 3.436742126941681,\n",
              " 3.6751914024353027,\n",
              " 3.0669191479682922,\n",
              " 3.266341984272003,\n",
              " 4.290837645530701,\n",
              " 2.1522103250026703,\n",
              " 3.6028096079826355,\n",
              " 3.40247243642807,\n",
              " 2.796972095966339,\n",
              " 3.148147165775299,\n",
              " 2.2152602672576904,\n",
              " 4.7041842341423035,\n",
              " ...]"
            ]
          },
          "execution_count": 142,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "fts_sc"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 153,
      "metadata": {
        "id": "fa2zlhlUe-5g"
      },
      "outputs": [],
      "source": [
        "with open(\"res2.txt\", \"w\") as rfile:\n",
        "  for i in fts_sc:\n",
        "    rfile.write(str(i)+\"\\n\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 154,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7iAv3iAHfSqC",
        "outputId": "3af2ac98-f8f9-4494-8fdb-ee2dc2b33116"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 88.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 94.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 94.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 100.\n",
            "Use of uninitialized value in subtraction (-) at correlation.pl line 100.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Use of uninitialized value in multiplication (*) at correlation.pl line 106.\n",
            "Pearson: 0.63135\n"
          ]
        }
      ],
      "source": [
        "!perl correlation.pl sts-dev.csv res2.txt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "l20ZmbbUfd9Q"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "003dd16d6f8d462db3041b5d8357bc36": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0062ce6289af4927ac483959550cff3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1535b2eae132474a85c48b39675d4785",
            "placeholder": "​",
            "style": "IPY_MODEL_e7903433d3544f918a166a130b1e582d",
            "value": "Downloading: 100%"
          }
        },
        "037afda0efac4953a0eeb8e7828c92b1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "055510d446f14a289c774f73daa4c998": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1a74639222c245338c5e98589442d223",
              "IPY_MODEL_720aeb8ba97a4914961199e615e7c31d",
              "IPY_MODEL_5c3d340902844f90be1d4a81eb14642c"
            ],
            "layout": "IPY_MODEL_c639e828361b4554b9b62991f9136c54"
          }
        },
        "0606c62286ed49f39e031bc729358f12": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1cb973e5fc3e41f2a517f6feca8b3059",
            "max": 437998343,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_dc17fb3ce0504681af7bb884ca0b8987",
            "value": 437998343
          }
        },
        "08504531b202485aaa18ea068312dd51": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "09e427231287419c90ca7ea2763dc9b6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0b0621a877dc47818c0ac64b04ee4514": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0b7db0af91dd428f9f103c499e7c7953": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "116933b0cb8a42caafd96830cc607756": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "11df1f5a82774061ba0eb515f25e2506": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f9830dfe20574ea5ad71dd18335c625f",
            "max": 278,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e8f1b2422df9457986f1f1c1a01329dd",
            "value": 278
          }
        },
        "13b855ba59694ce994c12a5007f85fa2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "13ce8aafcb764c1a9e62e24775744a13": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1535b2eae132474a85c48b39675d4785": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "15d8611be2504abe82b2f127f50243b5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "178038682e2942568f69e21fea3030d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1817e141ef1b41778e56d9ff39d8099f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a74639222c245338c5e98589442d223": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e494279972be47de898169df3d55b748",
            "placeholder": "​",
            "style": "IPY_MODEL_94488ec867944b2daa02b863fb217bfb",
            "value": "Downloading special_tokens_map.json: 100%"
          }
        },
        "1cb973e5fc3e41f2a517f6feca8b3059": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1cce355f121c43b98882524738eb9ce9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e1efd4eb87b44e8bd7f9b47783b44e1",
            "max": 252,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1dab331710b549488aab2b48bf0df477",
            "value": 252
          }
        },
        "1dab331710b549488aab2b48bf0df477": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1fc63bf478694de6951563c7e896f9be": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "20724fe595014b50956b20edc2796e82": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b0c1f8486e647bb80ac3b9b9bc1e453",
            "max": 1261,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_420c5adf8fdd42858aeb5d7baa24c600",
            "value": 1261
          }
        },
        "260d20d9c62d4365b7b5852d798a8fe9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "276bdee57f53446cae940b9ea1841d5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2a92d364647b483e8515c593410c0550": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2ca9725c94b94774aaf0bdb229b2a5e7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "328e5c80e0bb4e7d91bcf52fea7c4eaf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3317c3066dd049d4b286acbf2cc23f75": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "334210826f2d447aa2a307802dcb740b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_13ce8aafcb764c1a9e62e24775744a13",
            "placeholder": "​",
            "style": "IPY_MODEL_1fc63bf478694de6951563c7e896f9be",
            "value": "Downloading config.json: 100%"
          }
        },
        "35640a4f096b4d3289157979a56c14ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "36e5b462fa7c414da7048f03b78f8075": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e93b47091cc040a19baa7234400737b8",
            "max": 231508,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_116933b0cb8a42caafd96830cc607756",
            "value": 231508
          }
        },
        "37fe97af735a4d48a90f13a83bc9285d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5bbac811c4624e0ca9c6add3d67d1adb",
            "placeholder": "​",
            "style": "IPY_MODEL_4d5a76e4166d4c368277951bf503e5f6",
            "value": "Downloading vocab.txt: 100%"
          }
        },
        "3c50628623cc47039ada0b6b32d77717": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3d8c6d73026342dd8ed3abc68b95d804": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "41b122010cdf4b2fb77a0a210ea336ec": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "420c5adf8fdd42858aeb5d7baa24c600": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4334fd8a1161434ca7b8e43c92b93995": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_003dd16d6f8d462db3041b5d8357bc36",
            "placeholder": "​",
            "style": "IPY_MODEL_fd34ccc53bea4031bb0324b88cc55333",
            "value": "Downloading tokenizer_config.json: 100%"
          }
        },
        "446a973acca84f16a76d5dc6cf16062f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df3f7f04bdbe4f9f8e61e8b9d5e280b9",
            "max": 3836889,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_aac6a03ce30545cd92e684f29ea89c61",
            "value": 3836889
          }
        },
        "4d59af8a27de4ef6ad0b72fb0e9fb667": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_334210826f2d447aa2a307802dcb740b",
              "IPY_MODEL_74a4ab12824a4fa5b4d6d1a75fa32f23",
              "IPY_MODEL_7e85cac165ab4a0db95208bea46aa87a"
            ],
            "layout": "IPY_MODEL_41b122010cdf4b2fb77a0a210ea336ec"
          }
        },
        "4d5a76e4166d4c368277951bf503e5f6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "50774ee5e96a45a1bc188227ef6ed9a9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3c50628623cc47039ada0b6b32d77717",
            "placeholder": "​",
            "style": "IPY_MODEL_2a92d364647b483e8515c593410c0550",
            "value": "Downloading: 100%"
          }
        },
        "5220e601079f44839322b07a6d229aa7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "525f2014d82641799d13fd1c9e27991d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0062ce6289af4927ac483959550cff3d",
              "IPY_MODEL_885696668fc5460c8956c7da863be929",
              "IPY_MODEL_edad53b846fd4a67aeafe2b55910e29e"
            ],
            "layout": "IPY_MODEL_3d8c6d73026342dd8ed3abc68b95d804"
          }
        },
        "567eef54b8264b8c9f581715f04a2b5f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5bbac811c4624e0ca9c6add3d67d1adb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5c3d340902844f90be1d4a81eb14642c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6db521a8a1a845268e7d53769d13ec0c",
            "placeholder": "​",
            "style": "IPY_MODEL_c0b70f8fa29d48d5a067dce22e7455fd",
            "value": " 112/112 [00:00&lt;00:00, 3.48kB/s]"
          }
        },
        "66b1d8c3d1ae426fb2e302529e5a696b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6744c1358205465c888b9fa75353bfe0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6b0c1f8486e647bb80ac3b9b9bc1e453": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6cc9a9c6f06d4b18b1b7f7e49cd3415a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7fbb1cfe230544448f7eda3e92574373",
              "IPY_MODEL_20724fe595014b50956b20edc2796e82",
              "IPY_MODEL_a2a1003a84ea4c2aac79f4b909e071b1"
            ],
            "layout": "IPY_MODEL_769aab99065240c8a6a4bd65b6d9c17f"
          }
        },
        "6db521a8a1a845268e7d53769d13ec0c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "720aeb8ba97a4914961199e615e7c31d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fcab23c459e846f9af532cc51f018fff",
            "max": 112,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8b9927f8e01a416d9a24a10dd4cd6a0d",
            "value": 112
          }
        },
        "74a4ab12824a4fa5b4d6d1a75fa32f23": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2ca9725c94b94774aaf0bdb229b2a5e7",
            "max": 689,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_08504531b202485aaa18ea068312dd51",
            "value": 689
          }
        },
        "769aab99065240c8a6a4bd65b6d9c17f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7e85cac165ab4a0db95208bea46aa87a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b362abe5cfa7435a8cad51e76c33971d",
            "placeholder": "​",
            "style": "IPY_MODEL_13b855ba59694ce994c12a5007f85fa2",
            "value": " 689/689 [00:00&lt;00:00, 21.3kB/s]"
          }
        },
        "7fbb1cfe230544448f7eda3e92574373": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_15d8611be2504abe82b2f127f50243b5",
            "placeholder": "​",
            "style": "IPY_MODEL_35640a4f096b4d3289157979a56c14ae",
            "value": "Downloading: 100%"
          }
        },
        "823947b3ab524cec992c0e504c279dea": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c34cc9ae1d3d44e08c7b7cced0845d89",
              "IPY_MODEL_11df1f5a82774061ba0eb515f25e2506",
              "IPY_MODEL_a0d6654d739e413ab5dad65e325412d7"
            ],
            "layout": "IPY_MODEL_66b1d8c3d1ae426fb2e302529e5a696b"
          }
        },
        "885696668fc5460c8956c7da863be929": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_037afda0efac4953a0eeb8e7828c92b1",
            "max": 92668928,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d3ae4937d3a843d294c38d7529035a94",
            "value": 92668928
          }
        },
        "8b9927f8e01a416d9a24a10dd4cd6a0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8d448dac7e664c5790cc5bcbb5c1b72e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8e1efd4eb87b44e8bd7f9b47783b44e1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "94488ec867944b2daa02b863fb217bfb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9474bf1b7f2e41d6b7f02265ab211508": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a637c28d7f3449f7b90821702173dba1",
              "IPY_MODEL_0606c62286ed49f39e031bc729358f12",
              "IPY_MODEL_d13809843a1d498b987b8523971d0b0f"
            ],
            "layout": "IPY_MODEL_8d448dac7e664c5790cc5bcbb5c1b72e"
          }
        },
        "96ff0a3c81e94c1792e7a183a506b339": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a0d6654d739e413ab5dad65e325412d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1817e141ef1b41778e56d9ff39d8099f",
            "placeholder": "​",
            "style": "IPY_MODEL_0b7db0af91dd428f9f103c499e7c7953",
            "value": " 278/278 [00:00&lt;00:00, 8.26kB/s]"
          }
        },
        "a0f4fc1727d140f79e19cdec457fd395": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_09e427231287419c90ca7ea2763dc9b6",
            "placeholder": "​",
            "style": "IPY_MODEL_276bdee57f53446cae940b9ea1841d5a",
            "value": " 252/252 [00:00&lt;00:00, 7.04kB/s]"
          }
        },
        "a2a1003a84ea4c2aac79f4b909e071b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_260d20d9c62d4365b7b5852d798a8fe9",
            "placeholder": "​",
            "style": "IPY_MODEL_c8ab752034fd423b92a814171647a577",
            "value": " 1.26k/1.26k [00:00&lt;00:00, 36.6kB/s]"
          }
        },
        "a637c28d7f3449f7b90821702173dba1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_567eef54b8264b8c9f581715f04a2b5f",
            "placeholder": "​",
            "style": "IPY_MODEL_d2098375206f41bfada8829e15930be9",
            "value": "Downloading pytorch_model.bin: 100%"
          }
        },
        "aac6a03ce30545cd92e684f29ea89c61": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ac61e88d126149d7aec525043a473e1c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b0f81c177e8f46b3b0063659d0e6e603": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f92dbe50635141ffa7810f07ecbede0c",
            "placeholder": "​",
            "style": "IPY_MODEL_328e5c80e0bb4e7d91bcf52fea7c4eaf",
            "value": " 226k/226k [00:00&lt;00:00, 279kB/s]"
          }
        },
        "b362abe5cfa7435a8cad51e76c33971d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be57c23214644b9b854443c78eb8bfa6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_96ff0a3c81e94c1792e7a183a506b339",
            "placeholder": "​",
            "style": "IPY_MODEL_178038682e2942568f69e21fea3030d2",
            "value": " 3.84M/3.84M [00:01&lt;00:00, 3.04MB/s]"
          }
        },
        "be9ca588b16b49efa7fa20254df146c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_37fe97af735a4d48a90f13a83bc9285d",
              "IPY_MODEL_36e5b462fa7c414da7048f03b78f8075",
              "IPY_MODEL_b0f81c177e8f46b3b0063659d0e6e603"
            ],
            "layout": "IPY_MODEL_d355ebce9ef44c5ea20704f8a386ca6f"
          }
        },
        "c0b70f8fa29d48d5a067dce22e7455fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c34cc9ae1d3d44e08c7b7cced0845d89": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3317c3066dd049d4b286acbf2cc23f75",
            "placeholder": "​",
            "style": "IPY_MODEL_dda9fa4615cc4c0c86ff1f8f8c7aa2b6",
            "value": "Downloading: 100%"
          }
        },
        "c639e828361b4554b9b62991f9136c54": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c8ab752034fd423b92a814171647a577": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d13809843a1d498b987b8523971d0b0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0b0621a877dc47818c0ac64b04ee4514",
            "placeholder": "​",
            "style": "IPY_MODEL_fd06b534c85e4c02aea2400660b12e2f",
            "value": " 418M/418M [00:10&lt;00:00, 24.8MB/s]"
          }
        },
        "d2098375206f41bfada8829e15930be9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d355ebce9ef44c5ea20704f8a386ca6f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d3ae4937d3a843d294c38d7529035a94": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d8006ec1ce484b2db878efa2de803346": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4334fd8a1161434ca7b8e43c92b93995",
              "IPY_MODEL_1cce355f121c43b98882524738eb9ce9",
              "IPY_MODEL_a0f4fc1727d140f79e19cdec457fd395"
            ],
            "layout": "IPY_MODEL_ac61e88d126149d7aec525043a473e1c"
          }
        },
        "dc17fb3ce0504681af7bb884ca0b8987": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "dda9fa4615cc4c0c86ff1f8f8c7aa2b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "df3f7f04bdbe4f9f8e61e8b9d5e280b9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e494279972be47de898169df3d55b748": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e7903433d3544f918a166a130b1e582d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e8f1b2422df9457986f1f1c1a01329dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e93b47091cc040a19baa7234400737b8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed8995012b8342829991156406f97898": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_50774ee5e96a45a1bc188227ef6ed9a9",
              "IPY_MODEL_446a973acca84f16a76d5dc6cf16062f",
              "IPY_MODEL_be57c23214644b9b854443c78eb8bfa6"
            ],
            "layout": "IPY_MODEL_f2dddfa584274ee69b0681b896104c46"
          }
        },
        "edad53b846fd4a67aeafe2b55910e29e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6744c1358205465c888b9fa75353bfe0",
            "placeholder": "​",
            "style": "IPY_MODEL_5220e601079f44839322b07a6d229aa7",
            "value": " 92.7M/92.7M [00:06&lt;00:00, 18.8MB/s]"
          }
        },
        "f2dddfa584274ee69b0681b896104c46": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f92dbe50635141ffa7810f07ecbede0c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f9830dfe20574ea5ad71dd18335c625f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fcab23c459e846f9af532cc51f018fff": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fd06b534c85e4c02aea2400660b12e2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fd34ccc53bea4031bb0324b88cc55333": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}