9_Neural_Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_MZAf9xt4FcE"
   },
   "source": [
    "Welcome to the AI4ALL notebook on neural networks!\n",
    "\n",
    "This notebook is intended to span two days. A good stopping point for the first day would be midway through the section on backpropagation, but feel free to proceed at your own pace(if you get done early there are some open-ended questions you could pursue at the bottom).\n",
    "\n",
    "Deep learning (and thus neural networks) are currently very popular in AI and machine learning, but the core algorithms involved are actually fairly old and very simple. \n",
    "\n",
    "Here, we will be implementing a simple neural network, alternately known as a fully-connected(FC) network or multilayer perceptron, and then training it using the backpropagation algorithm.\n",
    "\n",
    "To simplify the process, we will use the Pytorch framework, which provides us with an environment and pre-implemented neural network algorithms.\n",
    "\n",
    "We will also use numpy and matplotlib for handling our data and visualizing it, respectively.\n",
    "\n",
    "Portions of this code were adapted with permission from the Princeton COS485 Neural Networks course taught by Sebastian Seung."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "p3Xa-UdqBuQP"
   },
   "outputs": [],
   "source": [
    "#First, let's set up our environment\n",
    "# http://pytorch.org/\n",
    "from os import path\n",
    "from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
    "\n",
    "import torch, torchvision\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython import display \n",
    "import time, math\n",
    "\n",
    "#Allow inline plotting within the notebook and use a grayscale colormap\n",
    "%matplotlib inline\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "#Fix the numpy random seed so we know what we're getting\n",
    "np.random.seed(seed=485)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "P_kuHM4qCZSm"
   },
   "source": [
    "For illustration purposes, we will use the MNIST dataset of 60,000 handwritten digits, each labeled 0 through 9. \n",
    "\n",
    "The problem we will try to solve on this dataset is to classify each example image as one of the 10 digits.\n",
    "\n",
    "This dataset is considered a standard for developing and testing machine learning methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 255,
     "status": "ok",
     "timestamp": 1532468114727,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "SEyRoOjaCyfh",
    "outputId": "f72e926f-b471-45e3-db55-7876ab067e62"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data shape: torch.Size([60000, 28, 28])\n",
      "Labels shape: torch.Size([60000])\n"
     ]
    }
   ],
   "source": [
    "#load the data\n",
    "mnist_train = torchvision.datasets.MNIST(root='data', train=True, download=True) # train data only\n",
    "mnist_test = torchvision.datasets.MNIST(root='data', train=False, download=True) # test data only\n",
    "\n",
    "digits_train = \n",
    "digits_test = \n",
    "\n",
    "#Let's take a look at the shape of this data\n",
    "print(\"Data shape: {}\".format(mnist_train.train_data.size()))\n",
    "print(\"Labels shape: {}\".format(mnist_train.train_labels.size()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "46O1Gb9fn7Om"
   },
   "source": [
    "The dataset is ordered (starting with examples labeled '0', ending with examples labeled '9'). To generate train examples, use 60,000 random indices into the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "xeMgQ0V5oi2b"
   },
   "outputs": [],
   "source": [
    "indices = torch.randperm(60000)\n",
    "trainimages = mnist_train.train_data[indices]\n",
    "trainlabels = mnist_train.train_labels[indices]\n",
    "\n",
    "testindices = torch.randperm(10000)\n",
    "testimages = mnist_test.test_data[testindices]\n",
    "testlabels = mnist_test.test_labels[testindices]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "i0n_oFBDoshp"
   },
   "source": [
    "Before we train anything, it helps to get a look at our data. Let's plot the first image in our training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "4FOqSwK3Wcnb"
   },
   "outputs": [],
   "source": [
    "#A plotting function to visualize images\n",
    "def plot_image(image):\n",
    "    nr, nc = image.size() \n",
    "    extent = [-0.5, nc-0.5, nr-0.5, -0.5]\n",
    "    plt.imshow(image.numpy(), extent=extent, origin='upper', interpolation='nearest') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 340,
     "status": "ok",
     "timestamp": 1532468118991,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "wi6B9QdVorCV",
    "outputId": "c31d78bf-6a56-42eb-cd7e-1294d7bdd4b9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFAFJREFUeJzt3X2wVPV9x/H3B4VqwVxRKgFUwIcaox1NBOyEaLBGo9EEM6OppEk0xqA2piYxUx1b60OjpW2IiZM2Iz5io0StSgzRIQ7VGDOtShiGIORBDQhyuYRnMAIq3/5xzk0WvHt2797de+7l93nN7Nzd8z1nz/eeez97nnb3KCIws/QMKLsBMyuHw2+WKIffLFEOv1miHH6zRDn8ZolKKvySDpW0VdJeZffSG/Lf9bCy++gtkq6X9L2y++gv9sjwS1om6Y38n7/zNjIiXo2IIRHxdgPPeaGkZ1vRb/78T0u6uJnPmf+urzTQyxhJIWnBbsOHSdohaVnFsGWSOiQNrhh2saSnKx6HpCPy+/tLukvSaklbJP1a0lUVL8ydt5D0esXjkxpZBgW/4z2Svt7M5yxzPo3YI8Of+1j+z995W1U0sjJ9dnlI2ruE2Q6WdGzF408Bv+1ivL2BK+p8zluAIcDRQBvwceDlihfmIRExJB/3uIphP23wd7BqImKPuwHLgA93MXwMEMDe+eOngZuAnwFvAEcAFwKvAFvI/tH/huwfdRvwNrAV2Fhlvm3AnUA78BrwdWCvvHYh8CzwDWBD/txn5rWb8ufelj//d/LhAXwR+E0+/n8A03eb5w+BL1fpJ4Aj8vv35NP/KP/dngMOrzJd53L6R+DfK4bPB/4BWLbbsr4aWA/snw+7GHi6Sh+LgXPq+Bv+YZqCccYCP8l/nyeB7wDfq6g/BKwGNgHPAMfkw6cCbwI78uX9w3z41cDL+fMtAT5R8VxH5PPaBKwFHqiovSef/3rgV8Ani+bTV26lN9CSX6p74X8VOIZs7dUGbAaOyusjKv5hLgSerTHf2cBtwGDgIOB54JKK6d8EvgDsBVwGrAJU0cvFXQTgSeAAYF9gQj7NgLw+DPg9MLxKP7uHf33+HHsD9wHfrzJd53IaA6zI+z06/8f+MO8M/4eBR4Cv58OKwn8H8CLwOeDIgmVZT/j/F/gm8CfAyXloK8N/EbBfXv8WsLCidk9nvxXDzgNGkm0R/zXwOjAir80ie+EbAOwDfDAfPjhfRp/Ll+v7yV4cjqk2n75y67ObuU0wW9LG/Da7YLx7IuLFiHgLeAvYCRwrad+IaI+IF+uZmaThwJlka+HXI2IN2Sbu+RWjLY+I2yM75jCT7MVleI2n/peIWB8Rb0TE82RrnlPz2vlkIeuop0fgkYh4Pv9d7wOOrzH+Sv4Y+AuAewvG/SfgS5L+rMZzfimf9+XAEkkvSTqzru4rSDoUGA9cGxHbI+IZsq2gP4iIuyJiS0RsB64HjpPUVu05I+KhiFgVETsj4gGyLa4JeflNYDQwMiK2RUTn8Z+zyV4M746ItyJiAfAwcG53f6fetieH/5yI2D+/nVMw3orOOxHxOtkr/qVAu6QfSXpPnfMbDQzMp9soaSPZVsBBFeOsrpjX7/O7Qyi2YrfHM4FP5/c/DfxXnf3tMn+yLYZa84Ys8BcCU4CqR9IjYjEwh2zTuar8RezmiDgBOBB4EHhI0gF19FJpJLAh/5t1Wt55R9JekqZJelnSZrItFMi2lrok6bOSFlb8/Y6tGP/vAQHPS3pR0kX58NHAiRUrmo1ku4rv7ubv0+v25PDXa5ePNUbE3Ig4jWyt/Evg9q7G68IKYDswrOJF510RcUwjfRQM/x4wWdJxZJviRVs1zfAwcBbwSkQsrzHudWS7NaPqeeKI2AzcTLbpPLabfbUDQyvPMgCHVtz/FDCZbKuljWwXBrIAw27LVdJosr/15cCBEbE/2fEJ5b2ujogvRMRI4BLgP/MzGCuAn1T8zfeP7ADlZV3Npy9x+CtIGi7p4/k/1HaygzSdpwU7gIMlDepq2ohoB34MTJf0LkkDJB0u6UN1zr4DqHlOPiJWAi+QrfEfjog36nz+huRr1r8i24+vNe5LwAPA31UbR9K1ksZLGiRpH7KzBBvJdi+609dysgOQN+TP9UHgYxWj7Ef2N1wH/CnZi0yl3Zf3YLKg/i7v83Nka/7Ovs+TdHD+cEM+7ttkWzt/Lukzkgbmt/GSjq4ynz7D4d/VAOBKsoNq64EPAX+b1/6H7EDVaklrq0z/WWAQ2ZHiDcB/k21B1OPbwLmSNki6tca4M4G/oHub/A2LiPkR8XKdo99IFqSqTwfcTXZQbBVwGnBWRGxtoLVPASeS/a2uY9djEveS7Qa8Rvb3+L/dpr0TeG/nMaGIWAJMJzuI2EG2fH9WMf544DlJW4HHgCsi4rcRsQU4nez4yyqyXat/JTvI+I75NPA7tkznkWbrRySdTLb5PyYidpbdj/VPXvP3M5IGkm0q3+HgW084/P1Ivh+5kWxX4lslt2P9nDf7zRLlNb9Zonr1wyKSvJlh1mIRodpj9XDNL+kMSb/K36JZ+M4uM+tbGt7nz78Q49dk52k733gyJT9fWm0ar/nNWqw31vwTgJci4pWI2AF8n+ztlGbWD/Qk/KPY9UMnK+niPd2SpkqaL2l+D+ZlZk3WkwN+XW1avGOzPiJmADPAm/1mfUlP1vwrgUMqHh9M9t5mM+sHehL+F4AjJY3NP+l2PtkHHsysH2h4sz8i3pJ0OTCX7Gue7qr3W2/MrHy9+vZe7/ObtV6vvMnHzPovh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiWr4Et3WPwwdOrSwfu211xbWt23bVli/5pprut2T9Q09Cr+kZcAW4G3grYgY14ymzKz1mrHmPyUi1jbhecysF3mf3yxRPQ1/AD+W9HNJU7saQdJUSfMlze/hvMysiXq62T8xIlZJOgh4UtIvI+KZyhEiYgYwA0BS9HB+ZtYkPVrzR8Sq/Oca4FFgQjOaMrPWazj8kgZL2q/zPnA6sLhZjZlZaymisS1xSYeRre0h2324PyJuqjGNN/tb4AMf+EDV2vTp0wunPfHEEwvr8+cXH6r5yEc+UljfsGFDYd2aLyJUz3gN7/NHxCvAcY1Ob2bl8qk+s0Q5/GaJcvjNEuXwmyXK4TdLVMOn+hqamU/1NaStra2wvnTp0qq1HTt2FE67aNGiwvrJJ59cWN+0aVNhffLkyVVrCxcuLJzWGlPvqT6v+c0S5fCbJcrhN0uUw2+WKIffLFEOv1miHH6zRPmru/uB444r/vBk0fsAzj777MJpn3rqqcL6gQceWFi/5ZZbCuv3339/1dopp5xSOG1HR0dh3XrGa36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFE+z98PTJo0qbA+a9asqrVa5/FrWbduXWH9q1/9amF97ty5VWvjx48vnHbOnDmFdesZr/nNEuXwmyXK4TdLlMNvliiH3yxRDr9Zohx+s0T5PH8/cNFFFxXWa31mv5XWrl1bWF+8eHHV2ujRo5vdjnVDzTW/pLskrZG0uGLYAZKelPSb/OfQ1rZpZs1Wz2b/PcAZuw27GpgXEUcC8/LHZtaP1Ax/RDwDrN9t8GRgZn5/JnBOk/sysxZrdJ9/eES0A0REu6SDqo0oaSowtcH5mFmLtPyAX0TMAGaAL9Rp1pc0eqqvQ9IIgPznmua1ZGa9odHwPwZckN+/APhBc9oxs95Sc7Nf0ixgEjBM0krgOmAa8KCkzwOvAue1ssnURRTvLW3YsKGXOrE9Sc3wR8SUKqVTm9yLmfUiv73XLFEOv1miHH6zRDn8Zoly+M0S5Y/09gOrV68uu4WqRo4cWVifOHFi1drs2bOb3Y51g9f8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1mifJ6/H1ixYkVh/Wtf+1rV2le+8pVmt7OLyZMnF9Y3b95ctfb44483ux3rBq/5zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEqdbXQjd1Zr5iT0MOO+ywwvoTTzxRtXb33XcXTjtt2rTCeltbW2F95cqVhfUpU6p9+TPMmTOncFprTESonvG85jdLlMNvliiH3yxRDr9Zohx+s0Q5/GaJcvjNEuXz/HuAonPx8+bNK5x27dq1PZr3m2++WVg/99xzq9a2b9/eo3lb15p2nl/SXZLWSFpcMex6Sa9JWpjfPtqTZs2s99Wz2X8PcEYXw2+JiOPzm7+SxayfqRn+iHgGWN8LvZhZL+rJAb/LJS3KdwuGVhtJ0lRJ8yXN78G8zKzJGg3/d4HDgeOBdmB6tREjYkZEjIuIcQ3Oy8xaoKHwR0RHRLwdETuB24EJzW3LzFqtofBLGlHx8BPA4mrjmlnfVPM8v6RZwCRgGNABXJc/Ph4IYBlwSUS015yZz/P3un322aewPnfu3ML62LFjC+snnXRSYX358uWFdWu+es/z17xoR0R09W0Md3a7IzPrU/z2XrNEOfxmiXL4zRLl8JslyuE3S5Qv0b2HGzhwYGF92LBhhfV99923sL5o0aLC+qOPPlq1dsUVVxROu2nTpsK69YzX/GaJcvjNEuXwmyXK4TdLlMNvliiH3yxRDr9Zonyefw93xhldfffqH61bt66wPmFC8fe01KrfeOONVWvnnXde4bR33HFHYd16xmt+s0Q5/GaJcvjNEuXwmyXK4TdLlMNvliiH3yxRvkT3Hu6JJ54orM+ePbuwftttt/Vo/hMnTqxamzVrVuG0hx56aI/mnaqmXaLbzPZMDr9Zohx+s0Q5/GaJcvjNEuXwmyXK4TdLVM3P80s6BLgXeDewE5gREd+WdADwADCG7DLdn4yIDa1r1aoZNWpU1dppp51WOO2VV17Z7HZ2ccQRR1SttbW1tXTeVqyeNf9bwJURcTTwl8AXJb0XuBqYFxFHAvPyx2bWT9QMf0S0R8SC/P4WYCkwCpgMzMxHmwmc06omzaz5urXPL2kM8D7gOWB4RLRD9gIBHNTs5sysder+Dj9JQ4CHgS9HxGaprrcPI2kqMLWx9sysVepa80saSBb8+yLikXxwh6QReX0EsKaraSNiRkSMi4hxzWjYzJqjZviVreLvBJZGxDcrSo8BF+T3LwB+0Pz2zKxV6tnsnwh8BviFpIX5sGuAacCDkj4PvAoUfw+zlWLr1q2F9SVLlrR0/qNHj65ae+ihh1o6bytWM/wR8SxQbQf/1Oa2Y2a9xe/wM0uUw2+WKIffLFEOv1miHH6zRDn8ZonyJbr3cAMGFL++Dxo0qLC+Y8eOwvpRRx1VWL/sssuq1s4666zCaa21vOY3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8/x7gG3btlWt1boE+6233lpYnzt3bmH9qquuKqxfeumlVWsLFiwonNZay2t+s0Q5/GaJcvjNEuXwmyXK4TdLlMNvliiH3yxRqnUeuKkzk3pvZgbA6aefXli/+eabC+snnHBCj6a/4YYbqtZqfVeANSYi6rqWntf8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1miap7nl3QIcC/wbmAnMCMivi3peuALwO/yUa+JiMdrPJfP85u1WL3n+esJ/whgREQskLQf8HPgHOCTwNaI+Ea9TTn8Zq1Xb/hrfpNPRLQD7fn9LZKWAqN61p6Zla1b+/ySxgDvA57LB10uaZGkuyQNrTLNVEnzJc3vUadm1lR1v7df0hDgJ8BNEfGIpOHAWiCAfybbNbioxnN4s9+sxZq2zw8gaSAwB5gbEd/soj4GmBMRx9Z4HoffrMWa9sEeSQLuBJZWBj8/ENjpE8Di7jZpZuWp52j/B4GfAr8gO9UHcA0wBTiebLN/GXBJfnCw6Lm85jdrsaZu9jeLw2/Wev48v5kVcvjNEuXwmyXK4TdLlMNvliiH3yxRDr9Zohx+s0Q5/GaJcvjNEuXwmyXK4TdLlMNvliiH3yxRNb/As8nWAssrHg/Lh/VFfbW3vtoXuLdGNbO30fWO2Kuf53/HzKX5ETGutAYK9NXe+mpf4N4aVVZv3uw3S5TDb5aossM/o+T5F+mrvfXVvsC9NaqU3krd5zez8pS95jezkjj8ZokqJfySzpD0K0kvSbq6jB6qkbRM0i8kLSz7+oL5NRDXSFpcMewASU9K+k3+s8trJJbU2/WSXsuX3UJJHy2pt0MkPSVpqaQXJV2RDy912RX0Vcpy6/V9fkl7Ab8GTgNWAi8AUyJiSa82UoWkZcC4iCj9DSGSTga2Avd2XgpN0r8B6yNiWv7COTQiruojvV1PNy/b3qLeql1W/kJKXHbNvNx9M5Sx5p8AvBQRr0TEDuD7wOQS+ujzIuIZYP1ugycDM/P7M8n+eXpdld76hIhoj4gF+f0tQOdl5UtddgV9laKM8I8CVlQ8XkmJC6ALAfxY0s8lTS27mS4M77wsWv7zoJL72V3Ny7b3pt0uK99nll0jl7tvtjLC39WlhPrS+caJEfF+4Ezgi/nmrdXnu8DhZNdwbAeml9lMfln5h4EvR8TmMnup1EVfpSy3MsK/Ejik4vHBwKoS+uhSRKzKf64BHiXbTelLOjqvkJz/XFNyP38QER0R8XZE7ARup8Rll19W/mHgvoh4JB9c+rLrqq+yllsZ4X8BOFLSWEmDgPOBx0ro4x0kDc4PxCBpMHA6fe/S448BF+T3LwB+UGIvu+grl22vdll5Sl52fe1y96W8wy8/lfEtYC/groi4qdeb6IKkw8jW9pB93Pn+MnuTNAuYRPaRzw7gOmA28CBwKPAqcF5E9PqBtyq9TaKbl21vUW/VLiv/HCUuu2Ze7r4p/fjtvWZp8jv8zBLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNE/T9O3NiuFizoEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plot first image\n",
    "plt.figure()\n",
    "ax = plt.subplot(111)\n",
    "ax.grid(False)\n",
    "plot_image(trainimages[0])\n",
    "plt.title(\"First entry in MNIST dataset\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5TgDtjZYo98v"
   },
   "source": [
    "Looks like a number, right? What about the mean(average) of all images in the set?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 778,
     "status": "ok",
     "timestamp": 1532468121028,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "6PhywVxupGLt",
    "outputId": "364a2848-079d-48b6-c02f-c2dbe1bef6b6"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGBRJREFUeJzt3XvwXGV9x/H3h9xv5EJCSMIliKHjpSM6Ea2gxjtlakErFloVWjBOK6N2aAulVGiLlTreaJ2xxkK5VFAqgrRjVWCkaK1IoFRA5JpAbgQCuf0C4ZJ8+8c50eXHb59ns5ffbvJ8XjOZ7O989+w+e/Z895zd73meRxGBmZVnn343wMz6w8lvVignv1mhnPxmhXLymxXKyW9WKCd/F0g6StL9koYkHd+H5z+4fu4xg/yYg07SJZLO73c7RstAJr+klZKelTR72PI7JIWkhf1pWVN/A3wpIqZGxLXdfOB6W7w9dZ+IeKR+7h3det5OHlPSkvp9+taw5a+ql9/UsCwk3Slpn4Zl50u6pL69sL7P2PrvAyVdLWmDpM31uqdIemP9YTUkaVu9zlDDv4Pb3RZNXuNNkk7r5mOO9vMMZPLXVgAn7fpD0q8Dk/rXnKRDgLv78cS7kmIAPQ68QdJ+DctOBu4b4b7zgRNbfNzLgVVU23w/4EPA+oj4Yf1hNRV4RX3fGbuWRcQjbb2KvVlEDNw/YCVwDnBrw7LPAn8JBLCwXjahXv4IsB74J2BSHZsJ/AfVTrixvn1gw+PdBPwt8N/AVuD7wOxEmz4MPAA8CVwHzK+XPwjsBJ4GhoAJI6w7H7i6bssK4GMNsfOAq4DL6nbcDSyuY5cPe+w/BxbW2+DU+nXf3LBsbL3edOAiYB2wBjgfGFPHXgr8F7AZ2AB8o8nrHf6YLW8vYAmwun4/PlovG1Mv+yRwU8N9AzgTuL/huc4HLmnSjiHgiMz+84J1Evd7NXB7/Xq+AXwdOD+3/wCfAnYA2+v2fKlefiHVB9MW4DbgjQ3PdSSwvI6tBz7fEHs98GNgE/B/wJLU83Qtz/qd6InkfztwL/CyesfZ9WnfmPxfpErEWcA04N+BT9ex/YDfASbXsX8Drh2W/A8Ch1OdUdwEXNCkPW+tE+U1VB84/wjcPLy9Tdbdp94RPgmMB14CPAS8qyH5twPH1q/z08BPmj12w459GTClbvvwBLkW+Eod3x/4KfCROnYl1YfoPsBE4OjdSP5Wt9cSqkR/A3BLvexY4HvAabw4+RfV2+i0FpL/BqoPoBOBg9tN/vq9eBj4E2Ac8D7gOX6V/K3sP6cNe8wP1OuNBc4AHgUm1rH/AT5Y354KvL6+vQB4ot4++wDvqP+e0+x5Skn+c+pkOAa4vt6oUb+5ArYBhzWs9xvAiiaPeQSwcdibd07D338MfLfJuhcBn2n4e2q9oyxsbG+TdV8HPDJs2V8A/1LfPg+4oSH2cuDp4dtihB37JSPt7MBc4BnqM6A6fhLwg/r2ZcAyGs6CWkmg3dxeS4DV9e37gV+jOqr+PiMn/0vrnf8Rqg/XVPLPBC6gOkPaAdwBvDbV9iZtfBOwFlDDsh9TJ3+L+08yKanOGF5V374Z+GuGnS1RnfVcPmzZ94CTW32edv8N8nd+qE57fw84hWqnbTSH6lP5NkmbJG0CvlsvR9JkSV+R9LCkLVQbf8awX68fbbj9FFVSj2Q+1VECgIgYovp0XtDCazgEmL+rjXU7z6ZK0mbtmNjCd/lViecbB6xreL6vUJ0BQPXVQcBPJd0t6Q9beA3N2tlsezW6HDgdeAtwTbM7RcR3qJJ/aerBImJjRJwVEa+g2oZ3ANdKUgttaTQfWBN1htV++R63uP+8gKQzJN1T/xC5ierr164frU+lOmv6haRbJf1WvfwQ4IRh+8fRwLzdfD27bVB/LAIgIh6WtILqqHDqsPAGqu/Cr4iINSOsfgbVEed1EfGopCOA/6Xa8XfXWqo3CQBJU6hO70Z63uFWUZ2NLGrjeaE6gu3O8lVUR/7ZEfH8i1aKeJTq9wskHQ3cIOnmiHigzfblXE71W8llEfFUJkfPoTpDuKKVB46IDZI+S/VD4iyqD+RWrQMWSFLDB8DBVF9tIL//vGD7S3oj1VH8bcDdEbFT0sZd94+I+4GT6qrGe4Fv1j+GrqI68n+42cvcjde0Wwb9yA9V0r81IrY1LoyIncBXgS9I2h9A0gJJ76rvMo3qw2GTpFnAuR204QrgDyQdIWkC8HdU32VXtrDuT4Etks6UNEnSGEmvlPTaFp97PdXvBC2JiHVUP8Z9TtK+kvaRdJikNwNIOkHSgfXdN1LtXF0rEY7QnhXAm6l+Z8jd9ybgTqpkHpGkv6+331hJ04A/Ah6IiN1JfKi+gz8PfKx+rPdS/Si3S27/Gf6+TKsf73FgrKRPAvs2tPsDkubU++2mevEO4F+Bd0t6V71vTKxLpbveo916/3fHwCd/RDwYEcubhM+kOqr8pD41u4Hq0xqqHwMnUZ0h/ITqK0G7bbgR+CuqX+zXAYfRYmkqqjr5u6m+M66o2/PPVKeErfg0cE59SvinLa7zIaoftH5OleDf5Fenka8FbpE0RPVj6cfrBO2ZiPhRRKxt8e7nUB3Fm5lM9fVhE9UPp4cAv91Gm56lOgKfQrWNfhdovC4ht/9cCLxP0kZJ/0D1Pf0/qUqZD1P9iNv41ewY4O56u18InBgR2yNiFXAc1VfBx+t1/oxf5ebw5+kavfArj5mVYuCP/GbWG05+s0I5+c0K5eQ3K9So1vkl+ddFsx6LiJauZenoyC/pGEn3SnpA0lmdPJaZja62S331ZY73UXVEWA3cCpwUET9PrOMjv1mPjcaR/0iqK6seqi+Y+DrVxQpmtgfoJPkX8MIrmFYzQkcXSUslLZfU7Co9M+uDTn7wG+nU4kWn9RGxjKoLqU/7zQZIJ0f+1cBBDX8fSNX7zcz2AJ0k/63AIkmHShpP1dHluu40y8x6re3T/oh4XtLpVL2ZxgAXR0RfBrE0s903qr36/J3frPdG5SIfM9tzOfnNCuXkNyuUk9+sUE5+s0I5+c0KNdDj9lslNx9FKt7Jut2IdyJXht65c2fb6+ceu4SBbX3kNyuUk9+sUE5+s0I5+c0K5eQ3K5ST36xQLvV1QaflsH32SX8G5+JjxjSdMp6xY9Nvcafx1HPn7NiRnhz4+edfNMP4Czz33HNtx3PPnYvvDaVAH/nNCuXkNyuUk9+sUE5+s0I5+c0K5eQ3K5ST36xQrvO3KFWrz9W6c7XyCRMmJOOTJk1KxqdMmdI0Nm3atOS6M2bMSManTp2ajOfanup2+9RTTyXX3bx5czK+ZcuWttcfGhpKrvvMM88k47lrDDrpbjxafOQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFBOfrNCuc5fy/WZT9Xqc7XuyZMnJ+PTp09PxmfNmpWMz5s3r2lswYIFyXVz8Tlz5iTjudeW6he/adOm5Lpr1qxJxletWpWMr169umns0UcfTa67cePGZDx3jULuOoHUWAWjdQ1AR8kvaSWwFdgBPB8Ri7vRKDPrvW4c+d8SERu68DhmNor8nd+sUJ0mfwDfl3SbpKUj3UHSUknLJS3v8LnMrIs6Pe0/KiLWStofuF7SLyLi5sY7RMQyYBmApP73ZjAzoMMjf0Ssrf9/DLgGOLIbjTKz3ms7+SVNkTRt123gncBd3WqYmfVWJ6f9c4Fr6n7uY4ErIuK7XWlVD3RSx4d0LT/XZz5Xpz/ggAOS8YMOOigZP/TQQ9uKARxyyCHJ+Ny5c5PxXJ0/Vc9+8sknk+vm6vgzZ85MxidOnNg0lptLodPpwXPj/qfWz63bLW0nf0Q8BLyqi20xs1HkUp9ZoZz8ZoVy8psVyslvVignv1mh9pouvZ1Og50r9aWGz953332T6+a6xeZKeYcffngyvmjRoqaxhQsXJtdNdQeG/GsbN25cMp4a4jo3JHluSPTc8NlPP/1001hu2O+tW7cm49u2bUvGt2/fnozn9tfR4CO/WaGc/GaFcvKbFcrJb1YoJ79ZoZz8ZoVy8psVynX+Wq5eneoe2mmdPzd89sEHH5yMz58/v2ksNyx4rvtortttrutr6n3JrZt7T3JdqVPxTqcez12DMAh1/Bwf+c0K5eQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFCu87cYT9Wcc/3Sc9cB7Lfffsn4jBkzkvHUWAS5abCHhoaS8dxU1Lntnto2uVp7TidDXHc6NHenQ3eP1jTcKT7ymxXKyW9WKCe/WaGc/GaFcvKbFcrJb1YoJ79ZofaaOn9OJ/3Oc/Fcv/PcdQC5eO4ahNQY8xs3bkyuu2HDhmT8mWeeScZzU3TnptFOyfWZT43LD+lrFHLj6qemFm8lnrsOYI+o80u6WNJjku5qWDZL0vWS7q//b/8dNrO+aOW0/xLgmGHLzgJujIhFwI3132a2B8kmf0TcDAwfy+k44NL69qXA8V1ul5n1WLvf+edGxDqAiFgnaf9md5S0FFja5vOYWY/0/Ae/iFgGLAOQ1P9fOcwMaL/Ut17SPID6/8e61yQzGw3tJv91wMn17ZOBb3enOWY2WrKn/ZKuBJYAsyWtBs4FLgCuknQq8AhwQi8b2Ypc3bTTeKrW3smY/5AfIz5XM968eXPT2Pr169teF/KvbcqUKcn4+PHj24pBvpaeq/OnxirYtm1bct1nn302Ge/1/jYasskfESc1Cb2ty20xs1Hky3vNCuXkNyuUk9+sUE5+s0I5+c0Ktdd06e11aSVV6suVrHJddnPr50p9qW63uSGkc6W6WbNmJeNz585NxlPTk+dKoLnpwXPdclOlvlxX5U6GBW/FIJT6fOQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFBOfrNC7TV1/k7lhsdO1eI77bKbG6I6V+dPmTZtWjKeq+MvWLAgGc/V+VPXEeS6zeamF8/V+VOP38k2hc6nfE8NBT9a1wD4yG9WKCe/WaGc/GaFcvKbFcrJb1YoJ79ZoZz8ZoXaa+r8ndZGc7X2VJ0/N7x1ruab6zuei48d2/xtnD17dnLdhQsXJuO5On9uCu7U8Nu56cM7nQY7VUvPvd+5eCdTug8KH/nNCuXkNyuUk9+sUE5+s0I5+c0K5eQ3K5ST36xQe02dPydXd03VynPx3GPn6tW5fum56wRS1xnk+vNPnz49GZ88eXIynvPUU081jaXG1c+tC/k6f6pWn3u/O+mPv6fIHvklXSzpMUl3NSw7T9IaSXfU/47tbTPNrNtaOe2/BDhmhOVfiIgj6n/f6W6zzKzXsskfETcD6XmTzGyP08kPfqdL+ln9taDpBd6SlkpaLml5B89lZl3WbvJ/GTgMOAJYB3yu2R0jYllELI6IxW0+l5n1QFvJHxHrI2JHROwEvgoc2d1mmVmvtZX8kuY1/Pke4K5m9zWzwZSt80u6ElgCzJa0GjgXWCLpCCCAlcBHetjGluTqrrm6ba7um6oZ5+rNuXr15s2bk/HcdQKpWnxuHvrcc6fGMYD8dt+2bVvT2JNPpn9HTq0L+e2eek9z+0NOp+NHDMK4/dnkj4iTRlh8UQ/aYmajyJf3mhXKyW9WKCe/WaGc/GaFcvKbFaqYLr29nFI5N7R2rmSVG/o7N5V1Lp6Se93PPfdcMt5J27du3drRc+ekSma58mkunnvPR6tc1wkf+c0K5eQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFB7TZ0/17U0N+VyJ0N35+Tq1Z3GU912c9cY5GrtEydOTMZzQ3un6t2ddsPO1dJTtXrX+X3kNyuWk9+sUE5+s0I5+c0K5eQ3K5ST36xQTn6zQrnO34V4rk97rlY+adKkZHzKlCltr5+bonvq1KkdPXeuzp+6RiE3rHhuaO6nn346GU9d45AbTj03RkLuOoBc2wfhOgAf+c0K5eQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFCtTNF9EHAZcACwE1gWERdKmgV8A1hINU33+yNiY++amm1nMp6rq+b6Z6ceP9fXP1drnzlzZjI+a9asZHzfffdtGpsxY0Zy3dmzZ7f92JDf7lu2bGkay9X5N23alIw/8cQTyXhqCvDc1OS5awhyYyzk6vyDoJUj//PAGRHxMuD1wEclvRw4C7gxIhYBN9Z/m9keIpv8EbEuIm6vb28F7gEWAMcBl9Z3uxQ4vleNNLPu263v/JIWAq8GbgHmRsQ6qD4ggP273Tgz652Wr+2XNBW4GvhERGzJfddrWG8psLS95plZr7R05Jc0jirxvxYR36oXr5c0r47PAx4bad2IWBYRiyNicTcabGbdkU1+VYf4i4B7IuLzDaHrgJPr2ycD3+5+88ysV1o57T8K+CBwp6Q76mVnAxcAV0k6FXgEOKE3TWxNp6W8XOlm+/btbT92rrtwrhSYK8fNmTOnaSxXJpw+fXoynhs+e2hoKBlPldTWrFmTXHflypXJ+KpVq5Lx9evXt9UuyHf5zXXpze2Pg9ClN5v8EfEjoNkX/Ld1tzlmNlp8hZ9ZoZz8ZoVy8psVyslvVignv1mhnPxmhdprhu7OdaHM1fFz9epU99Bed9lN1fEh3602JVfPTl3fALB27dpk/MEHH2wau/fee5Pr3nfffcn4ww8/nIw//vjjTWO5qclz3Y1z13bsLV16zWwv5OQ3K5ST36xQTn6zQjn5zQrl5DcrlJPfrFB7TZ0/1z861/86V+/esGFD01gndXbI14xz00Wnata5obdz2y3X73316tXJ+IoVK5rGcv31c/39c0N3p67dyNXxc/tL7j0bhP76OT7ymxXKyW9WKCe/WaGc/GaFcvKbFcrJb1YoJ79ZoTSa9UhJA1v8zI1PP27cuKaxSZMmJdfN1do77c+fGi9g4sSJyXVz73/u+ofcNNobNzaftT13DUGuz31uGu3U9RG5On6uP/4g1/EjoqULT3zkNyuUk9+sUE5+s0I5+c0K5eQ3K5ST36xQTn6zQmXr/JIOAi4DDgB2Assi4kJJ5wEfBnYNjn52RHwn81iDWxzNSF0HMGbMmOS648ePT8YnTJjQUTx1DULu+oWcXD08F0/Nl5CbSyEX76RWvyfX8XNarfO3MpjH88AZEXG7pGnAbZKur2NfiIjPtttIM+ufbPJHxDpgXX17q6R7gAW9bpiZ9dZunRNKWgi8GrilXnS6pJ9JuljSiNeYSloqabmk5R211My6quXklzQVuBr4RERsAb4MHAYcQXVm8LmR1ouIZRGxOCIWd6G9ZtYlLSW/pHFUif+1iPgWQESsj4gdEbET+CpwZO+aaWbdlk1+VUPTXgTcExGfb1g+r+Fu7wHu6n7zzKxXWin1HQ38ELiTqtQHcDZwEtUpfwArgY/UPw6mHmvPrZ8k5Mpp/Yx3Oqx4bv/Ilcx6WW7bm8t1nWi11Of+/F3g5G8v7uTvDffnN7MkJ79ZoZz8ZoVy8psVyslvVignv1mhXOorXKelwJxSy2395FKfmSU5+c0K5eQ3K5ST36xQTn6zQjn5zQrl5DcrVCuj93bTBuDhhr9n18sG0aC2ravt6nIdflC3GZTTtkNaveOoXuTzoieXlg/q2H6D2rZBbRe4be3qV9t82m9WKCe/WaH6nfzL+vz8KYPatkFtF7ht7epL2/r6nd/M+qffR34z6xMnv1mh+pL8ko6RdK+kBySd1Y82NCNppaQ7Jd3R7/kF6zkQH5N0V8OyWZKul3R//f+IcyT2qW3nSVpTb7s7JB3bp7YdJOkHku6RdLekj9fL+7rtEu3qy3Yb9e/8ksYA9wHvAFYDtwInRcTPR7UhTUhaCSyOiL5fECLpTcAQcFlEvLJe9hngyYi4oP7gnBkRZw5I284Dhvo9bXs9m9S8xmnlgeOBU+jjtku06/30Ybv148h/JPBARDwUEc8CXweO60M7Bl5E3Aw8OWzxccCl9e1LqXaeUdekbQMhItZFxO317a3Armnl+7rtEu3qi34k/wJgVcPfq+njBhhBAN+XdJukpf1uzAjm7poWrf5//z63Z7jstO2jadi08gOz7dqZ7r7b+pH8I40vNkj1xqMi4jXAbwIfrU9vrTUtTds+WkaYVn4gtDvdfbf1I/lXAwc1/H0gsLYP7RhRRKyt/38MuIbBm3p8/a4Zkuv/H+tze35pkKZtH2laeQZg2w3SdPf9SP5bgUWSDpU0HjgRuK4P7XgRSVPqH2KQNAV4J4M39fh1wMn17ZOBb/exLS8wKNO2N5tWnj5vu0Gb7r4vV/jVpYwvAmOAiyPiU6PeiBFIegnV0R6q7s5X9LNtkq4EllB1+VwPnAtcC1wFHAw8ApwQEaP+w1uTti1hN6dt71Hbmk0rfwt93HbdnO6+K+3x5b1mZfIVfmaFcvKbFcrJb1YoJ79ZoZz8ZoVy8psVyslvVqj/BzFtAk+o2fP3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plot mean image\n",
    "mean = torch.mean(trainimages.float(), dim=0)\n",
    "plt.figure()\n",
    "ax = plt.subplot(111)\n",
    "ax.grid(False)\n",
    "plot_image(mean)\n",
    "plt.title(\"Mean of entries in MNIST dataset\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SQcdYceQpOGR"
   },
   "source": [
    "Basically a blur, but it lets us see what parts of the images usually contain non-0 values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UxtZdenh9OLz"
   },
   "source": [
    "We can also look at this property by visualizing the pixels that are always zero(black) or ever non-zero(white)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 785,
     "status": "ok",
     "timestamp": 1532468122995,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "nWVMd_wiWFS2",
    "outputId": "c0d79a2d-3c08-4928-fe20-087259aed09d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAACxRJREFUeJzt3UGIXId9x/Hvr1ZycXKQiSyE49ZpMKWlUCdaTMGluA0Kri9yDi3xoai0oBxiSKCHmF5iKAFTmrQ9BZRaRIXEoWC7FqE0MSbULRRjyZhEjtrIGLdRLCQZH+Kcgu1/D/tUNvKuZrTzZt6o/+8Hlpl5O5r5M+i77828mXmpKiT180tTDyBpGsYvNWX8UlPGLzVl/FJTxi81ZfxSU8YvNWX8UlN7VnlnSXw74YodPHhw0vs/ffr0pPffUVVlnutlkbf3JrkP+DvgJuDvq+rRGdc3/hWb+u3byVz/DzWipcef5CbgR8Ah4DzwAvBgVf3wGv/G+FfM+PuZN/5FnvPfDbxSVa9W1c+BbwGHF7g9SSu0SPy3AT/ecvn8sOwXJDma5FSSUwvcl6SRLfKC33abFu/ZxqyqY8AxcLNfWieLrPnPA7dvufxh4PXFxpG0KovE/wJwZ5KPJHk/8Gng5DhjSVq2XW/2V9XbSR4CvsPmrr7jVfXyaJONbOpXvbvq+rjfCHs5FtrPf913NuFz/q7/CTWNKeNfxa4+STcw45eaMn6pKeOXmjJ+qSnjl5pa6ef5F+XuOt0oFvyo/IiT7Mw1v9SU8UtNGb/UlPFLTRm/1JTxS03dULv6pA5m7SYca1ega36pKeOXmjJ+qSnjl5oyfqkp45eaMn6pqbXaz+9HdqXZrtXJxsbG3Lfjml9qyvilpoxfasr4paaMX2rK+KWmjF9qaqH9/EleA94C3gHerqr5dzJuf3vX/L3vA5DGM8abfH6vqt4Y4XYkrZCb/VJTi8ZfwHeTnE5ydIyBJK3Gopv991TV60luBZ5J8p9V9dzWKwx/FPzDIK2Zhdb8VfX6cHoJeAq4e5vrHKuqjUVfDJQ0rl3Hn+TmJB+8ch74JHBmrMEkLdcim/37gaeG3XN7gG9W1b+MMpWkpdt1/FX1KvBbI87ifnxphdzVJzVl/FJTxi81ZfxSU8YvNWX8UlMrjf/gwYNU1Y4/klbHNb/UlPFLTRm/1JTxS00Zv9SU8UtNGb/UlPFLTRm/1JTxS00Zv9SU8UtNGb/UlPFLTRm/1JTxS00Zv9SU8UtNGb/UlPFLTRm/1JTxS00Zv9TUzPiTHE9yKcmZLctuSfJMknPD6d7ljilpbPOs+b8O3HfVsoeBZ6vqTuDZ4bKkG8jM+KvqOeDNqxYfBk4M508AD4w8l6Ql2+1z/v1VdQFgOL11vJEkrcLSX/BLcjTJqSSnLl++vOy7kzSn3cZ/MckBgOH00k5XrKpjVbVRVRv79u3b5d1JGttu4z8JHBnOHwGeHmccSasyz66+x4H/AH4tyfkkfwY8ChxKcg44NFyWdAPZM+sKVfXgDr/6xMizSFoh3+EnNWX8UlPGLzVl/FJTxi81ZfxSU8YvNWX8UlPGLzVl/FJTxi81ZfxSU8YvNWX8UlPGLzVl/FJTxi81ZfxSU8YvNWX8UlPGLzVl/FJTxi81ZfxSU8YvNWX8UlPGLzVl/FJTxi81ZfxSUzPjT3I8yaUkZ7YseyTJT5K8NPzcv9wxJY1tnjX/14H7tln+N1V11/Dzz+OOJWnZZsZfVc8Bb65gFkkrtMhz/oeSfH94WrB3tIkkrcRu4/8q8FHgLuAC8OWdrpjkaJJTSU5dvnx5l3cnaWy7ir+qLlbVO1X1LvA14O5rXPdYVW1U1ca+fft2O6ekke0q/iQHtlz8FHBmp+tKWk97Zl0hyePAvcCHkpwHvgjcm+QuoIDXgM8scUZJSzAz/qp6cJvFjy1hFkkr5Dv8pKaMX2rK+KWmjF9qyvilpoxfamrmrr4xnT59miQ7/r6qVjiN1Jtrfqkp45eaMn6pKeOXmjJ+qSnjl5oyfqmplcZ/8OBBqmrHH0mr45pfasr4paaMX2rK+KWmjF9qyvilpoxfasr4paaMX2rK+KWmjF9qyvilpoxfasr4paaMX2pqZvxJbk/yvSRnk7yc5HPD8luSPJPk3HC6d9ZtXfne/p1+JK3OPGv+t4E/r6pfB34b+GyS3wAeBp6tqjuBZ4fLkm4QM+OvqgtV9eJw/i3gLHAbcBg4MVztBPDAsoaUNL7res6f5A7gY8DzwP6qugCbfyCAW8ceTtLyzH2sviQfAJ4APl9VP533OXqSo8DR3Y0naVnmWvMneR+b4X+jqp4cFl9McmD4/QHg0nb/tqqOVdVGVW2MMbCkcczzan+Ax4CzVfWVLb86CRwZzh8Bnh5/PEnLMs+a/x7gj4HfT/LS8HM/8ChwKMk54NBw+Zr86m5pfcx8zl9V/w7s9AT/E+OOI2lVfIef1JTxS00Zv9SU8UtNGb/UlPFLTRm/1JTxS00Zv9SU8UtNGb/UlPFLTRm/1JTxS01llZ+jT7LQnfmZf4mZX3NfVXN9x55rfqkp45eaMn6pKeOXmjJ+qSnjl5oyfqkp45eaMn6pKeOXmjJ+qSnjl5oyfqkp45eaMn6pqZnxJ7k9yfeSnE3ycpLPDcsfSfKTJC8NP/cvf1xJY5n5ZR5JDgAHqurFJB8ETgMPAH8E/Kyq/nruO/PLPKSFjfVlHntmXaGqLgAXhvNvJTkL3DbPjUtaX9f1nD/JHcDHgOeHRQ8l+X6S40n27vBvjiY5leTUQpNKGtXc3+GX5APAvwJfqqonk+wH3gAK+Es2nxr86YzbcLNfWtBYm/1zxZ/kfcC3ge9U1Ve2+f0dwLer6jdn3I7xSwta2Rd4ZvOeHgPObg1/eCHwik8BZ+a5Q0nrYZ5X+38H+DfgB8C7w+K/AB4E7mJzs/814DPDi4PXuq2lrbrdKtD/F7PW7LOMutk/FuOXZltV/L7DT2rK+KWmjF9qyvilpoxfasr4paZmfrDnRjHHu55WNIm0+O66VXDNLzVl/FJTxi81ZfxSU8YvNWX8UlPGLzW16v38bwD/veXyh4ZlS7eL/a4rm+06retc4Gy7NeZsvzLvFVf6ef733Hlyqqo2JhvgGtZ1tnWdC5xtt6aazc1+qSnjl5qaOv5jE9//tazrbOs6Fzjbbk0y26TP+SVNZ+o1v6SJTBJ/kvuS/FeSV5I8PMUMO0nyWpIfDEcenvQQY8Nh0C4lObNl2S1Jnklybjjd9jBpE822FkduvsaRpSd97NbtiNcr3+xPchPwI+AQcB54AXiwqn640kF2kOQ1YKOqJt8nnOR3gZ8B/3DlaEhJ/gp4s6oeHf5w7q2qL6zJbI9wnUduXtJsOx1Z+k+Y8LEb84jXY5hizX838EpVvVpVPwe+BRyeYI61V1XPAW9etfgwcGI4f4LN/zwrt8Nsa6GqLlTVi8P5t4ArR5ae9LG7xlyTmCL+24Afb7l8nvU65HcB301yOsnRqYfZxv4rR0YaTm+deJ6rzTxy8ypddWTptXnsdnPE67FNEf9277Ndp10O91TVx4E/AD47bN5qPl8FPsrmYdwuAF+ecpjhyNJPAJ+vqp9OOctW28w1yeM2Rfzngdu3XP4w8PoEc2yrql4fTi8BT7H5NGWdXLxykNTh9NLE8/yfqrpYVe9U1bvA15jwsRuOLP0E8I2qenJYPPljt91cUz1uU8T/AnBnko8keT/waeDkBHO8R5KbhxdiSHIz8EnW7+jDJ4Ejw/kjwNMTzvIL1uXIzTsdWZqJH7t1O+L1JG/yGXZl/C1wE3C8qr608iG2keRX2Vzbw+YnHr855WxJHgfuZfNTXxeBLwL/BPwj8MvA/wB/WFUrf+Fth9nu5TqP3Lyk2XY6svTzTPjYjXnE61Hm8R1+Uk++w09qyvilpoxfasr4paaMX2rK+KWmjF9qyvilpv4XYLGaWylVB1sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nonzero = torch.gt(torch.sum(trainimages.float(), dim=0), 0.0)\n",
    "plt.figure()\n",
    "ax = plt.subplot(111)\n",
    "ax.grid(False)\n",
    "plot_image(nonzero)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Y9GXmHPHWxJM"
   },
   "source": [
    "Let's look at a few more examples from the dataset. We'll define a function to display multiple images at once to help with that. Montage takes a list containing a square number of images from our data (e.g. 1, 4, 9, etc.) and displays them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "9rK8ZX3rWvr_"
   },
   "outputs": [],
   "source": [
    "#Takes in an array(list, in python terminology) of images\n",
    "def montage(imgstack):\n",
    "    plt.figure()\n",
    "    width = int(np.ceil(np.sqrt(len(imgstack))))\n",
    "    height = int(np.floor(np.sqrt(len(imgstack))))\n",
    "    for i in range(0, len(imgstack)):\n",
    "        ax = plt.subplot(height, width, i+1)\n",
    "        ax.grid(False)\n",
    "        plot_image(imgstack[i])\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "b3HwMUavXTW6"
   },
   "source": [
    "**QUESTION**\n",
    "Now let's display the first 6 training examples\n",
    "\n",
    "Can you complete the function call below to do that?\n",
    "\n",
    "Try passing a subset of trainimages. You can select only a slice of the whole array like so:\n",
    "\n",
    "trainimages[start:end]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 132
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 236,
     "status": "error",
     "timestamp": 1532468125613,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "ab-RhQYyXaSZ",
    "outputId": "0db87c88-d516-46fb-e605-7884ccd75cbb"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD7CAYAAAB68m/qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHOFJREFUeJzt3X1wVNX5B/DvIwZfwMo7RggkIlDAF+g4SkecIgwFRIU6vgGl/CxjnKpFKXUMgqjYqpWq7bR0JC2vLYIoFmK1pZgJAo44giMIRF60goEYREt9AdTg+f2R9XjOJbu52b337t2z389MJs/Zs7v3gSc5XM6ee48opUBERLnvpGwnQEREweCATkTkCA7oRESO4IBOROQIDuhERI7ggE5E5AgO6EREjshoQBeRESKyU0T2iEhZUElRdrGu7mJt3SbpXlgkIi0A7AIwDEANgNcBjFVK7QguPYoa6+ou1tZ9J2fw2osB7FFKvQsAIrIMwGgASX84RISXpcaEUkqSdLGuOSxFXYFm1pZ1jZVDSqmOTT0pkymXLgDeN9o1iccsIlIqIptEZFMGx6LosK7uarK2rGts7fXzpEzO0Bs7EzjhX3SlVDmAcoD/4ucI1tVdTdaWdc1tmZyh1wAoMtpdARzILB2KAdbVXayt4zIZ0F8H0FNESkSkJYAbAVQEkxZlEevqLtbWcWlPuSil6kXkdgCrAbQAMF8ptT2wzCgrWFd3sbbuS3vZYloH45xcbDSxGqJZWNf4YF2dtVkpdVFTT+KVokREjuCATkTkCA7oRESOyGQdOhFR7BQXF+v4qquusvquueYaqz148GAdf/3111bfk08+qePy8nKrb8uWLRlmGQ6eoRMROYIDOhGRI7hsMaFz5846/tWvfmX19e7dW8fdu3e3+hYvXpz0PXfu3Gm1n376aR17/97r6+v9JxsALm9zU77UtW/fvjpeuXKl1XfaaafpuLCwMOX7iHz715VqLDx06JDVHjJkiI537IjkZpVctkhElE84oBMROYIDOhGRI/J2Dv2kk+x/y37xi1/o+NFHHw3lmG+//baOP/roI6vv97//vY6fffbZUI5vype51nzjal1nzZpltcePH69jc5kikHou3MvvHLrXH//4Rx3feeedvl+XAc6hExHlEw7oRESOyNsrRW+99VarHcY0y9GjR632d7/73aTPbdOmjY5PPfVUq++pp57SsfdqNkqtoKBAx95ptpKSEh2b/4UHgMsuu8xqr1+/PukxJk+erON58+ZZfVVVVTr+5z//afVFvVQ119x777069tbHu3yYGvAMnYjIERzQiYgcwQGdiMgRebVssX379jp+5ZVXrL5evXoFcoyNGzfq2LucqbS0VMc//elPk76HuZQKsPP++OOPM00RQG4vb/N+xnD22Wfr+JZbbrH6rr32Wh2bc+bZ4P2cpqysLPBj5HJdL7zwQqu9evVqHXfo0CHp68wlhABQXV2t47lz56Y85s9//nMdP/HEE77y9B6TyxaJiChwHNCJiBzh9LLFk0+2/3g333yzjtOdYvniiy+s9gcffGC1zeWQb775ptU3YMAAHaeacslXrVq10rF3I4Lvfe97Oh4+fLjVl2o5KOUO71LE008/XcfeJZ/mNMc777yT9jH79OmT9mvjiGfoRESO4IBOROQIDuhERI5weg69Y8eOVvuhhx5K632OHTum4xUrVlh9EyZMsNqtW7fWsXeZmjmHn4p3Kal5GfqqVat8vUdcmZ9rmHeYBIBBgwbp+Pzzz48sJ4qHiooKq33llVfq2LsrkHcHIb+8Y4K5zLU5S7jNDaTjhGfoRESO4IBOROSIJqdcRGQ+gCsBHFRKnZd4rB2ApwEUA3gPwPVKqf+Gl2Z23XfffTqePXu21WdOsQDAn//8Zx3fcMMNaR3PezXo2rVr03qfVLJV19GjR+v4Zz/7WZBv3WzmVFpdXZ3VZ25GAgB79+7VsXnFb3P873//S+t1zeXK7+y6desCf8/p06en9TrvpjPen4+48HOGvhDACM9jZQAqlVI9AVQm2pRbFoJ1ddVCsLZ5qckBXSm1DoD3BiKjASxKxIsAjAk4LwoZ6+ou1jZ/pbvKpbNSqhYAlFK1ItIp2RNFpBRAev9Hpaixru7yVVvWNbeFvmxRKVUOoByI/u5t//2vPUVYWVmp46FDh/p+n23btun4jDPOsPq8d3NLd97c3DR6zBj75CmqudfmSLeu/fv3Dy2nxpjz5Bs2bLD6zGWlu3btsvq8ef7yl7/MOJc//OEPGb9H2LL5+xoGcycw4MQ7Opq7WHl3Azt8+LCO//SnP4WQXfDSXeVSJyKFAJD4fjC4lCiLWFd3sbZ5IN0BvQLAxEQ8EUBuX+1C32Bd3cXa5gE/yxaXAhgMoIOI1AC4D8AjAJaLyCQA+wBcF2aS6TL/uw0ADzzwgI69//VKdQN983XHjx+3+i655BLf+ZjTKlOmTLH6zDszmlM8Ycnluu7fv99qm9Ml5eXlVt+RI0d0/J///Mfq+8lPfqLjZcuWWX3t2rXLOE8AeOGFF3Ts/XkMSy7XNgjmcti77rrL6uvWrZvVNqdZvFeKmpt/h7GEMgxNDuhKqbFJuvxPQlPssK7uYm3zF68UJSJyBAd0IiJHOH23RS9z2Zp3uWGqS4IvuqjJvVkbtWTJEqv9m9/8RsdRzJPH0YEDB3Tsvdz+q6++0vG7775r9f3ud7/T8RtvvGH17du3T8c9evSw+qZNm6bjbOwSZe6m1LVrV6vvvffeizgbd5h3TVy0aJHVZ96109z1qCneXZHMTapzBc/QiYgcwQGdiMgR0pybumd8sBhdeeZdplhVVaXjfv36pf2+e/bs0fFNN91k9b3yyitpv2/QlFIS1HulW1fv5s7msr7mTEcUFxfr+NVXX7X6OnfunE5qofBOI5l//vr6+kCOEYe6hsG7MYU5HeJdgtycMU3k278u78bxmWw+HYLNSqkm5355hk5E5AgO6EREjuCATkTkiLxatmjyXrKfyby56dxzz9Wxd9PblStX6ti8rBgAPv/880COn0uC2vXF3DUqTnPmXuecc47VNjcYX7BgQdTpxJ752cjzzz9v9fXt21fH5h0TgRPvmmjyXsI/Z84cHcdszjwtPEMnInIEB3QiIkdwQCcicoTTc+innnqq1R479tub0P32t79N6z29a8kPHrT3CRg8eLCO27Zta/WZ69IHDhxo9d1+++06NtfEU/x4fwbOOussHRcVFVl9LVu2TPo+JSUlwSaWg8y58B//+MdWn/kZQ2FhodVnrjX3zpmbv5Pey/m9t62O425gmeAZOhGRIzigExE5wukpF/OuawAwb968tN5n6dKlOr7lllusvs8++8xqm0vT/vGPf1h95qXeffr0sfpmzJih440bN1p9R48ebWbG+cW8pN5b40mTJun4tddes/oOHTqk4/nz51t95l0hvbZs2WK127dvr+NRo0ZZfebG0AUFBUnfM194L9N/7rnndNy9e/e03tO83QYArF27Vsfe31fX8QydiMgRHNCJiBzBAZ2IyBHOzaGfcsopOi4rKwvkPf/2t7/p2Dtn7tW7d28dN2e3lMsvv1zH5i47ADBz5kzf75OPjhw5ouM77rjD6kt1afenn34ayPH379+v4/Lycqvv4Ycf1rF3GWu+MJcmmnPmQPrz5ibzdy4T9957r9V+5plndBzUbSrCxjN0IiJHcEAnInKEc1Mu99xzj46HDBmS1nv89a9/tdrmMijvFWvmFZ4AMHXqVB2nukowFXPTY2oe79XB5lWDQU2xpGJejQwAZ555ZtLn7tixI+x0YsG84tO8g2JQzN9PAFixYoWOS0tLrT5z+gew79ToveL0/vvvT3pM83XeDd9nz56t48WLFyd9jzDwDJ2IyBEc0ImIHNHkgC4iRSJSJSLVIrJdRO5IPN5ORNaIyO7E9/z8CD9Hsa5uYl3zm5859HoAU5VSb4jIGQA2i8gaAP8HoFIp9YiIlAEoA3B3eKn6853vfEfH5o7eXql2Bvdeav/vf/9bx945uKCWoi1ZskTHEe1ek1N1TcXcrX3NmjVW3wcffKBj7/JPc+f45ujRo4fVvu2223Q8fvx4q8+ca92+fbvV9+yzz6Z1/CbErq7mLS9S/d6l67LLLrPa3lt+mLzHN+fNm5Ob+TrvbTzGjBmj49jNoSulapVSbyTiTwFUA+gCYDSARYmnLQIwpvF3oDhiXd3Euua3Zq1yEZFiAAMAvAags1KqFmj4IRKRTkleUwqgtLE+igfW1U2sa/7xPaCLSGsAKwDcqZT6JNV0hkkpVQ6gPPEewf9/y8PcBNa8+hIALrjgAl/v4V3qFJRdu3bp+Morr7T6amtrdXz8+PFQjt+YXKlrKuaG394NJsy2uUk3YN/V0rtU1JwmAOzNp81NqQGgXbt2vvJ84YUXrHZ9fb2v16UjTnU1f5+8d7UcOXJkEIcIxZdffqnjvXv3Wn2pptLCGj/88LXKRUQK0PDDsUQp9c21u3UiUpjoLwRwMNnrKZ5YVzexrvnLzyoXATAPQLVS6nGjqwLAxEQ8EcCq4NOjsLCubmJd85ufKZdLAUwA8JaIvJl47B4AjwBYLiKTAOwDcF04KVJIWFc3sa55TMJYRpT0YBHPtXo3nQ1jCdF7771ntZcvX67jv/zlL1ZfTU2Njo8dOxZ4Ls2hlPI3qepDtufQly1bpuPrr78+i5mcuFTW3N1o6NChVt9HH30U+PHjXtc2bdpYbe/vqF/XXHONjr13cDR16NDBans/G9mwYUPS1x4+fFjH5h1Xs2SzUuqipp7EK0WJiBzBAZ2IyBFOT7mcccYZVvuuu+5K+lzzarP169cnfZ53edvChQutdphL0YIU9/+aN8eUKVN0/Nhjj2UxkxOZyybNjTDC4lJdycIpFyKifMIBnYjIERzQiYgc4fQcOiXn0lzrWWedpeOXX37Z6uvZs2fox9+6dauOhw8fbvWZOyZF8bvmUl3Jwjl0IqJ8wgGdiMgRzm0STfnH3MSiX79+Vt+4ceN0fNNNN1l9JSUlOu7WrZvv482ZM8dqV1ZW6riurs73+xAFjWfoRESO4IBOROQIDuhERI7gssU8xeVtbmJdncVli0RE+YQDOhGRIzigExE5ggM6EZEjOKATETmCAzoRkSOivvT/EIC9ADok4jjIx1y6B/x+rGtqrGtw8jUXX7WNdB26PqjIJj9rKqPAXIITp/yZS3DilD9zSY1TLkREjuCATkTkiGwN6OVZOm5jmEtw4pQ/cwlOnPJnLilkZQ6diIiCxykXIiJHcEAnInJEpAO6iIwQkZ0iskdEyqI8duL480XkoIhsMx5rJyJrRGR34nvbCPIoEpEqEakWke0icke2cgkC62rl4kxtWVcrl5yoa2QDuoi0ADAHwEgAfQGMFZG+UR0/YSGAEZ7HygBUKqV6AqhMtMNWD2CqUqoPgIEAbkv8XWQjl4ywridworas6wlyo65KqUi+AHwfwGqjPQ3AtKiObxy3GMA2o70TQGEiLgSwMws5rQIwLA65sK6sLeuau3WNcsqlC4D3jXZN4rFs66yUqgWAxPdOUR5cRIoBDADwWrZzSRPrmkSO15Z1TSLOdY1yQG9sa6y8XjMpIq0BrABwp1Lqk2znkybWtREO1JZ1bUTc6xrlgF4DoMhodwVwIMLjJ1MnIoUAkPh+MIqDikgBGn4wliilnstmLhliXT0cqS3r6pELdY1yQH8dQE8RKRGRlgBuBFAR4fGTqQAwMRFPRMPcWKhERADMA1CtlHo8m7kEgHU1OFRb1tWQM3WN+IOEKwDsAvAOgOlZ+CBjKYBaAF+h4QxkEoD2aPh0enfie7sI8hiEhv++bgXwZuLrimzkwrqytqyrO3Xlpf9ERI7glaJERI7ggE5E5IiMBvRsXxpM4WBd3cXaOi6DDwlaoOHDknMAtASwBUDfJl6j+BWPL9bVza8gf2ez/Wfhl/X1oZ9xOZMz9IsB7FFKvauU+hLAMgCjM3g/igfW1V2sbe7a6+dJmQzovi4NFpFSEdkkIpsyOBZFh3V1V5O1ZV1z28kZvNbXpcFKqXIktmoSkRP6KXZYV3c1WVvWNbdlMqDH9dJgykxO17WgoMBqr1u3TseLFi2y+p588slIcoqRnK4tNS2TKZe4XhpMmWFd3cXaOi7tM3SlVL2I3A5gNRo+PZ+vlNoeWGaUFayru1hb92Uy5QKl1IsAXgwoF4qJXK7r1VdfbbUvueQSHS9cuDDibOInl2tLTeOVokREjuCATkTkCA7oRESOyGgOnSgOTjrp2/OSmTNnZjETouziGToRkSM4oBMROYJTLpTzLrjgAh2ff/75WcyEKLt4hk5E5AgO6EREjuCATkTkCM6h+zBy5EgdP/HEE1bf8uXLrfYDDzyg4+PHj4ebGAEAZs+enbRv3LhxOl65cmUU6VCa+vfvb7XXr1+v48mTJ1t9CxYsiCSnXMMzdCIiR3BAJyJyBKdcfHjwwQd13KtXL6tvxowZVvull17Ssbm5AoVnwIABOj569KjVt3XrVh0fO3Ysspyo+YYOHWq1W7VqpePRo+2tT8OYchkxYoTVnjBhgo7Hjx8f+PHCwDN0IiJHcEAnInIEB3QiIkdwDt2Hbt26Je2bNWuW1d6wYUPY6eS9iy++2Gq3bt1ax2+99ZbVt2PHjkhyosyNGjUqaV9RUVHSvkx06dJFx94lyebPVa7gGToRkSM4oBMROYJTLj5UVVXp+LrrrrP6xo4da7UfffRRHR85ciTcxPJUp06drHbLli11bP79U/yZUx69e/eO/PglJSVJj79///6o08kYz9CJiBzBAZ2IyBEc0ImIHME5dB+2b9+uY+8ces+ePa22OZ/LOfRwpNqVaNeuXRFmQplq27atjgsLCyM/fo8ePSI/Zph4hk5E5IgmB3QRmS8iB0Vkm/FYOxFZIyK7E9/bpnoPih/W1V2sbf7yc4a+EMAIz2NlACqVUj0BVCbahIYNi7/5irmFyNG69u/f3/qiEyxEjtY2aiNHjtRfLmhyQFdKrQPwsefh0QAWJeJFAMYEnBeFjHV1F2ubv9L9ULSzUqoWAJRStSLSKdkTRaQUQGmax6Fosa7u8lVb1jW3hb7KRSlVDqAcAEREhX08igbr6ibWNbelO6DXiUhh4l/6QgAHg0wqbjp06OD7ueYOOTkotnU9/fTTdexdOkq+xLa2fgT1ezVw4ECr7b1zp6mgoEDHbdq0sfoOHz4cSD5BS3fZYgWAiYl4IoBVwaRDWca6uou1zQN+li0uBfAqgN4iUiMikwA8AmCYiOwGMCzRphzCurqLtc1fTU65KKXGJukamuRx59x4443ZTiFwLtX1iy++aDTOhHc55K9//Wsdp7pSddOmTVZ72rRpOt65c2cguTXFpdp+49prr7Xa5557ro6XLFli9aXa0Nk75dKiRYukzzXv6nn55ZdbfX//+9+TJ5tFvFKUiMgRHNCJiBzBAZ2IyBG822KGnnnmGav9ySefZCmT/FVdXa3jt99+2/frTjrp2/OZKVOmWH1333231fa7dLVr165W25yL987D7t2719d7uuzDDz/UsfdOmb169dJxq1atrL5LL7200ThIFRUVOn7xxRdDOUbQeIZOROQIDuhERI7glEuGvMvkvv766yxl4rYvv/xSx6+++qrVd9ppp6X1nuPGjdPx7NmzUz73s88+0/HatWuTPu+HP/yh1S4uLtbxiBH2DRDnzp3rI0u31dXV6dj7d7dgwQIdd+zY0eo7++yzdbxnz56k7++dEr311luttrlJtJf5cxbUctiw8QydiMgRHNCJiBzBAZ2IyBGcQ6ecUF9fr+Oamhqr70c/+pGOzzvvPKtv27ZtSKZ79+5J+8w5c8C+w+Pq1auTvm7WrFlWe8aMGToeMmSI1cc5dNu+ffus9tCh396pwLts0dxQOtUcuteZZ55ptc36uIBn6EREjuCATkTkCA7oRESO4Bw65RzzUn/Ant+ePn261Tdx4kQdm2vZAeCqq65KeoyZM2da7VTz5qaXXnrJaptztD/4wQ+svlNOOUXHubLOOVs+//xzq92cefN8wjN0IiJHcEAnInIEp1wo5zz//PNWu6ysTMc33HCD1XfgwAEdP/TQQ1ZfUVFR0mMMHz7capu7DXnvCnj11Vfr+OGHH076ni+//LLV5jRLvP3rX//KdgrNxjN0IiJHcEAnInIEB3QiIkdwDp1yzubNm612VVWVjgcNGmT1TZ48WcfeW6eaywa9vHPow4YN0/Hx48etvoKCgiYybuDdnZ7ibffu3dlOodl4hk5E5AgO6EREjuCUC+W8kSNH6njq1KlWn3n3w+bsbCQiVtvcUNqMvQ4fPmy1zWkWc+kjxZ+5+fSaNWuymIl/PEMnInJEkwO6iBSJSJWIVIvIdhG5I/F4OxFZIyK7E9/bhp8uBYV1dRPrmt/8nKHXA5iqlOoDYCCA20SkL4AyAJVKqZ4AKhNtyh2sq5tY1zzW5By6UqoWQG0i/lREqgF0ATAawODE0xYBWAvg7lCyzDJzhxzv7uO5ytW6PvbYY1bbXHp28803W32jRo1K+j61tbVW27zbonfZojlP7r0L4Pvvv99ExsFyta5hMT8rUUpZfY8//riOvT87GzduDDexNDXrQ1ERKQYwAMBrADonfniglKoVkU5JXlMKoDSzNClMrKubWNf843tAF5HWAFYAuFMp9Yl3FUAySqlyAOWJ91BNPJ0ixrq6iXXNT74GdBEpQMMPxxKl1HOJh+tEpDDxr30hgINhJZltK1eu1HH//v2zmEmw8qGuFRUVjcYuy4e6puupp56y2qk2ie7Xr5+OzSuFgfhOufhZ5SIA5gGoVko9bnRVAPhmO5iJAFYFnx6FhXV1E+ua3/ycoV8KYAKAt0TkzcRj9wB4BMByEZkEYB+A65K8nuKJdXUT65rH/Kxy2QAg2QTc0GDToaiwrm5iXfMbL/0noryxb98+q71lyxYdX3jhhUlfN3jwYKv94IMPBppXUHjpPxGRIzigExE5glMuRJQ3jhw5YrXNuyimmnLxTtXEFc/QiYgcwQGdiMgRHNCJiBzBOXQfFi9erOP777/f6tu6dWvE2RBRUObOnavjAQMGWH3mnTrN58UZz9CJiBzBAZ2IyBHival7qAfj7ThjQynl736qPrCu8cG6OmuzUuqipp7EM3QiIkdwQCcicgQHdCIiR3BAJyJyBAd0IiJHcEAnInIEB3QiIkdwQCcicgQHdCIiR3BAJyJyRNR3WzwEYC+ADok4DvIxl+4Bvx/rmhrrGpx8zcVXbSO9l4s+qMgmP/cliAJzCU6c8mcuwYlT/swlNU65EBE5ggM6EZEjsjWgl2fpuI1hLsGJU/7MJThxyp+5pJCVOXQiIgoep1yIiBzBAZ2IyBGRDugiMkJEdorIHhEpi/LYiePPF5GDIrLNeKydiKwRkd2J720jyKNIRKpEpFpEtovIHdnKJQisq5WLM7VlXa1ccqKukQ3oItICwBwAIwH0BTBWRPpGdfyEhQBGeB4rA1CplOoJoDLRDls9gKlKqT4ABgK4LfF3kY1cMsK6nsCJ2rKuJ8iNuiqlIvkC8H0Aq432NADTojq+cdxiANuM9k4AhYm4EMDOLOS0CsCwOOTCurK2rGvu1jXKKZcuAN432jWJx7Kts1KqFgAS3ztFeXARKQYwAMBr2c4lTaxrEjleW9Y1iTjXNcoBXRp5LK/XTIpIawArANyplPok2/mkiXVthAO1ZV0bEfe6Rjmg1wAoMtpdARyI8PjJ1IlIIQAkvh+M4qAiUoCGH4wlSqnnsplLhlhXD0dqy7p65EJdoxzQXwfQU0RKRKQlgBsBVER4/GQqAExMxBPRMDcWKhERAPMAVCulHs9mLgFgXQ0O1ZZ1NeRMXSP+IOEKALsAvANgehY+yFgKoBbAV2g4A5kEoD0aPp3enfjeLoI8BqHhv69bAbyZ+LoiG7mwrqwt6+pOXXnpPxGRI3ilKBGRIzigExE5ggM6EZEjOKATETmCAzoRkSM4oBMROYIDOhGRI/4fNkVGYP3FrZQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "montage(trainimages[:6])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5o8t4l8wYE7O"
   },
   "source": [
    "**QUESTION**\n",
    "\n",
    "Next, can you print out the labels corresponding to each image above? \\\n",
    "You can select a range from trainlabels the same way as you did above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 132
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 268,
     "status": "error",
     "timestamp": 1532468128086,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "koYO5eMfYSGQ",
    "outputId": "b43fb8a6-2ce6-4148-d805-bb2597518467"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([8, 3, 2, 1, 6, 4])\n"
     ]
    }
   ],
   "source": [
    "print(trainlabels[:6])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cnuNDLIH6RDV"
   },
   "source": [
    "Okay, we've looked at our data, but before we train any neural networks on it, let's take a moment to familiarize ourselves with the basic neural network algorithm.\n",
    "\n",
    "---\n",
    "**Neurons 101**\n",
    "\n",
    "Let's illustrate with the simplest possible neural \"network.\"  \n",
    "Suppose we have a single neuron $y$ that computes a function of one input:\n",
    "$y = Wx + b$\n",
    "$y$, $x$, $b$, and $W$ are all scalar values(single numbers).\n",
    "\n",
    "All we do is take in a single value $x$, multiply it by some number we have learned, $W$, and add another number $b$ to it.\n",
    "\n",
    "Let's suppose for a moment that we want $y$ to be 1 or greater if $x$ is less than or equal to 0, and $y$ should be less than 1 if $x$ is greater than 0.\n",
    "If we want that to be true, we need to find values of $W$ and $b$ which make $y$ take the proper value as a function of $x$.\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "\n",
    "**QUESTION**\n",
    "Can you think of values for $W$ and $b$ that make $y$ behave the way we want?\n",
    "\n",
    "Sketch out the shape of this function ($y$ as a function of $x$)\n",
    "\n",
    "---\n",
    "\n",
    "**Parameter Fitting**\n",
    "\n",
    "In this case finding values by hand is easy, but what about harder cases? How can we find values for $W$ and $b$ which work for any desired function?\n",
    "\n",
    "There's many answers to thus question across the fields of machine learning and optimization(you may recall some such methods from previous modules), but for neural networks, we use **stochastic gradient descent** along with **backpropagation** to fit these values.\n",
    "\n",
    "---\n",
    "\n",
    "**Loss Functions**\n",
    "\n",
    "Suppose we randomly guess that $W$ should be 1 and $b$ should be 0.\n",
    "While we don't know what the right values of $W$ and $b$ are, we do know what the desired behavior of $y$ is.\n",
    "We also know what our current function is using $W=1$ and $b=0$. In fact, we can compute a difference between the current function and the desired behaviour. For example, the following:\n",
    "\n",
    "$ L = \\begin{cases}\n",
    "    1 - y\\ \\text{if}\\ x \\leq 0\\\\\n",
    "    y - 1\\ \\text{if}\\ x \\geq 0\\\\\n",
    " \\end{cases} $\n",
    "  \n",
    "Let's look at this function for a minute. We have two cases, but they're mostly the same, so let's just think about the top one. $L$ is a function that should be small when $y$ does what we want(negative values are small), and large when it does not. This lets us evaluate $y$'s behaviour and how it differs from the behaviour we want for any given value of $x$.\n",
    "\n",
    "If we have a function $L$ which describes how $y$ differs from the function we want, it turns out we can actually compute how $W$ and $b$ should change to make the value of $L$ smaller, and thus **make $y$ behave as desired.**\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "If we know how $L$(the thing we want to minimize) will change as a function of $W$, we can change the value of $W$ to make $L$ smaller. For example, if we know when $W = 1$ that making $W$ bigger makes $L$ smaller, we can make $W$ bigger and get closer to our goal of making $L$ as small as possible.\n",
    "\n",
    "Problem is, this relationship might not hold for other values of $W$, or for the same value of $W$ if $x$ or $b$ were different(think about this for a minute and confirm for yourself that it is true).\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "\n",
    "Turns out, it's easy to evaluate how to change $W$ with everything else held fixed, and not so easy otherwise, so let's do the following(this is called **stochastic gradient descent(SGD)**):\n",
    "\n",
    "1. Compute how $W$ affects $L$ (for example, we can compute the value of $L$ given $W = 1$ and $W = 1.01$ and see what the difference is) \n",
    "2. Change $W$ by a little bit(if this is too large we might overshoot the correct value of $W$ and end up making things worse, so we have to go slowly)\n",
    "3. Get a new value for $x$ (from our training set) and run the network using our new value of $W$.\n",
    "4. Go back to 1, repeat until $L$ is small(we can decide when it is small enough)\n",
    "\n",
    "We can also do the same process for $b$ (we can compute changes to each the same way).\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "Ok, this sounds good, but how should we figure out how changing $W$ affects $L$? Computing it discretely can be error-prone, and we have to do the same process separately for each variable we want to fit. What if we had hundreds or thousands of variables?\n",
    "\n",
    "Turns out, we can compute exactly how to change each variable in parallel, no matter how many we have, using **Backpropagation,** a.k.a. the chain rule for partial derivatives. If you take multivariate calculus this topic will be covered in more detail, but for now the basic intuition is simple:\n",
    "\n",
    "* The derivative of a function $L$ with respect to one of its variables is another function(let's call this $\\frac{dL}{dW}$ in the case of $W$) that describes exactly how that function changes as the variable changes. This is a *partial derivative*\n",
    "* If the entire function we want to compute is too big, we can decompose the derivative into steps: $L$ is a function of $y$, which is a function of $W$ in turn. We can say $\\frac{dL}{dW} =frac{dL}{dy}\\frac{dy}{dW}$. This is the chain rule- just multiply the individual derivatives!.\n",
    "* This means we only need to compute each of $\\frac{dL}{dy}$ and $\\frac{dy}{dW}$ individually, which ends up being much easier to do, especially if your full function is very large(as it is for most neural networks).\n",
    "* We can also compute a derivative $\\frac{dL}{db}$ for our other variable. In each case, we assume all other values($b$ and $x$ in the case of $\\frac{dL}{dW}$ and $W$ and $x$ in the case of $\\frac{dL}{db}$) are *constant* and do not change. Because of this, we still need to make small steps using SGD above.\n",
    "\n",
    "We won't go over how to compute deriviatives right now, but it's pretty easy to do(there's online tutorials for this, and it's the first thing any calculus course will cover). Suffice to say that any *continuous* function(a function $f(x)$ has this property if the function exists for all values of $x$) is *differentiable*(derivatives can be computed), which means we can compute a derivative for most functions. There's some fun problems where this is not the case, but these are outside our current scope.\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "Still alive? Congratulations! You now know the core math behind neural networks. This is what makes neural networks powerful- we compute how to change each variable to improve our objective function while making small changes to the variables, repeating for many different inputs. There's a few more wrinkles to it, but this is the driving intuition.\n",
    "\n",
    "Now, how can we apply this to our example network from before? Well, let me give you three derivative functions:\n",
    "\n",
    "* $\\frac{dL}{dy} = \\begin{cases}\n",
    "    -1 & \\text{if } x \\leq 0\\\\\n",
    "    1 & \\text{if } x > 0\n",
    " \\end{cases} $\n",
    "  \n",
    "* $\\frac{dy}{dW} = x$\n",
    "* $\\frac{dy}{db} = 1$ \n",
    "\n",
    "**Questions**\n",
    "\n",
    "Write down $\\frac{dL}{dW}$ and $\\frac{dL}{db}$. If $x = 2$, what will each of them be? How about for $x = -3$? $x = 10$?\n",
    "\n",
    "We can update our parameters as $W_{t+1} = W_t -eta *frac{dL}{dW}$\n",
    "$\\eta$(the Greek letter \"eta\") is a number, usually called the *learning rate(LR)* which specifies how much we should change for each iteration of SGD. $t$ indicates which iteration we are on(e.g. the first iteration, producing parameters to be used in the second iteration, and so on). Normally we might do some thousands of iterations, depending on our data.\n",
    "\n",
    "Using the sequence of values of $x$ above and $\\eta = 0.1$, what will $W$ and $b$ be after doing SGD for each of them in the given order? Will their values be different if the values of $x$ were presented in a different order?\n",
    "\n",
    "(Advanced) Now suppose we add another layer to our network, so we have $y_1 = W_1x + b_1$ and $y_2 = W_2y_1 + b_2$, with $L$ taking $y_2$ in place of $y$. What is $\\frac{dL}{dW_1}$? Remember that the chain rule/backpropagation can be applied multiple times, and that $y_2$ is a function of $y_1$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "txTqpG7MOBTf"
   },
   "source": [
    "Okay, now that we understand neural networks and have looked at our data, let's define a simple neural network, consisting of 10 neurons, one for each digit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "PQn8hE2bOVOg"
   },
   "outputs": [],
   "source": [
    "class SimpleNet(nn.Module):\n",
    "\n",
    "  #Initialize some components of our network\n",
    "  def __init__(self, n_inputs, n_neurons = 10):\n",
    "    super(SimpleNet, self).__init__()\n",
    "    \n",
    "    self.n_inputs = n_inputs\n",
    "    self.n_neurons = n_neurons\n",
    "    \n",
    "    #Pre-implemented FC layer. You will implement this yourself later.\n",
    "    #Takes in an MNIST image(usually 28 by 28 pixels) and outputs 10 values\n",
    "    \n",
    "    #QUESTION: If x is an image and y the output, with W and b being weights\n",
    "    #and biases, what function does this compute?\n",
    "    #QUESTION: What is the shape of the weight matrix of this network?\n",
    "    #Hint: For a fully-connected layer, it needs a value mapping each\n",
    "    #input value to each output value\n",
    "    self.fc = nn.Linear(self.n_inputs, self.n_neurons)\n",
    "    \n",
    "  #Computes the \"forward pass\" of the network.\n",
    "  #In this case, we take in an image x, flatten it into a vector,\n",
    "  #matrix multiply it with the weight matrix, and add the bias vector\n",
    "  #The result is 10 values, which, with the right loss function,\n",
    "  #we can interpret as the probability of the given image being each of 0-9.\n",
    "  def forward(self, x):\n",
    "    #Apply the network\n",
    "    x = self.fc(x)\n",
    "    return x\n",
    "  \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "J7in5_NzcLYe"
   },
   "source": [
    "**QUESTION**\n",
    "\n",
    "If each image was 2x2 pixels and we only had 3 digit classes, what would the above network look like?\n",
    "Draw(on paper) a sketch of this network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Vhkv4Ebucyw6"
   },
   "outputs": [],
   "source": [
    "#Let's define a function to visualize our training performance\n",
    "def visualize(t, errcl, w, x):\n",
    "    errorrate = np.divide(\n",
    "        np.cumsum(errcl[: t + 1]),\n",
    "        range(1, t + 2)\n",
    "    )\n",
    "\n",
    "    display.clear_output(wait=True)\n",
    "\n",
    "    plt.subplot(1, 3, 1)\n",
    "    plot_image(w.view((28,28)))\n",
    "    plt.title(\"weight vector at t = \" + str(t + 1))\n",
    "\n",
    "    plt.subplot(1, 3, 2)\n",
    "    plot_image(x.view((28,28)))\n",
    "    plt.title(\"input vector\")\n",
    "\n",
    "    plt.subplot(1, 3, 3)\n",
    "    plt.plot(errorrate, color='red')\n",
    "    plt.title(\"average class error = \" + str(errorrate[t]))\n",
    "    #plt.ylim(0, 1)\n",
    "\n",
    "    plt.subplots_adjust(wspace=.5)\n",
    "\n",
    "    plt.gcf().set_size_inches(18.5, 10.5)\n",
    "    display.display(plt.gcf())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I5J049ulcDyT"
   },
   "source": [
    "Let's define the parameters we need to train our network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "5Hnu6d-acIer"
   },
   "outputs": [],
   "source": [
    "n = 28*28   # number of image pixels\n",
    "m = 60000   # number of images in training set\n",
    "eta = 0.0000001  # learning rate parameter\n",
    "tmax = 60000   # number of learning updates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "kS-HuhVEzUDB"
   },
   "outputs": [],
   "source": [
    "#Let's initialize our network\n",
    "net = SimpleNet(n, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3ANy0uZoz2wj"
   },
   "source": [
    "\n",
    "Let's also define an optimizer. This object is responsible for changing the network's weights based on the gradient computed by the backwards pass.\n",
    "\n",
    "We will also need to define our loss function. We won't get into the math, but the cross-entropy loss is standard for classification. It takes in a set of values(one for each digit), weights them to produce a set of values that sum to 1(representing the probability that this image is each digit class), and then compares to the \"true probability\" (probability 1 for the correct class, 0 for the other 9)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Am8xs5msca3y"
   },
   "outputs": [],
   "source": [
    "opt = torch.optim.SGD(net.parameters(), lr = eta)\n",
    "\n",
    "loss_func = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "O-_2qhsEc6W6"
   },
   "source": [
    "Finally, here's our core training loop.\n",
    "We will do the following tmax times:\n",
    "\n",
    "1. At each iteration we get a new image at random\n",
    "2. We run our network on the image\n",
    "3. Then we compute the error, delta, by comparing the output to our known label for the digit \\\n",
    "4. We then update our network using that delta."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 3268
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 682,
     "status": "error",
     "timestamp": 1532468144773,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "PLpQ_5Qoc5dj",
    "outputId": "8abd49e5-ca5d-43bf-d5aa-43110bc7960a"
   },
   "outputs": [
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-26-001c04aa3eef>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m     \u001b[0mopt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mt\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0;36m1000\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m    \u001b[0;31m# display every 1000 time steps\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m         \u001b[0mvisualize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlosses\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnet\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mweight\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mlabel\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdetach\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m         \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msleep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0.01\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-22-0c7dfef044df>\u001b[0m in \u001b[0;36mvisualize\u001b[0;34m(t, errcl, w, x)\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m     \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgcf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_size_inches\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m18.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10.5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m     \u001b[0mdisplay\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdisplay\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgcf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/display.py\u001b[0m in \u001b[0;36mdisplay\u001b[0;34m(include, exclude, metadata, transient, display_id, *objs, **kwargs)\u001b[0m\n\u001b[1;32m    296\u001b[0m             \u001b[0mpublish_display_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    297\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 298\u001b[0;31m             \u001b[0mformat_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmd_dict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minclude\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minclude\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexclude\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mexclude\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    299\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mformat_dict\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    300\u001b[0m                 \u001b[0;31m# nothing to display (e.g. _ipython_display_ took over)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/formatters.py\u001b[0m in \u001b[0;36mformat\u001b[0;34m(self, obj, include, exclude)\u001b[0m\n\u001b[1;32m    178\u001b[0m             \u001b[0mmd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    179\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 180\u001b[0;31m                 \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mformatter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    181\u001b[0m             \u001b[0;32mexcept\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    182\u001b[0m                 \u001b[0;31m# FIXME: log the exception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<decorator-gen-9>\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, obj)\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/formatters.py\u001b[0m in \u001b[0;36mcatch_format_error\u001b[0;34m(method, self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    222\u001b[0m     \u001b[0;34m\"\"\"show traceback on failed format call\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    223\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 224\u001b[0;31m         \u001b[0mr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    225\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mNotImplementedError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    226\u001b[0m         \u001b[0;31m# don't warn on NotImplementedErrors\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/formatters.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, obj)\u001b[0m\n\u001b[1;32m    339\u001b[0m                 \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    340\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 341\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mprinter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    342\u001b[0m             \u001b[0;31m# Finally look for special method names\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    343\u001b[0m             \u001b[0mmethod\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_real_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprint_method\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/pylabtools.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(fig)\u001b[0m\n\u001b[1;32m    239\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    240\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;34m'png'\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mformats\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 241\u001b[0;31m         \u001b[0mpng_formatter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfor_type\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mFigure\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mprint_figure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'png'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    242\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;34m'retina'\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mformats\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m'png2x'\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mformats\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    243\u001b[0m         \u001b[0mpng_formatter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfor_type\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mFigure\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mretina_figure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/IPython/core/pylabtools.py\u001b[0m in \u001b[0;36mprint_figure\u001b[0;34m(fig, fmt, bbox_inches, **kwargs)\u001b[0m\n\u001b[1;32m    123\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    124\u001b[0m     \u001b[0mbytes_io\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mBytesIO\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 125\u001b[0;31m     \u001b[0mfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcanvas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprint_figure\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbytes_io\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    126\u001b[0m     \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbytes_io\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetvalue\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    127\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mfmt\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'svg'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/backend_bases.py\u001b[0m in \u001b[0;36mprint_figure\u001b[0;34m(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)\u001b[0m\n\u001b[1;32m   2261\u001b[0m                 \u001b[0morientation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morientation\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2262\u001b[0m                 \u001b[0mbbox_inches_restore\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0m_bbox_inches_restore\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2263\u001b[0;31m                 **kwargs)\n\u001b[0m\u001b[1;32m   2264\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2265\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mbbox_inches\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mrestore_bbox\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py\u001b[0m in \u001b[0;36mprint_png\u001b[0;34m(self, filename_or_obj, *args, **kwargs)\u001b[0m\n\u001b[1;32m    511\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    512\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mprint_png\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfilename_or_obj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 513\u001b[0;31m         \u001b[0mFigureCanvasAgg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    514\u001b[0m         \u001b[0mrenderer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_renderer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    515\u001b[0m         \u001b[0moriginal_dpi\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdpi\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    431\u001b[0m             \u001b[0;31m# if toolbar:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    432\u001b[0m             \u001b[0;31m#     toolbar.set_cursor(cursors.WAIT)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 433\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfigure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    434\u001b[0m             \u001b[0;31m# A GUI class may be need to update a window using this draw, so\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    435\u001b[0m             \u001b[0;31m# don't forget to call the superclass.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/artist.py\u001b[0m in \u001b[0;36mdraw_wrapper\u001b[0;34m(artist, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m     53\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     54\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 55\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     56\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     57\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0martist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_agg_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/figure.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self, renderer)\u001b[0m\n\u001b[1;32m   1473\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1474\u001b[0m             mimage._draw_list_compositing_images(\n\u001b[0;32m-> 1475\u001b[0;31m                 renderer, self, artists, self.suppressComposite)\n\u001b[0m\u001b[1;32m   1476\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1477\u001b[0m             \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose_group\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'figure'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36m_draw_list_compositing_images\u001b[0;34m(renderer, parent, artists, suppress_composite)\u001b[0m\n\u001b[1;32m    139\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mnot_composite\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mhas_images\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    140\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0martists\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 141\u001b[0;31m             \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    142\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    143\u001b[0m         \u001b[0;31m# Composite any adjacent images together\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/artist.py\u001b[0m in \u001b[0;36mdraw_wrapper\u001b[0;34m(artist, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m     53\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     54\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 55\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     56\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     57\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0martist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_agg_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_base.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self, renderer, inframe)\u001b[0m\n\u001b[1;32m   2605\u001b[0m             \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstop_rasterizing\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2606\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2607\u001b[0;31m         \u001b[0mmimage\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_draw_list_compositing_images\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0martists\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2608\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2609\u001b[0m         \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose_group\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'axes'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36m_draw_list_compositing_images\u001b[0;34m(renderer, parent, artists, suppress_composite)\u001b[0m\n\u001b[1;32m    139\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mnot_composite\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mhas_images\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    140\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m \u001b[0;32min\u001b[0m \u001b[0martists\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 141\u001b[0;31m             \u001b[0ma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    142\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    143\u001b[0m         \u001b[0;31m# Composite any adjacent images together\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/artist.py\u001b[0m in \u001b[0;36mdraw_wrapper\u001b[0;34m(artist, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m     53\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     54\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 55\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mdraw\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martist\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     56\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     57\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0martist\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_agg_filter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36mdraw\u001b[0;34m(self, renderer, *args, **kwargs)\u001b[0m\n\u001b[1;32m    591\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    592\u001b[0m             im, l, b, trans = self.make_image(\n\u001b[0;32m--> 593\u001b[0;31m                 renderer, renderer.get_image_magnification())\n\u001b[0m\u001b[1;32m    594\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mim\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    595\u001b[0m                 \u001b[0mrenderer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdraw_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mim\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36mmake_image\u001b[0;34m(self, renderer, magnification, unsampled)\u001b[0m\n\u001b[1;32m    839\u001b[0m         return self._make_image(\n\u001b[1;32m    840\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_A\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbbox\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransformed_bbox\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0maxes\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbbox\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagnification\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 841\u001b[0;31m             unsampled=unsampled)\n\u001b[0m\u001b[1;32m    842\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    843\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_check_unsampled_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrenderer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/image.py\u001b[0m in \u001b[0;36m_make_image\u001b[0;34m(self, A, in_bbox, out_bbox, clip_bbox, magnification, unsampled, round_to_pixel_border)\u001b[0m\n\u001b[1;32m    509\u001b[0m             \u001b[0;31m# (of int or float)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    510\u001b[0m             \u001b[0;31m# or an RGBA array of re-sampled input\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 511\u001b[0;31m             \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_rgba\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbytes\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnorm\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    512\u001b[0m             \u001b[0;31m# output is now a correctly sized RGBA array of uint8\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    513\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/cm.py\u001b[0m in \u001b[0;36mto_rgba\u001b[0;34m(self, x, alpha, bytes, norm)\u001b[0m\n\u001b[1;32m    273\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mnorm\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    274\u001b[0m             \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnorm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 275\u001b[0;31m         \u001b[0mrgba\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0malpha\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbytes\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbytes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    276\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mrgba\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    277\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/anaconda3/lib/python3.6/site-packages/matplotlib/colors.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, X, alpha, bytes)\u001b[0m\n\u001b[1;32m    538\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    539\u001b[0m         \u001b[0mrgba\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mempty\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mxa\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlut\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 540\u001b[0;31m         \u001b[0mlut\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxa\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'clip'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrgba\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    541\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mvtype\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'scalar'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    542\u001b[0m             \u001b[0mrgba\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrgba\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABC8AAAJqCAYAAAAL9GrYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XmcHWWZ9//vt/dO0tlIYAgEIou7gE40UR8EERfc/akgMqM4wyCDo/KoMzrzuKDo6Ogg6rggCoKjgAzi7qCILAoEDIyyKoRtEkIgQMjaS7r7+v1R1eakOd11p/t01+nTn/fr1a/uc+qq+76qTp1Tfa66q8oRIQAAAAAAgHrVVHYCAAAAAAAAo6F4AQAAAAAA6hrFCwAAAAAAUNcoXgAAAAAAgLpG8QIAAAAAANQ1ihcAAAAAAKCuUbwAAAAAgBqyHbYPKDsPoJFQvAAAAAAAlML2IbZvtL0t/33IKLFbhv0M2P6PiulH277D9mbbt9t+fcW04/P4yvkPz6ftbvsC22ttb7R9je1lFfMebntw2Lxvr5j+HdsP2t5k+07bJ1RMa7N9se378qLW4cOW6cW2r8j7va/KMi/Jp2+z/UfbR1ZMe4vtP+XzPmz7PNuz82ntts+2fX++Pv7H9lHD2o1hy/SRiunzbX/P9iP5z3eH2s6n32e7u2LeXw7L+//aXpfndo7t9pR1PRqKFwAAAACmPNstZedQ76qto11db7Vcz7bbJP1I0nckzZN0nqQf5c8/QUTMGvqRtIekbkn/lbe1V97O+yTNlvSPks63vXtFE9dVthERV+bPz5L0O0l/KWl+nsfPbM+qmHftsHnPq5j2aUlLImK2pNdK+qTtv6yY/ltJfyVpXZXF2irpnDzfai6Q9D+SdpP0/yRdbHthPu0aSS+MiDmS9pPUIumT+bQWSaslHSZpjqSPSLrI9pJh7c+tWKbTKp7/pLLXZD9J+ytb36cOm/c1FfO+bOhJ2y+X9CFJL5G0JG/j4/nklHVdFcULAAAAABPK9ods311xRPwN+fPtth+3/cyK2IX5Ed3d88evtv37PO5a2wdVxN5n+4O2b5a01XbLSH3l8c22T8+PJN9r+x/yo88t+fQ5+dHqB20/YPuTtptHWKZm2/9S0deNthdXiXtVftR7k+3Vtk+tmNaRH7V/NF++39neI592vO178rbvtX3cCHk0VSzzo7Yvsj0/nzZ0dP1vbf+vpF9Xey6Pfa3t2/I8rrT9tNHWc+GLnuZwZV+yvxARvRHxJUmWdETCvG+S9LCk3+SP95b0eET8d2R+pqwwsH9RQxFxT0R8PiIejIiBiDhLUpukp6QsRETcFhG9Qw/zn/3zaX0R8YWI+K2kgSrz3hAR/ynpnuHTbD9Z0nMkfSwiuiPi+5JukfTGfN7VEfFIxSwDkg7Ip22NiFMj4r6IGIyIn0q6V1nRIMWTJP0wIjZFxEZJP5D0jMR53y7p7Hy9bJB0mqTj87zGvK4pXgAAAACYaHdLOlTZEeCPS/qO7T3zL3yXSDq2IvZoSVdFxMO2n6PsqPQ7lR15/rqkHw8NQc8dK+lVyo4g94/UVx77d5KOknSIsi+Fr9fOzpPUr+wL4LMlvUzSCarufXnfr1R2pP9vJG2rErdV0tskzc3z/HvvOJ3h7Xmei/PlO0lSt+2Zkr4k6aiI6JL0Akm/HyGP9+TLcZikRZI2SPrKsJjDJD1N0surPZd/Sb5A0imSFkr6uaSfDBsBMXw978T2zXnho9rPV0fI/RmSbo6IqHjuZqV9SX67pG9XzLtS0h15EaY5X8e9eXtDnp0Xru60/ZGRijDOTl1pk7Sq4undbT+UF5LOyF+jynm+anubpD9KelDZOhyvZ0i6JyI2Vzz3B1WsH9v/x/ZGSZuVFTW+MMIy7SHpyZJuGzbpfttrbH/L9oKK578i6dW259mel7f938Pm/a7t9bZ/afvgYXn/YVjOe9jerUpe1dZ1VRQvAAAAAEyoiPiviFibHwH+nqS7JD0vn3y+di5evDV/TsqKDV+PiOvzo7TnKftCurwi/kv5EejuhL6OlvTFiFiTHxH+zFAj+Ze7oySdkh+1fljSGZLeMsJinSDpwxHxp/xI/x8i4tEqy35lRNyS53OzsiLBYfnk7cqKFgfky3djRGzKpw1Keqbtzvwo9fAvnUPeKen/5cvUq2xo/5uGfTE/NV+m7hGeO0bSzyLisojYLunfJXUqK5oM2Wk9V1nOgyJi7gg/J4+Q+yxJG4c9t1FS1wjxkiTb+yhbh38+dSMiBiR9W9m205v/fmdEbM1Drpb0TEm7K/sifqyqnKrh7LoO/ynp4/mIAykrSBwiaU9lo0L+UtLnhy3/yXnehyoryPVq/ArXT0T8Nj9tZG9Jn5N03/BGbLdK+q6k8yLij/nTj0h6rqR9lS1PVx4z5CZlRYVH858BSZVFqOOUnRKyr6QrJP3C9twR8h76e6fXdYR1PSKKFwAAAAAmlO23VZz68biyL5FDR3l/LanT9jLb+yr7kviDfNq+kt5feRRf2SiFRRXNr96FvhYNi6/8e19JrZIerJj368q+7FazWNkoj6JlX+bsgovr8yPkJ1Xk85+SfiHpQmcXMPys7db8C/cxeeyDtn9m+6kjdLGvpB9U5HyHsi+ae4ywnNWeWyTp/qEHETGYT9+roI3x2qJs1Eql2cpGEYzmbZJ+GxH3Dj3h7EKWn1V2KkqbsuLGN/Mj+0OnK9ybF5FukfQJZaeeqKKNTkk/kbQiIj499HxErIuI2/N575X0T8PnzeMG8tND9pb094VLXyx5/UTEA5IulXRh5fO2m5RtZ32S/qEifktErIyI/oh4KJ/2Mu+4KOd/SbpTWcFhtrJt/TsV81+Tn8qyLV9Xjysr3FTLe+jvP+c90roeDcULAAAAABMmL0h8Q9mXo90iYq6kW5Vd22Doi/JFyo6Ev1XSTyuGya+W9KlhR/FnRMQFFV1Eal/KhvPvXTFv5TUqVis7Wr6goq/ZETHSKQyrlXA9BWUjAH4saXF+hPzMimXfHhEfj4inKxvl8GplX8wVEb+IiJcqO9r/x3y5RsrjqGHrqCP/MjskqsxX+dxaZUUQSZJtK1s3RW38WX69jOF3Axn6OXOE2W6TdFDe35CD9MRTG4Z7mypGXeQOkXR1/oV8MCJ+J+l6SUc+Ye4dy/PnfvNTkX6obJnfWdD/TvNW0aK0baPIbZL2s105YuFgjbx+duo3X69nKytkvTEfVTOSodd3aLkOVjbqaWtEbFG23b6yYP6heW/L56/M+aGhkUm7uK7/jOIFAAAAgIk0U9kXm/WSZPsdykZDVDpf2UiD47TjlBEp+8J+Uj56wbZnOrsA5kinFRT1dZGk99reKx/i/sGhCRHxoKRfSjrd9mxnF8Lc3/Zhqu6bkk6zfWCe20HVzulXduT6sYjosf08ZQUa5fm92PaznF0UdJOy00gGbO+RX7thprKCyhZVudhj7kxJn8oLN0MXPH3dCLEjuUjSq2y/JD/F4P15v9emNhARz4id78ZR+XPSCLNdqWy53uPs4q1DIwN+PVI/tl+gbETIfw2b9DtJhw6NtLD9bGUjAW7OHx/lHRdDfaqyu2/8KH/cKuliZXcveVteUKvs83Db++Sv82JlpxsNzbu7s1uWznJ2rY2XKyvE/bpi/nbbHfnDNmcXanU+rSmf1po9dIfza41ExJ3KrnXysfz5Nygr7nw/n/e4irz2lfQpSZdXpP41Zdc1ec3w033y99RT8v53U3aNlSsrTt/4naQTbHfmoyROVH4di7zPFzq7DWyH7X9UNpromnzeb0v6W9tPd3a9jA9LOjdlXY+G4gUAAACACRMRt0s6XdJ1kh6S9Czt+JIzFHO9sgtbLlLFRQEjYqWy6158WdmFKFcpv2vBGPv6hrICxc3Kbj/5c2UX6BwqDLxN2SkHt+f9Xaxs5EM1n1f2pf+XygoPZyu7TsRwJ0v6hO3Nkj6azzPkL/I+Nik73eMqZUPzm5QVENZKekzZKRAjXTfii8pGdvwy72OFpGUjxFYVEX9SdivP/1B2LYTXKPvC27cr7eyqvP3XK1vvjyu76Onrh/p1djeX4ReJfLukS2Lni1gqIq5Sdr2Pi/P18H1J/xoRv8xDXiLpZttblb3ul0j613za0KiXl0l6vGLEyNBpEM9Rtk1tVVbQuVXZhVKlrFj295LWKNtm/l3ZdVN+VJHen5R9Wd9L2WlC3dox0uVF+eOfS9on//uXFfO+RdLSvO3PSHpTRKzPpz09z2eLsu38T8reL0OjkN6pbETKuoplGrprzX7KTjPZnC9Pr3a+9szfKLumxRplIyT20473XpeywsiGfNorlI3+eVSSIuJSZafwXKHsdKT7JX0scV2PyBGjjv4BAAAAgIZk+yhJZ0bEvoXBAErFyAsAAAAA00I+BP6Vtlts76XsaPAPiuYDUD5GXgAAAACYFmzPUHZqxlOVDc//maT3VtyeFECdongBAAAAAADqGqeNAAAAAACAukbxAgAAAAAA1LWWshMAAAAAML0sWLAglixZUnYaAEp04403PhIRC1PjKV4AAAAAmFRLlizRypUry04DQIls378r8Zw2AgAAAAAA6hrFCwAAAAAAUNcoXgAAAAAAgLpG8QIAAAAAANQ1ihcAAAAAAKCuUbwAAAAAUJXtxbavsH2H7dtsv3eU2OfaHrD9psnMEcD0wK1SAQAAAIykX9L7I+Im212SbrR9WUTcXhlku1nSv0n6RRlJAmh8jLwAAAAAUFVEPBgRN+V/b5Z0h6S9qoS+W9L3JT08iekBmEYoXgAAAAAoZHuJpGdLun7Y83tJeoOkMwvmP9H2Stsr169fP1FpAmhQFC8AAAAAjMr2LGUjK06JiE3DJn9B0gcjYmC0NiLirIhYGhFLFy5cOFGpAmhQXPMCAAAAwIhstyorXHw3Ii6pErJU0oW2JWmBpFfa7o+IH05imgAaHMULAAAAAFU5q0icLemOiPh8tZiIeFJF/LmSfkrhAkCtUbwAAAAAMJIXSvprSbfY/n3+3L9I2keSImLU61wAQK1QvAAAAABQVUT8VpJ3If74icsGwHTGBTsBAAAAAEBdo3gBAAAAAADqGsULAAAAAABQ1yheAAAAAACAukbxAgAAAAAA1DWKFwAAAAAAoK5RvAAAAAAAAHWN4gUAAAAAAKhrFC8AAAAAAEBdo3gBAAAAoH7Z2Q+AaY3iBQAAAAAAqGsULwAAAAAAQF2jeAEAAAAAAOoaxQsAAAAAAFDXKF4AAAAAAIC6RvECAAAAAADUNYoXAAAAAACgrlG8AAAAAAAAdY3iBQAAAAAAqGsULwAAAAAAQF2jeAEAAAAAAOoaxQsAAAAAAFDXKF4AAAAAAIC6RvECAAAAAADUNYoXAAAAAACgrlG8AAAAAAAAdY3iBQAAAAAAqGsULwAAAAAAQF2jeAEAAAAAAOoaxQsAAAAAAFDXKF4AAAAAqH+zZpWdAYASUbwAAAAAUP+2bi07AwAlongBAAAAAADqGsULAAAAAABQ1yheAAAAAACAukbxAgAAAAAA1DWKFwAAAACmhpkzy84AQEkoXgAAAACYGrZtKzsDACWheAEAAABg6rDLzgBACSheAAAAAJhaKGAA0w7FCwAAAAAAUNcoXgAAAACYGlpby84AQEkoXgAAAACofxFSX9+Ox/Pnl5cLgElH8QIAAADA1LNhQ9kZAJhEFC8AAAAATB1HHrnjby7cCUwbFC8AAAAATB2XXVZ2BgBKQPECAAAAwNTygheUnQGASUbxAgAAAMDUcs01O/7m1BFgWqB4AQAAAAAA6hrFCwAAAABTT8SOvxl9ATQ8ihcAAAAAAKCuUbwAAAAAMDUx+gKYNiheAAAAAACAukbxAgAAAMDUVTn6AkDDongBAAAAoDFw6gjQsCheAAAAAACAukbxAgAAAMDUxqkjQMOjeAEAAACgKtuLbV9h+w7bt9l+b5WY42zfnP9ca/vgMnL9s/b2UrsHMDFayk4AAAAAQN3ql/T+iLjJdpekG21fFhG3V8TcK+mwiNhg+yhJZ0laVkaykqS+vtK6BjBxGHkBAAAAoKqIeDAibsr/3izpDkl7DYu5NiI25A9XSNp7crPMvfnNpXQLYHJQvAAAAABQyPYSSc+WdP0oYX8r6b8nI58nuOiiHX//8IelpABg4nDaCAAAAIBR2Z4l6fuSTomITSPEvFhZ8eL/jDD9REknStI+++wzQZnm3vAGLuIJNBhGXgAAAAAYke1WZYWL70bEJSPEHCTpm5JeFxGPVouJiLMiYmlELF24cOHEJQygIVG8AAAAAFCVbUs6W9IdEfH5EWL2kXSJpL+OiDsnM78nYLQF0LA4bQQAAADASF4o6a8l3WL79/lz/yJpH0mKiDMlfVTSbpK+mtU61B8RS0vIdWczZkjbtpWdBYAaoXgBAAAAoKqI+K0kF8ScIOmEycloF3R3l50BgBritBEAAAAAjeOZzyw7AwATgOIFAAAAgMZxyy1lZwBgAlC8AAAAAFC/mpqk2bPHNu+xx9Y2FwCloXgBAAAAoH4NDEgbN45t3gsvrG0uAEpD8QIAAAAAANQ1ihcAAAAAGstb3lJ2BgBqjOIFAAAAgMZywQVlZwCgxiheAAAAAGhchx1WdgYAaoDiBQAAAIDGdfXVZWcAoAYoXgAAAABoPE181QEaCe9oAAAAAI1nYKDsDADUEMULAAAAAI3txz8uOwMA40TxAgAAAEBje93rys4AwDhRvAAAAAAAAHWN4gUAAACAxvSjH5WdAYAaoXgBAAAAoDG99rVlZwCgRiheAAAAAGh88+aVnQGAcaB4AQAAAKDxPf542RkAGAeKFwAAAAAaV1dX2RkAqAGKFwAAAAAa16ZNZWcAoAYoXgAAAAAAgLpG8QIAAADA9HDEEWVnAGCMKF4AAAAAmB6uuKLsDACMEcULAAAAAABQ1yheAAAAAGhsP/lJ2RkAGCeKFwAAAAAa26tfXXYGAMaJ4gUAAACA6ePOO8vOAMAYULwAAAAAMH085SllZwBgDCheAAAAAACAukbxAgAAAEDj+8hHys4AwDhQvAAAAADQ+D7xibIzADAOFC8AAAAAAEBdo3gBAAAAYHr52tfKzgDALqJ4AQAAAGB6OfnksjMAsIsoXgAAAAAAgLpG8QIAAADA9MAdR4Api+IFAAAAgOmBO44AUxbFCwAAAADTz09/WnYGAHYBxQsAAAAA089rXlN2BgB2AcULAAAAAABQ1yheAAAAAJg+5s8vOwMAY0DxAgAAAMD08eijZWcAYAwoXgAAAAAAgLpG8QIAAADA9PTRj5adAYBEFC8AAAAATE+nnVZ2BgASUbwAAAAAAAB1jeIFAAAAgOnluOPKzgDALqJ4AQAAAGB6+c53ys4AwC6ieAEAAAAAAOoaxQsAAAAA05dddgYAElC8AAAAAAAAdY3iBQAAAIDpp7W17AwA7AKKFwAAAACmn1tvLTsDALuA4gUAAACA6efJTy47AwC7gOIFAAAAgOnt5z8vOwMABSheAAAAAJjeXvWqsjMAUIDiBQAAAAAAqGsULwAAAABUZXux7Sts32H7NtvvrRJj21+yvcr2zbafU0auY9LeXnYGABK1lJ0AAAAAgLrVL+n9EXGT7S5JN9q+LCJur4g5StKB+c8ySV/Lf9e/nh7JLjsLAAkYeVEC22fa/khi7Lm2PznROQEAGlt+xPTwsvMAMLVExIMRcVP+92ZJd0jaa1jY6yR9OzIrJM21veckpwqgwVG8KEFEnBQRp9WiLdth+4BatLULfR5v+7eT3OeoyzkZOdk+0HaP7e9UPLen7R/bXpvnuGTYPO22z7G9yfY62+8bNv0ltv9oe1s+JHPfimlH2742n3blRC4bgMYXEc+IiCsnup+yiu5l7A+B6Sb/P+fZkq4fNmkvSasrHq/REwsc9e/rXy87AwCjoHiBSWd7qp6u9BVJvxv23KCkSyW9cYR5TlU2hHJfSS+W9E+2XyFJthdIukTSRyTNl7RS0vcq5n1M0hckfaY26QMAqpnC+yVg0tieJen7kk6JiE3DJ1eZJaq0caLtlbZXrl+/fiLSHJ+TTio7AwCjoHixC2y/w/ZPKh6vsn1RxePVtg/J/36q7ctsP2b7T7aProjb6aiU7X+y/WB+9P6EKkeP5tn+me3Ntq+3vX8+39X59D/Y3mL7mGH5ttt+3PYzK55baLvb9u7541fb/n0ed63tgypiF9u+xPZ624/a/rLtp0k6U9Lz8z4fz2Pn2P52Hnu/7Q/bbsqnHW/7Gttn2H5M2Rf64ev2ebavy/N4MO+rLXE5q+ZUS7bfIulxSZdXPh8RD0XEV/XEosaQt0k6LSI2RMQdkr4h6fh82v8n6baI+K+I6FG2Xg62/dS87V9FxEWS1tZ6eQBMP7bvs31k/vepti/KP7c356eULB0W+8+2b7e9wfa3bHfk054w0m1ov2X7REnHKSvUbqncZ1bEnmn734c996OhkWm2F9n+fr4/udf2eyrimm3/i+2787xvzPdVVfcTtv8u31c/5myU3KJhOb/L9l2S7hrf2gUam+1WZYWL70bEJVVC1khaXPF4b1X5/yUizoqIpRGxdOHChROTLICGRfFi11wl6VDbTc7O42uV9EJJsr2fpFmSbrY9U9Jlks6XtLukYyV91fYzhjfo7Cj8+yQdKekASYdV6fdYSR+XNE/SKkmfkqSIeFE+/eCImBURlUftFRG9yo7sH1vx9NGSroqIh51dCfocSe+UtJukr0v6cV70aJb0U0n3S1qibOjfhfkX8JMkXZf3OTdv9z8kzZG0X74Mb5P0jop+l0m6J18fn6qyjAOS/q+kBZKeL+klkk5OXM6RctqJ7a/mxZFqPzdXmyefb7akT0h6/0gxI8w3T9IiSX+oePoPkoa2g2dUTouIrZLurpgOABPptZIulDRX0o8lfXnY9OMkvVzS/pKeLOnDRQ1GxFmSvivps/nn8WuqhJ0v6Rg7u0Je/ln5MkkX5kXvnyj7bNxL2b7gFNsvz+d9n7J92islzZb0N5K2VdtP2D5C0qeV7ff2VLY/u3BYLq9Xtn96etGyAdNV/l49W9IdEfH5EcJ+LOltziyXtDEiHpy0JMfrPe8pjgFQOooXuyAi7pG0WdIhyr6g/0LSA/mR8sMk/SYiBiW9WtJ9EfGtiOjPL3L0fUlvqtLs0ZK+FRG3RcQ2ZUWK4S6JiBsiol/ZP4WH7ELa52vn4sVb8+ck6e8kfT0iro+IgYg4T1KvpOWSnqfsi/c/RsTWiOiJiKrXlMgLHcdI+ueI2BwR90k6XdJfV4StjYj/yNdH9/A2IuLGiFiRT79PWSGlWiFnzCLi5IiYO8LPQaPMepqksyNi9Sgx1czKf2+seG6jpK6K6Ru1s8rpADCRfhsRP4+IAUn/KengYdO/HBGrI+IxZUXnY5/Qwtj8Rtlw8kPzx29SVnxeK+m5khZGxCcioi/f735D0lvy2BMkfTgi/pRfGPAPEfHoCP0cJ+mciLgpL+b/s7IReksqYj4dEY9V2y8B+LMXKvuf7oh8tO7vbb/S9km2h86z+Lmyg1SrlL1nTy4p17H54hfLzgBAAs7x3HVXSTpc2SiJq5SdSnCYstECV+Ux+0paNuz0hRZl/xwOt0jZtQ6GVPuCvK7i723a8aU4xa8lddpelrdziKQfVOT5dtvvrohvy3MakHR/XjApsiCf7/6K5+7XzhdqGvWLv+0nS/q8pKWSZihbXzcm9D2hnJ0GdKSyi1Ptqi3579mSeir+3lwxffaweSqnA8BEGr5v6bDdUvG5X/m5fb+yfcO4RUTYvlBZMeRqZUX1oQsh7ytp0bD9Z7OygoeUDUu/O7GrRZJuquh3i+1Hle2b7suf3tWiNDDt5AevRr2XaESEpHdNTkYApitGXuy6oeLFofnfVykrXhymHcWL1cpOzag8sj8rIv6+SnsPKjsvcMjiKjFjlo8EuUjZP4lvlfTT/DZXQ3l+alieMyLignzaPq5+EbPhF2B6RNJ2Zf90DtlH0gOjzDPc1yT9UdKBETFb0r+oYEdZkNMT5OdZbxnh57YRZjtc2Wkz/2t7naQPSHqj7ZtGiN+RUMQGZa9v5dHMgyUN9XVb5bT8dKP9K6YDQJkq90f7aMf561uVFZklSbb/Yth8hZ/Hki6Q9CZnd1hapmx0opTte+4dtl/qiohXVkzfPzH/tarYL+Wfsbtp1/ZNAKabGTOKYwCUguLFrrtK2V0jOiNijbKjQa9Q9g/R/+QxP5X0ZNt/bbs1/3lufmHJ4S6S9A7bT7M9Q9JHdzGfh5RdZ2I05ys7reM47ThlRMqG9Z1ke1l+juJM26+y3SXpBmVfvD+TP99h+4UVfe7t/IKa+ZDjiyR9ynZX/s/o+7TjSFqKLkmbJG3JT8MZXugpWs6dcqomv0XtrBF+RrrOxFnK/lE+JP85U9LPlJ0HLknKL2LXnj9sH7qoXe7bkj5se16+XH8n6dx82g8kPdP2G/N5Pirp5oj4Y95uc/58i6Sm/DVoHWUdAEAtvcv23rbnKysoD11v6A+SnmH7kPwz6tRh8xXulyLifyStl/RNSb+IiKGRFjdI2mT7g7Y788/BZ9p+bj79m5JOc3brats+yPZuI/R7vrL96yG22yX9q6Tr81MTAaC6bs4iA+oVxYtdFBF3Khvu/5v88SZl5/hdk3+JVz6y4WXKztFdq2xo7r9pxxfcyvb+W9KXJF2h7DzB6/JJvYkpnSrpvPyik0dXC4iI65UdKVsk6b8rnl+p7Mv0lyVtyPs/Pp82IOk1yk6P+V9lV5EeusvHr5WNDlhn+5H8uXfnfdwj6bfK/mk8J3EZpGxEw1uVnTLxDe18y9CU5ayW07hFxLaIWDf0o+y174mIyvt7dWvHKSJ/zB8P+ZiyIc73Kyt8fS4iLs3bXq/sFqufUrb+l2nHed1Sdn5pt7JRKYfmf3+jVssGAAXOl/RLZZ/r90j6pPTn/eAnJP1K2V06hl8P6WxJT88/r384SvsXKDst789F9Yp9zyGS7lU2su+byi4ILWWnF16U57Up76szn3aqKvYTEXG5sltRf19ZMX5/7fwZCwAAphBnp6ihXuSjM26V1J54vQkAAGrK9n2STojfUyFoAAAgAElEQVSIX5WdC4DGtHTp0li5cmVx4GS56y7pyU/O/ub7ETApbN8YEUuLIzOMvKgDtt9guy2/Xdy/SfoJhQsAAABgkhx4YNkZAChA8aI+vFPZub93K7vLR7ULewIAAAAAMC1xq9Q6EBGvKDsHAACGRMSSsnMAgNLcdRcjMYA6xMgLAAAAABgydO0LAHVlUkdezJgxI+bMmTNqTMoFRFMvMtrUVFybSWnLdmHM4OBgUk4pUvqrpVqtg5SYWl4gdrJflxQp21xKTrXcBmr1+tbKxo0b1d3dPbkbOeqKba6EBkARwb4AAJBsXMUL26+Q9EVJzZK+GRGfGS1+zpw5Ov7440dts7+/+DqVAwMDSfl1dHQUxqT0l/KFtDvxntDNzc01ianlF9KUddDW1lYYk7KeUl+7lNxr9bqktCOlrfP29ifcDfcJ+vr6CmNStoHUnFLWecrrm/raFa3Pb3/720ntYOrY1X0BAAB16cgjpV9xkyWgXo35tBHbzZK+IukoSU+XdKztp9cqMQBA/WNfAABoGJddVnYGAEYxnmtePE/Sqoi4JyL6JF0o6XW1SQsAMEWwLwAAAMCEG0/xYi9Jqyser8mfAwBMH+wLAACN5+MfLzsDAMOMp3hR7aIETzgB3/aJtlfaXrlt27ZxdAcAqEO7vC+YhJwAABifU08tOwMAw4yneLFG0uKKx3tLWjs8KCLOioilEbF0xowZ4+gOAFCHdnlfMGmZAQAAoGGMp3jxO0kH2n6S7TZJb5H049qkBQCYItgXAAAax/OfX3YGAEYw5lulRkS/7X+Q9Atlt8c7JyJuG22ewcFB9fT0jNpuyi0ya3kbyZT+anlb0pRbUqa0lXIL0M7OzqScUm6BOTg4WJN2UqXcvjVl+Vpaijfx1LxbW1sLY1Jug5qyLlNipNptTyn9pSy/JPX29o46PeX9hKljLPsCAADq1rXXSon/1wOYXGMuXkhSRPxc0s9rlAsAYApiXwAAAICJNp7TRgAAAACgMa1aVXYGACpQvAAAAACA4Z761LIzAFCB4gUAAAAADFfD67kBGD+KFwAAAAAAoK5RvAAAAACAIaedVnYGAKqgeAEAAAAAQz784bIzAFAFxQsAAAAAAFDXWia7Q9ujTm9ubi5so6WldmkPDg4WxgwkXKynaLmG9PX11SSnzs7Owpjt27cn5dTUVFzDSlkHKa9dSjuS1N7eXhizbdu2wpiIKIxpa2tLyillfaZsB62trYUxKXlLadtTyrbS3d1dGJOynUjF20HqewUAAKB0q1ZJBxxQdhYAxMgLAAAAAKjuWc8qOwMAOYoXAAAAAFBNT0/ZGQDIUbwAAAAAAAB1jeIFAAAAAFR6znPKzgDAMBQvAAAAAKDSjTeWnQGAYSheAAAAAACAukbxAgAAAABGcumlZWcAQBQvAAAAAGBkRx1VdgYARPECAAAAAADUuZayExiLiEiKGxwcLIzZvn17Tfprbm5Oyqm1tbUm/W3evLkwpqOjIymnlHXQk3CP67a2tsKY3t7epJxsF8Y0NRXX3lLWwcDAQFJOKespZTuoVYyUto2nvHYp66m/vz8pp5TXBQAAoO4dcIC0alXZWQDI8S0DAAAAAIa7666yMwBQgeIFAAAAAACoaxQvAAAAAGA0nD4ClI7iBQAAAACM5sADy84AmPYoXgAAAAAAgLpG8QIAAAAAANQ1ihcAAAAAUM1pp5WdAYAcxQsAAAAAqObDHy47AwC5lsnucHBwcFzTJamlJS3tpqbi2kxKW83NzYUx27ZtS8ppYGCgMCYiCmP6+voKY7Zu3ZqUk+3CmLa2tpr019/fn5RTyjrv7OwsjNm+fXthTMp2IqVtK7NmzSqMaW9vL4xJ2U6ktHWQ0lZ3d3dSfylStl8AqJXly5cXxrz5zW8ujDnllFOS+rv++usLY44++ujCmDVr1iT1BwAAMoy8AAAAAIAie+1VdgbAtEbxAgAAAACKrF1bdgbAtEbxAgAAAAAA1DWKFwAAAAAwkne8o+wMAIjiBQAAAACM7Jxzys4AgCheAAAAAACAOkfxAgAAAABSXHpp2RkA0xbFCwAAAABIcdRRZWcATFstk91hU9Po9ZKi6ZJkO6mv1Lgi27dvL4wZGBhIaisiCmNaW1sLYzo7OwtjWlrSXt6urq7CmJR10NHRURjT3d2dlFPK+mxubi6MmTt3bmHMnDlzknJKaStlHcycObMwpr+/PymnwcHBwpiHHnqoMGbdunWFMSnbriT19vbWpB0AWL58eWHMhRdeWBizePHiwpiUz1NJWrZsWU1i1qxZk9QfAADIMPICAAAAAADUtXGNvLB9n6TNkgYk9UfE0lokBQCYWtgfAAAa2l13SQceWHYWwLRWi9NGXhwRj9SgHQDA1Mb+AADQmA44oOwMgGmP00YAAAAAVGX7HNsP2751hOlzbP/E9h9s32b7HZOdI4DpYbzFi5D0S9s32j6xFgkBAKYk9gcA0JjOlfSKUaa/S9LtEXGwpMMlnW67bRLyKs+LXlR2BsC0NN7TRl4YEWtt7y7pMtt/jIirKwPyf2JPlNLuagEAmJJG3R9U7gsAAFNHRFxte8loIZK6nN3mb5akxySl3TZtqvrNb8rOAJiWxjXyIiLW5r8flvQDSc+rEnNWRCyNiKUpt/cEAEw9RfuDyn1BGfkBACbMlyU9TdJaSbdIem9EpN17GAB2wZiLF7Zn2u4a+lvSyyRVPRcOANC42B8AwLT2ckm/l7RI0iGSvmx7drVA2yfaXml75fr16yczx9p41rPKzgCY1sZz2sgekn6QjRBTi6TzI+LS0WZoampSe3v7qI329vaOI6Wdbd26tTCmubm5MKa7u7swJl8PNYlLiZk7d25hTGtra1JOM2bMKIzp6+urSUxPT09SThFRGDNz5szCmJT1VLRNDlmwYEFhzOzZVffVO0nJu60t7VTRlG08pb/BweIDJI8++mhSTgMDA6NOT32vYErZ5f0Bpqbly5cXxixevLgw5pRTTknq7/nPf35hTMr+IuVzp6kp7XhOrfbjQAN5h6TPRPZmXGX7XklPlXTD8MCIOEvSWZK0dOnS4jdvvbn5Zon3N1CaMRcvIuIeSQfXMBcAwBTE/gAAprX/lfQSSb+xvYekp0i6p9yUADSi8V6wEwAAAECDsn2BsruILLC9RtLHJLVKUkScKek0SefavkWSJX0wIh4pKd3Jc/fd0v77l50FMK1QvAAAAABQVUQcWzB9rbJrHU0vBxwgJZy2BqB2xnW3EQAAAAAAgIlG8QIAAAAAANQ1ihcAAAAAkGLVqrIzAKYtihcAAAAAkIKLdAKloXgBAAAAALvq7rvLzgCYViheAAAAAMCueuYzy84AmFYm9VapEaHt27ePu52+vr4aZJMZHBwsjOnt7S2MaW9vT+pvYGCgMGbmzJmFMa2trYUxs2fPTspp1qxZhTEp6+nRRx8tjOnp6UnKadu2bYUxW7duLYxJ2Va6urqSckrR0lL8lmprayuMmTt3blJ/8+fPL4zp6OgojEl57R566KGknFK2FQD1Z/ny5YUxF154YWHM4sWLC2NSPyci4TaEKW01NRUfqzn99NOTcrrhhhsKY1asWJHUFoApLvH/WgC1wcgLAAAAAABQ1yheAAAAAECqD3yg7AyAaYniBQAAAACk+tznys4AmJYoXgAAAAAAgLpG8QIAAAAAxuLss8vOAJg2KF4AAAAAwFiccELZGQDTBsULAAAAAABQ1yheAAAAAMCu6OwsOwNg2mmZ7A5tjzq9o6OjsI1t27Yl9RURhTHd3d2FMSk59fX1JeU0c+bMmvQ3a9aswpjBwcGknLZv314Ys2nTpsKYxx57rDCmp6cnKaeUuIGBgZq08/DDDyflNGfOnMKYlHXe3t5eGJOyXUpp21Nra2tNYlL6kqRHHnlk1Okp70sAtfO9730vKe7oo48ujEn5jCvaz0tSU1PasZOUtlasWFEYc8kllxTGnHHGGUk5AYAkads2KeEzCkDtMPICAAAAAADUNYoXAAAAAACgrlG8AAAAAICx2nPPsjMApgWKFwAAAAAwVuvWlZ0BMC1QvAAAAAAAAHWN4gUAAAAA7KrDDis7A2BaoXgBAAAAALvqyivLzgCYViheAAAAAACAutYymZ3ZVnNz86gxPT09Neuvra2tMKa/v78wJiWn9vb2pJxmzpxZk5hZs2YVxnR1dSXllLJ8g4ODhTEp6/uRRx5Jyqmvr68wZuvWrYUxKXlHRFJOKesppa2i94AkdXd3J+W0adOmwpjt27fXJCZlfUtSR0fHqNObmqiZApMp9TMu5fMyJSblPX766acn5XTDDTcUxqxYsaIwZs2aNUn9AcCYHXSQdPPNZWcBNDS+RQAAAADAeNxyS9kZAA2P4gUAAAAAAKhrFC8AAAAAYCz+8R/LzgCYNiheAAAAAMBYfPazO/5OvMYQgLGheAEAAAAA42WXnQHQ0CheAAAAAMB4veQlZWcANDSKFwAAAAAwXr/+ddkZAA2N4gUAAAAAAKhrLZPZ2eDgoHp7e8fdTlNTWs0lpa+2trbCmEi4+M6sWbOScmppKV7lu+22W2HMvHnzatKOJHV0dBTG9Pf3F8asW7euMKavry8pp40bNxbGbN26tTBm1apVhTFbtmxJymnmzJmFMYsWLSqMWbNmTWHMvvvum5TTXnvtVRiT8vo+9thjhTHNzc1JORW971LeTwBqx4nnYKfuW4t84AMfKIw544wzatIXANSFt7xFuvDCsrMAGh4jLwAAAABgrC64oOwMgGmhsHhh+xzbD9u+teK5+bYvs31X/rt4GAAAYEpjfwAAAICypIy8OFfSK4Y99yFJl0fEgZIuzx8DABrbuWJ/AADAyE46qewMgIZVWLyIiKslDT8h/nWSzsv/Pk/S62ucFwCgzrA/AACgwNe/XnYGQMMa6zUv9oiIByUp/7177VICAEwh7A8AAAAw4Sb8gp22T7S90vbK7u7uie4OAFCHKvcFZecCAEDNHXFE2RkADW+sxYuHbO8pSfnvh0cKjIizImJpRCzt7OwcY3cAgDqVtD+o3BdManYAAEyGyy8vOwOg4Y21ePFjSW/P/367pB/VJh0AwBTD/gAAAAATLuVWqRdIuk7SU2yvsf23kj4j6aW275L00vwxAKCBsT8AACDBZz9bdgZAQ2opCoiIY0eY9JJd7cy2WlpG77KpqXgwyMDAQHJ/RVL6a21trUlfkpRy6syMGTMKY+bNm1cYs2DBgqSc5s+fXxiTsp4GBwcLYx599NGknFavXl0Ys2XLlsKYjRs31qSvVOvXry+M2W+//WrW3+OPP14Y09XVVRjT3t5eGNPb25uUU39//6jTIyKpHdSfWu4PMHlS33Mpn+EpMWeccUZSfwDQsD74Qemf/qnsLICGM+EX7AQAAAAAABgPihcAAAAAMF5PfWrZGQANjeIFAAAAAIzXHXeUnQHQ0CheAAAAAACAukbxAgAAAABq6VvfKjsDoOFQvAAAAACAWvqbvyk7A6DhULwAAAAAAAB1jeIFAAAAANTCokVlZwA0rJbJ7CwiNDAwMGpMU1NxPcV2Un8tLcWL19/fXxgTETXpS5JmzpxZk7ZS2tlzzz2Tcpo3b15hzIYNGwpjUl67np6epJwGBwcLY5qbmwtjurq6CmP+4i/+IimnzZs3F8bMmDGjMCbl9d22bVtSTinbZmdnZ2FMyvpO3caLtoPU9y+A2kh9z6XEpXzOX3vttYUxZ5xxRlJOq1evLoxZsWJFUlsAMCkeeEDifx1gQjDyAgAAAAAA1DWKFwAAAABQa7/6VdkZAA2F4gUAAAAA1NpLX1p2BkBDoXgBAAAAAADqGsULAAAAAFXZPsf2w7ZvHSXmcNu/t32b7asmM7+6tHx52RkADYniBQAAAICRnCvpFSNNtD1X0lclvTYiniHpzZOUV/267rqyMwAaEsULAAAAAFVFxNWSHhsl5K2SLomI/83jH56UxABMOxQvAAAAAIzVkyXNs32l7Rttv63shOrKkUeWnQHQMFrKTmC4wcHBwpi+vr6ktmwXxrS0FK+C/v7+wpi2traa5ZSyDnp6egpjNm/enJRTV1dXYUx3d3dhzMMPFxfaH3/88aScHnnkkcKYlO1gjz32KIxZuHBhUk69vb2FMR0dHYUx7e3tNWlHStt+t2/fXpN2UrbdlLZS2wFQbHnCedXLli1LaisiCmNS9k8p/Z1//vlJOa1du7Yw5phjjimMWbFiRVJ/AMakRdJfSnqJpE5J19leERF3Dg+0faKkEyVpn332mdQkS3P55WVnADQMRl4AAAAAGKs1ki6NiK0R8YikqyUdXC0wIs6KiKURsTT14BEADKF4AQAAAGCsfiTpUNsttmdIWibpjpJzKt+nP112BkDDoXgBAAAAoCrbF0i6TtJTbK+x/be2T7J9kiRFxB2SLpV0s6QbJH0zIka8req08aEPlZ0B0HDq7poXAAAAAOpDRBybEPM5SZ+bhHQATGOMvAAAAACAiTJ7dtkZAA2B4gUAAAAATJTEOwACGB3FCwAAAACotSa+agG1xDsKAAAAAGpt1aqyMwAayqResLOpqUnt7e2jxvT09BS209nZmdRfRBTGDAwMFMY0JVRNt2zZkpRTX19fYczg4GBhTNF6lCTbSTlt2LChMGb9+vWFMXfffXdhzJ133pmU0+OPP14Yk7KeWlqKN/Hdd989KafW1taaxKRs4yntSNKMGTMKY1LeBynrKWXblYrfUymvG4A0ixcvrkmMlLbPSNkf1qodKS33a665pjDm+uuvL4w5+uijk3Jas2ZNUhwA6ElPKjsDoKEw8gIAAAAAJtLpp5edATDlUbwAAAAAgIn0gQ+UnQEw5VG8AAAAAAAAdY3iBQAAAABMhJNPLjsDoGFQvAAAAACAifCVr5SdAdAwKF4AAAAAwES7996yMwCmNIoXAAAAADDRDjyw7AyAKY3iBQAAAABMtIGBsjMApjSKFwAAAAAwUWbOLDsDoCG0TGZng4OD2rZt26gxbW1the3YTuqvr68vKaci/f39hTFFyzWkpaV4lW/evLkwZu3atYUxc+bMScopZfm2bNlSGLNmzZrCmJS8JWnjxo2FMTMTdgRdXV2FMXvssUdSTq2trYUxKetp9uzZhTEp266Utj2lxKTo7OxMihsoOKqQ+v4FUBsp+zlJamoqPp6R0lat2qllW8uWLSuMueCCC5JyOvTQQ5PiAODPtmyR+P8HGLfC/wpsn2P7Ydu3Vjx3qu0HbP8+/3nlxKYJACgT+wIAAACUKeW0kXMlvaLK82dExCH5z89rmxYAoM6cK/YFAACMz1/9VdkZAFNWYfEiIq6W9Ngk5AIAqFPsCwAAqIHvfrfsDIApazwX7PwH2zfnQ4nn1SwjAMBUwr4AAAAAE26sxYuvSdpf0iGSHpR0+kiBtk+0vdL2yu7u7jF2BwCoQ2PaF0xWcgAA1I3LLy87A2DKG1PxIiIeioiBiBiU9A1Jzxsl9qyIWBoRS1PvVgAAqH9j3RdMXoYAANSJI44oOwNgyhtT8cL2nhUP3yDp1pFiAQCNiX0BAABjcO+9ZWcATEktRQG2L5B0uKQFttdI+pikw20fIikk3SfpnROYIwCgZOwLAACokf32kyLKzgKYcgqLFxFxbJWnzx5rh83NzaMn1FKYknp7e2vSlyQ1NRUPPknpb3BwMCmndevWFcZs2bKlMGbDhg2FMR0dHUk5pazzlJz6+/uT+ksxc+bMwpj58+cXxsydO7cmfUlSW1tbYUxra2thTE9PT036ktLWeSTsHGsVIxVvT7aT2kF9qfW+ALWxevXqwpi1a9cmtbV48eLCmGOPrbYZ7Oziiy9O6i/F8uXLC2O+973vFcakLNsLXvCCpJxS+jvmmGOS2gIAAOnGc7cRAAAAAECKk08uOwNgSqN4AQAAAAAT7StfKTsDYEqjeAEAAAAAAOoaxQsAAAAAmEyzZ5edATDlULwAAAAAgMm0eXPZGQBTDsULAAAAAJgMe+5ZdgbAlEXxAgAAAAAmQ+LtqwE8EcULAAAAAABQ11omszPbamoaf72kFm0MaWtrq0lMR0dHUn8puT/wwAOFMWvWrCmMaW5uTsqppaV4M2htbU1qq8jcuXNrFrf77rsXxqS8LintSFJ/f39hTMq67OrqKozZsGFDUk61ei+ktBMRSW319vaOOn1wcDCpHQDFVqxYURhz3XXXJbW19957F8akfg7USq2WL2XZUj+bJnsdAGhgV14pHX542VkAUwYjLwAAAABgsr34xWVnAEwpFC8AAAAAAEBdo3gBAAAAAJPl3nvLzgCYkiheAAAAAMBkWbKk7AyAKYniBQAAAACUYdassjMApgyKFwAAAABQhq1by84AmDIoXgAAAADAZPrIR8rOAJhyKF4AAAAAwGT6xCfKzgCYclomu8OmptHrJb29vYVttLe3J/W1NWEYVmdnZ2FMS0vxatpzzz2TckrJfePGjYUxRetRktavX5+UU8ryNTc3F8bMmDGjMGbhwoVJOc1KOP8vJWbu3LmFMdu3b0/KqaOjozAmIgpjuru7C2NSXpPU/lKkbE99fX1JbbW1tY27LwC1Y7tmcaltTaZa5Z362VSP6wDAFNfaKiX+PwpMZ3yLAAAAAICy9PeXnQEwJVC8AAAAAIDJ9qIXlZ0BMKVQvAAAAACAyXbVVWVnAEwpFC8AAAAAoEz33Vd2BkDdo3gBAAAAAGV60pPKzgCoexQvAAAAAABAXaN4AQAAAABluOKKsjMApgyKFwAAAABQhsMP3/E3hQxgVBQvAAAAAKBsRxxRdgZAXWuZzM4iQr29vaPGdHZ2FrZT1MaQ9vb2wpjt27cXxsybN68wZsGCBUk5LVy4sDBm5syZhTG33357Ycxjjz2WlNPg4GBhTH9/f2FMW1tbYUxfX19STimvXUpMc3NzYUzKNidJTU3Ftb6U/lLaaWmp3Vsz5f2SEmM7qb+BgYFRp0dEUjsAaiP1PZcS9973vrcw5uKLL07qL8Xy5csLY5YtW1YYk7JsKfvC1LYAAEDtMfICAAAAAMpSeZtUCqTAiCheAAAAAEBZ9t13x98Jo8KB6YriBQAAAADUg4RTzIHpiuIFAAAAANSDTZvKzgCoWxQvAAAAAKBML35x2RkAdY/iBQAAAACU6de/LjsDoO5RvAAAAABQle1zbD9s+9aCuOfaHrD9psnKrWHdf3/ZGQB1ieIFAAAAgJGcK+kVowXYbpb0b5J+MRkJNbwlS8rOAKhLLZPdYVPT+OslLS1paQ8ODhbGbNu2rTCmv7+/MMZ2Uk4LFiwojJk/f35hzKJFiwpj1q1bl5RTyjp44IEHCmM2btxYGJP6+nd1dRXGNDc3F8bsvvvuSf2l6O3tLYzp6ekpjElZB52dnTXLKWX7TclpYGAgKaeiuOD+5cCkSt0/pcS94AUvKIxJ2femfg6k5JTSVko7qfun66+/PikOaBQRcbXtJQVh75b0fUnPnfCEAExbjLwAAAAAMCa295L0BklnJsSeaHul7ZXr16+f+OSmmi99qewMgLpWWLywvdj2FbbvsH2b7ffmz8+3fZntu/Lf8yY+XQBAGdgXAABG8AVJH4yIwiGaEXFWRCyNiKULFy6chNSmmHe/u+wMgLqWMvKiX9L7I+JpkpZLepftp0v6kKTLI+JASZfnjwEAjYl9AQCgmqWSLrR9n6Q3Sfqq7deXm1ID6OgoOwOg7hQWLyLiwYi4Kf97s6Q7JO0l6XWSzsvDzpPEhxQANCj2BQCAaiLiSRGxJCKWSLpY0skR/397dx9k2VnXCfz7m7ckhoQECCFkEl+AwvclMCRZ2XUDrEhw1+yuYZOogFthcdnElyotQYs1KNaWojFbikAFTQluIGZDFEqzIqVxVTQJk4i8OCtERDJJNpOYBGaKJPP27B/3jrdn0j33zEz3Ped2fz5VXX3uuU/f+z23e/r0/c45z2m/23Os+ddhXjNYa45ozovxZD3nJLk9yemttfuT0R+1SRadGXHhuW2PPfbYsaUFoHfHui+YVU4Ajl1VfSDJXyZ5flVtr6rLq+q/VNV/6TvbqnTOOX0ngMHqfLWRqnpKRrMI/2hr7ctdZy9vrV2b5NokOf30011mAGCOLce+oKrsCwDmRGvtsiMY+wMrGGVtuOuupOO+FdaaTkdeVNXGjP5Yvb61dvN49QNVdcb4/jOS7FiZiAAMgX0BAMzQr/5q3wlgULpcbaSS/EaSba21X15w14eTvG68/LokH1r+eAAMgX0BAMzYD/9w3wlgULqcNvKSJK9J8qmq+sR43U8l+fkkN1bV5Um+mOTVXZ5w2iHGrU0/mnj9+vVdnip79+5dlsd6+OGHp47Zvn17p0xdnu9Zz3rW1DHPec5zpo455ZRTOmW67777po7p8n056aSTpo7ZtWtXp0wnn3zy1DGnn3761DGnnjr9qo1d52J5osPESevWTT+YqcuYnTt3dsrU5fvS5d/BcccdN3VM19MDNm3adNj7u2w/g7Ss+wJm55prruk07uKLL546Zv/+/VPHdPk33uVxlvOxujzO1Vdf3SlT19cT4Kht2JB0+PsN1pqp5UVr7c+TLPWu5eXLGweAIbIvAIAZ2bPHvBewCP8FCgAAMETPfnbfCWAwlBcAAABDdP/9fSeAwVBeAAAADMlb39p3Ahgc5QUAAMCQXHXVZPn66/vLAQOivAAAABiq7//+vhPAICgvAAAAgEFTXgAAAAzNF77QdwIYlA2zfLJ169bl+OOPP+yYPXv2TH2cLmOSZP/+/VPHbNy4ceqYRx99tNPzdfHEE09MHfPYY49NHXP22WdPHbNp06ZOmZ71rGdNHXPyySdPHbN79+6pY7p8T5Ju35cTTjhh6pidO3dOHdP15+nxxx+fOmbv3r1Tx3zlK19ZlsdJRv+mlkOX713X59q3b99h72+tdXocYHncdtttncbddNNNU+6WduUAABzpSURBVMecf/75U8ecddZZU8d0/X1SVVPHdNm+m2++eeqYa665plMmgJn56q/uOwEMiiMvAAAAhuykk/pOAL1TXgAAAAzZrl19J4DeKS8AAACG6LLL+k4Ag6G8AAAAGKL3v7/vBDAYygsAAICh+/Ef7zsB9Ep5AQAAMHRXX913AuiV8gIAAGConvKUvhPAICgvAAAAhmrnzr4TwCAoLwAAAOaBeS9YwzbM8slaa9m7d+8xP86mTZs6jdu9e/fUMevWTe9vTjzxxKlj9uzZ0ynTI488siyP1eV1/Kqv+qpOmU466aRleayqmjqmtdYp064O17LuMubBBx+cOmZnxza7y/bt27dv6pguP5ddnitJ9u/fvyyP1SV31+/dcccdd8x5gNm75JJLpo45//zzp4752Mc+NnXM1R3P277jjjumjrntttumjtm+fXun5wMYvKuvTn7pl/pOAb1w5AUAAMCQvf71fSeA3ikvAAAAhuw975ks//mf95cDeqS8AAAAmBf/8l/2nQB6obwAAAAABk15AQAAMHR/9md9J4BeKS8AAACG7l/8i8nyi1/cXw7oifICAABgnmzd2ncCmDnlBQAAwDz41m/tOwH0ZsOsn3D//v3HdP+R2Lhx49Qxjz322NQxJ5xwwtQxjz/+eKdMXR5r7969U8fcd999U8e01pYtU5fvy6ZNm6aO2b17d6dMX/rSl6aO2bVr19QxXXJXVadMXV7PLj9zXcbs27evU6Yu2bt8X5ZT19cTmD+33Xbb1DHr16+fQRKANeqv/zrxtxZrlCMvAAAA5s1JJ/WdAGZKeQEAADBvOhyFDKuJ8gIAAGBe3HBD3wmgF8oLAACAeXHJJZPle+7pLwfMmPICAABgHp19dt8JYGaUFwAAAPPk+OP7TgAzp7wAAACYJ4891ncCmDnlBQAAwLw69dS+E8BMbJjlk7XWsnfv3sOO2bBheqRpj7Hw+aY54YQTpo55bBmbzS6P9cQTT0wd89BDD00ds25dt26qy7iqWpbH6bJtXR9ruSznz9Pu3bunjjnuuOOmjuny76Crffv2TR2zZ8+eZXu+TZs2LdtjAQAwxaOP9p0AZsKRFwAAAPPmYx/rOwHM1NTyoqrOqqpbq2pbVX2mqn5kvP6tVXVvVX1i/PGqlY8LQB/sCwBgYL7t2ybL55/fXw6YkS7Hpu9N8mOttbuq6qQkd1bVR8f3XdNa+6WViwfAQNgXAMBQ3X573wlgxU098qK1dn9r7a7x8s4k25KcudLBABgO+wIAGKAbbug7AczMEc15UVVfk+ScJAeqvSur6pNVdV1VLTrNbVW9oaq2VtXW5Zz4EoB+HOu+YEYxAWD1u+SSyfJrXtNfDpiBzuVFVT0lyQeT/Ghr7ctJ3pXkOUlekOT+JFcv9nWttWtba1taa1u6XNkDgOFajn3BzMICwFryP/9n3wlgRXUqL6pqY0Z/rF7fWrs5SVprD7TW9rXW9id5T5JzVy4mAH2zLwCAAfrO7+w7AcxEl6uNVJLfSLKttfbLC9afsWDYv0/y6eWPB8AQ2BcAwED9wR9Mlv/iL/rLASusy9VGXpLkNUk+VVWfGK/7qSSXVdULkrQkX0jyg9MeqKqyadOmw45Zt276wSCjv6Gna60ty5jjjz++0/N1sXv37qljurwGXR6na+4uj7V+/fqpY/bt2zd1TJdtW04bNkz/Ed+4ceOyPd9yvU579+5dtufbs2fP1DFd/h0s17+7Ls/FIC3bvgAAWCEveUniby1Wqanv7Fprf55ksXcttyx/HACGyL4AAIA+zfa/wQEAAFhejrZgDVBeAAAArBYdT/WFeaO8AAAAAAZNeQEAADDvnDrCKqe8AAAAWE2mXOER5pHyAgAAYDXZs6fvBLDslBcAAACrwcJTR7Zv7y8HrADlBQAAsKiquq6qdlTVp5e4//uq6pPjj7+oqn8264ws4ayz+k4Ay2rDLJ+stZa9e/ce8+OsW9etc9mwYfrm7d69+1jjJBltWxebOpx/1iV3F10zVYfLKa1fv35ZxnTV5bH27ds3dUyX7+8JJ5zQKdNy/OwmyZ4Oh/Ft3Lix02N1+R53GdPlNejyend9PgBgbvxmkncked8S9/99kn/VWnukqi5Mcm2S82aUjcVcfHFy0019p4Bl58gLAABgUa21P03y8GHu/4vW2iPjm7cl2TyTYCztf/2vyfLb395fDlhmygsAAGA5XJ7kf/cdggXe9Ka+E8CyUV4AAADHpKpemlF5seS75ap6Q1VtraqtDz744OzCrUUXX9x3Alh2ygsAAOCoVdW3Jvn1JBe11v5xqXGttWtba1taa1tOO+202QVcixaeOnLbbf3lgGWkvAAAAI5KVZ2d5OYkr2mtfbbvPCzin//zvhPAspjp1UYAAID5UVUfSHJBkmdU1fYkVyXZmCSttXcn+ekkT0/yzvEV7Pa21rb0k5aDfP3XJ//3//adApaN8gIAAFhUa+2yKfe/PsnrZxSHI7FtWzIqlJLv+q7k93+/3zxwjJw2AgAAsJrdckvfCeCYzfTIix07djx0zTXX/MMhq5+R5KFZ5lgmcs+W3LO1krm/eoUel/nxUBL7gn7JPVtyP5l9AczCPfckZ501Wr733uTMM/vNA8dgpuVFa+1J0wpX1dZ5PC9O7tmSe7bmNTfzwb6gf3LPltxAbzZvPni5tf6ywDFy2ggAAMBq9d3f3XcCWBbKCwAAgNXqQx+aLJ9wQn854BgNoby4tu8AR0nu2ZJ7tuY1N/NrXn/m5J4tuWdrXnMDS3n88b4TwFGr5rwnAABghrZs2dK2bt3ad4y15cBlU2+/PTn33H6zQJKquvNI5lYawpEXAAAAzMJ55/WdAI6K8gIAAGC1e8tb+k4Ax6S38qKqXllVf1tVd1fVm/vKcaSq6gtV9amq+kRVDfpYt6q6rqp2VNWnF6x7WlV9tKo+N/58ap8ZF7NE7rdW1b3j1/0TVfWqPjMeqqrOqqpbq2pbVX2mqn5kvH7Qr/dhcg/69Wb1sC9YefYFs2NfAAza2942Wb7ssv5ywFHqZc6Lqlqf5LNJviPJ9iQfT3JZa+1vZh7mCFXVF5Jsaa091HeWaarq25PsSvK+1to3j9e9PcnDrbWfH79ROLW19qY+cx5qidxvTbKrtfZLfWZbSlWdkeSM1tpdVXVSkjuT/LskP5ABv96Hyf0fM+DXm9XBvmA27Atmx74AujPnRU8OzHuRJOY+pGfzMufFuUnubq19vrW2O8kNSS7qKcuq1Vr70yQPH7L6oiTvHS+/N6M/TgZlidyD1lq7v7V213h5Z5JtSc7MwF/vw+SGWbAvmAH7gtmxLwAGz6kjzLG+yoszk9yz4Pb2zM9OsiX5w6q6s6re0HeYo3B6a+3+ZPTHSpJn9pznSFxZVZ8cH0o8qENuF6qqr0lyTpLbM0ev9yG5kzl5vZlr9gX9mZvfTYuYi99N9gXAIC08deTbv72/HHAU+iovapF183Lc0ktaay9McmGSK8aHtbLy3pXkOUlekOT+JFf3G2dxVfWUJB9M8qOttS/3naerRXLPxevN3LMv4EjNxe8m+wJgLvzZn/WdAI5IX+XF9iRnLbi9Ocl9PWU5Iq21+8afdyT5nYwOe54nD4zPbT1wjuuOnvN00lp7oLW2r7W2P8l7MsDXvao2ZvRH3/WttZvHqwf/ei+Wex5eb1YF+4L+DP5302Lm4XeTfQEweLffPn0MDFBf5cXHkzyvqr62qjYluTTJh3vK0llVnTieyCpVdWKSVyT59OG/anA+nOR14+XXJflQj1k6O/BH39i/z8Be96qqJL+RZFtr7ZcX3DXo13up3EN/vVk17Av6M+jfTUsZ+u8m+wJgLpy7oIesxQ6ChGHa0MeTttb2VtWVST6SZH2S61prn+kjyxE6PcnvjPbx2ZDk/a21P+g30tKq6gNJLkjyjKranuSqJD+f5MaqujzJF5O8ur+Ei1si9wVV9YKMDin/QpIf7C3g4l6S5DVJPlVVnxiv+6kM//VeKvdlA3+9WQXsC2bDvmCm7AuA+XDeeY7AYO70cqlUAABg7XKp1AE4cNTF5s3JPfccfiysgHm5VCoAAAB927697wTQifICAABgrbn33snyfXMxXzZrnPICAABgrXn2syfLZ57ZXw7oSHkBAACwFr3qVX0ngM6UFwAAAGvR7//+ZPn5z+8vB3SgvAAAAFjrPvvZvhPAYSkvAAAA1qrWJssm7mTAlBcAAACYuJNBU14AAACsZT/4g30ngKmUFwAAAGvZu989Wa7qLwcchvICAACAia1b+04AT6K8AAAAWOsWTtz54hf3lwOWoLwAAAAgef3r+04AS1JeAAAAkLznPZNlc18wMMoLAAAARk48cbJ8//395YBDKC8AAAAY2bVrsvzsZ/eXAw6hvAAAAGDinHP6TgBPorwAAABg4q67JsvmvmAglBcAAAAc7JnP7DsBHER5AQAAwMEeeGCy7OgLBkB5AQAAwJMtvPII9Ex5AQAAwJMtvPLIySf3lwOivAAAAGAp69ePPu/c2W8O1jzlBQAAAIvbu3eybO4LeqS8AAAAAAZNeQEAAMDSWpssO/qCnigvAAAAOLyzz+47AWuc8gIAAIDD+4d/mCyfe25/OVizlBcAAAB09/GP952ANUh5AQAAwHTmvqBHygsAAABg0JQXAAAAdOPoC3qivAAAAKC7Cy+cLP+//9dfDtYU5QUAAADd3XLLZPmMM/rLwZqivAAAAODIfO/3TpYvuaS/HKwZygsAAACOzPXXT5ZvvLG/HKwZygsAAACOnMk7mSHlBQAAAEdn8+bJsgKDFaS8AAAA4Ojcc0/y1Kf2nYI1QHkBAAAsqqquq6odVfXpJe6vqvqVqrq7qj5ZVS+cdUYG4NFHJ8uOvmCFKC8AAICl/GaSVx7m/guTPG/88YYk75pBJobo6U+fLL/xjf3lYNVSXgAAAItqrf1pkocPM+SiJO9rI7clOaWqzphNOgbloYcmy+9+d385WLWUFwAAwNE6M8k9C25vH69jLfrDP5wsO32EZaa8AAAAjtZi71DbIutSVW+oqq1VtfXBBx9c4Vj04ju+I3nRiya3Tz21vyysOsoLAADgaG1PctaC25uT3LfYwNbata21La21LaeddtpMwtGDrVsny48+mtx8c39ZWFWUFwAAwNH6cJLXjq86cn6SL7XW7u87FD1rCw6++Z7vSXbs6C8Lq8aGvgMAAADDVFUfSHJBkmdU1fYkVyXZmCSttXcnuSXJq5LcneQrSf5TP0kZnNYm816cfvrBhQYcBeUFAACwqNbaZVPub0mumFEc5s0f/VHy8pePlr/v+5Lrr+83D3PNaSMAAAAsv5e9bLL8/vcn/+E/9JeFuae8AAAAYGUsPF3kd34ncaUZjpLyAgAAgJWzsMB45jP7y8FcU14AAACwsn76pyfLBybyhCOgvAAAAGBl/czPJBsWXC/iWc/qLwtzSXkBAADAytuzZ7L8wAPJ//k//WVh7igvAAAAmI2F819ccEFvMZg/ygsAAABmZ8eOybL5L+hIeQEAAMDsnHZa8qu/OrmtwKAD5QUAAACzdeWVySmnTG4rMJhCeQEAAMDsPfJI8uM/Prl98cX9ZWHwlBcAAAD04xd/cbL8wQ/2l4PBU14AAADQn4VXIFm/vr8cDJryAgAAgH6tG7813b8/OfPMfrMwSMoLAAAA+rVv32T5vvv6y8FgKS8AAADo38LTR1x9hEMoLwAAABiGV7xisqzAYAHlBQAAAMPwkY8cfHvjxn5yMDjKCwAAAIZj4ekje/f2l4NBUV4AAAAwLIfOf7HOW9e1zk8AAAAAw3PLLZPl1pLTTusvC71TXgAAADA8F16YvPGNk9sPPZQ8//n95aFXygsAAACG6Z3vPPgUks9+Nlm/vr889EZ5AQAAwLAtLDD273cZ1TVIeQEAAMDwLSwwkuRFL+onB71QXgAAADAfFhYYd92V3HFHf1mYKeUFAAAA82NhgXHeef3lYKaUFwAAAMyXf/zHybL5L9YE5QUAAADz5WlPS666anK7Knnkkf7ysOKUFwAAAMyft741ee5zJ7ef9rTkzjt7i8PKUl4AAAAwnz73ueR3f3dye8uW0VEYTiVZdZQXAAAAzK+LLkpOPPHJ6xUYq4ryAgAAgPm2a9foKiRbtx68XoGxaigvAAAAWB1e9KKDL6WajAqMyy/vJw/LRnkBAADA6tJacumlk9vXXecojDmnvAAAAGD1+cAHFj8K47/+137ycEyUFwAAAKxerSUbNkxuv+tdyZ/8SW9xODrKCwAAAFa3PXsOPgrjpS9N3vzm/vJwxJQXAAAArA0LC4xf+AXzYMwR5QUAAABrx2LzYBz4YLCUFwAAAKwthxYYBygwBkt5AQAAwNrT2ujj0UcPXl+V/Nt/208mlqS8AAAAYO166lNHJcY3f/Nk3e/93qjEeOc7+8vFQZQXAAAA8KlPJffcc/C6K64YlRjveEc/mfgnygsAAABIks2bF58P44d+aFRibN06+0wkUV4AAADAwQ7Mh/GTP3nw+he/eFRinH56P7nWMOUFAAAALOa///dRiXHiiQev37Hj4EusViV3391PxjVCeQEAAACHs2vX0pdXPeB5zxuVGG9/+2wyrTHKCwAAAOjiwOkkBz4+97knj3nTm558VAbHTHkBAAAAR+O5zz24zFhKVXLJJbPLtQopLwAAAGA5HCgxvv7rn3zfjTeOSoxNm2afaxVQXgAAAMBy2rZt6SMy9uyZnE6iyOhMeQEAAAAraanTSg4UGV/3dbPPNGeUFwAAADALC4/GOO64yfq///uDJ/i8/vr+Mg6U8gIAAABm7fHHk09+cvH7vv/7J0XGT/zEbHMNlPICAAAA+vAt3zI6CuMrX0nOPnvxMb/4i5Mi44wzkne9KznllOTWW2ebtWfVDnc5FwAAgGW2ZcuWtnXr1r5jwLBt3Jjs3dtt7BNPzN3kn1V1Z2ttS9fxjrwAAACWVFWvrKq/raq7q+rNi9x/dlXdWlV/VVWfrKpX9ZETVp09eybzYzz72Ycfe9xxB8+ZsfDj0ktnk3eFKS8AAIBFVdX6JL+W5MIk35jksqr6xkOGvSXJja21c5JcmuSds00Ja8C99x482eeBjwsumP61v/3byebNKx5xpSkvAACApZyb5O7W2udba7uT3JDkokPGtCQnj5efmuS+GeaDte3WW0clxo03jm7/8A8nl1ySfNu3Jbt3T8bde+/kSIznPjfZtaufvMdgQ98BAACAwTozyT0Lbm9Pct4hY96a5A+r6oeSnJjkX88mGvBPXv3qUYlxqNaSX/u15MorJ+v+7u+Sk06a3K5K3vKW5Gd/duVzHgNHXgAAAEupRdYd+g7psiS/2VrbnORVSX6rqp70PqOq3lBVW6tq64MPPrgCUYFFXXHFqMQ466xk/fon399a8ra3jUqM9euTO+6YfcYOlBcAAMBStic5a8HtzXnyaSGXJ7kxSVprf5nk+CTPOPSBWmvXtta2tNa2nHbaaSsUF1jSF784unrJgfky3vGO5IUvHJUWB+zfn5x33mjdyScnX/rSaN0AKC8AAIClfDzJ86rqa6tqU0YTcn74kDFfTPLyJKmqb8iovHBoBQzdFVckd945KidaS77new6+f+fO5JRTRkdj/NzPTUqMRx8dLS+cU2MGlBcAAMCiWmt7k1yZ5CNJtmV0VZHPVNXPVtV3j4f9WJL/XFV/neQDSX6gtcVOvgcG7aabJkdl7Nx58H3/7b+NSoyq5NRTR8sLL8/6W7+VbNu2ovHK7xUAAGCWtmzZ0rZu3dp3DGCa1pLPf350hZIu/viPk5e+tNPQqrqztbalaxRHXgAAAABPVpU85zmjEmPfvsmRGQc+vvzl5L3vnYz/pm9asSjKCwAAAODw1i1SH5x0UvLa107KjGc+c+WefsUeGQAAAGAZKC8AAACAQVNeAAAAAIOmvAAAAAAGTXkBAAAADJryAgAAABg05QUAAAAwaMoLAAAAYNCUFwAAAMCgKS8AAACAQVNeAAAAAIOmvAAAAAAGTXkBAAAADJryAgAAABg05QUAAAAwaMoLAAAAYNCUFwAAAMCgKS8AAACAQVNeAAAAAIOmvAAAAAAGTXkBAAAADJryAgAAABg05QUAAAAwaMoLAAAAYNCqtdZ3BgAAYA2pqgeT/MMRfMkzkjy0QnGGwjauHmthO5djG7+6tXZa18HKCwAAYNCqamtrbUvfOVaSbVw91sJ29rGNThsBAAAABk15AQAAAAya8gIAABi6a/sOMAO2cfVYC9s582005wUAAAAwaI68AAAAAAZNeQEAAAxSVb2yqv62qu6uqjf3naeLqrquqnZU1acXrHtaVX20qj43/nzqeH1V1a+Mt++TVfXCBV/zuvH4z1XV6xasf1FVfWr8Nb9SVTXj7Turqm6tqm1V9Zmq+pHVto3jDMdX1R1V9dfj7fyZ8fqvrarbx5l/u6o2jdcfN7599/j+r1nwWD85Xv+3VfWdC9YP4ue7qtZX1V9V1e+Nbw9yG5UXAADA4FTV+iS/luTCJN+Y5LKq+sZ+U3Xym0leeci6Nyf5o9ba85L80fh2Mtq2540/3pDkXcmoCEhyVZLzkpyb5KoDZcB4zBsWfN2hz7XS9ib5sdbaNyQ5P8kV4+/LatrGJHkiyctaa/8syQuSvLKqzk/yC0muGW/nI0kuH4+/PMkjrbXnJrlmPC7j1+bSJN+U0Xa8c1wWDOnn+0eSbFtwe5DbqLwAAACG6Nwkd7fWPt9a253khiQX9ZxpqtbanyZ5+JDVFyV573j5vUn+3YL172sjtyU5parOSPKdST7aWnu4tfZIko9m9Ob5jCQnt9b+so0mL3zfgseaidba/a21u8bLOzN603tmVtE2Jsk4767xzY3jj5bkZUluGq8/dDsPbP9NSV4+PmLkoiQ3tNaeaK39fZK7M/rZHsTPd1VtTvJdSX59fLsy0G1UXgAAAEN0ZpJ7FtzePl43j05vrd2fjN78J3nmeP1S23i49dsXWd+L8WkD5yS5PatwG8dHD3wiyY6MypW/S/Joa23vItn+aXvG938pydNz5Ns/a/8jyU8k2T++/fQMdBuVFwAAwBAtNs/BartU4lLbeKTrZ66qnpLkg0l+tLX25cMNXWTdXGxja21fa+0FSTZndBTBNyw2bPx57razqv5Nkh2ttTsXrl5k6CC2UXkBAAAM0fYkZy24vTnJfT1lOVYPjE+HyPjzjvH6pbbxcOs3L7J+pqpqY0bFxfWttZvHq1fVNi7UWns0yZ9kNMfHKVW1YXzXwmz/tD3j+5+a0elDR7r9s/SSJN9dVV/I6JSOl2V0JMYgt1F5AQAADNHHkzxvfOWDTRlNCPjhnjMdrQ8nOXA1jdcl+dCC9a8dX5Hj/CRfGp9y8ZEkr6iqU8eTWL4iyUfG9+2sqvPHcw28dsFjzcT4eX8jybbW2i8vuGvVbGOSVNVpVXXKePmEJP86o/k9bk1y8XjYodt5YPsvTvLH4zk7Ppzk0vGVOr42owlI78gAfr5baz/ZWtvcWvua8fP/cWvt+zLQbdwwfQgAAMBstdb2VtWVGb3JXZ/kutbaZ3qONVVVfSDJBUmeUVXbM7qixs8nubGqLk/yxSSvHg+/JcmrMprg8CtJ/lOStNYerqq3ZfTmL0l+trV2YBLQN2Z0RZMTkvzv8ccsvSTJa5J8ajwfRJL8VFbXNibJGUneO75ixrokN7bWfq+q/ibJDVX1c0n+KqMiJ+PPv1VVd2d0NMKlSdJa+0xV3ZjkbzK6UssVrbV9STLgn+83ZYDbWKOiBAAAAGCYnDYCAAAADJryAgAAABg05QUAAAAwaMoLAAAAYNCUFwAAAMCgKS8AAACAQVNeAAAAAIOmvAAAAAAG7f8Duumzdh2tOBMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1332x756 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "losses = []\n",
    "for t in range(0, tmax):     # iterate over the train steps\n",
    "    i = torch.floor(m*torch.rand(1)).long()    # choose a random example\n",
    "    x = trainimages[i].view(-1)                # flatten image\n",
    "    x = x.float() / 255.0                      # normalize max value to one \n",
    "    label = trainlabels[i]  # get true label as a float\n",
    "    probs = net(x)\n",
    "    loss = loss_func(probs.unsqueeze(0), label)\n",
    "    losses.append(loss.item())\n",
    "    loss.backward()    #Call backwards to do backprop\n",
    "    opt.step()\n",
    "    if t % 1000 == 0:    # display every 1000 time steps\n",
    "        visualize(t, np.array(losses), net.fc.weight[label,:].detach(), x)        \n",
    "        time.sleep(0.01)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XsklwIextD9x"
   },
   "source": [
    "Take a look at the output above. The left plot shows the weights learned for the current image class. Note that it resembles a \"canonical\" or \"average\" digit of that class. Why do you think that is? How might this set of weights make sense as a way to detect the respective digit class?\n",
    "\n",
    "Let's also test our network on the test set!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 237
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 599,
     "status": "error",
     "timestamp": 1532468083739,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "UFN6-9TktIuB",
    "outputId": "0750411c-aa24-4a25-9836-042e8df1f751"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:8: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.\n",
      "  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on 10000 test images: 88.5%\n"
     ]
    }
   ],
   "source": [
    "correct = 0.0\n",
    "total = 0.0\n",
    "for i in range(len(testimages)):\n",
    "  x = testimages[i].view(-1)\n",
    "  x = x.float() / 255.0                      # normalize max value to one \n",
    "  label = testlabels[i]  # get true label as a float\n",
    "  probs = net(x)\n",
    "  predicted = F.softmax(probs).max(0)[1]\n",
    "  total += 1\n",
    "  correct += (predicted == label).sum().item()\n",
    "  \n",
    "print(\"Accuracy of the network on \" + str(len(testimages)) + \" test images: \" + str(100 * correct/total) + \"%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "p3S8fUsVuxK7"
   },
   "source": [
    "How did we do? Much better than chance, certainly!\n",
    "\n",
    "Okay, so we've now got a simple neural network for classifying MNIST digits!\n",
    "However, we did it using a lot of built-in code in pytorch. Let's go back and implement some of that for our own satisfaction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9DGADEiqbQfw"
   },
   "source": [
    "**Question** \\\n",
    "Here's a new version of the network we defined above. Modify the forward(self, x) function to fill in the blanks.\n",
    "\n",
    "**HINT**: Use the torch.matmul() function to multiply the inputs by the weights in one pass.\n",
    "\n",
    "(**ADVANCED**) Let's add two more layers to our network! They should look similar to the first, but will make our network much more powerful. The network should look something like:\n",
    "* layer 1: n_inputs -> 32 neurons\n",
    "* layer 2: 32 neurons -> 64 neurons\n",
    "* layer 3: 64 neurons -> n_neurons\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "C6iAuvDOfZoz"
   },
   "outputs": [],
   "source": [
    "#Let's define our own FC layer module\n",
    "class ourLinear(nn.Module):\n",
    "  \n",
    "  def __init__(self, n_inputs, n_outputs):\n",
    "    super(ourLinear, self).__init__()\n",
    "    self.n_inputs = n_inputs\n",
    "    self.n_outputs = n_outputs\n",
    "    \n",
    "    #Define parameters containing our weights and biases\n",
    "    #Parameter objects are Pytorch's way of indicating what variables\n",
    "    #need gradients computed via backpropagation\n",
    "    #Weight matrix needs to be specified with the number of outputs first\n",
    "    self.weight = nn.Parameter(torch.Tensor(self.n_outputs, self.n_inputs))\n",
    "    self.bias = nn.Parameter(torch.Tensor(self.n_outputs))\n",
    "    #also need a variable to hold the input to this layer for use in the backwards pass\n",
    "    self.input = None\n",
    "    \n",
    "    self.reset_parameters()\n",
    "    \n",
    "  #Define the forward pass for this layer\n",
    "  def forward(self, x):\n",
    "    #Save x for use in backwards pass gradient computation\n",
    "    #QUESTION: Why do we need x for the backward pass?\n",
    "    self.input = x.detach() #detach to avoid interfering with autograd\n",
    "    #Fill in here\n",
    "    #Multiply with weights\n",
    "    x = torch.matmul(x, self.weight.t())\n",
    "    #Add bias\n",
    "    x = x + self.bias\n",
    "    return x\n",
    "  \n",
    "  #Compute backwards pass given gradient of loss with respect to output\n",
    "  def backward(self, grad_output):\n",
    "    grad_output = grad_output.detach()\n",
    "    \n",
    "    #compute gradients with respect to each set of weights and the input\n",
    "    self.weight.grad = torch.matmul(self.input, grad_output)\n",
    "    \n",
    "    self.bias.grad = grad_output\n",
    "    \n",
    "    #Input is output of previous layer, so we can continue the backwards pass\n",
    "    grad_input = torch.matmul(self.weight, grad_output)\n",
    "   \n",
    "    return grad_input\n",
    "    \n",
    "  #We need a function to initialize our weights\n",
    "  def reset_parameters(self):\n",
    "    #A common solution for the weight matrix is to use a uniform distribution\n",
    "    stdv = 1.0 / math.sqrt(self.weight.size(1))\n",
    "    self.weight.data.uniform_(-stdv, stdv)\n",
    "    #bias is commonly initialized to zero\n",
    "    self.bias.data.zero_()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "GjIaefOcbpJj"
   },
   "outputs": [],
   "source": [
    "class SimpleNet(nn.Module):\n",
    "\n",
    "  #Initialize some components of our network\n",
    "  def __init__(self, n_inputs, n_neurons = 10):\n",
    "    super(SimpleNet, self).__init__()\n",
    "    \n",
    "    self.n_inputs = n_inputs\n",
    "    self.n_neurons = n_neurons\n",
    "    \n",
    "    #Let's use the layer we implemented above rather than the original linear layer\n",
    "    self.fc = ourLinear(self.n_inputs, self.n_neurons)\n",
    "    \n",
    "    #Advanced implement additional layers as described above\n",
    "    #self.fc2 = ourLinear(???)\n",
    "    \n",
    "    #self.fc3 = ourLinear(???)\n",
    "    \n",
    "    \n",
    "  #Computes the \"forward pass\" of the network.\n",
    "  #In this case, we take in an image x, flatten it into a vector,\n",
    "  #matrix multiply it with the weight matrix, and add the bias vector\n",
    "  #The result is 10 values, which, with the right loss function,\n",
    "  #we can interpret as the probability of the given image being each of 0-9.\n",
    "  def forward(self, x):\n",
    "    #Apply the network\n",
    "    #Fill-in here\n",
    "    x = self.fc(x)\n",
    "    #Advanced fill-in here- uncomment the lines\n",
    "    #Let's apply non-linearities to make our three-layer network more expressive\n",
    "    #x = F.relu(x)\n",
    "    #Apply second layer\n",
    "    #x = ???\n",
    "    #Another non-linearity\n",
    "    #x = F.relu(x)\n",
    "    #Third layer\n",
    "    #x = ???\n",
    "    \n",
    "    return x\n",
    "  \n",
    "  #Let's compute the backwards pass of our network manually as well this time\n",
    "  #Takes as input the gradient of the network with respect to the loss function\n",
    "  def backward(self, grad):\n",
    "  \n",
    "    #output layer backwards pass\n",
    "    grad = self.fc.backward(grad)\n",
    "  \n",
    "    #lower layers\n",
    "\n",
    "    grad = self.fc2.backward(grad)\n",
    "    \n",
    "    grad = self.fc1.backward(grad)\n",
    "    \n",
    "    return grad\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "i1VIXKnSeWCc"
   },
   "source": [
    "Once you've completed the above, run it using the previous training code and see what you get!\n",
    "\n",
    "If you filled in the advanced network, the performance should be much higher.\n",
    "\n",
    "Note that Pytorch is handling the backwards pass derivatives automatically based on the forward pass functions.\n",
    "\n",
    "**Question**\n",
    "Let's fix that and define it ourselves!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "IonoLuHMOZ3j"
   },
   "outputs": [],
   "source": [
    "#Let's initialize our network\n",
    "net = SimpleNet(n, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "3sGRdWadOTN0"
   },
   "outputs": [],
   "source": [
    "opt = torch.optim.SGD(net.parameters(), lr = eta)\n",
    "\n",
    "loss_func = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 2079
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 880,
     "status": "error",
     "timestamp": 1532469047309,
     "user": {
      "displayName": "Riley Simmons-Edler",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106443123906519806041"
     },
     "user_tz": 240
    },
    "id": "cIHzabOHOq0-",
    "outputId": "0e217a8a-f890-478b-87c3-24cadb7f1fec"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCsAAAJpCAYAAACEi7gMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XmcXFWZ8PFfpTt7urN1JyyBYT8K\nYRwRF1Q0UVHAuCGIg4o7buOgqOgoTGDkRQQRcBd9eREVhUERRHGc0QRQHBHRmQByIOxZIEtn3zrp\n7vePe7uprlTn3nS6+1Z1/76fTz7UPfXUuU9VF327njpLqaurC0mSJEmSpFoxqugEJEmSJEmSylms\nkCRJkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTGotOQJIkSZJGqhDC\nu4C3xxhfVXQuRQkhjAYuAs4C9osxLukjrgR8ErgQmBtj/F2VmH8CvhpjLFW0jwL+APwtxviutO05\nwDeBFmAV8MEY4/+m970f+BjQADwGvC/GuCSEcB7wT2l8t3+JMd5Ydq6JwH3A1THG8yryeC1wC3Bg\njPGxEMIPgeeVhUwG7owxvjmNf0ea4wdjjD8o6+d04NNAE3Bbmt+29DX6AvAmoAu4Mcb4LyGEhjSn\ncnsD5wDLgQsqX0qgOe3jq8CLgdHAv5bnMZgsVkiSJEmSinQT8Kcccd8kKR6sqHZnCGFv4Iw+Hvsh\nYCbwt7K2H5MUGn4WQng98EPgyBDC84HzgefFGJeHEC4Bvgi8LX3c1yqLEBWq3hdCmEBSlGnrbosx\nvq0i5pfA1entz5AUCWJFzGzgy8BzgSVp3mcDnwdOBeYAf09SaLgthHByjPEG4FllfTQBfwFuiDEu\nB24ou+8twKkxxg0hhC8CE4FnA/sAd4UQfh9jfHQXz39AWKyQJEmSVDdCCO8DPkHyWWY58A5gXXp7\n/xjjyjTucmAr8C/AuSQfNMcBPwPOijF2hBAWAr8HTgLeCzwMfA84ABhL8g39l9P+XgN8F9gIXAZ8\nCfj79NvxM0hGBYwj+fb+PTHGLVVy/zTwAWAHybfrn6i4f+Yuzv9PwEeAErAeeHeM8b6+2iv6HQtc\nAhwPjAGujDFemN73GHBV+vocB1xT8Zo8AHwLeA7QAXwvxvjF9LFdwGeBdwGHxxg7Kp9zTp+PMf4h\nhPCvGXHfS+Me6+P+K0hGCFxX3pgWMT5K8nN7btp2JDAlxvgzgBjjzSGE74QQng2sBN6afogHuAP4\ntzxPJITw98ArgWqjD84Dvg98uI/HngCMjTH+PG1aQFIkWVAR+grgtzHGJ9PHXQ5cSVKsOIVkRMe2\n9L7vp203VPRxDsnruby8MYQwjuQ1PCFtOg44J8bYCSwJIfwMeANweR8vwYBxzQpJkiRJdSGEMAP4\nGnBcjPFQYDFwboxxLckHunll4W8ErgfeDrwFeAFwcPrvQ2VxzwOOiDHeSfIB7tEY47NIPnB+IYSw\nXzqE/nvAGTHGZwOHknzbTAjhWJIPia+IMR5AUjj5fJXcXwq8j+RD/2zgpcDJFWF9nb8p7fMF6X2X\nAK/tq73KS3c2cDhwJHAEcHIIofy1mhVjDDHGJ6q8JhcCa2KMIc35w+lz6VZKH9urUBFC+HEI4YEq\n/yZUJhdj/EOVnHeyq7j0g35zjPH6KndfTjJSYl1Z22HAIxVxjwDPijE+FmO8vaz9BOCPZcevCiHc\nGUKIIYRL02JQ9zSVb5JME9lRkd+RJB/8L9vFUzyfsqJIjPGPMcauKnFdJCNMum0EDil7Xg+X3fcw\nZSMq0lxaSIp8V1Tp+73A72OM3X3s6lyDymKFJEmSpLoQY1xB8oG0e02DO4CD0ts3AK8HCCEcBWyP\nMd4DvA64Ksa4Lsa4g2R0xEll3f4y/dYY4J9JvoEnxvgI8BRwIMkHwLExxlvTuK/yzGep1wHXxRiX\npcffqui/24nAL2KMG2KM7SRD9X9aEdPX+beSfGh8bwhhZozx32OMF++ivdLrgG/EGLfFGDeRjJ4o\nz/GWivjy1+S1wDfSnNrSnF+9i8eSxr41xvisKv82V4vfEyGE8cClJCNMKu87HpgaY/xRxV0TSF6/\ncltIi1Blj38HSbFiftp0D3AjMBc4hqQI9un0vg8A96dFnvI+SiTvi4/GGLf38RzmkhR+buv7mfb4\nDXBcCGF2CKGR5HmP6+N57fScSN5jP4wxrq/IYRTJaJ8vlTX/J/CREMK4EML+JGthjGMIOA1EkiRJ\nUl1IRzj8W7q+QAPJ4oIPpnf/DPhyOoy9e1QFwBTgk+lUDUg+A60s67at7PbzSUYz7E8y5WFvkqLE\nVGBNWdyysttTgDeFELo/wI8imWpRqaX8cd0f2kMI5TFVzx9j3B5CeCXJlIvzQwj/C3w4xrior/aK\nc08BLgshXJgejwXu6uM1qDxurXjua0jWLujrsUX4V5IP3+UjCrqLGF8ieT9U2sTOH7onkIwc6H78\nh0mm97wixvgUJNNFyuK3hRAuAz4TQvg28HGSAkalM0iKGDstCFrmNKCyoFJVjPH+EMJHSdbc2EYy\njWdtH8+r13MqO9epVbo+BthYMY3o88BXgP8lGcl0K9CeJ889ZbFCkiRJUr04lWT0xMtijKvSHRve\nBsm3/iGEu0imT7yRZJg7JAWCm2OMX8vR/w9Ihul/K8bYFUJYmravByaVxe1VdnsZydz/T2b0vYqk\nYAFACGH6bpyfGONfgFNCCGNIpnV8C3hJX+0V/S4DvhRjrDoKIsPTwHSge4rI9LRtl0IIPwb+ocpd\nRw3C6IrXAy3pB/ju8z9FMs1mFvC7tCg0HhgTQmgFPkUyJag7vkQyveH+9PhdJNM5XlY2aoYQwiHA\nirJRCY3AduBVwAzg/vRck9L4vYD9gaNDCK9LH9MK/CmE8JYYY/d6FK8lGR2SS4zxeyRTkwghvAzo\nLlA9kD6P/0yPD+1+TmlsSHP7S5Vu5wG/rDjPJpKpId2Pv4pk95FBZ7FCkiRJUr2YATyWFiqmk6xF\nUV5EuAF4PzAmxvg/adtNJN98XxVj3BxC+ACwNf2wV63/P6eFgneSDJ+fRLJo5ugQwpwY40LggyTT\nLwBuBm4IIXwxxrgyhPAGknUPvljR983A9SGEc4ENJCNBvpnn/Ol6B/8KvC3G2B5CuBs4vq/2Ks/r\nJuB9IYRbgU7gc8DdMcZfVYmtdAvJyIAPpWsdnAS8OetBMca35uh7QMQYjyg/DiF0xRi7C0pTytrf\nBcyJz2xdujKEcFqM8VrgncDjMcYHQwj7kmz/+cLyQkXq34C2tDAylmTqxy9ijD8k2ZWj+1znpbmd\nV5lvukDonBjjY+nxDJKf/YOVsdWkBZMbSKYSbSIZWXN1evf1wOdCCNeQLLp6Rnp/t+cAD/SxFsZz\n2Hlx0k8DM2KMnwghHE5SlDkrT557ymKFJEmSpHrxI+AfQwiLSRZDPAe4OYRwaYzxEyRrCXyD5INm\nt5+RLCp5T/qN98OUfVNc4VzgxhDCauDb6b/vkCws+SHg6hDCWpJtIzuBrhjjPen0ioXpnP8VJB9g\ne4kx/ndItsD8K8nQ/VvT5/POnOd/FLgvhNBOUuz4CHBvH+2Vvk6yw8h9JB9g7yb/bg7nAN8MITyQ\nPueLYox3ZTwmt5DsgFL+Tf3CEMIOkhEyAP8RY5ydxt5L8hl2X+CHIYQtwOl7kM9pwHdCCOeTjBbp\n3kb0dJIi1a/LpunsSPP4GMnOGw+STNX5JbsxIqIPs4CVZeuEABBC+A/g70hGZhwWQjiHZKvVG0MI\nNwH/Q1I0+1F38S3GeEMI4Xkk77Mu4Nqy3UW6z/XULvKovO9q4MchhEdI1r84PV3QdtCVurqqFVQk\nSZIkSdWEECaSrAMwJca4Lite0u5zNxBJkiRJyhBC+FMIoXtRwlOBv1mokAaP00AkSZIkKdvHga+H\nED5PsuDmOzPiJe0Bp4FIkiRJkqSa4jQQSZIkSZJUUyxWSJIkSZKkmuKaFZIkSZKGzMqVG3LPQ586\ndQJr1mwezHT2SC3nZ279Y279s7u5tbY2lbJiHFkhSZIkqSY1NjYUncIu1XJ+5tY/5tY/g5GbxQpJ\nkiRJklRTLFZIkiRJkqSaYrFCkiRJkiTVFIsVkiRJkiSppliskCRJkiRJNcVihSRJkiRJqikWKyRJ\nkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTLFZIkiRJkqSaYrFCkiRJkiTVFIsVkiRJkiSppliskCRJ\nkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTLFZIkiRJkqSaYrFCkiRJ\nkiTVFIsVkiRJkiSppliskCRJkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiSp5kzdayqUSjQf\ncXDRqUgqgMUKSZIkSTWnobMDgDErVxaciaQiWKyQJEmSJEk1xWKFJEmSJEmqKRYrJEmSJElSTbFY\nIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmSpJpisUKSJEmSJNUUixWSJEmSJKmmWKyQJEmSJEk1xWKF\nJEmSJEmqKRYrJEmSJElSTbFYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmSpJpisUKSJEmSJNUUixWS\nJEmSJKmmWKyQJEmSJEk1xWKFJEmSJEmqKRYrJEmSJElSTWksOgFJkiRJtSuEcDFwLMlnhy/EGH9a\nJeYLwDExxjlDnJ6kYcqRFZIkSZKqCiHMBWbHGI8BjgcurxJzOPCyoc5N0vBmsUKSJElSX24HTklv\nrwUmhhAaKmIuBT430CcuDXSHkuqK00AkSZIkVRVj7AA2pYfvBX6ZtgEQQngXcBvw2JAnJ2lYs1gh\nSZIkaZdCCG8gKVa8uqxtGvBu4FXAvnn7mjp1Ao2NlYMz+lYCWlubcscPNXPrH3Prn5GUm8UKSZIk\nSX0KIbyGZJrH8THGdWV3vQJoBe4AxgIHhxAuizF+fFf9rVmzOdd5W9P/dgGrVm7Y3bSHRGtrEyvN\nbbeZW/8Mp9zyFDYsVkiSJEmqKoQwGbgEeFWMsa38vhjjDcANadwBwNVZhQpJystihSRJkqS+nAq0\nANeHELrbfgssijHeOJgn7iTZDaBrME8iqWZZrJAkSZJUVYzxSuDKHHGPAXMGIwd3BZFGJrculSRJ\nklRzShX/lTSyWKyQJEmSJEk1xWKFJEmSpJrjiAppZLNYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmq\nbU88UXQGkoaYxQpJkiRJNW3Ch95TdAqShpjFCkmSJEmSVFMsVkiSJEmqaaVtW4tOQdIQs1ghSZIk\nqaaVtliskEYaixWSJEmSalqpfVvRKUgaYhYrJEmSJNWcrrLbpc2bC8tDUjEsVkiSJEmqaaVtjqyQ\nRhqLFZIkSZJqWmn7jqJTkDTELFZIkiRJqjml8oPOjqLSkFQQixWSJEmSalpphyMrpJHGYoUkSZKk\nmtbQ4cgKaaSxWCFJkiRJkmqKxQpJkiRJNa2UHSJpmLFYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmq\nOV2VDZdfWkQakgpisUKSJElSzdl43Gt6HbdeeH5BmUgqgsUKSZIkSTVn6w//fae2aTOamTajuYBs\nJA01ixWSJEmSatP++/c6bEj/tc5ohi9eWEhKkoaGxQpJkiRJtenxx2lvrj6SovXSi4Y4GUlDyWKF\nJEmSpJq1bvGSvu/83lVDl4ikIWWxQpIkSVJdavnUx4pOQdIgsVghSZIkqW6sXLG+Z1vTEtB8xMFF\npiNpkFiskCRJklTTVr//Q72OV938q57bY1euHOp0JA0BixWSJEmSalrn//kiK1esZ+WK9UnDi15M\nZ9n9rTOa4bf/VUhukgaHxQpJkiRJdWf1b37f67j1rScxfUb1nUMk1R+LFZIkSZLqz5FH9hpdAX64\nkYYT/3+WJEmSVJdWp1NDtrW29rS1zmiG/76zwKwkDQSLFZIkSZLq2vr7Hu69hsXrjy8sF0kDw2KF\nJEmSpLq3unvxzVSr61dIdc1ihSRJkqRhYeWK9XSVHbdYsJDqlsUKSZIkScPGqrKCRQlHWEj1ymKF\nJEmSpGFlVcWUkGkzJxeUiaT+slghSZIkadhZuXhJz+2Gri6mO8JCqisWKyRJkiQNP83NrLzn/p7D\nUcDkIw8rLh9Ju8VihSRJkqThadasXotujnn6KfjTHwtNSVI+FiskSZIkDWuryqaEtLz2uAIzkZSX\nxQpJkiRJw1tzM+0tM4Bkh5Cps1qKzUdSJosVkiRJkoa9dfcv7pkO0tjeDuvX7zJeUrEsVkiSJEka\nEVaVLbjZesgs16+QapjFCkmSJEkjw6xZ7Ggc3XPY6voVUs2yWCFJkiRpxFizbDWdZcetM5oLy0VS\n3yxWSJIkSRpRVq9Y36tgMXW/1sJykVSdxQpJkiRJI055waJx2zYmnPL6QvOR1JvFCkmSJEkj0uoV\n63t2CJl420K4fWGB2UgqZ7FCkiRJ0oi16ltX9dxuOdnRFVKtsFghSZIkaeQ66WTWvOs9z4ywOPEV\nhaYjKWGxQpIkSdKItuPiy+kCSsD4u+8uOh1JWKyQJEmSJFZf9KWe0RWT5hxTaC6SLFZIkiRJErzn\njJ7RFePuv6/obKQRz2KFJEmSJAGrF/53T8Fiyt/NLDodaUSzWCFJkiRJAIcf3jMVZPSWLbBkSaHp\nSCOZxQpJkiRJSq1esb6nYDHtqMMLzUUaySxWSJIkSVKZ7VOnAtAAEGOhuUgjlcUKSZIkSSqzLj7e\nM7pi+rHPLzQXaaSyWCFJkiRJFdoPCz2LbfLdbxecjTTyWKyQJEmSpArrf/cnIClWtHz2U8UmI41A\nFiskSZIkqYq2j58NJAWLqW5lKg0pixWSJEmSVEXnv5xDR6kEQMOWLQVnI40sFiskSZIkqQ9tT6/r\nWWyz+ejZheYijSQWKyRJkiRpFzpJpoKMeeKJolORRgyLFZIkSZK0C20PL+3ZGWTKwfsVnY40Ilis\nkCRJkqRdaWqiM705esM62LCh0HSkkcBihSRJkiRlaPvrAz23px+8b4GZSCNDY9EJSJIkSapdIYSL\ngWNJPjt8Icb407L75gJfADqACLwvxthZtaN6t88+tLe0MGbVquQb32XLoDUUnZU0bDmyQpIkSVJV\naTFidozxGOB44PKKkCuBk2OMLwGa0phha939j/TsDNLyD88qNBdpuLNYIUmSJKkvtwOnpLfXAhND\nCA1l9z8vxrgkvb0SmD6UyRWh/bBkNEUJYD8X25QGS6mrqys7SpIkSdKIFkI4Azg2xviOKvftDdwB\nvDDGuHpX/ezY0dHV2Niwq5DaVyo9c9vPU1J/lLICXLNCkiRJ0i6FEN4AvBd4dZX7ZgA/Bz6cVagA\nWLNmc+7ztrY2sXJlDe688fUraf3IGQBsb25m7cNLC05oZzX72mFu/TWccmttbcqMcRqIJEmSpD6F\nEF4DfA44Ica4ruK+ZuBW4JwY46+LyK8Qp7yVjvSL4Ua3MZUGhcUKSZIkSVWFECYDlwDzYoxtVUIu\nBS6LMf5qaDMrXtvDyVIdJWDqfq3FJiMNQ04DkSRJktSXU4EW4PoQerbp/C2wCPgP4HTg0BDC+9L7\nro0xXjnkWRahqQlGjYLOThq2bSs6G2nYsVghSZIkqaq08LCr4sPYocqlJnV00FUqJaMr9p/BmidW\nFJ2RNGw4DUSSJEmS+qmzIdnZpGHr1oIzkYYXixWSJEmS1E9tDz5B9+alE0+eV2gu0nBisUKSJEmS\n+qupiS6ShTbH3X570dlIw4bFCkmSJEnaA2s/8rGeggUxFpyNNDxYrJAkSZKkPdAx/9+ApFgx/djn\nF5uMNExYrJAkSZKkPbR9xl5AOrrinrsLzUUaDixWSJIkSdIeWnfvgz1TQaYf/4qi05HqnsUKSZIk\nSRoA2/faB0g/ZC1bVmguUr1rLDoB7SyE8ADw8hjj07uIuRpYHGO8oMp9rwH+FmN8YhBznAm8MMZ4\n8yCe4/0xxu8M1rlDCDOAHwIHxhgPKWu/GngNsK4s/PQY410hhP2B/wv8HbAR+ESMcUH6uFcAXwIm\nAY8D744xLtnVuSSpCCGEFwCfjzG+ZhD6HvTrQ3qeqtcISSrSuv99gNYZzQBMe+6zaXt6XcYjJPXF\nkRU1KMb4rF0VKnL4OLD/QOXTh7nA6wer8xDCXsDZg3XuEMI04DZgUR8h/5L+HLr/3ZW2Xwn8IsZ4\nGPAe4EchhPEhhInAj4H3pff9HPhWznNJ0pCKMd41GIWK1KBeHwBCCA3AJYN5Dknqr+1NkwEY1dUF\nGzYUnI1UvxxZMUhCCE8Cc2OMi0MIpwLfB6bEGDeHEM4CDgW2A8cDY4ArY4wXpo/tAvYDlgFXAKcA\ni0k+AJ8QY5yTnmZaCOGXwGzgb8DJJB/wXwk8O4RwdozxurKc7gK+GGP8SXr8RuAzMcYXhRDeAFwA\nTEzPdVqMcVUIYTzwbeBYYCvwf4D7ga8BjSGESTHGt4YQTgHmk7ynlgHvjzE+HEI4D9gXeA5wbYzx\n8orX6fVpn2NIRiq8N8b4V+BOYFY6yuTvY4ztafxRlefux48HoAt4I7A3Of+oDiFMBl4BvBkgxvjX\nEMITwJz0eT8SY7wnDb8K+FIIoak/55KkwRRCmAN8N8Z4SPp7uoVnflevAt4QY1weQngM+AZwKkkR\n/FsxxnPLH1/eH/AW+vgdHUL4MMk17HXpcQPwNPBSkt//3wRCGn5mjPHWNO504Jy0/Y/A+4Bbgcnp\nNeIEoAP4DnAAybX14hjjNSGEA0iuJ9cBR8UYX77nr54k7drah5+kZUYzJWDaofvR9tTaolOS6pIj\nKwbPAuCY9PbLgD8DL0iPjwWeAg4HjgSOAE4OIcyr6ONEkj/CDiH5kPuuivtfDbwdOAiYAbwxxngu\nsBR4W3mhInUDvT8svwm4PoRwEEkx5R9jjAeluX8rjfkEMCbGeCBwHMkfoU+l/70hLVTsT/JH4htj\njM8CfkFS4Ch/HidWKVQ0At8jKWwE4CaSaRSQjFp4Ih3V0N79mLQY0HPuiv7GhBAeqPLvhorXgRjj\nmhh3uQn2aSGEP4UQ7g8hfDaEUCL5OayMMW4qi3sYeBZwWHq7u/+NwGrgkBznkqSinQJ8DDgYWEHy\nO7jbMSTXryOAj4QQntNXJ7v6HQ38FJgbQpiQHr8MWBZjfIDkWvDXdGTaicAPQgjT02LDl0iKwoGk\noP7PaX4d6TXiUZJRbwvTa8lrga+kj4WkEPNXCxWShtKOSU0AjOrsLDgTqX45smLwdBcrvg+8iOQb\no5cAC9PjA0jmC28DtoUQrgFOAm4p6+NY4Jb0gy8hhB+RfLPf7Zcxxrb0vnuBWRk53QB8Kv02q0Ty\nB905wOtI/si7N437FvB0GncicDFAjHFJCGFWjHFjCKG83+OABTHGxenxd4GL02IEwB9jjKsqk4kx\n7gghzIgxbk+b7mDngkxuaVHjWf19fJnbSAp5VwP7AP8JLAEeJRldUm4LyR/PpV3cJ0m17vYY4+MA\nIYS/0Hsq4TUxxg5gRQjhDuDFJKP5dkuM8akQwj0k14ybeKZgPpFk6sgpadzi9DyvBcYBd8YYl6W5\nnQbsoOx6F0IYnfZ5avr4x0MIC0iul78FRgM37m6+krQn1j6ytGd0xZRD92PtQ08WnZJUdyxWDJ4F\nwD+HEKYC7SR/MH0thPAs4AlgKnBZCOHCNH4scFdFH1NJPiR3W1px//qy2x1Aw64SijE+kk5PeTHJ\nH28xxvhkCGEK8LJ0OG23dcB0km+k1pb1sbFK163AmrKYdelIhJa0qW0Xaf1zCOGdJM9/HMmUiULF\nGP9f2eGTIYQrgXnARSQ5lptAMnx51C7uk6RaV74CXOX1pPx3+BqSa1N/dY/wuwl4A/AqYDJJwffO\nskL4JJLr5iR6X4O2AlQUzKcDpRhj+XNYQzLiEJIRGOXXS0kaEt3bmDauc5FNqT8sVgySGONjIYRJ\nJGtS/CEtFBxIMlriNyQFgy/FGG/ZRTfrSf5Q67b3AKTW/YfiWOD6tG0Z8F8xxpMrg0MIq3im6EAI\nYRY7Fx+e5pkpL6QFmk6Sec99CiG8GPg08IL09TqOZDpJv4QQxgD/W+Wue6s9t130Mxt4KB31Asn/\nJ9tJ1vJoSedhdxchDiVZn6KR9Fu9tI/JJH/QP7TbT0SSaktL2e1pJNeAyoJG3gLGT4DPhhCOBtpi\njA+lo/A6gKMrC+IhhDNIrpfdx83A+Io+VwGdIYSpMcbuwvl0kmuTJBVm9ZeuoPWTZ1IC+NUv4fgT\ni05JqiuuWTG47gDOBH6fHj9AMs/2NyTfKr0vhNAQQiiFEM4JIRxf8fi7gHnpbhNTSBYuy2M7MKWP\n+24g+SZrHvDvadt/AMema1cQQnhBCOGK9L6bgdPTHPcC/kLyh2v5Of6TZGTGQenxB4Ffxxh3ZOQ5\ng2Ru9BPpHOZ3AhPTURnbgUllU0kyn1+Msb1iB4/uf7kLFakrSeZEdxdeTifZAWR9+ly775sL7EUy\nbWQB8HchhJemfXycZArPJiSpvp0aQhiVbkn6UpJr23Jg7xDCjHTK4NvK4vu8BsUYlwKPAJ8jLZin\n14pfkFw7CCFMCCFcFULYD/gl8JIQwgHpteFbwHvTc4wKITSlj/8P4APp4w8mWQ/jvwbyRZCk3Xb6\nu3uGDE8/vb9rwksjl8WKwbUAeCHJSuSk/30uSfHi68DjwH0kRYxnA7+rePyNwN1AJPk26nryTZO4\nAfhxuutILzHGB0l+7ku75wDHGJcD7wduDCH8jWRxtO7FOS8jKSg8TrLexidjjE8AvwZeEUL4U4xx\nCcnq7DelU0leRvpHY4ZfkYzqeDjt73KSocg3kIyQaAOeShfwLNdz7hznqCqE8Lo012uA/dOFOH+T\n3n06cHwI4UGSn9m1wI/S+z4IvDKEsBi4FDglxrgtxrgFeCvw9fS+FwEfyXEuSap195EUz+8DvhJj\nvC9do+gqkgL270iK8N2yfkffQLJD0vVlbR8CXp7+rryHZHelJ9PryxkkU0IeJLkGfpmkWPI7kmL3\ni0l+N89JH38jyTbSThCXVLgdU5OBZ6WC85DqUamrq/AlArQLIYRSjLErvf0R4FUxxjcVnJYkaQRI\nty59e4yxspguSf22cuWG3B+z/pZnAAAgAElEQVRAWlubWLlyw2Cms0fy5NcyoxmATW8+lS3f7PeM\n591Wy6+dufXPcMqttbUps4bnyIoaFkL4B+DREMLUdDrEScAfCk5LkiRJUk7dC21O+Ml1WaGSylis\nqGExxr+S7D3/Z5Jt4paSTNGQJEmSVAe2zHt9T8FCUn7uBlLjYozzgflF5yFJGnlijAcUnYMk1bvN\nV/2ACTOaKQFTZ7WwZskuN8yTlHJkhSRJkiQNos6GZLfnhvb2gjOR6seQjKw4++yzd1pE56yzzuLL\nX/5yz3Fra2tmP5s3b851vjxxEydOzIwZNWrnWs4ZZ5zBlVdemSuPcs3NzZkxpVL24LBVq7IrsVu2\nbNmprfL1BmhszP7xNzU1ZcZs3bo1M2b79u2ZMdXiPvGJT3DppZf2amtpacnsZ9y4cZkx69aty5VT\nnvfK6tWrex2feeaZXHHFFb3apqarQe/KsmXLcuV00EEHZcasWLEiM6Zygd2PfvSjfPWrX+3VVu3/\ng2rac1x8L7roIkdAjnClUqnXm27RokUceeSRRaXTb+Y9tOo1b6jf3Ac7766uLq8HGlHavvhlWj95\nZjIV5Gc/hTeeVHRKUs3r98iKEMJlIYQ/hBDuDCE8f3cfv9dee/X31IWaMWNG0Sn0S72+3vWa98yZ\nM4tOoV/qNW8VZ0+vBbNnzx6MtAadeQ+tes0b6jf3es1bqlmnv5vuav30M95VZCZS3ehXsSKE8HLg\n0BjjMcB7ga8MaFaSpJrntUCSpPw2H38CkC60uXFjoblI9aC/IyteCfwMIMb4N2BqCCF7noMkaTjx\nWiBJUk6br7muZ1eQqc8+sOh0pJrX32LFXsDKsuOVaZskaeTwWiBJ0m7oHD0agIZt2wrORKp9A7XA\n5i4XSTrrrLOqrj1w8cUXD9Dph9Y555xTdAr9Uq+v9yWXXFJ0Cv1y4YUXFp1Cv1xwwQVFp6D6lblg\n3qJFi3aaC1+50Gu9MO+hVa95Q/3mPlh551lQXBqu2uLjtB60DyVg0pvnsfEntxSdklSz+lusWEbv\nb8/2AZb3FVy5CwUkH5zPPvvsnuN62Q3knHPO6deHuaJ3A6l8vaE+dgO55JJL+NSnPtWrrR52A7nw\nwgv57Gc/26utHnYDueCCC3Yqxg3wbiC5+lLd2K1rAbDT7gJdXV11+cHFvIdWveYN9Zt7veYt1bxJ\nk3qmgoy743ZcuULqW3+ngfwaOBkghHAUsCzGuGHAspIk1QOvBZIk7aatx76s6BSkutCvkRUxxjtD\nCH8OIdwJdAIf2VV8X9tPlrfnqd7n/XZ+7733zozJMxqgr1EMa9eu7bk9ZsyYXDnlMX78+MyYPN9y\n95VTZXue0Qd5ctqwIfuzSZ6RJVD9fVA5kiLPiJA876c8o0YAtuWYU1jttaxsW758l184A9DQ0JAr\np02bNmXG5Mm72s+lMu88/68AzJo1K1echo/dvRZIkiTY+JNbGDejmRLQMqOZVSvWF52SVJP6vWZF\njPEzA5mIJKn+eC2QJGn3dYwZQ2N7e7LY01NPQR9f7kojWX+ngUiSJEmS+mHNklV0rxw27XlHFJqL\nVKssVkiSJEnSEOuYMAGAUTmnuksjjcUKSZIkSRpiax57Ckh2Bml+yfOKTUaqQRYrJEmSJKkA3VNB\nRj/0UKF5SLXIYoUkSZIkFWDTG95MF8noChYvLjgbqbZYrJAkSZKkAmz5zv8DkmLF9BcfVWwyUo2x\nWCFJkiRJBekYMwbwg5lUqXEoTlIqlTLbn3766cx+Ojo6cp2voaEhM2bjxo397qe8vampKVdO69at\ny4xpbMz+cWzbti0zZsaMGVXbW1paeh2vWLEis6/m5ubMmK6ursyYVatWZcZA9dwr3z95Xsu+3nPl\n2tvbc+U0ffr0zJhqP5fK12Xq1KmZ/UyaNClXTnleg/333z8zZunSpTu1bdq0qddxa2trrpw6Oztz\nxUmSJOkZa5asonVG8jd387EvZP0dfyw4I6k2WMCTJEmSpAJ1f+UzJv6t0DykWmKxQpIkSZIKtPas\nTwPpQpv/89dCc5FqhcUKSZIkSSpQx2c+17ONactxLys0F6lWWKyQJEmSpIJ1TJxYdApSTbFYIUmS\nJEkFW/Po8p7RFWM+84lCc5FqgcUKSZIkSaoBXSTrVjRd9Z2iU5EKZ7FCkiRJkmrAlnmv7ylYSCOd\nxQpJkiRJqgGbr/oBkBQrpoQDi01GKpjFCkmSJEmqEZ2lZFxF45rVBWciFatxKE6yYcOGzPZJkyYN\n2PlWrlyZGbPXXntlxvSV95QpU3pujx49OldOHR0dmTGrV2f/Qho1Kru+1Ne5KttnzpyZ2df69esz\nYyZPnpwZs2bNmswYgM7Ozsy2rVu3ZvYzMcdqynliAJ5++unMmGqv+bp163od53mPb9q0KVdOY8eO\nzYzJ8zpt3749s2358uW5cmppackVJ0mSpL61/fhGWk59Y9FpSIVzZIUkSZIk1Yq5rwCSqSDNr55T\naCpSkSxWSJIkSVIN6d7CdPRf7yk0D6lIFiskSZIkqYbsOPhQwF1BNLJZrJAkSZKkGrLuD3/uGV0x\n8Z2nFpqLVBSLFZIkSZJUY7pIRlaMv/XWolORCmGxQpIkSZJqzNYTTig6BalQFiskSZIkqcZs+t51\nz4yu+OiHik5HGnIWKyRJkiSpBnWvWzHhuh8WmodUhMahOMmUKVMy2zs7OzP7Wb58ea7zTZo0KTNm\ny5YtmTGjR4/ObG9ra8uV03777ZcZM2bMmMyY7du3Z8aUStXXDR47dmyv4+nTp2f21dzcnBnT1dWV\nGTNr1qzMGKj+c6nMc/369Zn9rF27NjOmo6MjV07Tpk3LjNm0adNObQ0NDb2O87zn8tqxY0dmzObN\nmzNjqv18J0yY0Ot469atuXLK8/6V6lW133N5fvcNprlz52bGLFy4cPATkSQNmvYXvJCxd/0x2RVk\n0yaYOLHolKQh48gKSZIkSapBG275TyCZCjL1uc8uNhlpiFmskCRJkqQa1dmYDIZvyDFyWBpOLFZI\nkiRJUo1q+/XtPWtXUGXqsTRcWayQJEmSpFo1ezaQTgU54pBic5GGkMUKSZIkSaphPVNBNjuyQiOH\nxQpJkiRJqmFt372GLnhmVxBpBLBYIUmSJEm17MR5PTenHD27wESkoWOxQpIkSZJqXFcyroLG1asL\nzkQaGo1DcZIdO3Zkti9ZsiSzn7Fjx+Y6X0dHR2ZMV1dXZkxf/ZTnvffee+fKady4cZkx48ePz4xp\nbW3NjDn00EOrtp944om9jjdu3JjZ14QJEzJjmpqaMmNWrlyZGQPw6KOP7tS211579TrO8zpNnDhx\nwHJ68sknM2Pa29sz26ZNm5bZz/Lly3Pl1NiY/b/ulClTMmPGjBmT2ZanH4BSqZQrTtLAWLBgQWbM\nwoULcz3utttuy+zrvPPOy5OWJGmQrJn/eaaff05Ssrjnbjjq6KJTkgaVIyskSZIkqcZ1fuSfe7Yw\nnX7CKwvNRRoK/RpZEUKYA/w7cF/atCjG+NGBSkqSVPu8FkiSNLQ6Gxtp2LGDUo5R4lK925NpILfF\nGE8esEwkSfXIa4EkDXMhhIuBY0k+O3whxvjTsvteBVwIdAC/jDF+vpgsR4a2719H6z++GSffaiRw\nGogkSZKkqkIIc4HZMcZjgOOByytCvgK8GXgJ8OoQwuFDnOLI8srjeqaCTD7meYWmIg22PSlWHB5C\nuDmE8LsQwnEDlpEkqZ54LZCk4e124JT09lpgYgihASCEcBDQFmN8MsbYCfwScDGFQdZdrGh8+KFC\n85AGWynPrhiVQgj7Ai8FrgcOAhYAh8QYd94SAVi1alVXS0vLnuQpaXhw1OIwsrvXAoB77723a/Zs\n94eXRrJSqURXV5fXgzoUQjgDODbG+I70+MXAp2KMb0qP3wscHGP87K762bGjo6uxsWHQ8x22zjwT\nvvKV5LZrV6h+ZV4H+rVmRYxxKXBdevhwCOEpYF9g530ngWuuuWantrPOOosvf/nLPccDuXVpnrg8\nW3J2dnbu1PaZz3yGiy66qOc4zxaZUPzWpQcddBCPPPJIr7Z62Lr0da97HT//+c97ta1bty6zn6He\nunT9+vW9jq+44grOPPPMXm21uHVp5fvynHPO4YILLujVluc9ADBqVPZArY997GO5+lJ92N1rAcCR\nRx7Z67irq6sutr3tT2G/FlRuXTpnzpyq25nW+tal9fI+qaZec6/XvDV4QghvAN4LvHoXYbneNGvW\nbM593tbWJlau3JA7fqgVkt85F9Dyla9QArYe8xI23Pyr2sktJ3Prn+GUW2tr9mfIfk0DCSG8LYTw\nyfT2XsBMYGl/+pIk1SevBZI0MoQQXgN8Djghxlj+rdEyYK+y433TNg2y7hL62P++s9A8pMHU391A\nbgauTSusY4AP7WrYb1/fYJe355kmkucbfIBVq1ZlxmzYkF312Xvvvau2Nzc399zed999c+WU51vn\nPM+v/Nx96WsUR2V7Q0P28LuOjo7MmC1btmTG5P1W8uCDD85sW7t2bWY/Tz/9dGZMnvcAwMyZMzNj\nqv3sKt8bed6XeUbXALm+7cozAqXa6KHNm3t/27F9+/ZcOeV9PTWs7Na1oJ7NnTu31/GCBQt2apsz\nZ84QZgTz58/PjKmWU962Si9/+cszY2p9hIak3RdCmAxcArwqxthWfl+M8bEQQnMI4QBgCTAPeNvQ\nZznybD3hBCbcemvRaUiDqr/TQDYArxvgXCRJdcRrgSSNCKcCLcD1IYTutt8Ci2KMNwIfAn6Utl8X\nY3xw6FMceTZ97zrGz2imBIw573O0n/d/ik5JGnD9HVkhSZIkaZiLMV4JXLmL+28Hjhm6jNSti2SR\nkOZvfJVVFis0DO3J1qWSJEmSpALsOPQwwK3WNHxZrJAkSZKkOrPu93f33G5607wCM5EGh8UKSZIk\nSapDPbuC/P72QvOQBoPFCkmSJEmqQ5tPfguQTgXZtKnQXKSBZrFCkiRJkurQ5m98t2d0xdQjDi40\nF2mgWayQJEmSpDrVXaxo2Ly50DykgWaxQpIkSZLq1Oof/eSZg0cfLS4RaYA1DsVJpk2bltm+ffv2\nzH6WL1+e63yTJk3KjJkyZUq/+ylv37ZtW66cmpubM2Pa2toyY9rb2zNjNmzYsFPbPvvsw0MPPdSr\nbfHixZl9jR07NjNm7dq1mTFbtmzJjAFobW3tdXz44Ydz11139WobM2ZMZj+dnZ2ZMTNnzsyV05NP\nPpkZ09i48/9KpVLvjaTyvAYdHR25csrzHs/z/9SECRMy2/L0A9DU1JQrTqpHCxcuzGyrFjOY8pxv\nzpw5vY7nz5/P+eefv1Pc/Pnzd7uv/sbkORfA3LlzM/se6tdckmrSK4+ji2TdimkveyFtT64oOiNp\nQDiyQpIkSZLqWOe4cQCM2ra14EykgWOxQpIkSZLqWNttf+wZXcHvf1dwNtLAsFghSZIkSfXswAN7\nbra86cQCE5EGjsUKSZIkSapznekaaqWMOKleWKyQJEmSpDrXtqytZxvTpnnHFZqLNBAsVkiSJEnS\nMNBdrBhz1x8LzUMaCBYrJEmSJGkY2PF3BwBOBdHwYLFCkiRJkoaBdX/6357RFcybV2Qq0h5rHIqT\nrFu3brfa+zJlypRcce3t7Zkx69evz4yZMGFC1fbOzs6e25s2bcqV06pVqzJjtm7N3hd5xYoVmTHL\nly/fqe3lL385V111Va+2vp5fuaVLl2bGlErZtds85wI4+OCDd2pbvHhxr+MjjjgiV19Znn766Vxx\n++yzT2ZMtffy2LFjex03NTVl9rN69epcOa1duzYzZtasWZkx1X4ulW153rsA48ePzxUnaWAsXLhw\nt2Pmz5/Peeedt1Nctbb+xOQxf/78XHELFizY5THkew3mzp2b63ySNFx0kX4j/YtfFJyJtGccWSFJ\nkiRJw8SOQw575mDz5uISkfaQxQpJkiRJGibW3Xl3z1SQKS85utBcpD1hsUKSJEmShpGudJp2w9Il\nBWci9Z/FCkmSJEkaRjZ8/JOAu4KovlmskCRJkqRhpP0z5/bcHvXdKwvMROo/ixWSJEmSNAyVgKnn\nnF10GlK/WKyQJEmSpOHmoIMAKHV2FpyI1D8WKyRJkiRpuFm0CEhGV4y59KJic5H6oXEoTjJz5szM\n9s059gAulfItEbNx48bMmH322SczZvHixZntnTkrlXniHn300cyYJUuyV/TdsWNH1fYHHnig1/FT\nTz2V2deGDRsyYw477LDMmFmzZmXGABx44IGZbdOmTcvsZ/To0QMSA/Dggw9mxlR7zVevXt3rOM/r\n3dfPrlJDQ0OuuCzt7e2Zbdu3b8/VV97XU1J9Ou+88wakn4ULF+aKW7BgQWbMnDlzMmO6uroyY+bO\nnZsnpdy5S1JNmDCBLpJiRdPFF7H6E58pOiNptziyQpIkSZKGoR37Jl8YjupyKojqj8UKSZIkSRqG\n1v7+7p7bTgVRvbFYIUmSJEnD0YQJdI+paLr4C4WmIu0uixWSJEmSNEy1P/9FAJRyrOEj1RKLFZIk\nSZI0TG34xa97Ftqc+IF3FZyNlJ/FCkmSJEkaAcbd+NOiU5Bys1ghSZIkScPYljedBCSjK6R6YbFC\nkiRJkoaxTd++umcqyOSXPr/odKRcGofiJGvXrs1s37x5c2Y/o0ePznW+mTNnZsasX78+M2bs2LGZ\n7UuXLs2VU1+vQbk8fd1///39Ptddd93V63jChAmZfR111FGZMc95znMyY4477rjMGICGhoad2ip/\nns3NzZn95Hk/bdiwIVdO48ePz4y55557dmp76KGHeh339X4qN2nSpFw5jRqVXWfM8x6fPHnyTm3b\nt2/PjKmmo6MjV5ykkW3hwoW54s4///ye2/Pnz+91XN4+EObMmZMrLm/uklSLuosVjQ/GolORcnFk\nhSRJkiQNc2vPv7CnYMETTxScjZQt18iKEMJs4Cbgshjj10II+wHfBxqA5cA7YozbBi9NSVLRvBZI\nklS/Oj70TzD/s5SAacccRdvSVUWnJO1S5siKEMJE4KvAb8qa/w34eozxWGAx8J7BSU+SVAu8FkiS\nVP92zNofgIbt7QVnImXLMw1kG3AisKysbQ5wc3r758CrBjYtSVKN8VogSVKdW3vPvT23pzz38AIz\nkbJlFitijDtijFsqmieWDfVdAew94JlJkmqG1wJJkoaHjsZkJYDGpUsKzkTatYHYDSRzu94PfOAD\nzJgxY6f2c889dwBOP/TqNe+urq6iU+iXefPmFZ1Cv1x77bVFp9Av9fr+VuFybd2+aNEiZs+e3aut\nXn83mffQGqidP/ak7/7mUK+v+WDlXSrl+nUhaZC0XfNjWk47OTnYvBly7BAoFaG/xYqNIYTx6bds\n+9J7WPBOvv3tb+/Udu655/L5z3++53ggty6dOnVqZsymTZsyY6pdpCvzHsitS5csya5u3nffff06\nV1dX105/HNTD1qXz5s3jlltu6dU2ffr0zH7yvJ8effTRXDnl+dlVbl167bXXctppp/Vqy7N1aZ5t\nUiHf1qXjxo3LjKnclrTy/Q35/6jMs3XpYH7YUCF261oAcOSRR/Y6rva7qR6Y9+A777zzem4P9tal\n1frOyimvenrNy9Vr3pJyeNWrgeRbhqnPPZw18bFC05H60t+tS/8LeHN6+83ArwYmHUlSHfFaIElS\nHeocMwaAhjVtBWci9S1zZEUI4XnApcABwPYQwsnA24CrQwgfAB4HvjeYSUqSiuW1QJKk4aPtp7+g\nZV468vmJJ2D//YtNSKois1gRY/wzyYrvlfKN6we2bKlck23n9h07dmT2k2f4P0BnZ2dmzIYNG/p9\nvvLh/Fu3bs2VUx55phusW7cuM2bKlCm52ivnjVdz0kknZcaEEDJjjj766MwYqP6an3DCCb2O80y9\nufvuuzNj2tryVZIXLVqUGTNx4sTMtjzTMvJOdcrzHs8TU22qU2VbtedWzfLly3PFqT4NxLVAgvxT\nKSqneAzmNLKFCxcOWt+SVJNe8EIgnQoy50WseSRzJqc05Po7DUSSJEmSVKc6xyZfpDVs3FhwJlJ1\nFiskSZIkaYRp+9Z3qc+9ijRSWKyQJEmSpJHmta8Hkqkgza+eU2gqUjUWKyRJkiRpBOoeWTH6r/cU\nmodUjcUKSZIkSRqBdhxyKOCHQtUm35eSJEmSNAKt+687njn46Q3FJSJVYbFCkiRJkkaiCRN6poK0\nfPA9haYiVbJYIUmSJEkj1I6DDik6BamqxqE4yahR1Wsi5e2lUimzny1btuQ637hx4/IllmHp0qWZ\n7a2trXvUV7k8r0Ge8x1xxBFV25/73Of2On7jG9+Y2decOXMyY/bdd9/MmLFjx2bGADQ0NGS27dix\nI7OfPK/lE088kSunpqamzJitW7fu1NbR0dHr+Mknn8zs59BDD82VU57XoLOzMzOm2s+lsm3lypW5\ncmpsHJJfJ5IKsmDBgsyYPNeMobZw4cIBiZGk4Wrtf99Dy4xmSsDE976DTf/3+0WnJAGOrJAkSZKk\nEa17Ksj4n99UaB5SOYsVkiRJkjSCdew7C4ASQM7Rx9Jgs1ghSZIkSSPY2r/c33N72ov+ocBMpGdY\nrJAkSZKkEa4zXU9wVI610aShYLFCkiRJkka41Vd8o2ftCqkWWKyQJEmSpJHu1NOAZN2KiR94V6Gp\nSGCxQpIkSZJE2a4gN/600DwksFghSZIkSQJ2zD6y6BSkHo1DcZJx48Zltk+dOjWzn+3bt+c6X6lU\nGpCYhoaGzPYlS5bkymnr1q2ZMXvttVdmzKRJkzJj5s6dm6v9wAMPzOxrwoQJmTGNjdlvo7Vr12bG\nADz22GO9jmfPns29997bqy3GmNnPxo0bM2P233//XDktW7YsM+app57aqa3yObe0tGT2s3r16lw5\n5emrr/fv7mpqasoVt3nz5gE5n6Sh19U1fGcp93VNlCTtbN1vf0/LjOZkKsg7/5FN3/tR0SlpBHNk\nhSRJkiQJgK70S91xt/6i4Ew00lmskCRJkiQBsOGz59JFstCmVCSLFZIkSZIkANrP/CSQFCsmv/j5\nxSajEc1ihSRJkiSpR+foMQCMXpy9Vpw0WCxWSJIkSZJ6tP3ohp5tTNmypchUNIJZrJAkSZIkPeNl\nc4BkKsiUuS8uNBWNXEOydakkSZKk+hVCmA3cBFwWY/xaxX0fAd4OdAB3xxg/VkCKGiQNjzxcdAoa\noRxZIUmSJKlPIYSJwFeB31S5rxn4FHBsjPGlwOEhhBcNcYoaBJtPe7u7gqhQQzKyYsKECZnta9eu\n7Xc/lcaMGZMZs23btsyYpqamqu2TJk3quT1jxoxcOS1atCgzZubMmZkx++67b2ZMQ0NDrvbGxuwf\n/8MPZ1dSH3vsscyYe+65JzMGYMqUKb2OZ8+ezZ133tmrrb29PbOf5ubmzJjOzs5+5VRNtffB0Ucf\n3et41apVmf10dHTkymnr1q254rJMnjx5p7aurq5ex+PHj8/V18aNGwckJ0kD57zzzsvVNpxV/k6r\nZu7cubn6Wrhw4R5mI9WtbcCJwKer3Nee/psUQtgITADahjA3DZLNl3+DCdf+AIAJH/swmy//RsEZ\naaRxZIUkSZKkPsUYd8QYq66yGGPcCpwPPAI8DvwxxvjgUOanwdNVKlECxv/oh0WnohHINSskSZIk\n9Us6DeSzwGHAeuC3IYTnxBj/p6/HTJ06gcbG6iOBq2ltrT7auVbUcn57nNu73w1XXcWorq4Bf57D\n+nUbRCMpN4sVkiRJkvrr2cAjMcZVACGEO4DnAX0WK9as2Zy789bWJlau3LCnOQ6aWs5vQHK76HJa\nrrqKErDpbe8YsKkgw/51GyTDKbc8hQ2ngUiSJEnqr8eAZ4cQuhe5Ohp4qLh0NFjGp+tXSEPFkRWS\nJEmS+hRCeB5wKXAAsD2EcDJwM/BojPHGEMIlwIIQwg7gzhjjHcVlq4G26Z8+zqSvXea33BpyFisk\nSZIk9SnG+Gdgzi7u/zbw7SFLSENqy7+ez6SvXQbA5DkvZt3COzMeIQ0MC2SSJEmSpD51NiTfcY++\n/96CM9FIYrFCkiRJktSn1Vd8g67ugy1Vd7GVBpzFCkmSJElS397yVgBKwNSjDi82F40YQ7JmxbZt\n2zLbt+So0LW3t+c63/r16zNjOjs7M2NGjapeyylv7+joyJVTV1dXZsyDDz6YGTNp0qTMmHHjxlVt\nf+KJJzIfW6mvn125PHmvW7cu1/kqcz/jjDP41a9+1avt8MOzf0GOGTMmM2bWrFm5cpo8eXJmzJNP\nPrlT25QpU3odt7W1ZfazcePGXDmVSqXMmNbW1syYxsadfwVUtuU5F8DWrVtzxUkaOgsXLux1PH/+\n/J3auttHsgULFuSKmzt3bmZMtddXkoaDzklNNGzcQMPq1UWnohEiV7EihDAbuAm4LMb4tRDC1ST7\nJ3e/Uy+JMf5icFKUJNUCrwWSJI1cbXcvovVZB1Di/7d35+GRVXX+x9+VrbektyRNswgI6BFsxnFQ\nfzLqEBZREWVGcUVHEH+44Q7jgkrDKIyC4ogrj7iM6LgvIOhgs7nPD0FUEA4iyt6Q7nS6O71lq98f\nVR3S6Urf20ml7q3K+/U8PM+9p07u/eSm0pf65px7gCuvgOe/IOtIanCJxYoQwgLgYuCaCS+9N8b4\noxlJJUnKFe8FkiTNckuXUqQ0FaTzta9m7cP9WSdSg0vzzIptwHHAgzOcRZKUX94LJEma5Ubnzweg\nUEyeUi9NV2KxIsY4HGOs9ECJ00MI14YQvhFC6JqBbJKknPBeIEmS+lb9AiiNruCBBzLNosZXSPPg\nR4AQwkpgTXme8tHA2hjjLSGE9wD7xBhPn+xre3t7i2ke9iep4aV7Wqdyazr3AoBbb721uGLFilpE\nlZRThUKBYrHo/WAW6+3dmO4DCNDd3UFv78aZjDMtec43U9m6ly0EYKS9g767p1awmI3XrRoaKVt3\nd0fifWBKq4HEGMfPWR/nwgAAACAASURBVL4c+Oyu+n/hC1/Yqe29730v559//th+mpUSmpubU+Wb\nbBWP8TZt2pTYZ8mSJTu1rVy5kpUrV47tr1mzJlWm2267LbFPmlVM0qwGcvDBB+/U9rnPfY43vOEN\nO7Ttu+++icfKejWQ733ve7zoRS/aoS3NaiBpvrf55WFsSaayGsib3vQmPvOZz+zQdvfddyceJ817\nAKq3GsjEFUvOOOMMLrzwwh3aKq0YUkma1WY+/vGPpzqW6sPu3gsADj300B32i8Vi6hVn8qRecvf0\n9Oywf91111Vc0SLtahiz3VRWA6mX98pE9Zpb0swb7uykZe1amgby+aFZjSPNMyt2EkL4bgjhgPJu\nD3Br1RJJkuqC9wJJkmafdat+MfagTbZUmiEqVUea1UAOAz4G7A8MhRBOpPRE+G+GEDYDA8ApMxlS\nkpQt7wWSJAmAvfce21xy+JNZd8sdGYZRI0ssVsQYb6L0F7OJvpv2JBs3Vh4iNL59r732SjzOQw89\nlOp8aYatp5kCMNl0kvHtaYftL1iwILFPR0dHYp/e3t7EPjfddFOq9jTTEgYHBxP7TJwCUcnatWsT\n+8DO0xIAbr755h320/x899lnn8Q+T3jCE1JlSnOdKr1/J7aNjiY/NTnNVAqY/HdqvDTX6f77709s\n6+pK98zENNdc9asa9wLV3sQpCZO11XK4//iplLtyxBFHjG339PRUzJ3GxKkw05FmukylqSITM0z1\ne5GkvBhta6N5cJDmB10kTDNnStNAJEmSJEmzU9+XvsbYU1KdCqIZYrFCkiRJkpTes58DlJ5bsfTx\nyQ+2l6bCYoUkSZIkabcM77sfAM0pVg+UpsJihSRJkiRpt/T//P+NbXfutTTDJGpUFiskSZIkSbtn\n3ryxzabh4QyDqFFZrJAkSZIk7bahAx/36M6vf5VdEDUkixWSJEmSpN3W/+ubxlYFWXriCzLNosZj\nsUKSJEmSNCVD4QkANA0NZZxEjaalFidpa2tLbG9pSY4ynHIu1F577ZXYZ+vWrYl9Jss0f/78se1i\nsVixz0TNzc2JfdIca8OGDYl91q9fX7H9oYce2mH/gQceSDzW4OBgYp9169Yl9lmyZEliH4COjo7E\ntvHXfzKbNm1K7LMl5ZrQ7e3tiX1aW1t3aluwYMEO+wMDA4nHWb58eapMaX4XJvu9G6/S9zaxLc3v\nJuz8/pKkSlauXLnbX1MsFjnyyCOndL6enp7EPtddd92Ujp32WBPbCoVC1c4nSVlbf/UNdO+3BwWA\nH3wX/vnFWUdSg3BkhSRJkiRpaubNG5sK0nnaKZlGUWOxWCFJkiRJmrLRhYsAcNyYqslihSRJkiRp\nyvruuo8ipWLF/Le/Kes4ahAWKyRJkiRJVTH/65dlHUENwmKFJEmSJGla+j7+qXE7fdkFUcOwWCFJ\nkiRJmpbRV/3r2FSQJc84LOs4agAWKyRJkiRJ0zba3gFA89q1GSdRI7BYIUmSJEmatr6rbwDKq4L8\n+MpMs6j+tdTiJCMjI4nt27ZtSzzOfvvtl+p8mzZtSuyzbNmyxD7r16+v2F4oPLooT9pM/f39iX0e\nfPDBxD5tbW2JfYaGhlK1NzUl16q6u7sT+wwMDCT2SfPzhcrZp5L7oIMOSuyzZs2aVJmGh4cT+3R0\ndOzUNvF9n+a9cs8996TKtHXr1lT9koyOjia2zZ8/P9WxNm/eXJVMkmqvp6enKn2uv/76qvSpplqf\nT5JmtYMOGpsK0nnyK1n7cOXPU1IajqyQJEmSJFXFaFcXAIViMeMkqncWKyRJkiRJVdH3p7uB8lSQ\nVVdnmkX1zWKFJEmSJKlqto+p6DzpJZnmUH2zWCFJkiRJqpqhxwcAmpwKommwWCFJkiRJqpr1v7hx\nbHvev70zwySqZxYrJEmSJElVtX1MxYIvfyHTHKpfFiskSZIkSVW16eTXAeUHbd55R6ZZVJ8sVkiS\nJEmSqmrLRz8+Nrpi6fOOzjSL6pPFCkmSJElS1Y3s8xgAmjZuzDiJ6lFLLU6yadOmxPaRkZHE48yb\nNy/V+ebPn5/Yp7OzM7HPxkl+qZqaHq3xzJ07N1WmvffeO7FPf39/Yp/99tsvsc9k3/9BBx20w/7d\nd9+deKwtW7Yk9lm4cGFinzTfP8CyZct2apv4Pac51tatWxP7pP3ZLV26NLHP5s2bd2obGhpKdfzx\n0uQG2HPPPRP7jI6O7vb5K1m/fn2qfkuWLKnK+SRVT09PT6q26667rirnO/vssxP7FAqFqpwrrUrf\nrySpNtZd+0u6Hr9vaWfV1XDMsdkGUl1xZIUkSZIkqfoWLwZKz63ofM0rss2iumOxQpIkSZI0I7Y+\n+zkAFKYw6lizm8UKSZIkSdKMGPjatylSGl3RftJLso6jOmKxQpIkSZI0Y4qtrQDM/en/ZJxE9cRi\nhSRJkiRpxqz9yn+PLWPK6tVZRlEdsVghSZIkSZo55VVACsDSZz412yyqGxYrJEmSJEkzavjAAwFo\n2rA+4ySqFxYrJEmSJEkzqv+6Xz86FeSDH8wyiupESy1O0lp+oMqu2hcuXJh4nPb29lTna2lJ/ra2\nbt2a2OdJT3rSbrXvysMPP5zYZ+7cuYl9HnnkkcQ+g4ODFdsXLVq0w/7BBx+ceKxt27Yl9knzc1m2\nbFliH4BDDz10p7ajjz56h/3HPe5xicfp6upK7DM8PJwqU7FYTOxz1113JbZ1dnYmHifN9U7bb2Rk\nJLHPgeUK93jLly/fYb9QKKTKlOY9Lqm2enp6UrXV0sqVK6v2dUcccUTi19X6+73++ut3Ov/ENkma\nlebOpVgo0FQswoc+BG85M+tEyjlHVkiSJEmSZtymM99XGl1RLPqgTSVKNbIihPBR4Fnl/ucDNwJf\nBZqBh4BXxxjT/UlYklSXvBdIkqTp2HrGu2n/6IcB6Pz7J7B2dX/GiZRniSMrQghHAitijIcDzwU+\nAZwLfDrG+CzgLuC1M5pSkpQp7wWSJKkahvfdH4Cm0dFsgyj30kwD+RnwkvJ2P7AA6AEuL7ddARxT\n9WSSpDzxXiBJkqat/7d/GNteeGzys4c0exXSPDxwuxDCaZSGAD8nxris3HYg8NUY4z9O9nWrV68u\nTnxon6RZKd3TOpVrU70XANx6663FFStW1CClpLwqFAoUi0XvB7NYb+/G1B9Aurs76O3dOJNxpiXP\n+XKdbVlpcYUisOaRDdmGmSDX162BsnV3dyTeB1KvBhJCOAE4FTgW+PO4lxJPctFFF+3U9pGPfIR3\nv/vdY/tLlixJzFDN1UAmroxRyR577LFT21FHHcW1116bKsd4aVZKWJ3iITNpVgP5/e9/v1PbVVdd\nxXHHHbdDW5oVUbJeDeTf/u3f+OhHP7pDWx5XA5l4zd/xjnfs9L5PsxrI7bffnirTTK0G8ta3vpVP\nfvKTO7RVczWQD33oQ6mOpfyazr0Adv4dLxaLqd9jeVIvuSeuoHH22Wdzzjnn7NTv7LPPrlEiKp4/\nyWS563U1kCOPPLJ2gaaoXt7jkurUBRfAmWf6VyztUqrVQEIIzwHOAp4XY1wPDIQQ5pVf3ht4cIby\nSZJywnuBJEmqijPOGNtc9MynZhhEeZbmAZuLgAuA42OMfeXmVcCLy9svBn4yM/EkSXngvUCSJFXT\naFPpo2jrnTHjJMqrNNNAXgZ0Ad8KIWxvew3whRDC64F7gK/s6gBtbW2J7ZP1Ge++++5LERcOOeSQ\nxD4DAwOJfSabKjJ++kSa6SsAaZ7Z0dramthn8eLFiX3mz59fsf3www/fYT/NNIEFCxYk9tmyZUti\nn7lz5yb2ATjggAMS29JcpzTvpzS5Ae68887EPpWmZUxsu+uuuxKPszvPkEmS5hr09++8XNTEtjTv\nOUj3Hlddm/a9QIKpTzmp5VSV6Zg4XaWnp2dKU18kqdGtvexbdL3yxNLOnXfA45+QbSDlTmKxIsZ4\nCXBJhZeeXf04kqQ88l4gSZKq6phjgdJDr5Y+/1j6/nxvtnmUO6meWSFJkiRJUjWN7LUXAE3rdx7t\nK1mskCRJkiTV3Lrrf0OR8pJiq67OOI3yxmKFJEmSJKn2xj0bbez5FVKZxQpJkiRJUia2/VMPUB5d\nUeHh75q9LFZIkiRJkjKx8TuXs309vEUvODbTLMoXixWSJEmSpMyMzp0LQGu8I+MkyhOLFZIkSZKk\nzPRd9q2x0RWsW5dlFOVISy1OsmTJksT2Rx55JPE4zc3Nqc63Zs2axD6LFi1K7NM/yZyp8e1NTenq\nPe3t7Yl9nvGMZyT2Wb9+fWKfwcHBiu0nnHBC4tdOtHr16t3+mkoWj3t4zq5s2bJlp7bu7u4d9kdG\nRhKPc/fddyf2GRgYSJVpsuuZdKyJbW1tbYnH2bx5c6pMad6/afqsq3AzmHh9t23blipTS0tN/jmR\ntBtWrly5w/7ZZ5+9U9v29tns+uuvT9XvhhtumNKx0h5fkmatcc+tWPKsp7Hu1j9nGkf54MgKSZIk\nSVKmRruXAdD8yMMZJ1FeWKyQJEmSJGWq75e/fXQqyK9+kWUU5YTjtiVJkiRNKoSwAvghcFGM8VMT\nXnsM8N9AG3BzjPENGURUIyhPGy8AS1/xYvrucYTFbOfICkmSJEkVhRAWABcD10zS5WPAx2KMTwNG\nQgj71iycGs62pz4dgKYKz7HT7GOxQpIkSdJktgHHAQ9OfCGE0AQ8C7gcIMb45hjjvbWNp0ay8XtX\njE0FmfPFSzPNouxZrJAkSZJUUYxxOMY42Z+5u4GNwEUhhF+EEM6vYTQ1ojlzKDY1UwDa339m1mmU\nMZ9ZIUmSJGkqCsDewH8CfwOuDCE8P8Z45a6+aMmS+bS0NKc+SXd3x3Qyzrg856vLbP9+Lpx1Fk3D\nw3QvbIM5c2objDq9bjlQ7WwWKyRJkiRNxRrgnhjjXwBCCNcATwR2WaxYt25z6hN0d3fQ27txOhln\nVJ7z1W22//sWus46iwJQnDuXNY9syE+2jDVStjSFDaeBSJIkSdptMcZh4O4QwuPKTYcBMcNIahAj\n3cuA0tAdtm3LNIuyU5ORFc3NlYd5jW9vaUmOMtlxJmpqSq7BPPDAA4l9FixYULH9wQcffb7Qpk2b\nUmVatmxZYp/29vbEPvvtt19inw0bKlcfFy1atMP+unXrqpIpTZ977033rKX+/v6d2u67774d9oeG\nhhKP88gjjyT26ezsTJVpzZo1iX3mzp27U1tbW9sO+1u3bk08zsKFC1NlKhQKiX3Gv08nU+n3buL1\n3ZLyacxpfocl5dM555yT2OeII45I7NPT01OFNOmlyX399ddXpY80W4UQDqO04sf+wFAI4URKD9T8\na4zx+8DbgS+XH7b5R+CKrLKqcay77S66l5X+v3jpkw+m7093Z5xIWfDThSRJkqSKYow3AT27eP0u\n4Jk1C6RZY3ivvWl58AGaUvzhUI3JaSCSJEmSpFxZ97+3UKQ0FWTpIQdkHUcZsFghSZIkScqXOXMY\nLU+RdnTF7GSxQpIkSZKUO323/WVsdMWC15+SdRzVmMUKSZIkSVL+zJlDsW0OAPO+/92Mw6jWLFZI\nkiRJknJp4O3vAlzGdDayWCFJkiRJyqVtZ7xnbHvx0S48M5tYrJAkSZIk5dbwnnsC0HJnzDiJaqml\nFie57777EttHR0cTj9PW1jat8423Z/kNvysDAwOJ7cuXL0+V6c9//nNin0KhkNjn9ttvT+zT1LRz\nDWq//fbjN7/5zQ5t/f39icfq7OxM7HPHHXck9uno6EjsA9DX17dT28RrNzQ0lHicRYsWJfa59dZb\nU2VatmxZYp9K79+JP4eRkZHE46T5mQAsLD8ZeVfSvJ8q5Z7Y1t7enipTc3Nzqn6S8mflypVZR9il\nYrGY6t80SVJjWvfzG+k6aJ/SztU/gWOfm20g1YQjKyRJkiRJ+bVwIcVCgQLQ9aqXZp1GNWKxQpIk\nSZKUa1te+nLAB23OJhYrJEmSJEm5tvniz49tdx64T4ZJVCsWKyRJkiRJuTeytPQ8vcKgIytmA4sV\nkiRJkqTc6/v9HRS371z9kyyjqAYsVkiSJEmS8m/OHKD03IrO007JNotmnMUKSZIkSVJd2NpzFACF\nzZsyTqKZZrFCkiRJklQXBr71g7GpIHO+8NlMs2hmtWQdYLv58+cn9lmyZEmqY80pDw/alXvuuSex\nT2tra8X2wcHBse37778/VaYNGzYk9rnxxhsT+yxbtiyxz0MPPbRT28te9jJ+//vf79CW5pqvX78+\nsc/mzZsT+2zdujWxD8DixYt3auvo6Nhhf+PGjYnHSZN7ZGQkVaahoaHEPgsWLNipbe7cuTvsp8md\n5r0LMDAwkNhn4nWrZFuFZZ+amnasYa5evTpVpjTnkyRJkqar2NxM08gI7e9/H9te98as42iGOLJC\nkiRJklQ3Nl5wEQBNoyMQ78g4jWZKqpEVIYSPAs8q9z8feCFwGLC23OWCGOOVM5JQkpQL3gskSVIe\nDL7qZIrvfGvpQZvP6WHt39KNBFZ9SSxWhBCOBFbEGA8PIXQCvwOuBd4bY/zRTAeUJGXPe4EkScqT\nob97Em1/+D2FFFPSVZ/STAP5GfCS8nY/sABonrFEkqQ88l4gSZJyY/2qn1OktIzp4qc9Kes4mgGF\nYrGY3KsshHAapSHAI8ByoA14BDg9xrhmsq9bvXp1cfny5dOMKqkBFLIOoOmb6r0A4NZbby2uWLFi\n5kNKyq1CoUCxWPR+MIv19m5M/QGku7uD3t7kh5RnJc/5ZkO2rmULKQBFYM0jyQsapDEbrttM2N1s\n3d0difeB1KuBhBBOAE4FjgWeAqyNMd4SQngPsBI4fbKvvfDCCyu2nXHGGWP7E1dOqCQPq4Gcd955\nvO997xvbX7hwYapMaVYDaW5O/iPlVFcDmZgb0q0G0tbWltinv78/sc/EFSYmM3E1kDPOOGOn90+a\nVTXSrOCR5jgAXV1diX0mrgZSKXdvb2/icdJepzTf31RWA6n0PhkeHk6VKc35PvCBD6Q6lvJrOvcC\ngEMPPXSH/WKxSKFQf59ZzF1b9Zob6jd7veaWNLusuet+ug/ahwLQ8p1vMnziy7KOpCpK+4DN5wBn\nAc+NMa4Hrhn38uWAC9xKUoPzXiBJknJl4UJGm5poGh1l0dvfzFqLFQ0l8c+4IYRFwAXA8THGvnLb\nd0MIB5S79AC3zlhCSVLmvBdIkqQ82nzy6wAoDA5mnETVlmZkxcuALuBbIYTtbV8CvhlC2AwMAKfs\n6gCTTfEY377HHnskBunr60sRt/I0iInSTKeYbPjj+OkoaaZJQLqh9C0tyT+Oe++9d8rHmfh8kk2b\nNiUea82aXU4/3+X5xksz5QRgYGAgsS3NlJp58+Yl9kkz7SZtv0qZJrbtvffeicdJc71h5+kylaR5\nb1b63iY+XyZtpolTYdRwpn0vkCRJqrYt/3EhC754SelBm085lP7f/jHrSKqSxE+ZMcZLgEsqvPSV\n6seRJOWR9wJJkpRXxeZmCiMjtNyb/FxC1Y90T/OTJEmSJCmH1v7hTqC07FzbNy7LNoyqxmKFJEmS\nJKl+dXezfcJ7xzvfmmkUVY/FCkmSJElSXRt427soAoUUzwpUfbBYIUmSJEmqa1vPOhsoTQVpf80r\nsw2jqrBYIUmSJEmqe6NLOwGY++MfZZxE1WCxQpIkSZJU9/quWgWURlc0Xbsq2zCaNosVkiRJkqT6\nd8CBjDY3A7DkX1+ecRhNl8UKSZIkSVJD2PjxT5YetDk4CIODWcfRNLTU4iTd3d2J7Q8//HDicZYu\nXZrqfK2trYl9tm7dmtinvb29YntzuVoHMDo6WrVM8+fPr8px1q5dW7G9UCjs9rE6OjoS+2zYsCGx\nT9rrtGjRosQMW7ZsSTxOmms5nUwTFYvFndq6urp22J/s5zLetm3bUmVqa2tL7JPm5ztc4WnJE9sm\n+/2daGhoKFU/SZIkaaYMvuLV8LY3UwC6HtPNmofXZx1JU+TICkmSJElSwxjZex8ACsUi9PVlnEZT\nZbFCkiRJktQw1v3uT2wf99x56OMzzaKps1ghSZIkSWoo2448BoCmoUH4+Q0Zp9FUWKyQJEmSJDWU\njd/83qOjK05+ZaZZNDUWKyRJkiRJDWfTm94CQNPGjT67og5ZrJAkSZIkNZwtKz88tr342J7sgmhK\nLFZIkiRJkhrStn98JgCt9/4t2yDabRYrJEmSJEkNacO3fjC2Pfc/PryLnsqbQrFYTO4lSZIkSVXQ\n27sx9QeQ7u4Oens3zmScaclzPrM9qnPfZTRt3UoRWPPIhl329bpNze5m6+7uKCT1cWSFJEmSJKlh\nbbzgkwAUgKZrV2UbRqlZrJAkSZIkNazBl7380WVMX/6iTLMoPYsVkiRJkqSGtuFjnxzbnve+MzNM\norQsVkiSJEmSGtrgq08eG12x4NJLMs2idCxWSJIkSZIa3prf/A6AQrFI26c/mdBbWbNYIUmSJElq\nfAccSLGp9BF44bkfyDiMkliskCRJkiTNCmtW/bw0HaRYpOWqK7KOo11oyeKkIYSLgKcDReBtMcYb\ns8ixO0IIPcC3gdvKTX+MMb4lu0S7FkJYAfwQuCjG+KkQwmOArwLNwEPAq2OM27LMWEmF3F8GDgPW\nlrtcEGO8Mqt8kwkhfBR4FqXfqfOBG6mP6z0x9wupg+utxuC9YObV670A6vN+UK/3AvB+IGkWWXEo\nxZYWmoaHWfSm/8vav63OOpEmUfNiRQjhCOBxMcbDQwgHA18EDq91jim6IcZ4YtYhkoQQFgAXA9eM\naz4X+HSM8dshhPOA1wKfzSLfZCbJDfDeGOOPMoiUSgjhSGBF+T3dCfyO0veQ9+tdKfe15Px6qzF4\nL5h59XovgPq8H9TrvQC8H0iafTa99V10fPwjNG3eDKtXw/LlWUdSBVlMAzka+AFAjPF2YEkIYWEG\nORrZNuA44MFxbT3A5eXtK4BjapwpjUq568HPgJeUt/uBBdTH9a6Uuzm7OJplvBfMvHq9F0B93g/q\n9V4A3g8kzTJb33PW2MogXX/3+EyzaHJZTANZDtw0br+33LYhgyy765AQwuXAUuCcGONPsw5USYxx\nGBgOIYxvXjBu6OkjwJ41D5ZgktwAp4cQ3kkp9+kxxjU1D7cLMcYRYFN591TgKuA5dXC9K+UeIefX\nWw3De8EMq9d7AdTn/aBe7wXg/UDS7LTh85ey6PWnUgD49S/h8GdkHUkT5OEBm4WsA6T0Z+Ac4ATg\nNcClIYS2bCNNWb1ccyjN9X1PjPEo4BZgZbZxJhdCOIHS/+SdPuGlXF/vCbnr5nqr4eT692Qc7wXZ\nqYt/n+r1XgDeDyTNLoP/8pJHR1f883GZZlFlWYyseJDSX8+224vSQ6dyLcb4APDN8u5fQgirgb2B\nv2aXarcMhBDmxRi3UMpdF0NrY4zj5ytfTg7n+gKEEJ4DnAU8N8a4PoRQF9d7Ym52nB+e2+uthuC9\nIBt18W9TJfVwP6jXewF4P5A0O2163etp/8LnKRSLMDAA7e1ZR9I4WYysuBo4ESCE8A/AgzHGjRnk\n2C0hhJNCCGeUt5cDewAPZJtqt6wCXlzefjHwkwyzpBZC+G4I4YDybg9wa4ZxKgohLAIuAI6PMfaV\nm3N/vSvlrofrrYbhvSAbuf+3aTJ5//epXu8F4P1A0uy15bwLxkZXdP79wZlm0c5qPrIixvirEMJN\nIYRfAaPAm2udYYouB75eHiLZBrwxxjiYcaaKQgiHAR8D9geGQggnAicBXw4hvB64B/hKdgkrmyT3\nxcA3QwibgQHglOwSTuplQBfwrXHzq18DfCHP15vKub9E/q+3GoD3gplXr/cCqNv7Qb3eC8D7gaRZ\nbNuRxzD3ulUUNqyHwVze0metQrFYTO4lSZIkSVXQ27sx9QeQ7u4OenvzO/Auz/nMll7XsoUUgNHm\nZpqGh3OVbby8Xbfxdjdbd3dH4rOc8vCATUmSJEmSMrHtiCMBaBoZgb6+hN6qFYsVkiRJkqRZa+O3\nfzj27Aqe+MQso2gcixWSJEmSpFlt6wtOKG2sXp1tEI2xWCFJkiRJmtUGLv3q2OiKpU88MNMsKrFY\nIUmSJEma9Yb33R+Apt7ebIMIsFghSZIkSRL9v/0DAAWg41UvzTaMLFZIkiRJkgTAP/4jAHOu/knG\nQWSxQpIkSZIkgOuvp0hpdMWCt74h6zSzmsUKSZIkSZIAWlsZDk8AYN43vp5xmNnNYoUkSZKkSYUQ\nVoQQ/hJCOH0Xfc4PIVxfw1jSjOm/9pdjoyvaT3111nFmLYsVkiRJkioKISwALgau2UWfQ4B/qlko\naaa1tjK6fDkAc6/4YcZhZi+LFZIkSZImsw04DnhwF30+BpxVmzhSbfTddBtQfnbFOyYdVKQZZLFC\nkiRJUkUxxuEY45bJXg8hnAzcAPytVpmkmmhtZWj/xwIw72v/lXGY2akl6wCSJEmS6k8IYSlwCnAM\nsHfar1uyZD4tLc2pz9Pd3bH74Wooz/nMNjVj2e6M0NZGAeh+bg/cdFOWsYA6uW5VYrFCkiRJ0lQc\nBXQDPwfmAAeGEC6KMb5jV1+0bt3m1Cfo7u6gt3fjtELOpDznM9vUTMzW1dREYXQUbr4588z1dN3S\n9E/iNBBJkiRJuy3G+J0Y4yExxqcD/wLcnFSokOrNmvt6x7YXvvSfM0wy+ziyQpIkSVJFIYTDKD1A\nc39gKIRwInA58NcY4/ezzCbVRGsrI9170Nz7MG3XX5t1mlnFYoUkSZKkimKMNwE9Kfr9LU0/qR71\n3fInuvfupAAsfsqh9P/2j1lHmhWcBiJJkiRJ0mRaWxnp7i5t3nsP/O7mjAPNDhYrJEmSJEnahb7b\n/jK23XniCzNMMntYrJAkSZIkKUH/RZ8GoGnjBlqvuTrjNI3PYoUkSZIkSQmGTno1xfL2wpNPyjTL\nbGCxQpIkSZKkFNZctYoi0LRtG61X/zjrOA3NYoUkSZIkSWk85WnQUlpUc9GrXpZxmMZmsUKSJEmS\npJTWXLkKgALQcPV43gAAERVJREFU+p1vZRumgVmskCRJkiQprSf/A6Pt7QAsetPrMg7TuCxWSJIk\nSZK0GzZcfAlQGl0x57OfyjZMg7JYIUmSJEnSbhh6/vEUy8+u6Dj7fRmnaUwWKyRJkiRJ2k1rbroN\nKI2uaPvyF7IN04AsVkiSJEmStLv23JORpUsBWPjeMzMO03gsVkiSJEmSNAV9P/0ZRYCREVq/+pWs\n4zQUixWSJEmSJE3FY/aluHgxBWDRu9+ZdZqGYrFCkiRJkqQpWnvF1aXRFcNDtP74R1nHaRgWKyRJ\nkiRJmqrwBIoLFlAAFr7xdVmnaRgWKyRJkiRJmoa1V64CoGnzZjpOPinjNI3BYoUkSZIkSdNxyBMp\ntrYCMOeqKzIO0xgsVkiSJEmSNE1rHlhLESgAXXssyjpO3bNYIUmSJElSFWx5xasAKBSLzH//ezJO\nU98sVkiSJEmSVAWb/vMzFFvbAFhwyWdo/cF3Mk5UvyxWSJIkSZJUJWvuWT22vfi012aYpL5ZrJAk\nSZIkqVpaWhh47wfGdtvf8sYMw9QvixWSJEmSJFXRlnecydbjXwjA3G9+Dfr7M05UfyxWSJIkSZJU\nZRu/eBnFlpbS6iAHPzbrOHXHYoUkSZIkSTOg/8tfB6AwMsKCd5yecZr6YrFCkiRJkqQZMHzscxlZ\nvicA87/2X7T8768yTlQ/LFZIkiRJkjRD+n5109j24he9MMMk9cVihSRJkiRJM6W9nXVXrQKgMDTI\nnEs/n3Gg+mCxQpIkSZKkGTT8lKcxOncuAAvfe2bGaeqDxQpJkiRJkmbY2pv/NLbdtceiDJPUB4sV\nkiRJkiTNtK4uBp92OACFYpGW3/6/jAPlm8UKSZIkSZJqYP2P/ofROXMAWHzcMRmnyTeLFZIkSZIk\n1cjan/4MgAIw74L/yDZMjlmskCRJkiSpVp5wMCN77AlA+wXnwfBwxoHyyWKFJEmSJEk11Pe728a2\nu/ZbnmGS/LJYIUmSJElSLbW0sOmt7wSgMDRIy/XXZBwofyxWSJIkSZJUY5vfv3Jse8lL/yW7IDll\nsUKSJEmSpAz0/vaPY9tL/+GJGSbJH4sVkiRJkiRlYd/92Hr8CwFouv8+eHh1xoHyw2KFJEmSJEkZ\n2fjFyxhdtIgC0HXYoVnHyQ2LFZIkSZIkZaj/25cDUBjcxrxPXJhxmnywWCFJkiRJUoZG/v7JbD3+\nBADazzsXBgczTpQ9ixWSJEmSJGVs4xe/Orbdte+yDJPkg8UKSZIkSZJyoP/r3wGgMDrK/PPOzThN\ntixWSJIkSZKUA0PHHEuxuRmABZ+4EEZGMk6UHYsVkiRJkiTlxJr714xtL372ERkmyZbFCkmSJEmS\n8qK5mU2nvx2A1lv/QNuPfphxoGxYrJAkSZIkKUc2f/BcRhe0A7Dota+G3t6ME9WexQpJkiRJknJm\n7R/vpFje7nzKikyzZMFihSRJkiRJedPeTv83vgdA05Yt0NmZcaDaslghSZIkSVIODR91DCP7PKa0\n09dH65VXZBuohixWSJIkSZKUU30330Zx7lwAFp12yqxZztRihSRJkiRJObbmrw9BUxOFoUGWHP2s\nrOPUhMUKSZIkSZLyrLkZLrkEgJY/3UrzH/+QcaCZZ7FCkiRJkqS8O/VUhvd/LABLj35mw08HsVgh\nSZIkSVIdWHflqrHtrv33hMHBDNPMLIsVkiRJkiTVg+5uNp57HgCFbVvp3qerYQsWFiskSZIkSaoT\nW99wOn0//PHYfvc+XdDbm2GimWGxQpIkSZKkOjJy+DPY8tJXjO13PmVFhmlmhsUKSZIkSZLqzMCn\nPk//l74GQNOWLXS84dSME1WXxQpJkiRJkurQ0PNfwMC7z6IIzPn+d2i6M2YdqWosVkiSJEmSVKe2\nvOvdDB79bArFIkuf+VQYHs46UlVYrJAkSZIkqY5t+Oo3ASgAnYccmG2YKrFYIUmSJElSPWtpYd3V\nNwDQ1L+ORS86PuNA02exQpIkSZKkOjf8909m86mnAdD2i58x7+KLMk40PRYrJEmSJElqAJvOv5CR\nzi4A2v/9bFp+d1PGiabOYoUkSZIkSQ2i7/a7GVm2DIAlzzkSenszTjQ1FiskSZIkSWogfbfcMbbd\n/cQDYXQ0wzRTY7FCkiRJkqRG0tLC2l/fPLbb8frXZhhmaixWSJIkSZpUCGFFCOEvIYTTK7x2ZAjh\nNyGEX4YQvhhC8POFlBOjBx7Ehk9fAsDcH36Ppjtjxol2j/+YSJIkSaoohLAAuBi4ZpIulwAnxhif\nAXQAz61VNknJtr3k5Qwd+iQAOp/51LqaDmKxQpIkSdJktgHHAQ9O8vphMcb7y9u9QGdNUklKrf+K\n/xnb7npMd4ZJdo/FCkmSJEkVxRiHY4xbdvH6BoAQwp7AscBVtcomKaX58xl4z/sBKAwN0fGGUzMO\nlE6hWCxmnUGSJElSjoUQVgJrYoyfqvDaMkpFivfFGK9OOtbw8EixpaW5+iEl7dpZZ8F555W2L7sM\nTjopyzSFxA4WKyRJkiTtymTFihDCQuA64KwY40/SHKu3d2PqDyDd3R309m7cnag1led8ZpuaRs+2\n9NDH0/zwakYXLGDtXx6ApupMttjdbN3dHYnFCqeBSJIkSZqqjwEXpS1USMpW3+/voAg0bdpE50GP\nyfUDN1uyDiBJkiQpn0IIh1EqSOwPDIUQTgQuB/4K/A/wr8DjQgivK3/J12OMl2SRVVIKTU2s/9YP\nWPzSf6ZpYCOLn/1P9F/zi6xTVWSxQpIkSVJFMcabgJ5ddJlToyiSqmSo5yg2fO5SOt5wKq1//AML\nX/FiNvz3d7OOtROngUiSJEmSNItse9FL2PTBfwdgzjU/pePUf8040c4sVkiSJEmSNMtsOf1tbD71\nNADmXvGD0jMscsRihSRJkiRJs9Cm8y9k8P88HYCmDeuZe9lXMk70KIsVkiRJkiTNUuuvuJpN7/sg\nAB3vfAtdey2FYuoVhmeMxQpJkiRJkmaxzW8/g8GjjgGgMDzMohNfmHEiixWSJEmSJM1667/xPdbe\ncjsAbT+/ge5lCykMbMwsj8UKSZIkSZLE6F57s/7Sr47tdx2wN00Pr84ki8UKSZIkSZIEwOALTmD9\nFy8b21/83KNgdLTmOSxWSJIkSZKkMYPHv5De1f2MLl1K8wP3s/Ckl9Q8g8UKSZIkSZK0o6Ym1l3/\nawDmXPNTupctrOmUEIsVkiRJkiRpJ6PL92TDpz4/tt956ONpuu/empzbYoUkSZIkSapo20tfwfrL\nvjm233nYCgpr1874eS1WSJIkSZKkSQ0e+zx6H3i0QNF18GNpuvsvM3pOixWSJEmSJGnXWlvpXd3P\nyP6PBaDz6U+m0L9uxk5nsUKSJEmSJCVraqLvVzeN7XY9fj8YGpqZU83IUSVJkiRJUuNpaaH3nofH\ndrv37oR7q//QTYsVkiRJkiQpvXnzWPPncQWK/far+iksVkiSJEmSpN1SXLSYdVetKu289a1VP35L\n1Y8oSZIkSZIa3vBTnkbvIxvo7u6A3o1VPbYjKyRJkiRJUq5YrJAkSZIkSblisUKSJEmSJOWKxQpJ\nkiRJkpQrFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJ\nkiRJuWKxQpIkSZIk5YrFCkmSJEmSlCsWKyRJkiRJUq5YrJAkSZIkSblisUKSJEmSJOWKxQpJkiRJ\nkpQrFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJkiRJ\nuWKxQpIkSZIk5YrFCkmSJEmSlCuFYrGYdQZJkiRJkqQxjqyQJEmSJEm5YrFCkiRJkiTlisUKSZIk\nSZKUKxYrJEmSJElSrliskCRJkiRJuWKxQpIkSZIk5UpL1gEkSZIkaaIQwkXA04Ei8LYY4401PPdH\ngWdR+rx0PvBC4DBgbbnLBTHGK0MIJwFvB0aBS2KMl4YQWoEvA/sBI8ApMca7q5SrB/g2cFu56Y/A\nR4GvAs3AQ8CrY4zbMsh2KvDqcU1PAX4LLAA2ldveFWO8KYRwJvASSj/bc2KMV4UQFgFfBxYBA8Ar\nY4x908y0AvghcFGM8VMhhMcwzWsVQngS8Nly9j/EGN9Y5XxfAlqBIeBVMcbVIYQh4JfjvvRoSgMP\nZixfhWxfZpq/A7ubzZEVkiRJknIlhHAE8LgY4+HAqcAna3juI4EV5XM/F/hE+aX3xhh7yv9dGUJY\nAHwQOAboAd4RQlgKvBLojzE+E/gwpWJHNd0wLsdbgHOBT8cYnwXcBbw2i2wxxku35wLOBr5SfumU\ncXlvCiE8Fng58EzgeODjIYRmSh94ry9n+x7w7unkKV+Di4FrxjVX41p9glLx7BnAohDC86qY70OU\nPvAfAXwfeGe5ff24a9gTYxyZyXyTZIPp/w7sVjaLFZIkSZLy5mjgBwAxxtuBJSGEhTU6988o/dUf\noJ/SyIDmCv3+D3BjjHF9jHELpb98P4NS9u+X+6wqt82kHuDy8vYVlD44Zp3tg8C/T/LakcCPY4yD\nMcZe4B7gkAnZtn8f07ENOA54cFxbD9O4ViGENuCx40b5TCdnpXxvAr5b3u4FOnfx9TOZr1K2Smb0\n2jkNRJIkSVLeLAduGrffW27bMNMnLv/Vevu0hVOBqygNZT89hPBO4BHg9HKe3nFf+giw5/j2GONo\nCKEYQmiLMQ5WKeIhIYTLgaXAOcCCGOO2yTLUOBshhKcC95WnLwCcG0LoAm6nNHoiMdu4timLMQ4D\nw+UM203rWpXb1lXoW5V8McZNAOWRJm+mNBIEYG4I4euUplV8N8b48ZnMN8m1g2n8DkwlmyMrJEmS\nJOVdodYnDCGcQKlYcTql5xy8J8Z4FHALsLLCl0yWsZrZ/0ypQHEC8BrgUnb8A/TuZpiJ6/o6Ss8r\nAPhP4MwY4z9ReqbBm1NmqMXPuxrXquo5y4WKrwLXxhi3T8M4AzgNOBY4KYTwlAzyVft3IDGbxQpJ\nkiRJefMgpb/EbrcXpQci1kQI4TnAWcDzykPcr4kx3lJ++XLg0AoZ9y63jbWXHzRYqNbIhRjjAzHG\nb8YYizHGvwCrKU2RmTdZhlplG6cH+FU57/fLOaE07D/VdRvXVm0D07lWlN6DnRX6VtOXgD/HGM/Z\n3hBj/FyMcaA88uIaJlzHWuSb7u/AVLJZrJAkSZKUN1cDJwKEEP4BeDDGuLEWJy6vSnEBcPz21ShC\nCN8NIRxQ7tID3Ar8L/DUEMLiEEI7pbn6Py9n3/7MixcA11Ux20khhDPK28uBPSh9uH1xucuLgZ9k\nka2caS9gIMY4GEIohBBWhRAWl1/uoXTdrgWeH0JoK/ffG/jThGzbv49qW8U0rlWMcQi4I4TwzHL7\ni6qZs7yyxmCM8exxbSGE8PXy9Wwp57ut1vmm+zswlWyFYrFYjeySJEmSVDUhhP8AxqYPxBh/X6Pz\nnkZpiPud45q/RGk6yGZKy2qeEmN8JIRwInAmpaUYL44xfq08jP8LwOMoPajw5BjjfVXK1kFpec/F\nQBulKSG/A/4LmEvpYZWnxBiHap2tnO8w4EMxxueV919KaVWPTcADwKkxxs0hhLcAJ5WzvT/GeE35\nw+5llP763k9p2c7108zyMWB/SsuAPlA+55eZxrUKIRwCfJ7SH/7/N8b4TqZgknzLgK08+myWP8UY\n3xRC+AhwFKXfhctjjB+eyXyTZLsYeA/T+B3Y3WwWKyRJkiRJUq44DUSSJEmSJOWKxQpJkiRJkpQr\nFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJkiRJufL/\nAQMF/DFK6uxaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fbfb5f336d0>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0mTraceback (most recent call last)",
      "\u001b[0;32m<ipython-input-93-001c04aa3eef>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0;36m255.0\u001b[0m                      \u001b[0;31m# normalize max value to one\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0mlabel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrainlabels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m  \u001b[0;31m# get true label as a float\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m     \u001b[0mprobs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnet\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m     \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mloss_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprobs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m     \u001b[0mlosses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.pyc\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    489\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    490\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 491\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    492\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mhook\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    493\u001b[0m             \u001b[0mhook_result\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-90-8f0c415bae84>\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x)\u001b[0m\n\u001b[1;32m     25\u001b[0m     \u001b[0;31m#Apply the network\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m     \u001b[0;31m#Fill-in here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 27\u001b[0;31m     \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     28\u001b[0m     \u001b[0;31m#Advanced fill-in here- uncomment the lines\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     29\u001b[0m     \u001b[0;31m#Let's apply non-linearities to make our three-layer network more expressive\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.pyc\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    489\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    490\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 491\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    492\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mhook\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    493\u001b[0m             \u001b[0mhook_result\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-89-8ddbf4ed7737>\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, x)\u001b[0m\n\u001b[1;32m     21\u001b[0m     \u001b[0;31m#Save x for use in backwards pass gradient computation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     22\u001b[0m     \u001b[0;31m#QUESTION: Why do we need x for the backward pass?\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 23\u001b[0;31m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdetach\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m#detach to avoid interfering with autograd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     24\u001b[0m     \u001b[0;31m#Fill in here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m     \u001b[0;31m#Multiply with weights\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.pyc\u001b[0m in \u001b[0;36m__setattr__\u001b[0;34m(self, name, value)\u001b[0m\n\u001b[1;32m    574\u001b[0m                     \u001b[0mbuffers\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    575\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 576\u001b[0;31m                     \u001b[0mobject\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__setattr__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    577\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    578\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__delattr__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCsAAAJpCAYAAACEi7gMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XmcXFWZ8PFfpTt7urN1JyyBYT8K\nYRwRF1Q0UVHAuCGIg4o7buOgqOgoTGDkRQQRcBd9eREVhUERRHGc0QRQHBHRmQByIOxZIEtn3zrp\n7vePe7uprlTn3nS6+1Z1/76fTz7UPfXUuU9VF327njpLqaurC0mSJEmSpFoxqugEJEmSJEmSylms\nkCRJkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTGotOQJIkSZJGqhDC\nu4C3xxhfVXQuRQkhjAYuAs4C9osxLukjrgR8ErgQmBtj/F2VmH8CvhpjLFW0jwL+APwtxviutO05\nwDeBFmAV8MEY4/+m970f+BjQADwGvC/GuCSEcB7wT2l8t3+JMd5Ydq6JwH3A1THG8yryeC1wC3Bg\njPGxEMIPgeeVhUwG7owxvjmNf0ea4wdjjD8o6+d04NNAE3Bbmt+29DX6AvAmoAu4Mcb4LyGEhjSn\ncnsD5wDLgQsqX0qgOe3jq8CLgdHAv5bnMZgsVkiSJEmSinQT8Kcccd8kKR6sqHZnCGFv4Iw+Hvsh\nYCbwt7K2H5MUGn4WQng98EPgyBDC84HzgefFGJeHEC4Bvgi8LX3c1yqLEBWq3hdCmEBSlGnrbosx\nvq0i5pfA1entz5AUCWJFzGzgy8BzgSVp3mcDnwdOBeYAf09SaLgthHByjPEG4FllfTQBfwFuiDEu\nB24ou+8twKkxxg0hhC8CE4FnA/sAd4UQfh9jfHQXz39AWKyQJEmSVDdCCO8DPkHyWWY58A5gXXp7\n/xjjyjTucmAr8C/AuSQfNMcBPwPOijF2hBAWAr8HTgLeCzwMfA84ABhL8g39l9P+XgN8F9gIXAZ8\nCfj79NvxM0hGBYwj+fb+PTHGLVVy/zTwAWAHybfrn6i4f+Yuzv9PwEeAErAeeHeM8b6+2iv6HQtc\nAhwPjAGujDFemN73GHBV+vocB1xT8Zo8AHwLeA7QAXwvxvjF9LFdwGeBdwGHxxg7Kp9zTp+PMf4h\nhPCvGXHfS+Me6+P+K0hGCFxX3pgWMT5K8nN7btp2JDAlxvgzgBjjzSGE74QQng2sBN6afogHuAP4\ntzxPJITw98ArgWqjD84Dvg98uI/HngCMjTH+PG1aQFIkWVAR+grgtzHGJ9PHXQ5cSVKsOIVkRMe2\n9L7vp203VPRxDsnruby8MYQwjuQ1PCFtOg44J8bYCSwJIfwMeANweR8vwYBxzQpJkiRJdSGEMAP4\nGnBcjPFQYDFwboxxLckHunll4W8ErgfeDrwFeAFwcPrvQ2VxzwOOiDHeSfIB7tEY47NIPnB+IYSw\nXzqE/nvAGTHGZwOHknzbTAjhWJIPia+IMR5AUjj5fJXcXwq8j+RD/2zgpcDJFWF9nb8p7fMF6X2X\nAK/tq73KS3c2cDhwJHAEcHIIofy1mhVjDDHGJ6q8JhcCa2KMIc35w+lz6VZKH9urUBFC+HEI4YEq\n/yZUJhdj/EOVnHeyq7j0g35zjPH6KndfTjJSYl1Z22HAIxVxjwDPijE+FmO8vaz9BOCPZcevCiHc\nGUKIIYRL02JQ9zSVb5JME9lRkd+RJB/8L9vFUzyfsqJIjPGPMcauKnFdJCNMum0EDil7Xg+X3fcw\nZSMq0lxaSIp8V1Tp+73A72OM3X3s6lyDymKFJEmSpLoQY1xB8oG0e02DO4CD0ts3AK8HCCEcBWyP\nMd4DvA64Ksa4Lsa4g2R0xEll3f4y/dYY4J9JvoEnxvgI8BRwIMkHwLExxlvTuK/yzGep1wHXxRiX\npcffqui/24nAL2KMG2KM7SRD9X9aEdPX+beSfGh8bwhhZozx32OMF++ivdLrgG/EGLfFGDeRjJ4o\nz/GWivjy1+S1wDfSnNrSnF+9i8eSxr41xvisKv82V4vfEyGE8cClJCNMKu87HpgaY/xRxV0TSF6/\ncltIi1Blj38HSbFiftp0D3AjMBc4hqQI9un0vg8A96dFnvI+SiTvi4/GGLf38RzmkhR+buv7mfb4\nDXBcCGF2CKGR5HmP6+N57fScSN5jP4wxrq/IYRTJaJ8vlTX/J/CREMK4EML+JGthjGMIOA1EkiRJ\nUl1IRzj8W7q+QAPJ4oIPpnf/DPhyOoy9e1QFwBTgk+lUDUg+A60s67at7PbzSUYz7E8y5WFvkqLE\nVGBNWdyysttTgDeFELo/wI8imWpRqaX8cd0f2kMI5TFVzx9j3B5CeCXJlIvzQwj/C3w4xrior/aK\nc08BLgshXJgejwXu6uM1qDxurXjua0jWLujrsUX4V5IP3+UjCrqLGF8ieT9U2sTOH7onkIwc6H78\nh0mm97wixvgUJNNFyuK3hRAuAz4TQvg28HGSAkalM0iKGDstCFrmNKCyoFJVjPH+EMJHSdbc2EYy\njWdtH8+r13MqO9epVbo+BthYMY3o88BXgP8lGcl0K9CeJ889ZbFCkiRJUr04lWT0xMtijKvSHRve\nBsm3/iGEu0imT7yRZJg7JAWCm2OMX8vR/w9Ihul/K8bYFUJYmravByaVxe1VdnsZydz/T2b0vYqk\nYAFACGH6bpyfGONfgFNCCGNIpnV8C3hJX+0V/S4DvhRjrDoKIsPTwHSge4rI9LRtl0IIPwb+ocpd\nRw3C6IrXAy3pB/ju8z9FMs1mFvC7tCg0HhgTQmgFPkUyJag7vkQyveH+9PhdJNM5XlY2aoYQwiHA\nirJRCY3AduBVwAzg/vRck9L4vYD9gaNDCK9LH9MK/CmE8JYYY/d6FK8lGR2SS4zxeyRTkwghvAzo\nLlA9kD6P/0yPD+1+TmlsSHP7S5Vu5wG/rDjPJpKpId2Pv4pk95FBZ7FCkiRJUr2YATyWFiqmk6xF\nUV5EuAF4PzAmxvg/adtNJN98XxVj3BxC+ACwNf2wV63/P6eFgneSDJ+fRLJo5ugQwpwY40LggyTT\nLwBuBm4IIXwxxrgyhPAGknUPvljR983A9SGEc4ENJCNBvpnn/Ol6B/8KvC3G2B5CuBs4vq/2Ks/r\nJuB9IYRbgU7gc8DdMcZfVYmtdAvJyIAPpWsdnAS8OetBMca35uh7QMQYjyg/DiF0xRi7C0pTytrf\nBcyJz2xdujKEcFqM8VrgncDjMcYHQwj7kmz/+cLyQkXq34C2tDAylmTqxy9ijD8k2ZWj+1znpbmd\nV5lvukDonBjjY+nxDJKf/YOVsdWkBZMbSKYSbSIZWXN1evf1wOdCCNeQLLp6Rnp/t+cAD/SxFsZz\n2Hlx0k8DM2KMnwghHE5SlDkrT557ymKFJEmSpHrxI+AfQwiLSRZDPAe4OYRwaYzxEyRrCXyD5INm\nt5+RLCp5T/qN98OUfVNc4VzgxhDCauDb6b/vkCws+SHg6hDCWpJtIzuBrhjjPen0ioXpnP8VJB9g\ne4kx/ndItsD8K8nQ/VvT5/POnOd/FLgvhNBOUuz4CHBvH+2Vvk6yw8h9JB9g7yb/bg7nAN8MITyQ\nPueLYox3ZTwmt5DsgFL+Tf3CEMIOkhEyAP8RY5ydxt5L8hl2X+CHIYQtwOl7kM9pwHdCCOeTjBbp\n3kb0dJIi1a/LpunsSPP4GMnOGw+STNX5JbsxIqIPs4CVZeuEABBC+A/g70hGZhwWQjiHZKvVG0MI\nNwH/Q1I0+1F38S3GeEMI4Xkk77Mu4Nqy3UW6z/XULvKovO9q4MchhEdI1r84PV3QdtCVurqqFVQk\nSZIkSdWEECaSrAMwJca4Lite0u5zNxBJkiRJyhBC+FMIoXtRwlOBv1mokAaP00AkSZIkKdvHga+H\nED5PsuDmOzPiJe0Bp4FIkiRJkqSa4jQQSZIkSZJUUyxWSJIkSZKkmuKaFZIkSZKGzMqVG3LPQ586\ndQJr1mwezHT2SC3nZ279Y279s7u5tbY2lbJiHFkhSZIkqSY1NjYUncIu1XJ+5tY/5tY/g5GbxQpJ\nkiRJklRTLFZIkiRJkqSaYrFCkiRJkiTVFIsVkiRJkiSppliskCRJkiRJNcVihSRJkiRJqikWKyRJ\nkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTLFZIkiRJkqSaYrFCkiRJkiTVFIsVkiRJkiSppliskCRJ\nkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiRJkmqKxQpJkiRJklRTLFZIkiRJkqSaYrFCkiRJ\nkiTVFIsVkiRJkiSppliskCRJkiRJNcVihSRJkiRJqikWKyRJkiRJUk2xWCFJkiSp5kzdayqUSjQf\ncXDRqUgqgMUKSZIkSTWnobMDgDErVxaciaQiWKyQJEmSJEk1xWKFJEmSJEmqKRYrJEmSJElSTbFY\nIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmSpJpisUKSJEmSJNUUixWSJEmSJKmmWKyQJEmSJEk1xWKF\nJEmSJEmqKRYrJEmSJElSTbFYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmSpJpisUKSJEmSJNUUixWS\nJEmSJKmmWKyQJEmSJEk1xWKFJEmSJEmqKRYrJEmSJElSTWksOgFJkiRJtSuEcDFwLMlnhy/EGH9a\nJeYLwDExxjlDnJ6kYcqRFZIkSZKqCiHMBWbHGI8BjgcurxJzOPCyoc5N0vBmsUKSJElSX24HTklv\nrwUmhhAaKmIuBT430CcuDXSHkuqK00AkSZIkVRVj7AA2pYfvBX6ZtgEQQngXcBvw2JAnJ2lYs1gh\nSZIkaZdCCG8gKVa8uqxtGvBu4FXAvnn7mjp1Ao2NlYMz+lYCWlubcscPNXPrH3Prn5GUm8UKSZIk\nSX0KIbyGZJrH8THGdWV3vQJoBe4AxgIHhxAuizF+fFf9rVmzOdd5W9P/dgGrVm7Y3bSHRGtrEyvN\nbbeZW/8Mp9zyFDYsVkiSJEmqKoQwGbgEeFWMsa38vhjjDcANadwBwNVZhQpJystihSRJkqS+nAq0\nANeHELrbfgssijHeOJgn7iTZDaBrME8iqWZZrJAkSZJUVYzxSuDKHHGPAXMGIwd3BZFGJrculSRJ\nklRzShX/lTSyWKyQJEmSJEk1xWKFJEmSpJrjiAppZLNYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmq\nbU88UXQGkoaYxQpJkiRJNW3Ch95TdAqShpjFCkmSJEmSVFMsVkiSJEmqaaVtW4tOQdIQs1ghSZIk\nqaaVtliskEYaixWSJEmSalqpfVvRKUgaYhYrJEmSJNWcrrLbpc2bC8tDUjEsVkiSJEmqaaVtjqyQ\nRhqLFZIkSZJqWmn7jqJTkDTELFZIkiRJqjml8oPOjqLSkFQQixWSJEmSalpphyMrpJHGYoUkSZKk\nmtbQ4cgKaaSxWCFJkiRJkmqKxQpJkiRJNa2UHSJpmLFYIUmSJEmSaorFCkmSJEmSVFMsVkiSJEmq\nOV2VDZdfWkQakgpisUKSJElSzdl43Gt6HbdeeH5BmUgqgsUKSZIkSTVn6w//fae2aTOamTajuYBs\nJA01ixWSJEmSatP++/c6bEj/tc5ohi9eWEhKkoaGxQpJkiRJtenxx2lvrj6SovXSi4Y4GUlDyWKF\nJEmSpJq1bvGSvu/83lVDl4ikIWWxQpIkSVJdavnUx4pOQdIgsVghSZIkqW6sXLG+Z1vTEtB8xMFF\npiNpkFiskCRJklTTVr//Q72OV938q57bY1euHOp0JA0BixWSJEmSalrn//kiK1esZ+WK9UnDi15M\nZ9n9rTOa4bf/VUhukgaHxQpJkiRJdWf1b37f67j1rScxfUb1nUMk1R+LFZIkSZLqz5FH9hpdAX64\nkYYT/3+WJEmSVJdWp1NDtrW29rS1zmiG/76zwKwkDQSLFZIkSZLq2vr7Hu69hsXrjy8sF0kDw2KF\nJEmSpLq3unvxzVSr61dIdc1ihSRJkqRhYeWK9XSVHbdYsJDqlsUKSZIkScPGqrKCRQlHWEj1ymKF\nJEmSpGFlVcWUkGkzJxeUiaT+slghSZIkadhZuXhJz+2Gri6mO8JCqisWKyRJkiQNP83NrLzn/p7D\nUcDkIw8rLh9Ju8VihSRJkqThadasXotujnn6KfjTHwtNSVI+FiskSZIkDWuryqaEtLz2uAIzkZSX\nxQpJkiRJw1tzM+0tM4Bkh5Cps1qKzUdSJosVkiRJkoa9dfcv7pkO0tjeDuvX7zJeUrEsVkiSJEka\nEVaVLbjZesgs16+QapjFCkmSJEkjw6xZ7Ggc3XPY6voVUs2yWCFJkiRpxFizbDWdZcetM5oLy0VS\n3yxWSJIkSRpRVq9Y36tgMXW/1sJykVSdxQpJkiRJI055waJx2zYmnPL6QvOR1JvFCkmSJEkj0uoV\n63t2CJl420K4fWGB2UgqZ7FCkiRJ0oi16ltX9dxuOdnRFVKtsFghSZIkaeQ66WTWvOs9z4ywOPEV\nhaYjKWGxQpIkSdKItuPiy+kCSsD4u+8uOh1JWKyQJEmSJFZf9KWe0RWT5hxTaC6SLFZIkiRJErzn\njJ7RFePuv6/obKQRz2KFJEmSJAGrF/53T8Fiyt/NLDodaUSzWCFJkiRJAIcf3jMVZPSWLbBkSaHp\nSCOZxQpJkiRJSq1esb6nYDHtqMMLzUUaySxWSJIkSVKZ7VOnAtAAEGOhuUgjlcUKSZIkSSqzLj7e\nM7pi+rHPLzQXaaSyWCFJkiRJFdoPCz2LbfLdbxecjTTyWKyQJEmSpArrf/cnIClWtHz2U8UmI41A\nFiskSZIkqYq2j58NJAWLqW5lKg0pixWSJEmSVEXnv5xDR6kEQMOWLQVnI40sFiskSZIkqQ9tT6/r\nWWyz+ejZheYijSQWKyRJkiRpFzpJpoKMeeKJolORRgyLFZIkSZK0C20PL+3ZGWTKwfsVnY40Ilis\nkCRJkqRdaWqiM705esM62LCh0HSkkcBihSRJkiRlaPvrAz23px+8b4GZSCNDY9EJSJIkSapdIYSL\ngWNJPjt8Icb407L75gJfADqACLwvxthZtaN6t88+tLe0MGbVquQb32XLoDUUnZU0bDmyQpIkSVJV\naTFidozxGOB44PKKkCuBk2OMLwGa0phha939j/TsDNLyD88qNBdpuLNYIUmSJKkvtwOnpLfXAhND\nCA1l9z8vxrgkvb0SmD6UyRWh/bBkNEUJYD8X25QGS6mrqys7SpIkSdKIFkI4Azg2xviOKvftDdwB\nvDDGuHpX/ezY0dHV2Niwq5DaVyo9c9vPU1J/lLICXLNCkiRJ0i6FEN4AvBd4dZX7ZgA/Bz6cVagA\nWLNmc+7ztrY2sXJlDe688fUraf3IGQBsb25m7cNLC05oZzX72mFu/TWccmttbcqMcRqIJEmSpD6F\nEF4DfA44Ica4ruK+ZuBW4JwY46+LyK8Qp7yVjvSL4Ua3MZUGhcUKSZIkSVWFECYDlwDzYoxtVUIu\nBS6LMf5qaDMrXtvDyVIdJWDqfq3FJiMNQ04DkSRJktSXU4EW4PoQerbp/C2wCPgP4HTg0BDC+9L7\nro0xXjnkWRahqQlGjYLOThq2bSs6G2nYsVghSZIkqaq08LCr4sPYocqlJnV00FUqJaMr9p/BmidW\nFJ2RNGw4DUSSJEmS+qmzIdnZpGHr1oIzkYYXixWSJEmS1E9tDz5B9+alE0+eV2gu0nBisUKSJEmS\n+qupiS6ShTbH3X570dlIw4bFCkmSJEnaA2s/8rGeggUxFpyNNDxYrJAkSZKkPdAx/9+ApFgx/djn\nF5uMNExYrJAkSZKkPbR9xl5AOrrinrsLzUUaDixWSJIkSdIeWnfvgz1TQaYf/4qi05HqnsUKSZIk\nSRoA2/faB0g/ZC1bVmguUr1rLDoB7SyE8ADw8hjj07uIuRpYHGO8oMp9rwH+FmN8YhBznAm8MMZ4\n8yCe4/0xxu8M1rlDCDOAHwIHxhgPKWu/GngNsK4s/PQY410hhP2B/wv8HbAR+ESMcUH6uFcAXwIm\nAY8D744xLtnVuSSpCCGEFwCfjzG+ZhD6HvTrQ3qeqtcISSrSuv99gNYZzQBMe+6zaXt6XcYjJPXF\nkRU1KMb4rF0VKnL4OLD/QOXTh7nA6wer8xDCXsDZg3XuEMI04DZgUR8h/5L+HLr/3ZW2Xwn8IsZ4\nGPAe4EchhPEhhInAj4H3pff9HPhWznNJ0pCKMd41GIWK1KBeHwBCCA3AJYN5Dknqr+1NkwEY1dUF\nGzYUnI1UvxxZMUhCCE8Cc2OMi0MIpwLfB6bEGDeHEM4CDgW2A8cDY4ArY4wXpo/tAvYDlgFXAKcA\ni0k+AJ8QY5yTnmZaCOGXwGzgb8DJJB/wXwk8O4RwdozxurKc7gK+GGP8SXr8RuAzMcYXhRDeAFwA\nTEzPdVqMcVUIYTzwbeBYYCvwf4D7ga8BjSGESTHGt4YQTgHmk7ynlgHvjzE+HEI4D9gXeA5wbYzx\n8orX6fVpn2NIRiq8N8b4V+BOYFY6yuTvY4ztafxRlefux48HoAt4I7A3Of+oDiFMBl4BvBkgxvjX\nEMITwJz0eT8SY7wnDb8K+FIIoak/55KkwRRCmAN8N8Z4SPp7uoVnflevAt4QY1weQngM+AZwKkkR\n/FsxxnPLH1/eH/AW+vgdHUL4MMk17HXpcQPwNPBSkt//3wRCGn5mjPHWNO504Jy0/Y/A+4Bbgcnp\nNeIEoAP4DnAAybX14hjjNSGEA0iuJ9cBR8UYX77nr54k7drah5+kZUYzJWDaofvR9tTaolOS6pIj\nKwbPAuCY9PbLgD8DL0iPjwWeAg4HjgSOAE4OIcyr6ONEkj/CDiH5kPuuivtfDbwdOAiYAbwxxngu\nsBR4W3mhInUDvT8svwm4PoRwEEkx5R9jjAeluX8rjfkEMCbGeCBwHMkfoU+l/70hLVTsT/JH4htj\njM8CfkFS4Ch/HidWKVQ0At8jKWwE4CaSaRSQjFp4Ih3V0N79mLQY0HPuiv7GhBAeqPLvhorXgRjj\nmhh3uQn2aSGEP4UQ7g8hfDaEUCL5OayMMW4qi3sYeBZwWHq7u/+NwGrgkBznkqSinQJ8DDgYWEHy\nO7jbMSTXryOAj4QQntNXJ7v6HQ38FJgbQpiQHr8MWBZjfIDkWvDXdGTaicAPQgjT02LDl0iKwoGk\noP7PaX4d6TXiUZJRbwvTa8lrga+kj4WkEPNXCxWShtKOSU0AjOrsLDgTqX45smLwdBcrvg+8iOQb\no5cAC9PjA0jmC28DtoUQrgFOAm4p6+NY4Jb0gy8hhB+RfLPf7Zcxxrb0vnuBWRk53QB8Kv02q0Ty\nB905wOtI/si7N437FvB0GncicDFAjHFJCGFWjHFjCKG83+OABTHGxenxd4GL02IEwB9jjKsqk4kx\n7gghzIgxbk+b7mDngkxuaVHjWf19fJnbSAp5VwP7AP8JLAEeJRldUm4LyR/PpV3cJ0m17vYY4+MA\nIYS/0Hsq4TUxxg5gRQjhDuDFJKP5dkuM8akQwj0k14ybeKZgPpFk6sgpadzi9DyvBcYBd8YYl6W5\nnQbsoOx6F0IYnfZ5avr4x0MIC0iul78FRgM37m6+krQn1j6ytGd0xZRD92PtQ08WnZJUdyxWDJ4F\nwD+HEKYC7SR/MH0thPAs4AlgKnBZCOHCNH4scFdFH1NJPiR3W1px//qy2x1Aw64SijE+kk5PeTHJ\nH28xxvhkCGEK8LJ0OG23dcB0km+k1pb1sbFK163AmrKYdelIhJa0qW0Xaf1zCOGdJM9/HMmUiULF\nGP9f2eGTIYQrgXnARSQ5lptAMnx51C7uk6RaV74CXOX1pPx3+BqSa1N/dY/wuwl4A/AqYDJJwffO\nskL4JJLr5iR6X4O2AlQUzKcDpRhj+XNYQzLiEJIRGOXXS0kaEt3bmDauc5FNqT8sVgySGONjIYRJ\nJGtS/CEtFBxIMlriNyQFgy/FGG/ZRTfrSf5Q67b3AKTW/YfiWOD6tG0Z8F8xxpMrg0MIq3im6EAI\nYRY7Fx+e5pkpL6QFmk6Sec99CiG8GPg08IL09TqOZDpJv4QQxgD/W+Wue6s9t130Mxt4KB31Asn/\nJ9tJ1vJoSedhdxchDiVZn6KR9Fu9tI/JJH/QP7TbT0SSaktL2e1pJNeAyoJG3gLGT4DPhhCOBtpi\njA+lo/A6gKMrC+IhhDNIrpfdx83A+Io+VwGdIYSpMcbuwvl0kmuTJBVm9ZeuoPWTZ1IC+NUv4fgT\ni05JqiuuWTG47gDOBH6fHj9AMs/2NyTfKr0vhNAQQiiFEM4JIRxf8fi7gHnpbhNTSBYuy2M7MKWP\n+24g+SZrHvDvadt/AMema1cQQnhBCOGK9L6bgdPTHPcC/kLyh2v5Of6TZGTGQenxB4Ffxxh3ZOQ5\ng2Ru9BPpHOZ3AhPTURnbgUllU0kyn1+Msb1iB4/uf7kLFakrSeZEdxdeTifZAWR9+ly775sL7EUy\nbWQB8HchhJemfXycZArPJiSpvp0aQhiVbkn6UpJr23Jg7xDCjHTK4NvK4vu8BsUYlwKPAJ8jLZin\n14pfkFw7CCFMCCFcFULYD/gl8JIQwgHpteFbwHvTc4wKITSlj/8P4APp4w8mWQ/jvwbyRZCk3Xb6\nu3uGDE8/vb9rwksjl8WKwbUAeCHJSuSk/30uSfHi68DjwH0kRYxnA7+rePyNwN1AJPk26nryTZO4\nAfhxuutILzHGB0l+7ku75wDHGJcD7wduDCH8jWRxtO7FOS8jKSg8TrLexidjjE8AvwZeEUL4U4xx\nCcnq7DelU0leRvpHY4ZfkYzqeDjt73KSocg3kIyQaAOeShfwLNdz7hznqCqE8Lo012uA/dOFOH+T\n3n06cHwI4UGSn9m1wI/S+z4IvDKEsBi4FDglxrgtxrgFeCvw9fS+FwEfyXEuSap195EUz+8DvhJj\nvC9do+gqkgL270iK8N2yfkffQLJD0vVlbR8CXp7+rryHZHelJ9PryxkkU0IeJLkGfpmkWPI7kmL3\ni0l+N89JH38jyTbSThCXVLgdU5OBZ6WC85DqUamrq/AlArQLIYRSjLErvf0R4FUxxjcVnJYkaQRI\nty59e4yxspguSf22cuWG3B+z/pZnAAAgAElEQVRAWlubWLlyw2Cms0fy5NcyoxmATW8+lS3f7PeM\n591Wy6+dufXPcMqttbUps4bnyIoaFkL4B+DREMLUdDrEScAfCk5LkiRJUk7dC21O+Ml1WaGSylis\nqGExxr+S7D3/Z5Jt4paSTNGQJEmSVAe2zHt9T8FCUn7uBlLjYozzgflF5yFJGnlijAcUnYMk1bvN\nV/2ACTOaKQFTZ7WwZskuN8yTlHJkhSRJkiQNos6GZLfnhvb2gjOR6seQjKw4++yzd1pE56yzzuLL\nX/5yz3Fra2tmP5s3b851vjxxEydOzIwZNWrnWs4ZZ5zBlVdemSuPcs3NzZkxpVL24LBVq7IrsVu2\nbNmprfL1BmhszP7xNzU1ZcZs3bo1M2b79u2ZMdXiPvGJT3DppZf2amtpacnsZ9y4cZkx69aty5VT\nnvfK6tWrex2feeaZXHHFFb3apqarQe/KsmXLcuV00EEHZcasWLEiM6Zygd2PfvSjfPWrX+3VVu3/\ng2rac1x8L7roIkdAjnClUqnXm27RokUceeSRRaXTb+Y9tOo1b6jf3Ac7766uLq8HGlHavvhlWj95\nZjIV5Gc/hTeeVHRKUs3r98iKEMJlIYQ/hBDuDCE8f3cfv9dee/X31IWaMWNG0Sn0S72+3vWa98yZ\nM4tOoV/qNW8VZ0+vBbNnzx6MtAadeQ+tes0b6jf3es1bqlmnv5vuav30M95VZCZS3ehXsSKE8HLg\n0BjjMcB7ga8MaFaSpJrntUCSpPw2H38CkC60uXFjoblI9aC/IyteCfwMIMb4N2BqCCF7noMkaTjx\nWiBJUk6br7muZ1eQqc8+sOh0pJrX32LFXsDKsuOVaZskaeTwWiBJ0m7oHD0agIZt2wrORKp9A7XA\n5i4XSTrrrLOqrj1w8cUXD9Dph9Y555xTdAr9Uq+v9yWXXFJ0Cv1y4YUXFp1Cv1xwwQVFp6D6lblg\n3qJFi3aaC1+50Gu9MO+hVa95Q/3mPlh551lQXBqu2uLjtB60DyVg0pvnsfEntxSdklSz+lusWEbv\nb8/2AZb3FVy5CwUkH5zPPvvsnuN62Q3knHPO6deHuaJ3A6l8vaE+dgO55JJL+NSnPtWrrR52A7nw\nwgv57Gc/26utHnYDueCCC3Yqxg3wbiC5+lLd2K1rAbDT7gJdXV11+cHFvIdWveYN9Zt7veYt1bxJ\nk3qmgoy743ZcuULqW3+ngfwaOBkghHAUsCzGuGHAspIk1QOvBZIk7aatx76s6BSkutCvkRUxxjtD\nCH8OIdwJdAIf2VV8X9tPlrfnqd7n/XZ+7733zozJMxqgr1EMa9eu7bk9ZsyYXDnlMX78+MyYPN9y\n95VTZXue0Qd5ctqwIfuzSZ6RJVD9fVA5kiLPiJA876c8o0YAtuWYU1jttaxsW758l184A9DQ0JAr\np02bNmXG5Mm72s+lMu88/68AzJo1K1echo/dvRZIkiTY+JNbGDejmRLQMqOZVSvWF52SVJP6vWZF\njPEzA5mIJKn+eC2QJGn3dYwZQ2N7e7LY01NPQR9f7kojWX+ngUiSJEmS+mHNklV0rxw27XlHFJqL\nVKssVkiSJEnSEOuYMAGAUTmnuksjjcUKSZIkSRpiax57Ckh2Bml+yfOKTUaqQRYrJEmSJKkA3VNB\nRj/0UKF5SLXIYoUkSZIkFWDTG95MF8noChYvLjgbqbZYrJAkSZKkAmz5zv8DkmLF9BcfVWwyUo2x\nWCFJkiRJBekYMwbwg5lUqXEoTlIqlTLbn3766cx+Ojo6cp2voaEhM2bjxo397qe8vampKVdO69at\ny4xpbMz+cWzbti0zZsaMGVXbW1paeh2vWLEis6/m5ubMmK6ursyYVatWZcZA9dwr3z95Xsu+3nPl\n2tvbc+U0ffr0zJhqP5fK12Xq1KmZ/UyaNClXTnleg/333z8zZunSpTu1bdq0qddxa2trrpw6Oztz\nxUmSJOkZa5asonVG8jd387EvZP0dfyw4I6k2WMCTJEmSpAJ1f+UzJv6t0DykWmKxQpIkSZIKtPas\nTwPpQpv/89dCc5FqhcUKSZIkSSpQx2c+17ONactxLys0F6lWWKyQJEmSpIJ1TJxYdApSTbFYIUmS\nJEkFW/Po8p7RFWM+84lCc5FqgcUKSZIkSaoBXSTrVjRd9Z2iU5EKZ7FCkiRJkmrAlnmv7ylYSCOd\nxQpJkiRJqgGbr/oBkBQrpoQDi01GKpjFCkmSJEmqEZ2lZFxF45rVBWciFatxKE6yYcOGzPZJkyYN\n2PlWrlyZGbPXXntlxvSV95QpU3pujx49OldOHR0dmTGrV2f/Qho1Kru+1Ne5KttnzpyZ2df69esz\nYyZPnpwZs2bNmswYgM7Ozsy2rVu3ZvYzMcdqynliAJ5++unMmGqv+bp163od53mPb9q0KVdOY8eO\nzYzJ8zpt3749s2358uW5cmppackVJ0mSpL61/fhGWk59Y9FpSIVzZIUkSZIk1Yq5rwCSqSDNr55T\naCpSkSxWSJIkSVIN6d7CdPRf7yk0D6lIFiskSZIkqYbsOPhQwF1BNLJZrJAkSZKkGrLuD3/uGV0x\n8Z2nFpqLVBSLFZIkSZJUY7pIRlaMv/XWolORCmGxQpIkSZJqzNYTTig6BalQFiskSZIkqcZs+t51\nz4yu+OiHik5HGnIWKyRJkiSpBnWvWzHhuh8WmodUhMahOMmUKVMy2zs7OzP7Wb58ea7zTZo0KTNm\ny5YtmTGjR4/ObG9ra8uV03777ZcZM2bMmMyY7du3Z8aUStXXDR47dmyv4+nTp2f21dzcnBnT1dWV\nGTNr1qzMGKj+c6nMc/369Zn9rF27NjOmo6MjV07Tpk3LjNm0adNObQ0NDb2O87zn8tqxY0dmzObN\nmzNjqv18J0yY0Ot469atuXLK8/6V6lW133N5fvcNprlz52bGLFy4cPATkSQNmvYXvJCxd/0x2RVk\n0yaYOLHolKQh48gKSZIkSapBG275TyCZCjL1uc8uNhlpiFmskCRJkqQa1dmYDIZvyDFyWBpOLFZI\nkiRJUo1q+/XtPWtXUGXqsTRcWayQJEmSpFo1ezaQTgU54pBic5GGkMUKSZIkSaphPVNBNjuyQiOH\nxQpJkiRJqmFt372GLnhmVxBpBLBYIUmSJEm17MR5PTenHD27wESkoWOxQpIkSZJqXFcyroLG1asL\nzkQaGo1DcZIdO3Zkti9ZsiSzn7Fjx+Y6X0dHR2ZMV1dXZkxf/ZTnvffee+fKady4cZkx48ePz4xp\nbW3NjDn00EOrtp944om9jjdu3JjZ14QJEzJjmpqaMmNWrlyZGQPw6KOP7tS211579TrO8zpNnDhx\nwHJ68sknM2Pa29sz26ZNm5bZz/Lly3Pl1NiY/b/ulClTMmPGjBmT2ZanH4BSqZQrTtLAWLBgQWbM\nwoULcz3utttuy+zrvPPOy5OWJGmQrJn/eaaff05Ssrjnbjjq6KJTkgaVIyskSZIkqcZ1fuSfe7Yw\nnX7CKwvNRRoK/RpZEUKYA/w7cF/atCjG+NGBSkqSVPu8FkiSNLQ6Gxtp2LGDUo5R4lK925NpILfF\nGE8esEwkSfXIa4EkDXMhhIuBY0k+O3whxvjTsvteBVwIdAC/jDF+vpgsR4a2719H6z++GSffaiRw\nGogkSZKkqkIIc4HZMcZjgOOByytCvgK8GXgJ8OoQwuFDnOLI8srjeqaCTD7meYWmIg22PSlWHB5C\nuDmE8LsQwnEDlpEkqZ54LZCk4e124JT09lpgYgihASCEcBDQFmN8MsbYCfwScDGFQdZdrGh8+KFC\n85AGWynPrhiVQgj7Ai8FrgcOAhYAh8QYd94SAVi1alVXS0vLnuQpaXhw1OIwsrvXAoB77723a/Zs\n94eXRrJSqURXV5fXgzoUQjgDODbG+I70+MXAp2KMb0qP3wscHGP87K762bGjo6uxsWHQ8x22zjwT\nvvKV5LZrV6h+ZV4H+rVmRYxxKXBdevhwCOEpYF9g530ngWuuuWantrPOOosvf/nLPccDuXVpnrg8\nW3J2dnbu1PaZz3yGiy66qOc4zxaZUPzWpQcddBCPPPJIr7Z62Lr0da97HT//+c97ta1bty6zn6He\nunT9+vW9jq+44grOPPPMXm21uHVp5fvynHPO4YILLujVluc9ADBqVPZArY997GO5+lJ92N1rAcCR\nRx7Z67irq6sutr3tT2G/FlRuXTpnzpyq25nW+tal9fI+qaZec6/XvDV4QghvAN4LvHoXYbneNGvW\nbM593tbWJlau3JA7fqgVkt85F9Dyla9QArYe8xI23Pyr2sktJ3Prn+GUW2tr9mfIfk0DCSG8LYTw\nyfT2XsBMYGl/+pIk1SevBZI0MoQQXgN8Djghxlj+rdEyYK+y433TNg2y7hL62P++s9A8pMHU391A\nbgauTSusY4AP7WrYb1/fYJe355kmkucbfIBVq1ZlxmzYkF312Xvvvau2Nzc399zed999c+WU51vn\nPM+v/Nx96WsUR2V7Q0P28LuOjo7MmC1btmTG5P1W8uCDD85sW7t2bWY/Tz/9dGZMnvcAwMyZMzNj\nqv3sKt8bed6XeUbXALm+7cozAqXa6KHNm3t/27F9+/ZcOeV9PTWs7Na1oJ7NnTu31/GCBQt2apsz\nZ84QZgTz58/PjKmWU962Si9/+cszY2p9hIak3RdCmAxcArwqxthWfl+M8bEQQnMI4QBgCTAPeNvQ\nZznybD3hBCbcemvRaUiDqr/TQDYArxvgXCRJdcRrgSSNCKcCLcD1IYTutt8Ci2KMNwIfAn6Utl8X\nY3xw6FMceTZ97zrGz2imBIw573O0n/d/ik5JGnD9HVkhSZIkaZiLMV4JXLmL+28Hjhm6jNSti2SR\nkOZvfJVVFis0DO3J1qWSJEmSpALsOPQwwK3WNHxZrJAkSZKkOrPu93f33G5607wCM5EGh8UKSZIk\nSapDPbuC/P72QvOQBoPFCkmSJEmqQ5tPfguQTgXZtKnQXKSBZrFCkiRJkurQ5m98t2d0xdQjDi40\nF2mgWayQJEmSpDrVXaxo2Ly50DykgWaxQpIkSZLq1Oof/eSZg0cfLS4RaYA1DsVJpk2bltm+ffv2\nzH6WL1+e63yTJk3KjJkyZUq/+ylv37ZtW66cmpubM2Pa2toyY9rb2zNjNmzYsFPbPvvsw0MPPdSr\nbfHixZl9jR07NjNm7dq1mTFbtmzJjAFobW3tdXz44Ydz11139WobM2ZMZj+dnZ2ZMTNnzsyV05NP\nPpkZ09i48/9KpVLvjaTyvAYdHR25csrzHs/z/9SECRMy2/L0A9DU1JQrTqpHCxcuzGyrFjOY8pxv\nzpw5vY7nz5/P+eefv1Pc/Pnzd7uv/sbkORfA3LlzM/se6tdckmrSK4+ji2TdimkveyFtT64oOiNp\nQDiyQpIkSZLqWOe4cQCM2ra14EykgWOxQpIkSZLqWNttf+wZXcHvf1dwNtLAsFghSZIkSfXswAN7\nbra86cQCE5EGjsUKSZIkSapznekaaqWMOKleWKyQJEmSpDrXtqytZxvTpnnHFZqLNBAsVkiSJEnS\nMNBdrBhz1x8LzUMaCBYrJEmSJGkY2PF3BwBOBdHwYLFCkiRJkoaBdX/6357RFcybV2Qq0h5rHIqT\nrFu3brfa+zJlypRcce3t7Zkx69evz4yZMGFC1fbOzs6e25s2bcqV06pVqzJjtm7N3hd5xYoVmTHL\nly/fqe3lL385V111Va+2vp5fuaVLl2bGlErZtds85wI4+OCDd2pbvHhxr+MjjjgiV19Znn766Vxx\n++yzT2ZMtffy2LFjex03NTVl9rN69epcOa1duzYzZtasWZkx1X4ulW153rsA48ePzxUnaWAsXLhw\nt2Pmz5/Peeedt1Nctbb+xOQxf/78XHELFizY5THkew3mzp2b63ySNFx0kX4j/YtfFJyJtGccWSFJ\nkiRJw8SOQw575mDz5uISkfaQxQpJkiRJGibW3Xl3z1SQKS85utBcpD1hsUKSJEmShpGudJp2w9Il\nBWci9Z/FCkmSJEkaRjZ8/JOAu4KovlmskCRJkqRhpP0z5/bcHvXdKwvMROo/ixWSJEmSNAyVgKnn\nnF10GlK/WKyQJEmSpOHmoIMAKHV2FpyI1D8WKyRJkiRpuFm0CEhGV4y59KJic5H6oXEoTjJz5szM\n9s059gAulfItEbNx48bMmH322SczZvHixZntnTkrlXniHn300cyYJUuyV/TdsWNH1fYHHnig1/FT\nTz2V2deGDRsyYw477LDMmFmzZmXGABx44IGZbdOmTcvsZ/To0QMSA/Dggw9mxlR7zVevXt3rOM/r\n3dfPrlJDQ0OuuCzt7e2Zbdu3b8/VV97XU1J9Ou+88wakn4ULF+aKW7BgQWbMnDlzMmO6uroyY+bO\nnZsnpdy5S1JNmDCBLpJiRdPFF7H6E58pOiNptziyQpIkSZKGoR37Jl8YjupyKojqj8UKSZIkSRqG\n1v7+7p7bTgVRvbFYIUmSJEnD0YQJdI+paLr4C4WmIu0uixWSJEmSNEy1P/9FAJRyrOEj1RKLFZIk\nSZI0TG34xa97Ftqc+IF3FZyNlJ/FCkmSJEkaAcbd+NOiU5Bys1ghSZIkScPYljedBCSjK6R6YbFC\nkiRJkoaxTd++umcqyOSXPr/odKRcGofiJGvXrs1s37x5c2Y/o0ePznW+mTNnZsasX78+M2bs2LGZ\n7UuXLs2VU1+vQbk8fd1///39Ptddd93V63jChAmZfR111FGZMc95znMyY4477rjMGICGhoad2ip/\nns3NzZn95Hk/bdiwIVdO48ePz4y55557dmp76KGHeh339X4qN2nSpFw5jRqVXWfM8x6fPHnyTm3b\nt2/PjKmmo6MjV5ykkW3hwoW54s4///ye2/Pnz+91XN4+EObMmZMrLm/uklSLuosVjQ/GolORcnFk\nhSRJkiQNc2vPv7CnYMETTxScjZQt18iKEMJs4Cbgshjj10II+wHfBxqA5cA7YozbBi9NSVLRvBZI\nklS/Oj70TzD/s5SAacccRdvSVUWnJO1S5siKEMJE4KvAb8qa/w34eozxWGAx8J7BSU+SVAu8FkiS\nVP92zNofgIbt7QVnImXLMw1kG3AisKysbQ5wc3r758CrBjYtSVKN8VogSVKdW3vPvT23pzz38AIz\nkbJlFitijDtijFsqmieWDfVdAew94JlJkmqG1wJJkoaHjsZkJYDGpUsKzkTatYHYDSRzu94PfOAD\nzJgxY6f2c889dwBOP/TqNe+urq6iU+iXefPmFZ1Cv1x77bVFp9Av9fr+VuFybd2+aNEiZs+e3aut\nXn83mffQGqidP/ak7/7mUK+v+WDlXSrl+nUhaZC0XfNjWk47OTnYvBly7BAoFaG/xYqNIYTx6bds\n+9J7WPBOvv3tb+/Udu655/L5z3++53ggty6dOnVqZsymTZsyY6pdpCvzHsitS5csya5u3nffff06\nV1dX105/HNTD1qXz5s3jlltu6dU2ffr0zH7yvJ8effTRXDnl+dlVbl167bXXctppp/Vqy7N1aZ5t\nUiHf1qXjxo3LjKnclrTy/Q35/6jMs3XpYH7YUCF261oAcOSRR/Y6rva7qR6Y9+A777zzem4P9tal\n1frOyimvenrNy9Vr3pJyeNWrgeRbhqnPPZw18bFC05H60t+tS/8LeHN6+83ArwYmHUlSHfFaIElS\nHeocMwaAhjVtBWci9S1zZEUI4XnApcABwPYQwsnA24CrQwgfAB4HvjeYSUqSiuW1QJKk4aPtp7+g\nZV468vmJJ2D//YtNSKois1gRY/wzyYrvlfKN6we2bKlck23n9h07dmT2k2f4P0BnZ2dmzIYNG/p9\nvvLh/Fu3bs2VUx55phusW7cuM2bKlCm52ivnjVdz0kknZcaEEDJjjj766MwYqP6an3DCCb2O80y9\nufvuuzNj2tryVZIXLVqUGTNx4sTMtjzTMvJOdcrzHs8TU22qU2VbtedWzfLly3PFqT4NxLVAgvxT\nKSqneAzmNLKFCxcOWt+SVJNe8EIgnQoy50WseSRzJqc05Po7DUSSJEmSVKc6xyZfpDVs3FhwJlJ1\nFiskSZIkaYRp+9Z3qc+9ijRSWKyQJEmSpJHmta8Hkqkgza+eU2gqUjUWKyRJkiRpBOoeWTH6r/cU\nmodUjcUKSZIkSRqBdhxyKOCHQtUm35eSJEmSNAKt+687njn46Q3FJSJVYbFCkiRJkkaiCRN6poK0\nfPA9haYiVbJYIUmSJEkj1I6DDik6BamqxqE4yahR1Wsi5e2lUimzny1btuQ637hx4/IllmHp0qWZ\n7a2trXvUV7k8r0Ge8x1xxBFV25/73Of2On7jG9+Y2decOXMyY/bdd9/MmLFjx2bGADQ0NGS27dix\nI7OfPK/lE088kSunpqamzJitW7fu1NbR0dHr+Mknn8zs59BDD82VU57XoLOzMzOm2s+lsm3lypW5\ncmpsHJJfJ5IKsmDBgsyYPNeMobZw4cIBiZGk4Wrtf99Dy4xmSsDE976DTf/3+0WnJAGOrJAkSZKk\nEa17Ksj4n99UaB5SOYsVkiRJkjSCdew7C4ASQM7Rx9Jgs1ghSZIkSSPY2r/c33N72ov+ocBMpGdY\nrJAkSZKkEa4zXU9wVI610aShYLFCkiRJkka41Vd8o2ftCqkWWKyQJEmSpJHu1NOAZN2KiR94V6Gp\nSGCxQpIkSZJE2a4gN/600DwksFghSZIkSQJ2zD6y6BSkHo1DcZJx48Zltk+dOjWzn+3bt+c6X6lU\nGpCYhoaGzPYlS5bkymnr1q2ZMXvttVdmzKRJkzJj5s6dm6v9wAMPzOxrwoQJmTGNjdlvo7Vr12bG\nADz22GO9jmfPns29997bqy3GmNnPxo0bM2P233//XDktW7YsM+app57aqa3yObe0tGT2s3r16lw5\n5emrr/fv7mpqasoVt3nz5gE5n6Sh19U1fGcp93VNlCTtbN1vf0/LjOZkKsg7/5FN3/tR0SlpBHNk\nhSRJkiQJgK70S91xt/6i4Ew00lmskCRJkiQBsOGz59JFstCmVCSLFZIkSZIkANrP/CSQFCsmv/j5\nxSajEc1ihSRJkiSpR+foMQCMXpy9Vpw0WCxWSJIkSZJ6tP3ohp5tTNmypchUNIJZrJAkSZIkPeNl\nc4BkKsiUuS8uNBWNXEOydakkSZKk+hVCmA3cBFwWY/xaxX0fAd4OdAB3xxg/VkCKGiQNjzxcdAoa\noRxZIUmSJKlPIYSJwFeB31S5rxn4FHBsjPGlwOEhhBcNcYoaBJtPe7u7gqhQQzKyYsKECZnta9eu\n7Xc/lcaMGZMZs23btsyYpqamqu2TJk3quT1jxoxcOS1atCgzZubMmZkx++67b2ZMQ0NDrvbGxuwf\n/8MPZ1dSH3vsscyYe+65JzMGYMqUKb2OZ8+ezZ133tmrrb29PbOf5ubmzJjOzs5+5VRNtffB0Ucf\n3et41apVmf10dHTkymnr1q254rJMnjx5p7aurq5ex+PHj8/V18aNGwckJ0kD57zzzsvVNpxV/k6r\nZu7cubn6Wrhw4R5mI9WtbcCJwKer3Nee/psUQtgITADahjA3DZLNl3+DCdf+AIAJH/swmy//RsEZ\naaRxZIUkSZKkPsUYd8QYq66yGGPcCpwPPAI8DvwxxvjgUOanwdNVKlECxv/oh0WnohHINSskSZIk\n9Us6DeSzwGHAeuC3IYTnxBj/p6/HTJ06gcbG6iOBq2ltrT7auVbUcn57nNu73w1XXcWorq4Bf57D\n+nUbRCMpN4sVkiRJkvrr2cAjMcZVACGEO4DnAX0WK9as2Zy789bWJlau3LCnOQ6aWs5vQHK76HJa\nrrqKErDpbe8YsKkgw/51GyTDKbc8hQ2ngUiSJEnqr8eAZ4cQuhe5Ohp4qLh0NFjGp+tXSEPFkRWS\nJEmS+hRCeB5wKXAAsD2EcDJwM/BojPHGEMIlwIIQwg7gzhjjHcVlq4G26Z8+zqSvXea33BpyFisk\nSZIk9SnG+Gdgzi7u/zbw7SFLSENqy7+ez6SvXQbA5DkvZt3COzMeIQ0MC2SSJEmSpD51NiTfcY++\n/96CM9FIYrFCkiRJktSn1Vd8g67ugy1Vd7GVBpzFCkmSJElS397yVgBKwNSjDi82F40YQ7JmxbZt\n2zLbt+So0LW3t+c63/r16zNjOjs7M2NGjapeyylv7+joyJVTV1dXZsyDDz6YGTNp0qTMmHHjxlVt\nf+KJJzIfW6mvn125PHmvW7cu1/kqcz/jjDP41a9+1avt8MOzf0GOGTMmM2bWrFm5cpo8eXJmzJNP\nPrlT25QpU3odt7W1ZfazcePGXDmVSqXMmNbW1syYxsadfwVUtuU5F8DWrVtzxUkaOgsXLux1PH/+\n/J3auttHsgULFuSKmzt3bmZMtddXkoaDzklNNGzcQMPq1UWnohEiV7EihDAbuAm4LMb4tRDC1ST7\nJ3e/Uy+JMf5icFKUJNUCrwWSJI1cbXcvovVZB1Di/7d35+GRVXX+x9+VrbektyRNswgI6BFsxnFQ\nfzLqEBZREWVGcUVHEH+44Q7jgkrDKIyC4ogrj7iM6LgvIOhgs7nPD0FUEA4iyt6Q7nS6O71lq98f\nVR3S6Urf20ml7q3K+/U8PM+9p07u/eSm0pf65px7gCuvgOe/IOtIanCJxYoQwgLgYuCaCS+9N8b4\noxlJJUnKFe8FkiTNckuXUqQ0FaTzta9m7cP9WSdSg0vzzIptwHHAgzOcRZKUX94LJEma5Ubnzweg\nUEyeUi9NV2KxIsY4HGOs9ECJ00MI14YQvhFC6JqBbJKknPBeIEmS+lb9AiiNruCBBzLNosZXSPPg\nR4AQwkpgTXme8tHA2hjjLSGE9wD7xBhPn+xre3t7i2ke9iep4aV7Wqdyazr3AoBbb721uGLFilpE\nlZRThUKBYrHo/WAW6+3dmO4DCNDd3UFv78aZjDMtec43U9m6ly0EYKS9g767p1awmI3XrRoaKVt3\nd0fifWBKq4HEGMfPWR/nwgAAACAASURBVL4c+Oyu+n/hC1/Yqe29730v559//th+mpUSmpubU+Wb\nbBWP8TZt2pTYZ8mSJTu1rVy5kpUrV47tr1mzJlWm2267LbFPmlVM0qwGcvDBB+/U9rnPfY43vOEN\nO7Ttu+++icfKejWQ733ve7zoRS/aoS3NaiBpvrf55WFsSaayGsib3vQmPvOZz+zQdvfddyceJ817\nAKq3GsjEFUvOOOMMLrzwwh3aKq0YUkma1WY+/vGPpzqW6sPu3gsADj300B32i8Vi6hVn8qRecvf0\n9Oywf91111Vc0SLtahiz3VRWA6mX98pE9Zpb0swb7uykZe1amgby+aFZjSPNMyt2EkL4bgjhgPJu\nD3Br1RJJkuqC9wJJkmafdat+MfagTbZUmiEqVUea1UAOAz4G7A8MhRBOpPRE+G+GEDYDA8ApMxlS\nkpQt7wWSJAmAvfce21xy+JNZd8sdGYZRI0ssVsQYb6L0F7OJvpv2JBs3Vh4iNL59r732SjzOQw89\nlOp8aYatp5kCMNl0kvHtaYftL1iwILFPR0dHYp/e3t7EPjfddFOq9jTTEgYHBxP7TJwCUcnatWsT\n+8DO0xIAbr755h320/x899lnn8Q+T3jCE1JlSnOdKr1/J7aNjiY/NTnNVAqY/HdqvDTX6f77709s\n6+pK98zENNdc9asa9wLV3sQpCZO11XK4//iplLtyxBFHjG339PRUzJ3GxKkw05FmukylqSITM0z1\ne5GkvBhta6N5cJDmB10kTDNnStNAJEmSJEmzU9+XvsbYU1KdCqIZYrFCkiRJkpTes58DlJ5bsfTx\nyQ+2l6bCYoUkSZIkabcM77sfAM0pVg+UpsJihSRJkiRpt/T//P+NbXfutTTDJGpUFiskSZIkSbtn\n3ryxzabh4QyDqFFZrJAkSZIk7bahAx/36M6vf5VdEDUkixWSJEmSpN3W/+ubxlYFWXriCzLNosZj\nsUKSJEmSNCVD4QkANA0NZZxEjaalFidpa2tLbG9pSY4ynHIu1F577ZXYZ+vWrYl9Jss0f/78se1i\nsVixz0TNzc2JfdIca8OGDYl91q9fX7H9oYce2mH/gQceSDzW4OBgYp9169Yl9lmyZEliH4COjo7E\ntvHXfzKbNm1K7LMl5ZrQ7e3tiX1aW1t3aluwYMEO+wMDA4nHWb58eapMaX4XJvu9G6/S9zaxLc3v\nJuz8/pKkSlauXLnbX1MsFjnyyCOndL6enp7EPtddd92Ujp32WBPbCoVC1c4nSVlbf/UNdO+3BwWA\nH3wX/vnFWUdSg3BkhSRJkiRpaubNG5sK0nnaKZlGUWOxWCFJkiRJmrLRhYsAcNyYqslihSRJkiRp\nyvruuo8ipWLF/Le/Kes4ahAWKyRJkiRJVTH/65dlHUENwmKFJEmSJGla+j7+qXE7fdkFUcOwWCFJ\nkiRJmpbRV/3r2FSQJc84LOs4agAWKyRJkiRJ0zba3gFA89q1GSdRI7BYIUmSJEmatr6rbwDKq4L8\n+MpMs6j+tdTiJCMjI4nt27ZtSzzOfvvtl+p8mzZtSuyzbNmyxD7r16+v2F4oPLooT9pM/f39iX0e\nfPDBxD5tbW2JfYaGhlK1NzUl16q6u7sT+wwMDCT2SfPzhcrZp5L7oIMOSuyzZs2aVJmGh4cT+3R0\ndOzUNvF9n+a9cs8996TKtHXr1lT9koyOjia2zZ8/P9WxNm/eXJVMkmqvp6enKn2uv/76qvSpplqf\nT5JmtYMOGpsK0nnyK1n7cOXPU1IajqyQJEmSJFXFaFcXAIViMeMkqncWKyRJkiRJVdH3p7uB8lSQ\nVVdnmkX1zWKFJEmSJKlqto+p6DzpJZnmUH2zWCFJkiRJqpqhxwcAmpwKommwWCFJkiRJqpr1v7hx\nbHvev70zwySqZxYrJEmSJElVtX1MxYIvfyHTHKpfFiskSZIkSVW16eTXAeUHbd55R6ZZVJ8sVkiS\nJEmSqmrLRz8+Nrpi6fOOzjSL6pPFCkmSJElS1Y3s8xgAmjZuzDiJ6lFLLU6yadOmxPaRkZHE48yb\nNy/V+ebPn5/Yp7OzM7HPxkl+qZqaHq3xzJ07N1WmvffeO7FPf39/Yp/99tsvsc9k3/9BBx20w/7d\nd9+deKwtW7Yk9lm4cGFinzTfP8CyZct2apv4Pac51tatWxP7pP3ZLV26NLHP5s2bd2obGhpKdfzx\n0uQG2HPPPRP7jI6O7vb5K1m/fn2qfkuWLKnK+SRVT09PT6q26667rirnO/vssxP7FAqFqpwrrUrf\nrySpNtZd+0u6Hr9vaWfV1XDMsdkGUl1xZIUkSZIkqfoWLwZKz63ofM0rss2iumOxQpIkSZI0I7Y+\n+zkAFKYw6lizm8UKSZIkSdKMGPjatylSGl3RftJLso6jOmKxQpIkSZI0Y4qtrQDM/en/ZJxE9cRi\nhSRJkiRpxqz9yn+PLWPK6tVZRlEdsVghSZIkSZo55VVACsDSZz412yyqGxYrJEmSJEkzavjAAwFo\n2rA+4ySqFxYrJEmSJEkzqv+6Xz86FeSDH8wyiupESy1O0lp+oMqu2hcuXJh4nPb29lTna2lJ/ra2\nbt2a2OdJT3rSbrXvysMPP5zYZ+7cuYl9HnnkkcQ+g4ODFdsXLVq0w/7BBx+ceKxt27Yl9knzc1m2\nbFliH4BDDz10p7ajjz56h/3HPe5xicfp6upK7DM8PJwqU7FYTOxz1113JbZ1dnYmHifN9U7bb2Rk\nJLHPgeUK93jLly/fYb9QKKTKlOY9Lqm2enp6UrXV0sqVK6v2dUcccUTi19X6+73++ut3Ov/ENkma\nlebOpVgo0FQswoc+BG85M+tEyjlHVkiSJEmSZtymM99XGl1RLPqgTSVKNbIihPBR4Fnl/ucDNwJf\nBZqBh4BXxxjT/UlYklSXvBdIkqTp2HrGu2n/6IcB6Pz7J7B2dX/GiZRniSMrQghHAitijIcDzwU+\nAZwLfDrG+CzgLuC1M5pSkpQp7wWSJKkahvfdH4Cm0dFsgyj30kwD+RnwkvJ2P7AA6AEuL7ddARxT\n9WSSpDzxXiBJkqat/7d/GNteeGzys4c0exXSPDxwuxDCaZSGAD8nxris3HYg8NUY4z9O9nWrV68u\nTnxon6RZKd3TOpVrU70XANx6663FFStW1CClpLwqFAoUi0XvB7NYb+/G1B9Aurs76O3dOJNxpiXP\n+XKdbVlpcYUisOaRDdmGmSDX162BsnV3dyTeB1KvBhJCOAE4FTgW+PO4lxJPctFFF+3U9pGPfIR3\nv/vdY/tLlixJzFDN1UAmroxRyR577LFT21FHHcW1116bKsd4aVZKWJ3iITNpVgP5/e9/v1PbVVdd\nxXHHHbdDW5oVUbJeDeTf/u3f+OhHP7pDWx5XA5l4zd/xjnfs9L5PsxrI7bffnirTTK0G8ta3vpVP\nfvKTO7RVczWQD33oQ6mOpfyazr0Adv4dLxaLqd9jeVIvuSeuoHH22Wdzzjnn7NTv7LPPrlEiKp4/\nyWS563U1kCOPPLJ2gaaoXt7jkurUBRfAmWf6VyztUqrVQEIIzwHOAp4XY1wPDIQQ5pVf3ht4cIby\nSZJywnuBJEmqijPOGNtc9MynZhhEeZbmAZuLgAuA42OMfeXmVcCLy9svBn4yM/EkSXngvUCSJFXT\naFPpo2jrnTHjJMqrNNNAXgZ0Ad8KIWxvew3whRDC64F7gK/s6gBtbW2J7ZP1Ge++++5LERcOOeSQ\nxD4DAwOJfSabKjJ++kSa6SsAaZ7Z0dramthn8eLFiX3mz59fsf3www/fYT/NNIEFCxYk9tmyZUti\nn7lz5yb2ATjggAMS29JcpzTvpzS5Ae68887EPpWmZUxsu+uuuxKPszvPkEmS5hr09++8XNTEtjTv\nOUj3Hlddm/a9QIKpTzmp5VSV6Zg4XaWnp2dKU18kqdGtvexbdL3yxNLOnXfA45+QbSDlTmKxIsZ4\nCXBJhZeeXf04kqQ88l4gSZKq6phjgdJDr5Y+/1j6/nxvtnmUO6meWSFJkiRJUjWN7LUXAE3rdx7t\nK1mskCRJkiTV3Lrrf0OR8pJiq67OOI3yxmKFJEmSJKn2xj0bbez5FVKZxQpJkiRJUia2/VMPUB5d\nUeHh75q9LFZIkiRJkjKx8TuXs309vEUvODbTLMoXixWSJEmSpMyMzp0LQGu8I+MkyhOLFZIkSZKk\nzPRd9q2x0RWsW5dlFOVISy1OsmTJksT2Rx55JPE4zc3Nqc63Zs2axD6LFi1K7NM/yZyp8e1NTenq\nPe3t7Yl9nvGMZyT2Wb9+fWKfwcHBiu0nnHBC4tdOtHr16t3+mkoWj3t4zq5s2bJlp7bu7u4d9kdG\nRhKPc/fddyf2GRgYSJVpsuuZdKyJbW1tbYnH2bx5c6pMad6/afqsq3AzmHh9t23blipTS0tN/jmR\ntBtWrly5w/7ZZ5+9U9v29tns+uuvT9XvhhtumNKx0h5fkmatcc+tWPKsp7Hu1j9nGkf54MgKSZIk\nSVKmRruXAdD8yMMZJ1FeWKyQJEmSJGWq75e/fXQqyK9+kWUU5YTjtiVJkiRNKoSwAvghcFGM8VMT\nXnsM8N9AG3BzjPENGURUIyhPGy8AS1/xYvrucYTFbOfICkmSJEkVhRAWABcD10zS5WPAx2KMTwNG\nQgj71iycGs62pz4dgKYKz7HT7GOxQpIkSdJktgHHAQ9OfCGE0AQ8C7gcIMb45hjjvbWNp0ay8XtX\njE0FmfPFSzPNouxZrJAkSZJUUYxxOMY42Z+5u4GNwEUhhF+EEM6vYTQ1ojlzKDY1UwDa339m1mmU\nMZ9ZIUmSJGkqCsDewH8CfwOuDCE8P8Z45a6+aMmS+bS0NKc+SXd3x3Qyzrg856vLbP9+Lpx1Fk3D\nw3QvbIM5c2objDq9bjlQ7WwWKyRJkiRNxRrgnhjjXwBCCNcATwR2WaxYt25z6hN0d3fQ27txOhln\nVJ7z1W22//sWus46iwJQnDuXNY9syE+2jDVStjSFDaeBSJIkSdptMcZh4O4QwuPKTYcBMcNIahAj\n3cuA0tAdtm3LNIuyU5ORFc3NlYd5jW9vaUmOMtlxJmpqSq7BPPDAA4l9FixYULH9wQcffb7Qpk2b\nUmVatmxZYp/29vbEPvvtt19inw0bKlcfFy1atMP+unXrqpIpTZ977033rKX+/v6d2u67774d9oeG\nhhKP88gjjyT26ezsTJVpzZo1iX3mzp27U1tbW9sO+1u3bk08zsKFC1NlKhQKiX3Gv08nU+n3buL1\n3ZLyacxpfocl5dM555yT2OeII45I7NPT01OFNOmlyX399ddXpY80W4UQDqO04sf+wFAI4URKD9T8\na4zx+8DbgS+XH7b5R+CKrLKqcay77S66l5X+v3jpkw+m7093Z5xIWfDThSRJkqSKYow3AT27eP0u\n4Jk1C6RZY3ivvWl58AGaUvzhUI3JaSCSJEmSpFxZ97+3UKQ0FWTpIQdkHUcZsFghSZIkScqXOXMY\nLU+RdnTF7GSxQpIkSZKUO323/WVsdMWC15+SdRzVmMUKSZIkSVL+zJlDsW0OAPO+/92Mw6jWLFZI\nkiRJknJp4O3vAlzGdDayWCFJkiRJyqVtZ7xnbHvx0S48M5tYrJAkSZIk5dbwnnsC0HJnzDiJaqml\nFie57777EttHR0cTj9PW1jat8423Z/kNvysDAwOJ7cuXL0+V6c9//nNin0KhkNjn9ttvT+zT1LRz\nDWq//fbjN7/5zQ5t/f39icfq7OxM7HPHHXck9uno6EjsA9DX17dT28RrNzQ0lHicRYsWJfa59dZb\nU2VatmxZYp9K79+JP4eRkZHE46T5mQAsLD8ZeVfSvJ8q5Z7Y1t7enipTc3Nzqn6S8mflypVZR9il\nYrGY6t80SVJjWvfzG+k6aJ/SztU/gWOfm20g1YQjKyRJkiRJ+bVwIcVCgQLQ9aqXZp1GNWKxQpIk\nSZKUa1te+nLAB23OJhYrJEmSJEm5tvniz49tdx64T4ZJVCsWKyRJkiRJuTeytPQ8vcKgIytmA4sV\nkiRJkqTc6/v9HRS371z9kyyjqAYsVkiSJEmS8m/OHKD03IrO007JNotmnMUKSZIkSVJd2NpzFACF\nzZsyTqKZZrFCkiRJklQXBr71g7GpIHO+8NlMs2hmtWQdYLv58+cn9lmyZEmqY80pDw/alXvuuSex\nT2tra8X2wcHBse37778/VaYNGzYk9rnxxhsT+yxbtiyxz0MPPbRT28te9jJ+//vf79CW5pqvX78+\nsc/mzZsT+2zdujWxD8DixYt3auvo6Nhhf+PGjYnHSZN7ZGQkVaahoaHEPgsWLNipbe7cuTvsp8md\n5r0LMDAwkNhn4nWrZFuFZZ+amnasYa5evTpVpjTnkyRJkqar2NxM08gI7e9/H9te98as42iGOLJC\nkiRJklQ3Nl5wEQBNoyMQ78g4jWZKqpEVIYSPAs8q9z8feCFwGLC23OWCGOOVM5JQkpQL3gskSVIe\nDL7qZIrvfGvpQZvP6WHt39KNBFZ9SSxWhBCOBFbEGA8PIXQCvwOuBd4bY/zRTAeUJGXPe4EkScqT\nob97Em1/+D2FFFPSVZ/STAP5GfCS8nY/sABonrFEkqQ88l4gSZJyY/2qn1OktIzp4qc9Kes4mgGF\nYrGY3KsshHAapSHAI8ByoA14BDg9xrhmsq9bvXp1cfny5dOMKqkBFLIOoOmb6r0A4NZbby2uWLFi\n5kNKyq1CoUCxWPR+MIv19m5M/QGku7uD3t7kh5RnJc/5ZkO2rmULKQBFYM0jyQsapDEbrttM2N1s\n3d0difeB1KuBhBBOAE4FjgWeAqyNMd4SQngPsBI4fbKvvfDCCyu2nXHGGWP7E1dOqCQPq4Gcd955\nvO997xvbX7hwYapMaVYDaW5O/iPlVFcDmZgb0q0G0tbWltinv78/sc/EFSYmM3E1kDPOOGOn90+a\nVTXSrOCR5jgAXV1diX0mrgZSKXdvb2/icdJepzTf31RWA6n0PhkeHk6VKc35PvCBD6Q6lvJrOvcC\ngEMPPXSH/WKxSKFQf59ZzF1b9Zob6jd7veaWNLusuet+ug/ahwLQ8p1vMnziy7KOpCpK+4DN5wBn\nAc+NMa4Hrhn38uWAC9xKUoPzXiBJknJl4UJGm5poGh1l0dvfzFqLFQ0l8c+4IYRFwAXA8THGvnLb\nd0MIB5S79AC3zlhCSVLmvBdIkqQ82nzy6wAoDA5mnETVlmZkxcuALuBbIYTtbV8CvhlC2AwMAKfs\n6gCTTfEY377HHnskBunr60sRt/I0iInSTKeYbPjj+OkoaaZJQLqh9C0tyT+Oe++9d8rHmfh8kk2b\nNiUea82aXU4/3+X5xksz5QRgYGAgsS3NlJp58+Yl9kkz7SZtv0qZJrbtvffeicdJc71h5+kylaR5\nb1b63iY+XyZtpolTYdRwpn0vkCRJqrYt/3EhC754SelBm085lP7f/jHrSKqSxE+ZMcZLgEsqvPSV\n6seRJOWR9wJJkpRXxeZmCiMjtNyb/FxC1Y90T/OTJEmSJCmH1v7hTqC07FzbNy7LNoyqxmKFJEmS\nJKl+dXezfcJ7xzvfmmkUVY/FCkmSJElSXRt427soAoUUzwpUfbBYIUmSJEmqa1vPOhsoTQVpf80r\nsw2jqrBYIUmSJEmqe6NLOwGY++MfZZxE1WCxQpIkSZJU9/quWgWURlc0Xbsq2zCaNosVkiRJkqT6\nd8CBjDY3A7DkX1+ecRhNl8UKSZIkSVJD2PjxT5YetDk4CIODWcfRNLTU4iTd3d2J7Q8//HDicZYu\nXZrqfK2trYl9tm7dmtinvb29YntzuVoHMDo6WrVM8+fPr8px1q5dW7G9UCjs9rE6OjoS+2zYsCGx\nT9rrtGjRosQMW7ZsSTxOmms5nUwTFYvFndq6urp22J/s5zLetm3bUmVqa2tL7JPm5ztc4WnJE9sm\n+/2daGhoKFU/SZIkaaYMvuLV8LY3UwC6HtPNmofXZx1JU+TICkmSJElSwxjZex8ACsUi9PVlnEZT\nZbFCkiRJktQw1v3uT2wf99x56OMzzaKps1ghSZIkSWoo2448BoCmoUH4+Q0Zp9FUWKyQJEmSJDWU\njd/83qOjK05+ZaZZNDUWKyRJkiRJDWfTm94CQNPGjT67og5ZrJAkSZIkNZwtKz88tr342J7sgmhK\nLFZIkiRJkhrStn98JgCt9/4t2yDabRYrJEmSJEkNacO3fjC2Pfc/PryLnsqbQrFYTO4lSZIkSVXQ\n27sx9QeQ7u4Oens3zmScaclzPrM9qnPfZTRt3UoRWPPIhl329bpNze5m6+7uKCT1cWSFJEmSJKlh\nbbzgkwAUgKZrV2UbRqlZrJAkSZIkNazBl7380WVMX/6iTLMoPYsVkiRJkqSGtuFjnxzbnve+MzNM\norQsVkiSJEmSGtrgq08eG12x4NJLMs2idCxWSJIkSZIa3prf/A6AQrFI26c/mdBbWbNYIUmSJElq\nfAccSLGp9BF44bkfyDiMkliskCRJkiTNCmtW/bw0HaRYpOWqK7KOo11oyeKkIYSLgKcDReBtMcYb\ns8ixO0IIPcC3gdvKTX+MMb4lu0S7FkJYAfwQuCjG+KkQwmOArwLNwEPAq2OM27LMWEmF3F8GDgPW\nlrtcEGO8Mqt8kwkhfBR4FqXfqfOBG6mP6z0x9wupg+utxuC9YObV670A6vN+UK/3AvB+IGkWWXEo\nxZYWmoaHWfSm/8vav63OOpEmUfNiRQjhCOBxMcbDQwgHA18EDq91jim6IcZ4YtYhkoQQFgAXA9eM\naz4X+HSM8dshhPOA1wKfzSLfZCbJDfDeGOOPMoiUSgjhSGBF+T3dCfyO0veQ9+tdKfe15Px6qzF4\nL5h59XovgPq8H9TrvQC8H0iafTa99V10fPwjNG3eDKtXw/LlWUdSBVlMAzka+AFAjPF2YEkIYWEG\nORrZNuA44MFxbT3A5eXtK4BjapwpjUq568HPgJeUt/uBBdTH9a6Uuzm7OJplvBfMvHq9F0B93g/q\n9V4A3g8kzTJb33PW2MogXX/3+EyzaHJZTANZDtw0br+33LYhgyy765AQwuXAUuCcGONPsw5USYxx\nGBgOIYxvXjBu6OkjwJ41D5ZgktwAp4cQ3kkp9+kxxjU1D7cLMcYRYFN591TgKuA5dXC9K+UeIefX\nWw3De8EMq9d7AdTn/aBe7wXg/UDS7LTh85ey6PWnUgD49S/h8GdkHUkT5OEBm4WsA6T0Z+Ac4ATg\nNcClIYS2bCNNWb1ccyjN9X1PjPEo4BZgZbZxJhdCOIHS/+SdPuGlXF/vCbnr5nqr4eT692Qc7wXZ\nqYt/n+r1XgDeDyTNLoP/8pJHR1f883GZZlFlWYyseJDSX8+224vSQ6dyLcb4APDN8u5fQgirgb2B\nv2aXarcMhBDmxRi3UMpdF0NrY4zj5ytfTg7n+gKEEJ4DnAU8N8a4PoRQF9d7Ym52nB+e2+uthuC9\nIBt18W9TJfVwP6jXewF4P5A0O2163etp/8LnKRSLMDAA7e1ZR9I4WYysuBo4ESCE8A/AgzHGjRnk\n2C0hhJNCCGeUt5cDewAPZJtqt6wCXlzefjHwkwyzpBZC+G4I4YDybg9wa4ZxKgohLAIuAI6PMfaV\nm3N/vSvlrofrrYbhvSAbuf+3aTJ5//epXu8F4P1A0uy15bwLxkZXdP79wZlm0c5qPrIixvirEMJN\nIYRfAaPAm2udYYouB75eHiLZBrwxxjiYcaaKQgiHAR8D9geGQggnAicBXw4hvB64B/hKdgkrmyT3\nxcA3QwibgQHglOwSTuplQBfwrXHzq18DfCHP15vKub9E/q+3GoD3gplXr/cCqNv7Qb3eC8D7gaRZ\nbNuRxzD3ulUUNqyHwVze0metQrFYTO4lSZIkSVXQ27sx9QeQ7u4OenvzO/Auz/nMll7XsoUUgNHm\nZpqGh3OVbby8Xbfxdjdbd3dH4rOc8vCATUmSJEmSMrHtiCMBaBoZgb6+hN6qFYsVkiRJkqRZa+O3\nfzj27Aqe+MQso2gcixWSJEmSpFlt6wtOKG2sXp1tEI2xWCFJkiRJmtUGLv3q2OiKpU88MNMsKrFY\nIUmSJEma9Yb33R+Apt7ebIMIsFghSZIkSRL9v/0DAAWg41UvzTaMLFZIkiRJkgTAP/4jAHOu/knG\nQWSxQpIkSZIkgOuvp0hpdMWCt74h6zSzmsUKSZIkSZIAWlsZDk8AYN43vp5xmNnNYoUkSZKkSYUQ\nVoQQ/hJCOH0Xfc4PIVxfw1jSjOm/9pdjoyvaT3111nFmLYsVkiRJkioKISwALgau2UWfQ4B/qlko\naaa1tjK6fDkAc6/4YcZhZi+LFZIkSZImsw04DnhwF30+BpxVmzhSbfTddBtQfnbFOyYdVKQZZLFC\nkiRJUkUxxuEY45bJXg8hnAzcAPytVpmkmmhtZWj/xwIw72v/lXGY2akl6wCSJEmS6k8IYSlwCnAM\nsHfar1uyZD4tLc2pz9Pd3bH74Wooz/nMNjVj2e6M0NZGAeh+bg/cdFOWsYA6uW5VYrFCkiRJ0lQc\nBXQDPwfmAAeGEC6KMb5jV1+0bt3m1Cfo7u6gt3fjtELOpDznM9vUTMzW1dREYXQUbr4588z1dN3S\n9E/iNBBJkiRJuy3G+J0Y4yExxqcD/wLcnFSokOrNmvt6x7YXvvSfM0wy+ziyQpIkSVJFIYTDKD1A\nc39gKIRwInA58NcY4/ezzCbVRGsrI9170Nz7MG3XX5t1mlnFYoUkSZKkimKMNwE9Kfr9LU0/qR71\n3fInuvfupAAsfsqh9P/2j1lHmhWcBiJJkiRJ0mRaWxnp7i5t3nsP/O7mjAPNDhYrJEmSJEnahb7b\n/jK23XniCzNMMntYrJAkSZIkKUH/RZ8GoGnjBlqvuTrjNI3PYoUkSZIkSQmGTno1xfL2wpNPyjTL\nbGCxQpIkSZKkFNZctYoi0LRtG61X/zjrOA3NYoUkSZIkSWk85WnQUlpUc9GrXpZxmMZmsUKSJEmS\npJTWXLkKgALQcPV43gAAERVJREFU+p1vZRumgVmskCRJkiQprSf/A6Pt7QAsetPrMg7TuCxWSJIk\nSZK0GzZcfAlQGl0x57OfyjZMg7JYIUmSJEnSbhh6/vEUy8+u6Dj7fRmnaUwWKyRJkiRJ2k1rbroN\nKI2uaPvyF7IN04AsVkiSJEmStLv23JORpUsBWPjeMzMO03gsVkiSJEmSNAV9P/0ZRYCREVq/+pWs\n4zQUixWSJEmSJE3FY/aluHgxBWDRu9+ZdZqGYrFCkiRJkqQpWnvF1aXRFcNDtP74R1nHaRgWKyRJ\nkiRJmqrwBIoLFlAAFr7xdVmnaRgWKyRJkiRJmoa1V64CoGnzZjpOPinjNI3BYoUkSZIkSdNxyBMp\ntrYCMOeqKzIO0xgsVkiSJEmSNE1rHlhLESgAXXssyjpO3bNYIUmSJElSFWx5xasAKBSLzH//ezJO\nU98sVkiSJEmSVAWb/vMzFFvbAFhwyWdo/cF3Mk5UvyxWSJIkSZJUJWvuWT22vfi012aYpL5ZrJAk\nSZIkqVpaWhh47wfGdtvf8sYMw9QvixWSJEmSJFXRlnecydbjXwjA3G9+Dfr7M05UfyxWSJIkSZJU\nZRu/eBnFlpbS6iAHPzbrOHXHYoUkSZIkSTOg/8tfB6AwMsKCd5yecZr6YrFCkiRJkqQZMHzscxlZ\nvicA87/2X7T8768yTlQ/LFZIkiRJkjRD+n5109j24he9MMMk9cVihSRJkiRJM6W9nXVXrQKgMDTI\nnEs/n3Gg+mCxQpIkSZKkGTT8lKcxOncuAAvfe2bGaeqDxQpJkiRJkmbY2pv/NLbdtceiDJPUB4sV\nkiRJkiTNtK4uBp92OACFYpGW3/6/jAPlm8UKSZIkSZJqYP2P/ofROXMAWHzcMRmnyTeLFZIkSZIk\n1cjan/4MgAIw74L/yDZMjlmskCRJkiSpVp5wMCN77AlA+wXnwfBwxoHyyWKFJEmSJEk11Pe728a2\nu/ZbnmGS/LJYIUmSJElSLbW0sOmt7wSgMDRIy/XXZBwofyxWSJIkSZJUY5vfv3Jse8lL/yW7IDll\nsUKSJEmSpAz0/vaPY9tL/+GJGSbJH4sVkiRJkiRlYd/92Hr8CwFouv8+eHh1xoHyw2KFJEmSJEkZ\n2fjFyxhdtIgC0HXYoVnHyQ2LFZIkSZIkZaj/25cDUBjcxrxPXJhxmnywWCFJkiRJUoZG/v7JbD3+\nBADazzsXBgczTpQ9ixWSJEmSJGVs4xe/Orbdte+yDJPkg8UKSZIkSZJyoP/r3wGgMDrK/PPOzThN\ntixWSJIkSZKUA0PHHEuxuRmABZ+4EEZGMk6UHYsVkiRJkiTlxJr714xtL372ERkmyZbFCkmSJEmS\n8qK5mU2nvx2A1lv/QNuPfphxoGxYrJAkSZIkKUc2f/BcRhe0A7Dota+G3t6ME9WexQpJkiRJknJm\n7R/vpFje7nzKikyzZMFihSRJkiRJedPeTv83vgdA05Yt0NmZcaDaslghSZIkSVIODR91DCP7PKa0\n09dH65VXZBuohixWSJIkSZKUU30330Zx7lwAFp12yqxZztRihSRJkiRJObbmrw9BUxOFoUGWHP2s\nrOPUhMUKSZIkSZLyrLkZLrkEgJY/3UrzH/+QcaCZZ7FCkiRJkqS8O/VUhvd/LABLj35mw08HsVgh\nSZIkSVIdWHflqrHtrv33hMHBDNPMLIsVkiRJkiTVg+5uNp57HgCFbVvp3qerYQsWFiskSZIkSaoT\nW99wOn0//PHYfvc+XdDbm2GimWGxQpIkSZKkOjJy+DPY8tJXjO13PmVFhmlmhsUKSZIkSZLqzMCn\nPk//l74GQNOWLXS84dSME1WXxQpJkiRJkurQ0PNfwMC7z6IIzPn+d2i6M2YdqWosVkiSJEmSVKe2\nvOvdDB79bArFIkuf+VQYHs46UlVYrJAkSZIkqY5t+Oo3ASgAnYccmG2YKrFYIUmSJElSPWtpYd3V\nNwDQ1L+ORS86PuNA02exQpIkSZKkOjf8909m86mnAdD2i58x7+KLMk40PRYrJEmSJElqAJvOv5CR\nzi4A2v/9bFp+d1PGiabOYoUkSZIkSQ2i7/a7GVm2DIAlzzkSenszTjQ1FiskSZIkSWogfbfcMbbd\n/cQDYXQ0wzRTY7FCkiRJkqRG0tLC2l/fPLbb8frXZhhmaixWSJIkSZpUCGFFCOEvIYTTK7x2ZAjh\nNyGEX4YQvhhC8POFlBOjBx7Ehk9fAsDcH36Ppjtjxol2j/+YSJIkSaoohLAAuBi4ZpIulwAnxhif\nAXQAz61VNknJtr3k5Qwd+iQAOp/51LqaDmKxQpIkSdJktgHHAQ9O8vphMcb7y9u9QGdNUklKrf+K\n/xnb7npMd4ZJdo/FCkmSJEkVxRiHY4xbdvH6BoAQwp7AscBVtcomKaX58xl4z/sBKAwN0fGGUzMO\nlE6hWCxmnUGSJElSjoUQVgJrYoyfqvDaMkpFivfFGK9OOtbw8EixpaW5+iEl7dpZZ8F555W2L7sM\nTjopyzSFxA4WKyRJkiTtymTFihDCQuA64KwY40/SHKu3d2PqDyDd3R309m7cnag1led8ZpuaRs+2\n9NDH0/zwakYXLGDtXx6ApupMttjdbN3dHYnFCqeBSJIkSZqqjwEXpS1USMpW3+/voAg0bdpE50GP\nyfUDN1uyDiBJkiQpn0IIh1EqSOwPDIUQTgQuB/4K/A/wr8DjQgivK3/J12OMl2SRVVIKTU2s/9YP\nWPzSf6ZpYCOLn/1P9F/zi6xTVWSxQpIkSVJFMcabgJ5ddJlToyiSqmSo5yg2fO5SOt5wKq1//AML\nX/FiNvz3d7OOtROngUiSJEmSNItse9FL2PTBfwdgzjU/pePUf8040c4sVkiSJEmSNMtsOf1tbD71\nNADmXvGD0jMscsRihSRJkiRJs9Cm8y9k8P88HYCmDeuZe9lXMk70KIsVkiRJkiTNUuuvuJpN7/sg\nAB3vfAtdey2FYuoVhmeMxQpJkiRJkmaxzW8/g8GjjgGgMDzMohNfmHEiixWSJEmSJM1667/xPdbe\ncjsAbT+/ge5lCykMbMwsj8UKSZIkSZLE6F57s/7Sr47tdx2wN00Pr84ki8UKSZIkSZIEwOALTmD9\nFy8b21/83KNgdLTmOSxWSJIkSZKkMYPHv5De1f2MLl1K8wP3s/Ckl9Q8g8UKSZIkSZK0o6Ym1l3/\nawDmXPNTupctrOmUEIsVkiRJkiRpJ6PL92TDpz4/tt956ONpuu/empzbYoUkSZIkSapo20tfwfrL\nvjm233nYCgpr1874eS1WSJIkSZKkSQ0e+zx6H3i0QNF18GNpuvsvM3pOixWSJEmSJGnXWlvpXd3P\nyP6PBaDz6U+m0L9uxk5nsUKSJEmSJCVraqLvVzeN7XY9fj8YGpqZU83IUSVJkiRJUuNpaaH3nofH\ndrv37oR7q//QTYsVkiRJkiQpvXnzWPPncQWK/far+iksVkiSJEmSpN1SXLSYdVetKu289a1VP35L\n1Y8oSZIkSZIa3vBTnkbvIxvo7u6A3o1VPbYjKyRJkiRJUq5YrJAkSZIkSblisUKSJEmSJOWKxQpJ\nkiRJkpQrFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJ\nkiRJuWKxQpIkSZIk5YrFCkmSJEmSlCsWKyRJkiRJUq5YrJAkSZIkSblisUKSJEmSJOWKxQpJkiRJ\nkpQrFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJkiRJ\nuWKxQpIkSZIk5YrFCkmSJEmSlCuFYrGYdQZJkiRJkqQxjqyQJEmSJEm5YrFCkiRJkiTlisUKSZIk\nSZKUKxYrJEmSJElSrliskCRJkiRJuWKxQpIkSZIk5UpL1gEkSZIkaaIQwkXA04Ei8LYY4401PPdH\ngWdR+rx0PvBC4DBgbbnLBTHGK0MIJwFvB0aBS2KMl4YQWoEvA/sBI8ApMca7q5SrB/g2cFu56Y/A\nR4GvAs3AQ8CrY4zbMsh2KvDqcU1PAX4LLAA2ldveFWO8KYRwJvASSj/bc2KMV4UQFgFfBxYBA8Ar\nY4x908y0AvghcFGM8VMhhMcwzWsVQngS8Nly9j/EGN9Y5XxfAlqBIeBVMcbVIYQh4JfjvvRoSgMP\nZixfhWxfZpq/A7ubzZEVkiRJknIlhHAE8LgY4+HAqcAna3juI4EV5XM/F/hE+aX3xhh7yv9dGUJY\nAHwQOAboAd4RQlgKvBLojzE+E/gwpWJHNd0wLsdbgHOBT8cYnwXcBbw2i2wxxku35wLOBr5SfumU\ncXlvCiE8Fng58EzgeODjIYRmSh94ry9n+x7w7unkKV+Di4FrxjVX41p9glLx7BnAohDC86qY70OU\nPvAfAXwfeGe5ff24a9gTYxyZyXyTZIPp/w7sVjaLFZIkSZLy5mjgBwAxxtuBJSGEhTU6988o/dUf\noJ/SyIDmCv3+D3BjjHF9jHELpb98P4NS9u+X+6wqt82kHuDy8vYVlD44Zp3tg8C/T/LakcCPY4yD\nMcZe4B7gkAnZtn8f07ENOA54cFxbD9O4ViGENuCx40b5TCdnpXxvAr5b3u4FOnfx9TOZr1K2Smb0\n2jkNRJIkSVLeLAduGrffW27bMNMnLv/Vevu0hVOBqygNZT89hPBO4BHg9HKe3nFf+giw5/j2GONo\nCKEYQmiLMQ5WKeIhIYTLgaXAOcCCGOO2yTLUOBshhKcC95WnLwCcG0LoAm6nNHoiMdu4timLMQ4D\nw+UM203rWpXb1lXoW5V8McZNAOWRJm+mNBIEYG4I4euUplV8N8b48ZnMN8m1g2n8DkwlmyMrJEmS\nJOVdodYnDCGcQKlYcTql5xy8J8Z4FHALsLLCl0yWsZrZ/0ypQHEC8BrgUnb8A/TuZpiJ6/o6Ss8r\nAPhP4MwY4z9ReqbBm1NmqMXPuxrXquo5y4WKrwLXxhi3T8M4AzgNOBY4KYTwlAzyVft3IDGbxQpJ\nkiRJefMgpb/EbrcXpQci1kQI4TnAWcDzykPcr4kx3lJ++XLg0AoZ9y63jbWXHzRYqNbIhRjjAzHG\nb8YYizHGvwCrKU2RmTdZhlplG6cH+FU57/fLOaE07D/VdRvXVm0D07lWlN6DnRX6VtOXgD/HGM/Z\n3hBj/FyMcaA88uIaJlzHWuSb7u/AVLJZrJAkSZKUN1cDJwKEEP4BeDDGuLEWJy6vSnEBcPz21ShC\nCN8NIRxQ7tID3Ar8L/DUEMLiEEI7pbn6Py9n3/7MixcA11Ux20khhDPK28uBPSh9uH1xucuLgZ9k\nka2caS9gIMY4GEIohBBWhRAWl1/uoXTdrgWeH0JoK/ffG/jThGzbv49qW8U0rlWMcQi4I4TwzHL7\ni6qZs7yyxmCM8exxbSGE8PXy9Wwp57ut1vmm+zswlWyFYrFYjeySJEmSVDUhhP8AxqYPxBh/X6Pz\nnkZpiPud45q/RGk6yGZKy2qeEmN8JIRwInAmpaUYL44xfq08jP8LwOMoPajw5BjjfVXK1kFpec/F\nQBulKSG/A/4LmEvpYZWnxBiHap2tnO8w4EMxxueV919KaVWPTcADwKkxxs0hhLcAJ5WzvT/GeE35\nw+5llP763k9p2c7108zyMWB/SsuAPlA+55eZxrUKIRwCfJ7SH/7/N8b4TqZgknzLgK08+myWP8UY\n3xRC+AhwFKXfhctjjB+eyXyTZLsYeA/T+B3Y3WwWKyRJkiRJUq44DUSSJEmSJOWKxQpJkiRJkpQr\nFiskSZIkSVKuWKyQJEmSJEm5YrFCkiRJkiTlisUKSZIkSZKUKxYrJEmSJElSrliskCRJkiRJufL/\nAQMF/DFK6uxaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fbfb5f336d0>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "losses = []\n",
    "for t in range(0, tmax):     # iterate over the train steps\n",
    "    i = torch.floor(m*torch.rand(1)).long()    # choose a random example\n",
    "    x = trainimages[i].view(-1)                # flatten image\n",
    "    x = x.float() / 255.0                      # normalize max value to one \n",
    "    label = trainlabels[i]  # get true label as a float\n",
    "    probs = net(x)\n",
    "    loss = loss_func(probs.unsqueeze(0), label)\n",
    "    losses.append(loss.item())\n",
    "    loss.backward()    #Call backwards to do backprop\n",
    "    #Let's do the backwards pass manually!\n",
    "    net.backward(loss.grad)\n",
    "    opt.step()\n",
    "    if t % 1000 == 0:    # display every 1000 time steps\n",
    "        visualize(t, np.array(losses), net.fc.weight[label,:].detach(), x)        \n",
    "        time.sleep(0.01) "
   ]
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "Copy of AI4ALL_neuralnets.ipynb",
   "provenance": [
    {
     "file_id": "1W-iZB_dCIXHHzEbJrHXiK6pqxi2qQ9k8",
     "timestamp": 1533520692325
    }
   ],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}