celebAbb.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "The celebA dataset comprises over 200,000 jpeg images of various sizes, containing 10,000 different celebrities. Labels are provided for 40 facial attributes (hair style etc.) as well as bounding box coordinates for the faces. \n",
    "\n",
    "The original compilers of the dataset used it to train a model to classify the attributes (Y. Sun, X. Wang, and X. Tang. \"Deep learning face representation from predicting 10,000 classes\". In CVPR, 2014.). Another interesting use was to use the detected attributes to score bounding box proposals for likelihood of containing a face (S. Yang, P. Luo, C. Change Loy and X. Tang. \"From Facial Parts Responses to Face Detection: A Deep Learning Approach\". arXiv:1509.06451).\n",
    "\n",
    "Popular approaches for segmenting an image like the YOLO and Faster R-CNN models use very deep models and learn to classify the images in parallel with learning to draw boundary boxes. \n",
    "\n",
    "Here I explore whether it's possible to perform rough boundary box regression alone on a scaled down version of this dataset with a smaller convNet of four layers.\n",
    "\n",
    "First the images and labels were scaled down to a maximum dimension of 128 pixels. For the training, any dimension lower than 128 will be zero padded so that all images are 128x128, so the boundary boxes are offset to reflect that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from PIL import Image\n",
    "\n",
    "dirPath = \"/Users/graeme/PycharmProjects/celebA/img_celeba/\"\n",
    "\n",
    "\n",
    "# create thumbnails\n",
    "size = 128, 128\n",
    "\n",
    "for infile in os.listdir(dirPath):\n",
    "    # outfile = os.path.splitext(infile)[0] + \".thumbnail\"\n",
    "    # if infile != outfile:\n",
    "    try:\n",
    "        im = Image.open(os.path.join(dirPath,infile))\n",
    "        width, height = im.size\n",
    "        im.thumbnail(size, Image.ANTIALIAS)\n",
    "        outpath = os.path.join(dirPath,'thumbs/',infile)\n",
    "        im.save(outpath, \"JPEG\")\n",
    "    except IOError:\n",
    "        print(\"cannot create thumbnail for\", infile)\n",
    "\n",
    "\n",
    "# # create new boundary box label dictionaries for scaled down version\n",
    "def create_dict(file):\n",
    "    dict = {}\n",
    "    handle = open(file)\n",
    "    for line in handle:\n",
    "        contents = line.split()\n",
    "        dict[contents[0]] = contents[1:]\n",
    "    return dict\n",
    "\n",
    "\n",
    "dictBB = create_dict('/Users/graeme/Downloads/Anno/list_bbox_celeba.txt')\n",
    "dictBBnew = dictBB\n",
    "imList = os.listdir(os.path.join(dirPath,'thumbs/'))\n",
    "\n",
    "n = 0\n",
    "for i in imList:\n",
    "    try:\n",
    "        im = Image.open(os.path.join(dirPath, i))\n",
    "        width, height = im.size\n",
    "        im = Image.open(os.path.join(dirPath,'thumbs/',i))\n",
    "        widthNew, heightNew = im.size\n",
    "        xScale = widthNew / width\n",
    "        xOffset = int((128 - widthNew) / 2)\n",
    "        yOffset = int((128 - heightNew) / 2)\n",
    "        yScale = heightNew / height\n",
    "        bbOld = [float(j) for j in dictBB[i]]\n",
    "        bbNew = [int(a * b) for a, b in zip(bbOld, [xScale,yScale,xScale,yScale])]\n",
    "        bbNew = [str(a + b) for a, b in zip(bbNew, [xOffset, yOffset, xOffset, yOffset])]\n",
    "        dictBBnew[i] = bbNew\n",
    "    except:\n",
    "        continue\n",
    "    n += 1\n",
    "    if n%1000==0:\n",
    "        print(n)\n",
    "\n",
    "# save new dictionary\n",
    "fout = dirPath+\"BBdictNew.txt\"\n",
    "fo = open(fout, \"w\")\n",
    "m=0\n",
    "for k, v in dictBBnew.items():\n",
    "    m+=1\n",
    "    bb = ' '.join(v)\n",
    "    fo.write(str(k)+ ' ' +  bb  + '\\n')\n",
    "\n",
    "fo.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model used has 3 convolutional layers, followed by one fully connected layer and then a classier. It can achieve a  classification accuracy upwards of 80% on the CIFAR-10 dataset if trained optimally. The convolutions are 5x5 with 64 features each and the fully connected layer has 32 features. There are 3x3 max-pooling operations (with a stride of 2) preceeding layer 1 and following layers 1 and 3, plus there is a skip connection from the output of layer 1 to layer 4. Batch normalisation is used after every layer, which speeds up learning and reduces hyperparameter selection sensitivity.\n",
    "\n",
    "First the label files are read and lists created in the same order as the image lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "import Model\n",
    "\n",
    "# model parameters\n",
    "convWidth = 5\n",
    "conv1feat = 64\n",
    "conv2feat = 64\n",
    "conv3feat = 64\n",
    "linear1feat = 32\n",
    "batchSize = 50\n",
    "numEpochs = 1\n",
    "etaIn = 0.1\n",
    "gammaIn = 0.9\n",
    "\n",
    "# data parameters\n",
    "dirPath = \"/Users/graeme/PycharmProjects/celebA/img_celeba/\"\n",
    "logPath = \"/Users/graeme/PycharmProjects/celebA/log\"\n",
    "cropWidth = 128#int(176/4)\n",
    "cropHeight = 128#int(208/4)\n",
    "imChanels = 3\n",
    "numLabels = 44\n",
    "min_fraction_of_examples_in_queue = 0.01\n",
    "num_preprocess_threads = 16\n",
    "NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN = 160000\n",
    "NUM_EXAMPLES_PER_EPOCH_FOR_EVAL = 19000\n",
    "\n",
    "\n",
    "def create_dict(file):\n",
    "    dict = {}\n",
    "    handle = open(file)\n",
    "    for line in handle:\n",
    "        contents = line.split()\n",
    "        dict[contents[0]] = contents[1:]\n",
    "    return dict\n",
    "\n",
    "\n",
    "# create label dictionaries\n",
    "dictBB = create_dict('/Users/graeme/PycharmProjects/celebA/BBdictNew.txt')\n",
    "dictAtt = create_dict('/Users/graeme/Downloads/Anno/list_attr_celeba.txt')\n",
    "for key, values in dictAtt.items():\n",
    "    try:\n",
    "        pos = key.find('.jpeg')\n",
    "        # valueList = values.split(',')\n",
    "        for element in range(len(values)):\n",
    "            if values[element] == '-1':\n",
    "                values[element] = 0\n",
    "            else:\n",
    "                values[element] = 1\n",
    "        dictAtt[key] = values#','.join(valueList)\n",
    "    except:\n",
    "        continue\n",
    "dictEval = create_dict('/Users/graeme/Downloads/list_eval_partition.txt')\n",
    "\n",
    "# count examples in each set\n",
    "trainExam = sum(i[0] == '0' for i in dictEval.values())\n",
    "valExam = sum(i[0] == '1' for i in dictEval.values())\n",
    "testExam = sum(i[0] == '2' for i in dictEval.values())\n",
    "\n",
    "# create file and label lists using dictionaries\n",
    "imList = os.listdir(os.path.join(dirPath,'thumbs/'))\n",
    "del imList[30088]  # .DS file\n",
    "trainFiles=[]\n",
    "trainLabels=[]\n",
    "evalFiles=[]\n",
    "evalLabels=[]\n",
    "testFiles=[]\n",
    "testLabels=[]\n",
    "for i in imList:\n",
    "    bb = [float(j) for j in dictBB[i]]\n",
    "    partition = dictEval[i]\n",
    "    if partition[0] == '0':\n",
    "        trainFiles.append(os.path.join(dirPath,'thumbs/', i))\n",
    "        trainLabels.append(bb + dictAtt[i])\n",
    "    elif partition[0] == '1':\n",
    "        evalFiles.append(os.path.join(dirPath,'thumbs/', i))\n",
    "        test = dictBB[i]\n",
    "        evalLabels.append(bb + dictAtt[i])\n",
    "    elif partition[0] == '2':\n",
    "        testFiles.append(os.path.join(dirPath,'thumbs/', i))\n",
    "        testLabels.append(bb + dictAtt[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After preparing the data, the model is loaded from the Model class where it is defined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## model training\n",
    "images = tf.placeholder(dtype=tf.float32, shape=[None,cropHeight,cropWidth,imChanels])\n",
    "labels = tf.placeholder(dtype=tf.int64, shape=[None,numLabels])\n",
    "\n",
    "# create model\n",
    "model = Model.Model(images,labels, convWidth, conv1feat, conv2feat, conv3feat, linear1feat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The classifier will be trained to output a vector giving the starting x & y coordinates and the height and width of the boundary box. The loss fuction used will add a penalty for incorrect end coordinates, so as to discourage the model from prioritising learning the boundary box size over position. \n",
    "\n",
    "The loss function used is mean squared error and optimised using Nesterov's accelerated gradient method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model._detect_bb(cropWidth,cropHeight,batchSize)\n",
    "model._use_nesterov(eta=etaIn,gamma=gammaIn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, the datasets are loaded using tensorflow functionality to create a preprocessing queue. The images are normalised, which makes learning easier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "restoreMode = False\n",
    "\n",
    "def _readFile2(fileName, labels):\n",
    "    # function to decode image, crop and normalise, can be used as a mapping method for a dataset\n",
    "    imageFile = tf.read_file(fileName)\n",
    "    image = tf.image.decode_image(imageFile,channels=imChanels)\n",
    "    if restoreMode is False:\n",
    "        image = tf.image.per_image_standardization(image)\n",
    "    image = tf.image.resize_image_with_crop_or_pad(image, cropHeight, cropWidth)\n",
    "    return image, labels\n",
    "\n",
    "\n",
    "def getData2(fileNames, fileLabels, batchSize, testFlag):\n",
    "    # function to create datasets for efficient processing, can be queued up by tf\n",
    "    if testFlag > 0:\n",
    "        num_examples_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_EVAL\n",
    "    else:\n",
    "        num_examples_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN\n",
    "    min_queue_examples = int(num_examples_per_epoch * min_fraction_of_examples_in_queue)\n",
    "\n",
    "    dataset = tf.data.Dataset.from_tensor_slices((fileNames, fileLabels))\n",
    "    dataset = dataset.map(_readFile2,num_parallel_calls=num_preprocess_threads)\n",
    "    dataset = dataset.shuffle(buffer_size=min_queue_examples)\n",
    "    dataset = dataset.batch(batchSize)\n",
    "    return dataset\n",
    "\n",
    "\n",
    "# create datasets\n",
    "with tf.device('/cpu:0'):\n",
    "    trainData = getData2(trainFiles, trainLabels, batchSize, testFlag=0)\n",
    "    valData = getData2(evalFiles, evalLabels, batchSize, testFlag=1)\n",
    "    testData = getData2(testFiles, testLabels, batchSize=testExam, testFlag=2)\n",
    "\n",
    "# create iterators\n",
    "iteratorTrain = tf.data.Iterator.from_structure(trainData.output_types, trainData.output_shapes)\n",
    "iteratorVal = tf.data.Iterator.from_structure(valData.output_types, valData.output_shapes)\n",
    "iteratorTest = tf.data.Iterator.from_structure(testData.output_types, testData.output_shapes)\n",
    "next_batchTrain = iteratorTrain.get_next()\n",
    "next_batchVal = iteratorVal.get_next()\n",
    "next_batchTest = iteratorTest.get_next()\n",
    "training_init_op = iteratorTrain.make_initializer(trainData)\n",
    "validation_init_op = iteratorVal.make_initializer(valData)\n",
    "test_init_op = iteratorTest.make_initializer(testData)\n",
    "\n",
    "# initialise tf\n",
    "sess = tf.InteractiveSession()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we train the model. Iterations are limited to 100 as we shall see the model converges by then."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step 0, Train Loss 0.237875, Val Loss 0.191324\n",
      "Step 10, Train Loss 0.102317, Val Loss 0.147486\n",
      "Step 20, Train Loss 0.0600575, Val Loss 0.116429\n",
      "Step 30, Train Loss 0.0560918, Val Loss 0.0716314\n",
      "Step 40, Train Loss 0.0278722, Val Loss 0.0412231\n",
      "Step 50, Train Loss 0.031112, Val Loss 0.0355213\n",
      "Step 60, Train Loss 0.037667, Val Loss 0.0409039\n",
      "Step 70, Train Loss 0.0231391, Val Loss 0.035265\n",
      "Step 80, Train Loss 0.0361555, Val Loss 0.0314031\n",
      "Step 90, Train Loss 0.0245483, Val Loss 0.0317307\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8XHW9//HXZ2ayNmvbtE2aNt1bCt3LvlssoEJRFgFF\nFJWLV3FBrz/cUHG97qKgIKDgVUDLVrCAgJSd0n3f9yRt2uxJmz3f3x9zJp0kM0naZJqQvJ+PRx+Z\nOdt8TwbOO9/lfI855xAREemIr7cLICIifZ/CQkREOqWwEBGRTiksRESkUwoLERHplMJCREQ6pbAQ\nEZFOKSxERKRTCgsREelUoLcL0FOGDh3qxowZ09vFEBF5T1mxYkWxcy6rs+36TViMGTOG5cuX93Yx\nRETeU8xsT1e2UzOUiIh0SmEhIiKdUliIiEinFBYiItIphYWIiHRKYSEiIp1SWIiISKcGfFhU1zXy\nqxe3snpfeW8XRUSkzxrwYVHf2MxdL29j9d6y3i6KiEifNeDDIiEQ/BXUNzX3cklERPouhYUXFnUN\nCgsRkWgGfFgE/D78PqOuUWEhIhLNgA8LgHi/j7rGpt4uhohIn6WwABLifKpZiIh0QGFBsN9CfRYi\nItEpLICEgF+joUREOqCwwKtZqM9CRCQqhQUQr2YoEZEOKSwI1SwUFiIi0SgsCPZZqBlKRCQ6hQXB\nobP1qlmIiESlsEDNUCIinVFYEGqGUliIiESjsCA0Gkp9FiIi0SgsUDOUiEhnFBaoGUpEpDMKCzQa\nSkSkMwoLgs1Q9U3NNDe73i6KiEifpLAg2MENerSqiEg0CguCfRagR6uKiESjsCDsOdya8kNEJCKF\nBeFhoZqFiEgkCgsgIc5rhlJYiIhEpLBAzVAiIp1RWHB0NJRqFiIikSksCKtZaDSUiEhECgvChs6q\nGUpEJCKFBUdrFpryQ0QkMoUFkBinPgsRkY7ENCzM7BIz22Jm283s9gjrbzOzjWa21sxeNrO8sHU3\nmtk279+NsSxnvF9DZ0VEOhKzsDAzP3A3cCkwFbjOzKa22WwVMNc5Nx1YCPzM23cw8F3gdOA04Ltm\nlhmrsibEaeisiEhHYlmzOA3Y7pzb6ZyrBx4FFoRv4Jx7xTl3xHv7DpDrvb4YeNE5V+qcKwNeBC6J\nVUE1GkpEpGOxDIuRwL6w9/nesmg+DTx3nPt2y9HRUAoLEZFIAr1dAAAz+zgwFzj/GPe7GbgZYPTo\n0cf9+fEaDSUi0qFY1iwKgFFh73O9Za2Y2UXAt4DLnXN1x7Kvc+4+59xc59zcrKys4y6o32fE+U19\nFiIiUcQyLJYBE81srJnFA9cCi8I3MLNZwL0Eg+Jg2KoXgPlmlul1bM/3lsVMvN+nZigRkShi1gzl\nnGs0sy8QvMj7gQedcxvM7E5guXNuEfBzIAX4p5kB7HXOXe6cKzWzHxAMHIA7nXOlsSorBGeeVc1C\nRCSymPZZOOcWA4vbLLsj7PVFHez7IPBg7ErXWkLAp9FQIiJR6A5uT0LAp2dwi4hEobDwJAT8qlmI\niEShsPAkxPnUZyEiEoXCwqPRUCIi0SksPMGahcJCRCQShYUnIaChsyIi0SgsPAkBn6b7EBGJQmHh\nSQioGUpEJBqFhSdeN+WJiESlsPCoz0JEJDqFhUfNUCIi0SksPBo6KyISncLCkxDw09TsaNT8UCIi\n7SgsPKHncGsyQRGR9hQWntCjVTUiSkSkPYWFJyHgB1C/hYhIBAoLT6gZSsNnRUTaU1h4EuK8PgvV\nLERE2lFYeNQMJSISncLCE69mKBGRqBQWngSNhhIRiUph4Tnawa2wEBFpS2HhOdpnoWYoEZG2FBae\n0Ggo1SxERNpTWHjUDCUiEp3CwhOvsBARiUph4Wnps2hQn4WISFsKC4+aoUREolNYeFqmKFdYiIi0\no7DwmBnxerSqiEhECoswCX6f7rMQEYlAYRFGz+EWEYlMYREmIeDX3FAiIhEoLMIkBNQMJSISicIi\nTHzAp9FQIiIRRA0LMxvUwbqRsSlO70qI86vPQkQkgo5qFq+HXpjZv9useyY2xeldGg0lIhJZR2Fh\nYa+zOlgX/QBml5jZFjPbbma3R1h/npmtNLNGM7uqzbomM1vt/VvUlc/rLo2GEhGJLNDBOhfldaT3\n7ZiZH7gbeD+QDywzs0XOuY1hm+0FPgl8LcIhapxzMzv7nJ6UEPBRUq2wEBFpq6OwGGZmXyRYiwi9\nxnvftqYRyWnAdufcTgAzexRYALSEhXNut7euT1yhEwJ+NUOJiETQUTPUnwmGwtCw16H3f+nCsUcC\n+8Le53vLuirRzJab2TtmdkWkDczsZm+b5YcOHTqGQ0eWEPBR39QncktEpE+JWrNwzn3nRBYkgjzn\nXIGZjQP+Y2brnHM7wjdwzt0H3Acwd+7cTpvGOhMf8OmmPBGRCDoaOnuTmU3wXpuZ3WdmJV6H9Iwu\nHLsAGBX2Ptdb1iXOuQLv505gCTCrq/serwRNJCgiElFHzVC3AXu81x8FTgWmAt8EfteFYy8DJprZ\nWDOLB64FujSqycwyzSzBez0UOJuwvo5YCd5noT4LEZG2OgqLRudcg/f6MuAh51yRc+55IKWzAzvn\nGoEvAC8Am4B/OOc2mNmdZnY5gJmdamb5wNXAvWa2wdv9JGC5ma0BXgF+2mYUVUyEahbOdbtFS0Sk\nX+lw6KyZDQfKgXnAT8LWJXXl4M65xcDiNsvuCHu9jGDzVNv93gKmdeUzelJCwIdz0NjsiPN36VYS\nEZEBoaOw+B6w0nv9nHNuPYCZnQvsinG5ekXLc7gbm4nza9osEZGQjkZDPW1mzwHpzrnwcamrCfY/\n9DvxoedwNzSRktBRjoqIDCxRr4ihfgXvdaRNTsgUHCdS6DncGhElItJaR38+P0WwFrHeex+eGI7+\nGBZxCgsRkUg6CotrCDY3TQGeBB51zvXLvoqQo30WGj4rIhIuai+uc26hc+4qgiOhCoDfm9lrXgd3\nvxRqhtIDkEREWuvKkJ/DQBFwEMigC/dYvFfFq89CRCSijjq4zyPYDHUWwek27nXOvXOCytUrWpqh\nND+UiEgrHfVZLAHWAq8S7NC+xsyuCa10zt0W26KdeEdHQ6nPQkQkXEdh8dkTVoo+QqOhREQi6+im\nvAdOZEH6glAzlDq4RURa05wWYdQMJSISmcIijEZDiYhEprAI01Kz0GgoEZFWOp0tz3v40E3AmPDt\nnXM3x65YvSPUZ1HboGYoEZFwXZla9WngHeANoF9fReP8Rrzfx+H6fn2aIiLHrCthMcg599WYl6QP\nMDPSkgJU1jZ0vrGIyADSlT6L58xsfsxL0kekJcZRUaOwEBEJ15WwuAV43syqzazUzMrMrDTWBest\naUlxVCosRERa6Uoz1NCYl6IPSUtSzUJEpK2OJhKc6JzbBpwcZZO1sSlS70pLDLCv9EhvF0NEpE/p\nqGZxO/Bp4O4I6xxwXkxK1MvS1QwlItJOR3NDfdr72W8fdhRJWlIclbUNOOeiPXtcRGTA6UqfBWY2\nBZgKJIaWOef+HqtC9aa0xDgamhw1DU0kx3fp1yMi0u915Q7ubwPzCT6L+wXgYoI36PXLsEhPigOg\nsqZRYSEi4unK0NmPAhcC+51zNwAzgEExLVUvSksKBoRuzBMROaorYVHjnGsCGs0sFTgA5MW2WL0n\nLTFYs9DwWRGRo7rSzrLKzDKAB4HlQCXwbkxL1YuONkMpLEREQjoMCwsOB/qec64cuNvMXgDSnHMr\nT0jpekFaKCzUDCUi0qLDsHDOOTN7ETjFe7/9hJSqF6UlBn8lFUcUFiIiIV3ps1htZrNiXpI+4mjN\norGXSyIi0nd0NN1HwDnXCMwClpnZDuAwYAQrHbNPUBlPqDi/j+R4v/osRETCdNQM9S4wG7j8BJWl\nz9A05SIirXUUFgbgnNtxgsrSZ6R7U36IiEhQR2GRZWa3RVvpnPtVDMrTJ6QlBaisUZ+FiEhIR2Hh\nB1LwahgDSVpiHAcqa3u7GCIifUZHYbHfOXdndw5uZpcAvyUYPPc7537aZv15wG+A6cC1zrmFYetu\nBL7tvf2hc+6h7pTlWKQlxbGlqOpEfZyISJ/X0dDZbtUozMxP8FkYlxKcsfY6M5vaZrO9wCdpMymh\nmQ0GvgucDpwGfNfMMrtTnmNxop5pseVAFQvufpOyw/Ux/ywRke7oKCzmdfPYpwHbnXM7nXP1wKPA\ngvANnHO7nXNrgeY2+14MvOicK3XOlQEvApd0szxdlpYYoKqukeZmF9PPeWdnCWv2lfPWjpJ263YX\nH8a52H6+iEhXRQ0L51xpN489EtgX9j7fWxbrfbstLSkO56CqLrad3AXlNQAs39P6V72+oIILfrGE\n17YVd7j/ixuLyC/TI2BFJPa6cgd3n2VmN5vZcjNbfujQoR47bloPTybonKOhqW3l6WhYrNxT1mr5\nq1uD57LlQGXUYzY2NfO5/1vBA2/s6pEyioh0JJZhUQCMCnuf6y3rsX2dc/c55+Y65+ZmZWUdd0Hb\nCk1T3lP3Wvxt6V7O/Ml/2gVGoRcWGworqalvaln+5vZgjWJPSfRaQ1FVHY3NruUYIiKxFMuwWAZM\nNLOxZhYPXAss6uK+LwDzzSzT69ie7y07IUIPQOqpu7iXbDlEcXUdBWWtL+wFZTUMS02gsdmxJr8c\ngNqGJpZ7NY29pdHD4kBF8Fj7KzTEV0RiL2Zh4c0r9QWCF/lNwD+ccxvM7E4zuxzAzE41s3zgauBe\nM9vg7VsK/IBg4CwD7uyBPpQuC3+0anc551i9LxgEe8Iu/nWNTRysquMD07IBWOEFxMo9ZdQ3NpOZ\nHNdhWBSW17b6KSISSzF9yLRzbjGwuM2yO8JeLyPYxBRp3wcJPnDphOvJZqj8shqKq+sA2FNyGAg2\nlxVVBJdNzUljwrCUlrB4c0cxfp9x+Ywc/rZ0L41NzQT87TN9v1ezKK6uo66xiYSAv9tlFRGJ5j3d\nwR0rPdnBHapVQOs+iPzy4OuRGUnMGZ3Jyr1lNDc73txewozcdKbmpHl9EpFrDuHLQ8EjIhIrCosI\nUhMCmPVMWKzaW05CwMf4rEGtwiJ0sR+ZkcScvEzKjzSwJr+ctfnlnD1hKKMHDwJgT+nhiMcN1SwA\nCivUyS0isaWwiMDnM1ITAj3yAKTV+8qYnpvO+KwUrxkqKDSKaUR6InPGBG9Ov2fJDpodnDV+KHlD\nkoHoI6IOVNSSm5kEtA4OEZFYUFhEkZbU/Wda1Dc2s76wkpmjMsgbksze0iMtd4UXltcwNCWBxDg/\n44YOIiM5jhc3FpEQ8DE7L4MRaYnE+33si9LJXVhRy+zRmd6x1MktIrGlsIiiJ+aH2rS/kvrGZmaO\nymT0kEHUNTZTVBW8sBeU1zAyIxEAM2OOd+E/dcxgEgJ+fD4jd3BSxJpFfWMzxdV1jMsKhoxqFiIS\nawqLKNISu/8ApFDn9qzRGYxp06xUUF7DSK8ZCWhpijprwpCWZXmDk1sNtw0pqqzFOchJTyI7PYn9\nPVizeGdnCf/11+U0RrjjXEQGLoVFFGlJgW43Q63aW8aw1ASy0xPJC3VYlwQnCCwsryEn/WhYXDh5\nGKkJAeZPHd6yLG/IIPaWtJ9QMHQjXnZGIjnpiRT24I15P3t+My9sKGJnceSOdREZmBQWUQSbobrX\nwb16XzmzRmdgZuRkJBLwGXtKjlB6uJ7ahmZyMo6GxUnZaaz7/sVMGJbasmz04GQO1zdR2mYK81Cz\nU3Z6ItkZiT3WDLViTxkr9wZrQ5v2R5+XSkQGHoVFFN1thio7XM/ukiPMHBVsXgr4feRmJrGn9MjR\nYbNhzVCRjB7sNV21aYoK7Z/tNUOVH2loNbfU8XrgjZ2kJQYI+IwtB/TwJxE5SmERRVpSHEfqmyLO\nFhuusLyGLQeq2jUVhforZo7KaFmWN2QQe0oOt8w2OzKj47AIDZ/d26aTe39FDWmJAQYlBMjxOsm7\ne6/FvtIjPL/+ANefnseEYSlsVliISJiYTvfxXpYedhf3kJSEduv3lR7h9//ZzsKV+TQ1O3Izk5g/\ndQR5Q5LZfKCSd3aW4jOYnpvesk/ekGRW7i1rCYucTsJi1ODI91oUlte27Jvt9XvsL69lfFbKcZ4t\n/PnN3fjM+ORZY9hfUcPy3WWd7yQiA4bCIorQzLOVtY3twuL+13fy0+c24/MZN5yRx5QRqby0qYj/\nW7qH+sZmMpLjOGlEGh8/I49BCUd/xXlDBlFV28iGggqS4vxkJsd1WIbEOD8j0hLbTSh4oLKG7PRg\njSLUSd6dmkVFTQOPLdvLZTNyGJGeyJQRaTy9upCKmoaW0BSRgU1hEUVoMsFII6IefGMXM0ZlcPf1\nsxnhXbSvPW00h+saqa5rZFhqAmbtH2Ge59UU3t5ZQk5GYsRt2ho9OJm9bab82F9ey7SRweat4enB\nIOvOcy2eWJnP4fomPn3OWACmjAh2sm85UMVpYwcf93FFpP9Qn0UU6VEmE6ysbaCwopZ5Jw1rCYqQ\nQQkBhqdFD4FQH8T+itpOm6BCRg9JbtUMVdvQRMnhenK8z04I+BmaktCtey1W7ytnZEYSp4wMNplN\nyQ6FhUZEiUiQwiKKlpln24yI2lZUDcDk4ant9unMqMHJhHKks87tkLzByRysqmsZ7XSg5R6Lo/vn\nZCR2qxlqW1E1k4Yf7e8YkZZIWmKATerkFhGPwiKKlmdatLnXYmtR8AI66TjCItQHAV0Pi9FebWRf\nWbB2EQqFnLBaTXZ64nE/Ma+p2bHjUDUTw87HzJiSnabhsyLSQmERRagZqm2fxZYDVQyK93f5Yt9W\nqCmqy81QXj/Hbu+O6lDNYkSrsEhif3lNu+G7XbGv9Ah1jc1MGNZ6JNWUEalsOVDVMvGhiAxsCoso\nEuN8xPmtXTPU1qIqJg5PxefrvHM6ktC0H10Ni4nDU0lNCPDYsn1A2FQf6a2boQ7XN3U6pfp3n17P\nrY+sarUsWk1pyog0qusaW4b5isjAprCIIjhFRxLbD1a3Wr61qKpV+/6xyhsarCnkdnL3dkhKQoDP\nv28CL28+yBvbiiksryEzOY6k+KOPUW2516KDfouXNxXx0Nt7eH79fmobjt7tvc07v3Y1C6+TWzfn\niQgoLDp0xtghvLOzhCavKaa4uo7i6vrj6q8IuWpOLt+9bGqXwwLgk2eNITcziR/+ayP5ZTWtahVA\ny13c0UZEVRxp4JtPriM53k9Dk2NdQUXLum1FVYzMSCIlofUo6tA5bu6BOaKcc/z8hc28ub2428cS\nkd6hsOjAWROGUFXbyHrv4hpqspk84vjDYlhqIp86e2yX7rEISYzzc/ulU9h8oIrXtx1qCYeQ7E5u\nzPvBvzZSXF3PHz4+BwhOGBiy7WB1u1oFBGs0owcns7mo+zWLx1cWcPcrO3jord3dPpaI9A6FRQfO\nHB98tsRbO0oA2Oo1yRzPsNnu+uC0bObkZdLsaHd/x7DUBHwWuWbxypaDLFyRzy3nj+P8SVmMGzqo\nJSyamh3bD1YzMUJYQDAUu1uzOFRVxw+e3QjA2vyKTrYWkb5KYdGBYamJTBqewls7gs0nW4qqyUiO\nIyu1/VxRsWZmfOdDU4GjneQhAb+PsUMHsXj9fo7UH+3kPlRVx9cXrmXisBS+OG8iALPzMlm5pwzn\nHPllwZFQ0ZrVThqRyq7iw62Oeay+/8wGauqbuP700RyorOVgpR4BK/JepLDoxFnjh7Jsdyl1jU1e\n53bqMTUh9aSZozJ45gvncP3po9ut+8GCU9hVfJg7nt4ABGsNX3lsNZU1Dfzu+lkkBIId4nPyMik5\nXM+ekiMtNxhOiNJhf9aEoTQ7uP3xdcc1hPaljUU8u3Y/t75vAh+ZNRJQ7ULkvUph0Ymzxg+htqGZ\nVXvL2XqgqleaoMJNy01vNTlhyFkThnLrhRNYuCKfx1fkc88r23ljezF3LjiZKSPSWrabkxd8vsaK\nPWVsPRhsVovWDHXGuCH8z8WTWbSmkJ+9sOWYyrmxsJJvPLmOycNT+a/zx3NyTjo+g7X55cd0HBHp\nGzSRYCdOHzcEnwUn26uqa2RSNzq3Y+2L8ybyzq5SvvXUOuobm/nwrJFcM3dUq20mZKWQmhhgxd4y\nauubyE5PJDUx+syy/33BeArLa/jjqzsYkZbAR+bk4jej2Tn2V9Syt+QIRVW1zB6dyZQRwVrXCxsO\n8JXHVpOaGOCu62YRHwj+TTJpeCprVLMQeU9SWHQiPSmOabkZPLW6EOidzu2uCvh93HXtLD5w1+vk\nZMTxwytOaddk5vMZs0cH+y0Cfms1zUckZsb3Lz+ZospavvfMRr73zMao247PGsT03AyeXFXAjNx0\n7vvEXIanHe2Mn56bzkubDuKc67WmPBE5PgqLLjhr/BDWeE++684NeSfCiPREnv/SuSTE+SM2V0Gw\nKerXL20lzu/jhjOGdHrMgN/H766bzaI1BVTVNtLU7HAE56QaPTiZwYPieW1bMc+uKeTp1QVcMTOH\nn145ncQ4f6vjTMvN4B/L88kvq2l5sFM0e0oOU1xdx5w8TZEu0hcoLLrg7PFD+cOSHQxLTSAjOb63\ni9OpYWmJHa6fk5eJc1Df2By1v6KtpHg/Hz21fcd6yA1DBnHDGXk0NDUT54/cFTbDe2rg2vyKTsPi\nl//eyls7Slj+7Yu6VD4RiS11cHfB3DGZxPt93boZry+ZMSqD0NRWnTVDHatoQQHB+zbi/T7WFnTe\nyb2n9AjF1XWUHa7vyeKJyHFSWHRBYpyfb35gSsuT5N7rUhICTPZGSEW6eztWEgJ+pmSnsnZf553c\nBd6U7NsPVXeypYicCGqG6qJPnt0/giLkwslZNDe7E/6M7em56Ty9qpDmZhd15t6a+iaKq4M1iu0H\nqzl1jPotRHqbahYD1FfnT+bZL55zwj93+sgMquoa2VVyOOo2BeVHHyPbdtZfEekdCosByu+zDvsX\nYmX6qFAnd/R+i31lwQkRAz5TWIj0EQoLOaEmZKWQFOfnlc2Hoj7ZL98Lizl5mQoLkT5CYSEnVMDv\n49rTRrFoTSFf/eca6hqb2m2TX3aEeL+PM8cPoaC8hpr69tuIyImlsJAT7o4PTeW290/iiZUF3PDA\nu5QfaT08Nr+shpyMxJbZcHdoRJRIr4tpWJjZJWa2xcy2m9ntEdYnmNlj3vqlZjbGWz7GzGrMbLX3\n74+xLKecWGbGF+dN5LfXzmT13nJuf3xdq/X5ZTXkZia3DOtVWIj0vpiFhZn5gbuBS4GpwHVmNrXN\nZp8GypxzE4BfA/8btm6Hc26m9++WWJVTes+CmSNZMDOHd3eXtuq/KCirITczibwhyfhMI6JE+oJY\n1ixOA7Y753Y65+qBR4EFbbZZADzkvV4IzDPNMDegTMtNp/RwPYUVwYci1TY0UVxdR25mEgkBP3lD\nBiksRPqAWIbFSGBf2Pt8b1nEbZxzjUAFEJrZbqyZrTKzV83s3BiWU3rRtJHBobTrvKnLQyOhcjOD\nc0eNz0pRWIj0AX21g3s/MNo5Nwu4Dfi7maW13cjMbjaz5Wa2/NChQye8kNJ9J2Wn4fcZ67z5ovK9\naT5yM5OA4HQku0sO09jU3GtlFJHYhkUBEP7knVxvWcRtzCwApAMlzrk651wJgHNuBbADmNT2A5xz\n9znn5jrn5mZlZcXgFCTWEuP8TByWwrqCSqB9zWLCsBQamhx7So9EPYaIxF4sw2IZMNHMxppZPHAt\nsKjNNouAG73XVwH/cc45M8vyOsgxs3HARGBnDMsqvWh6bjrrCypwzpFfVkOc3xiWmgAcnehQTVEi\nvStmYeH1QXwBeAHYBPzDObfBzO40s8u9zR4AhpjZdoLNTaHhtecBa81sNcGO71ucc6WxKqv0rmkj\ng53cBeU15JcdYWRGUsskg+OzBgEaPivS22I666xzbjGwuM2yO8Je1wJXR9jvceDxWJZN+o5TvE7u\n9QUVLfdYhKQmxjEiLVE1C5Fe1lc7uGUAOSk7jYDPWNcSFkmt1o8fNogdCguRXqWwkF6XGOdn4vBU\nlu0ua7nHItzEYalsLarmSH1jl47X2NTMW9uLaW6OPFGhiBw7hYX0CdNGprFsd7BbKrwZCuDSU0ZQ\n09DE4nUH2u0Xaebau17exvX3L+XxlfmxKazIAKSwkD5hWm4Goev+yDY1i9PGDmbMkGT+sXxfq+VP\nry7gtB+/zMq9ZS3LNhZWcs+SHQA8/PaeqNOgi8ixUVhInxC6kxto1wxlZlw9dxTv7iplV3HwCXu1\nDU38ZPFmDlXV8ak/L2PLgSoampr5n4VryEiO57b3T2JdQQWr90V/yJKIdJ3CQvqEKSNSCfjMu8ci\nsd36q+bk4jNaahcPv72bA5W1/Oyq6STG+bjhgaX84NmNbCis5IdXnMxN54wlJSHAX9/ec4LPpGdU\n1zVy/Z/e4clV3WtKyy87wmceWs7H719Kk/pwuqzscH27qfMHOoWF9AmJcX4mDU8lJyMJv6/9XJLD\n0xK5YPIwHl+RT9nheu5ZsoPzJmVxzdxRPHzT6dQ1NvPw23v44LRsLjklm5SEAFfOHsmza/dTXF3X\nC2fUPX97Zw9v7Sjha/9cy8ubio55/8amZu5/fSfv/9VrLNlykDe2F/PMmsIYlLT7mpsdDT08nUtj\nUzP3LNnOoapj/+4rahq47Pdv8JmHlvdomd7rFBbSZ3xx3gS+cOGEqOuvmTuKg1V1fObh5ZQfaeDr\nF08GYPKIVB666TQum5HD9xec3LL9DWeOob6pmceW7Yt2yD6ptqGJ+9/YxWljBzM1O43P/30lK/aU\ndb6jxznHrY+s4of/2sSZ44fwytcu4KTsNH714tYevyhD8MJ8wwNLeWXLwePa/ztPr2fB79/s0f6l\nt3eW8LPnt/DAG7uOaT/nHN98Yh35ZTUs31PG3hJNMxOisJA+45JTsrl67qio6+edNIyhKfGs2FPG\nB6dnt9zMBzBzVAa/u24WQ1MSWpZNGJbC2ROG8Ld39vSpiQifWVPIWT95mR/9ayMHK2vbrV+4Ip9D\nVXV8ed5E/vypUxmRlsinHwr2y3TF06sLeW79Ab42fxIP3DiXUYOT+Z+LJ7G39Ei7QQI9YU1+Ba9v\nK+bJlW0GEH6ZAAAThUlEQVSnfutcc7PjhQ0H2Li/krd2lPRYmV7eFAyuZ9YUHtMQ6keX7eNf6/bz\niTPzgvuv7Zu1sd6gsJD3jDi/j6vmjCLgM776/nbzSkb0iTPHUFhR2+n/9M3NLupftrUNTSxckc9V\nf3iL2x5b3a2/gBeuyOdLj67C7zceeGMX5/zsFe54ej2lh4Pt441Nzdz72g5mjsrgzPFDGJqSwMM3\nnU6c38eVf3iLlzZ23CR1sKqW7y7awOzRGXzuggmEHg9z4eRhzMnL5K6Xt1Hb0LPPNH9zezEAy3cf\n+4w8mw9UUVwdPPee6l9yzvHixiJSEgIUlNe0Gi3XkW1FVXz/mQ2cO3Eo37vsZObkZfbZprveoLCQ\n95QvXzSRl247n3FZKV3a/qKThjM9N50fL95MZW1DxG22H6zmrJ/+hwt+sYQ7n9nIW9uLeXdXKQ+9\ntZv/t3AtZ/zkZb72zzXsLT3CE6sKeOTdo3+dO+f4+Qub+cBvX+fRd/d2eCH++9K9fO2fazh7wlD+\n/eXz+c9XL+Ajs0by96V7mf/rV3l+/X6eXbuffaU1fP7Coxf60UOSefrzZzNmaDKf/etyfvfytoiB\n5Zzj20+up6ahiZ9fPaNV34+Z8T8XT6aosq7HO/3f8MKisKKWgvKaY9r39W3BRwtcMTOHFzcVcaCi\nfU3rWG0pqqKgvIYvXzSRxDgfT6/u/ILf3Oz4yj9Wk5IQ4JfXzMDnMy6bns3mA1VsLepaja6/U1jI\ne0pinJ8xQwd1eXu/z/jhFadQXF3Hr/69td36HYeque5P79DY7Bg7dBD/t3QP19+/lGvufZvvLtrA\n8xsOcPb4oTzy2TN45xvzOGfCUH7w7MaWIbz3vbaTu1/ZQcnhOm5/Yh3n/O8r/OalrawvqGiprSzd\nWcItf13BN59cx7wpw/jTJ+aSFB88j59eOZ1nbj2H4WmJ3PJ/K/nGE+uYNDyFeVOGtSpnTkYSC285\niwUzcvjli1s5539f4YuPrOKht3azeN1+nl5dwK9e3Mq/NxbxtfmTGB8hTM8YN4TzJmVx18vb2tUC\nVu8r58eLN/Hq1kPH1GR3uK6RVXvLOHfiUACW7Tq22sUb24uZNDyFr7x/Es3O8eiyvRG3e2pVATc+\n+C4VRyIHfrhQ7evyGTnMO2k4i9ftb+mrqahp4KP3vs0/2vRj/WvdftYXVPLtD05tGY33wek5+AwW\ndSFsetOKPaUnJNBiOpGgSF8wPTeDj5+ex8Nv7+bqubmcnBPs69h5qJrr7nsH5xyPfPYMJg5P5XBd\nI2/vKMHvM07KTmN4WgLhT/r9xdUzuPg3r/GVx1Zz/emj+clzm/nQ9Gx+e+0s3tlZwr2v7eQ3L23j\nNy9tY2hKAoMHxbG1qJqM5Di+cOEEvjhvIvGB1n+jnZSdxlOfP5s/LtnB71/Zzm3vn9Qy6264xDg/\nv/7oTM6dmMXLm4tYuquERW2aSU4bO5hPnzMu6u/iJx+ZxsfvX8rH7l/KPR+bzfumDOMvb+3mx4s3\n0dDkuO+1nWSlJnDFzBxuPm88WakJUY8F8O7uUhqaHJ8+Zyyr9pazbHcpV8xq+0DMyGobmli6q5Qb\nzsgjb8ggzp+UxSPv7uXzF04gzn/0d/Tw27u54+kNAPzwXxv5+dUzOjzuS5sOMmNUBsPSElkwI4d/\nrd3Pm9uLOX9SFrc/vpalu0pZk1/OGeOGMHpIMo1Nzfz6xa1MHp7K5TNyWo6TlZrAWeOHsmhNIV+d\nP4meeuJzQ1Mzr2w+yKtbDzFzVAYfmJbNoITjvxR/b9FGGpqaee5L5/ZYGSNRWMiA8LX5k3lu/X6+\n/dR6bj53HP/ZfJB/bywi4DP+7gUFwKCEABdNHR71OCPSE/nhFadw6yOrWL2vnLPGD+GX1wSbfM6e\nMJSzJwzlYFUtr28tZsnWQxSW1/DjD0/jw7NGkhTvj3rcOL+PW+dN5L8vnBBx6HCImXHlnFyunJOL\nc479FbVU1jYQ5/cR7/eRnZ7Y4f4jM5L45y1n8qk/L+Pmv65gbl4mS3eVMm/KMH7ykWms3FvGk6sK\n+PObu3n03X3cOm8CnzxrbLuAC3lzWzHxAR9njBvC7LxMlu/u+qitd3eVUt/YzDlereTjp+fxmYeX\n8/KmIi45JRuAe5Zs52fPb+Gik4YzZkgy97+xiw/NyOH8SZEfdnawqpbV+8pb+rTOn5xFWmKARWsK\n2VNyhOfWH+Az54zl0WX7+NZT63j4ptN4YmUBO4sPc+8Nc9qF9OUzcvj642tZk1/BzFEZXT63SPaW\nHOEvb+3m6dUFlByuJz7g429L93LH0xuYf/JwMpLiqKxtpLqukZvPG8epYwZ3esy1+eWsK6jgzgUn\nxzQoQGEhA0R6chzf/MBJ3PaPNXzubytJSwxw3qQsvjRvYktQdNVlM3J4d1cpW4qquPeGOSQEWofA\nsNTElgv6seroQt+WmZGTkUQOSZ1vHGZoSgKP3HwG//XX5byzs5TbL53CzeeOw+czLjkleJ/KjkPV\n/Ohfm/jx4s38felerp47iotPHtHyMKqQN3eUMDcvk8Q4P6fmZfLLF7dScaSB9OS4Vts1Nzu+8cQ6\nBqfE8/8umQIEm6Di/T5OHxu8KF44ZRgjM5K485mN/PHVnRyqqqOgvIYFM3P4xdUzaGp2LNl6iG88\nvpYXvnIeqYmtPwPglc3BUVChwE8I+Ln0lGwWrSnk2TX7ed+UYXzzAyeRNySZ7zy9gUeX7eP3/9nO\njNx05kf4I+HiU0bw7afW84cl2/ng9Bzi/T5yMhKZNjK9yxfn9QUV/PHVHSxetx+/z3j/1OFcOTuX\n8yZlsWZfOY+vzOf59QdwQFpiHOVH6tlWVMULXzmv3X9bbf196V6S4vxdrs11h8JCBowPe/9DjcxI\nYk5eJgH/8XfZ/eCKU3qqWL0iJSHAwzedTkl1HcPS2t8xPz4rhQc/eSpLthzkrpe38fMXtvDzF7Yw\naXgKv7x6JtNy0ymurmPT/kr+x7vf5VTvor98TynzTmp94f3T6zt5zBu2OyErhSvn5PLa1kPMycsk\nOT54GfL7jK+8fxIPvLGL1MQA44YOYmpOGp86eyx+nxHnh59dNZ0r//AWP31uMz/68LR25X5x40FG\nZiQxZcTRPwAun5nDY8v3kZ2eyC+vDnZef+z0PJ5eXcg3n1yHc8HmuUgX//SkOOafPJxn1+7nhQ1H\nR6KNGpzEZdNzuPSUbE7KTo3431JVbQM/fHYTjy3fR0pCgM+eN46bzh7L8LDf99wxg5k7ZjA/+cj0\nlmWvbT3EJx58l/tf38Xnw+472l9RQ2piHClek1VVbQOL1hRy2Yxs0iIEZ09TWMiAYWZ8ZPax/7Xf\nX/l9FjEowl0weRgXTB7GgYpa/r3xAPe+upNP/eVdnvjc2azOD867dc6EYDPSjNwM4vzGst1lrcJi\n5d4yfv7CFi45eQSlR+r59lPryc5IZPOBKr5+yeRWn3fVnFyu6qBGNnt0Jp8+eyz3v7GLISkJfHne\nxJamo8raBt7YfoiPzh3V6sJ/xrgh3Pq+CVx88ggyB8UD4PMZP71yGh/47RvMHJ3R0kEfya8/OpOv\nXzyF+qYm6hqb2VhYyTNr93Pvazu5Z8kOkuL8TMtNZ+aoDKZmpzElO5Wiyjq+8fhaDlTW8l/nj+O/\nL5hAelLXLujnTcrikpNH8Pv/bOfDs0aSk5HESxuL+MIjK8kbPIiFnzuT1MQ4nlpdyJH6Jq4/Pa9L\nx+0u6y+zcs6dO9ctX67b80Viacehaq78w1tkJsczcVgK7+wsYdUd81uazz58z5v4zVj4ubOA4Oij\nD971OgD/+uK51DY08YHfvs7h+kZqG5p59tZzWt1c2RV1jU1868n1LFyRz/ypw/nFNTN4ft0Bfv7v\nLRyqqmPhLWcytwvt/QCbD1SSnZbUrtmsK0qq63hjezGr9pazal85mworqQ8bSTYuaxC/uHoGs0dn\nHvOx88uOMO+Xr3LR1OGcNX4I33lqPeOzUthVfJizJwzlgRvnctnv38Rn8Oyt53Srv8LMVjjn5na6\nncJCRI7F8t2lfOz+pdQ1NnPJySP44w1zWtb9ePEm/vLmbtZ+bz6H6xq57R9reHN7Mf+85UxmeRfN\n17Ye4sY/v0tmcjzLv3VRxJFfnXHO8ec3d/OjxZsI+Iy6xmZmj87gOx+a2vI5J1pDUzO7ig+zaX8l\n1XWNXDk7l8S4jvscOnLXy9v41YvB4d7vmzKM318/i0WrC7n9iXWcO3Eor28r5kcfPoWPdbNm0dWw\nUDOUiByTuWMG89trZ/Hff1vB/JNb902cOmYw9722k58s3uQ1kzTy/QUnt7qAnzcpix8sOAWf2XEF\nBQSbFG86ZyyThqdyz5LtXHvaaC6bnh3zEUEdifP7mDQ8lUnHOGAimpvPG8erWw9xck4ad3xoKgG/\nj2tPG82uksPc++pOBsX7WTAz9h3bIapZiMhxKT9ST3pSXKsLdOnhemb/4EUATh87mB99+BQmDOuZ\ni6cENTc7frR4EyMzkrjpnLHdPp5qFiISUxnJ8e2WDR4Uz7c/eBJDUxJYMDOnV//S7698PuM7H5p6\nwj9XYSEiPeoz50a/g1zeuzQ3lIiIdEphISIinVJYiIhIpxQWIiLSKYWFiIh0SmEhIiKdUliIiEin\nFBYiItKpfjPdh5kdArrzJPqhQHEPFee9YiCeMwzM8x6I5wwD87yP9ZzznHORHz0Ypt+ERXeZ2fKu\nzI/SnwzEc4aBed4D8ZxhYJ53rM5ZzVAiItIphYWIiHRKYXHUfb1dgF4wEM8ZBuZ5D8RzhoF53jE5\nZ/VZiIhIp1SzEBGRTg34sDCzS8xsi5ltN7Pbe7s8sWJmo8zsFTPbaGYbzOxL3vLBZvaimW3zfvbO\nA4xjyMz8ZrbKzJ713o81s6Xed/6YmbV/is97nJllmNlCM9tsZpvM7Mz+/l2b2Ve8/7bXm9kjZpbY\nH79rM3vQzA6a2fqwZRG/Wwu6yzv/tWY2+3g/d0CHhZn5gbuBS4GpwHVmduIfQXViNAJfdc5NBc4A\nPu+d6+3Ay865icDL3vv+5kvAprD3/wv82jk3ASgDPt0rpYqt3wLPO+emADMInn+//a7NbCTwRWCu\nc+4UwA9cS//8rv8CXNJmWbTv9lJgovfvZuAPx/uhAzosgNOA7c65nc65euBRYEEvlykmnHP7nXMr\nvddVBC8eIwme70PeZg8BV/ROCWPDzHKBDwL3e+8NeB+w0NukP55zOnAe8ACAc67eOVdOP/+uCT75\nM8nMAkAysJ9++F07514DStssjvbdLgAedkHvABlmln08nzvQw2IksC/sfb63rF8zszHALGApMNw5\nt99bdQAY3kvFipXfAF8Hmr33Q4By51yj974/fudjgUPAn73mt/vNbBD9+Lt2zhUAvwD2EgyJCmAF\n/f+7Don23fbYNW6gh8WAY2YpwOPAl51zleHrXHBoXL8ZHmdmHwIOOudW9HZZTrAAMBv4g3NuFnCY\nNk1O/fC7ziT4V/RYIAcYRPummgEhVt/tQA+LAmBU2Ptcb1m/ZGZxBIPib865J7zFRaFqqffzYG+V\nLwbOBi43s90EmxjfR7AtP8NrqoD++Z3nA/nOuaXe+4UEw6M/f9cXAbucc4eccw3AEwS///7+XYdE\n+2577Bo30MNiGTDRGzERT7BDbFEvlykmvLb6B4BNzrlfha1aBNzovb4RePpEly1WnHPfcM7lOufG\nEPxu/+Oc+xjwCnCVt1m/OmcA59wBYJ+ZTfYWzQM20o+/a4LNT2eYWbL333ronPv1dx0m2ne7CPiE\nNyrqDKAirLnqmAz4m/LM7AME27X9wIPOuR/1cpFiwszOAV4H1nG0/f6bBPst/gGMJjhr7zXOubad\nZ+95ZnYB8DXn3IfMbBzBmsZgYBXwcedcXW+Wr6eZ2UyCnfrxwE7gUwT/OOy337WZfR/4KMGRf6uA\nzxBsn+9X37WZPQJcQHB22SLgu8BTRPhuveD8PcEmuSPAp5xzy4/rcwd6WIiISOcGejOUiIh0gcJC\nREQ6pbAQEZFOKSxERKRTCgsREemUwkLkOJjZt7wZTtea2WozO93Mvmxmyb1dNpFY0NBZkWNkZmcC\nvwIucM7VmdlQgvczvEVw1tPiXi2gSAyoZiFy7LKB4tDNXV44XEVwTqJXzOwVADObb2Zvm9lKM/un\nNy8XZrbbzH5mZuvM7F0zm+Atv9p7FsMaM3utd05NJDLVLESOkXfRf4PgNNgvAY8551715qCa65wr\n9mobTwCXOucOm9n/AxKcc3d62/3JOfcjM/sEwbttP2Rm64BLnHMFZpbhTSsu0ieoZiFyjJxz1cAc\ngg+TOQQ8ZmafbLPZGQQfqPWmma0mOF9PXtj6R8J+num9fhP4i5l9luD0MyJ9RqDzTUSkLedcE7AE\nWOLVCG5ss4kBLzrnrot2iLavnXO3mNnpBB/WtMLM5jjnSnq25CLHRzULkWNkZpPNbGLYopkEJ2+r\nAlK9Ze8AZ4f1Rwwys0lh+3w07Ofb3jbjnXNLnXN3EKyxhE8tLdKrVLMQOXYpwO/MLIPgDKfbCTZJ\nXQc8b2aFzrkLvaapR8wswdvv28BW73Wmma0F6rz9AH7uhZARfI7ymhNyNiJdoA5ukRMsvCO8t8si\n0lVqhhIRkU6pZiEiIp1SzUJERDqlsBARkU4pLEREpFMKCxER6ZTCQkREOqWwEBGRTv1/7WNxB8FT\nuVUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12c8b4160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4XOWd9vHvb2ZUrWpZtiXLlruNwd1002JiSgCTAAmQ\nEFLZZBPSNy9pZEPqkrZhExJYIIFsQgkQMMSB0DvGvfcuyU3Vsq2u5/1jzsgjaUYjWxpJlu7PdenS\nzGnzHA+cW085zzHnHCIiIh3x9XYBRESk71NYiIhITAoLERGJSWEhIiIxKSxERCQmhYWIiMSksBAR\nkZgUFiIiEpPCQkREYgr0dgG6y5AhQ9zo0aN7uxgiIieVZcuWlTrncmNt12/CYvTo0SxdurS3iyEi\nclIxs12d2U7NUCIiEpPCQkREYlJYiIhITAoLERGJSWEhIiIxKSxERCQmhYWIiMQ04MPicF0jv3ph\nMyv3VPZ2UURE+qwBHxb1jc3c9dIWVu6u6O2iiIj0WQM+LJICwX+C+qbmXi6JiEjfpbDwwqKuQWEh\nIhLNgA+LgN+H32fUNSosRESiGfBhAZDo91HX2NTbxRAR6bMUFkBSgk81CxGRDigsCPZbqM9CRCQ6\nhQWQFPBrNJSISAcUFng1C/VZiIhEpbAAEtUMJSLSIYUFoZqFwkJEJBqFBcE+CzVDiYhEp7AgOHS2\nXjULEZGoFBaoGUpEJBaFBaFmKIWFiEg0CgtCo6HUZyEiEo3CAjVDiYjEorBAzVAiIrEoLNBoKBGR\nWBQWBJuh6puaaW52vV0UEZE+SWFBsIMb9GhVEZFoFBYE+yxAj1YVEYlGYUHYc7g15YeISEQKC8LD\nQjULEZFIFBZAUoLXDKWwEBGJSGGBmqFERGJRWHBsNJRqFiIikSksCKtZaDSUiEhECgvChs6qGUpE\nJCKFBcdqFpryQ0QkMoUFkJygPgsRkY7ENSzM7FIz22RmW83stgjrv2Zm681stZm9ZGaFYetuNrMt\n3s/N8Sxnol9DZ0VEOhK3sDAzP/A74DJgCnCDmU1ps9kKYI5zbhrwOHCnt+9g4PvAmcAZwPfNLDte\nZU1K0NBZEZGOxLNmcQaw1Tm33TlXDzwCLAjfwDn3inPuqPf2XaDAe30J8IJzrtw5VwG8AFwar4Jq\nNJSISMfiGRYjgD1h74u8ZdF8GvjnCe7bJcdGQyksREQiCfR2AQDM7GPAHOCC49zvFuAWgFGjRp3w\n5ydqNJSISIfiWbMoBkaGvS/wlrViZhcD3wGucs7VHc++zrl7nXNznHNzcnNzT7igfp+R4Df1WYiI\nRBHPsFgCTDCzMWaWCFwPLAzfwMxmAvcQDIoDYaueB+abWbbXsT3fWxY3iX6fmqFERKKIWzOUc67R\nzL5I8CLvBx5wzq0zszuApc65hcDPgTTgb2YGsNs5d5VzrtzMfkgwcADucM6Vx6usEJx5VjULEZHI\n4tpn4ZxbBCxqs+z2sNcXd7DvA8AD8Stda0kBn0ZDiYhEoTu4PUkBn57BLSIShcLCkxTwq2YhIhKF\nwsKTlOBTn4WISBQKC49GQ4mIRKew8ARrFgoLEZFIFBaepICGzoqIRKOw8CQFfJruQ0QkCoWFJymg\nZigRkWgUFp5E3ZQnIhKVwsKjPgsRkegUFh41Q4mIRKew8GjorIhIdAoLT1LAT1Ozo1HzQ4mItKOw\n8ISew63JBEVE2lNYeEKPVtWIKBGR9hQWnqSAH0D9FiIiESgsPKFmKA2fFRFpT2HhSUrw+ixUsxAR\naUdh4VEzlIhIdAoLT6KaoUREolJYeJI0GkpEJCqFhedYB7fCQkSkLYWF51ifhZqhRETaUlh4QqOh\nVLMQEWlPYeFRM5SISHQKC0+iwkJEJCqFhaelz6JBfRYiIm0pLDxqhhIRiU5h4WmZolxhISLSjsLC\nY2Yk6tGqIiIRKSzCJPl9us9CRCQChUUYPYdbRCQyhUWYpIBfc0OJiESgsAiTFFAzlIhIJAqLMIkB\nn0ZDiYhEoLAIk5TgV5+FiEgECoswGg0lIhJZXMPCzC41s01mttXMbouw/nwzW25mjWZ2bZt1TWa2\n0vtZGM9yhmg0lIhIZFHDwsweDnv9kzbr/hnrwGbmB34HXAZMAW4wsyltNtsNfAL4a4RD1DjnZng/\nV8X6vO6QFPBpNJSISAQd1Swmh72+tM264Z049hnAVufcdudcPfAIsCB8A+fcTufcaqBPXKGTAn41\nQ4mIRNBRWLgTXBcyAtgT9r7IW9ZZyWa21MzeNbOrI21gZrd42yw9ePDgcRw6sqSAj/qmPpFbIiJ9\nSqCDdalmNpVgoKR4r837SemBshU654rNbCzwspmtcc5tC9/AOXcvcC/AnDlzOhNgHUpUM5SISEQd\nhUUpcHeE16H3sRQDI8PeF3jLOsU5V+z93m5mrwIzgW0d7tRFSZpIUEQkoqhh4Zyb28VjLwEmmNkY\ngiFxPXBjZ3Y0s2zgqHOuzsyGAOcCd3axPDEF77NQn4WISFsdjYaabWbDwt5/1MyeMLNfmVlWrAM7\n5xqBLwLPAxuAx5xz68zsDjO7yjvm6WZWBFwH3GNm67zdTwGWmtkq4BXgZ8659Sd6kp0Vqlk41+UW\nLRGRfqWjZqh7gfkAZjYX+AXwFWCGt+7DsQ7unFsELGqz7Paw10sINk+13e9tYGrs4nevpIAP56Cx\n2ZHgt57+eBGRPqujsAg458q819cD9zrnHgUe9f7i73dansPd2EyCXze3i4iEdHRF9Hs31gHMA17u\n5H4nrcTQc7gb1G8hIhKuo5rFY8ArZnYQqAfeADCzcUB1D5Stx4Wew60RUSIirXU0GuoOM3sZyAOe\nc86FrqAJwJd6onA9LSlBYSEiEknUsDCzDGC192Pee4CSnihYbzjWZ6FmKBGRcB01Q1UCe4HQlTN8\neJADRsWrUL0l1AylByCJiLTWUVjcDZwHvAY87Jx7p2eK1HsS1WchIhJR1FFNzrkvEpxi4xngs2a2\nwsx+YmaFPVa6HtbSDKX5oUREWulwCKxzrtk59wLBm/F+D9xC++nK+41jo6HUZyEiEq6jDu4U4Erg\nIwSnFn8KON05t6OHytbjNBpKRCSyjvosDhCc5fUR4CGCndpTvanKcc71yKNOe1KoGUod3CIirXUU\nFk8TDIhTvZ9wDuiHYaFmKBGRSDq6Ke9jPVmQvkCjoUREIuuXczydqJaahUZDiYi0orAIE+qzqNVE\ngiIircQMCzNr11QVaVl/kOA3Ev0+jtQrLEREwnWmZvFeJ5ed9MyMjJQAh2obersoIiJ9Skf3WQwl\nOONsijdcNjQ3VAaQ2gNl6xUZyQlU1SgsRETCddSc9AHgUwQfe/o7joVFNfC9OJer12SkJHBIYSEi\n0kpHQ2f/CPzRzD7snHusB8vUqzJSVLMQEWmrM30WQ0PPsjCzP5jZe2Y2L87l6jUZyQHVLERE2uhM\nWNzinDtkZvMJ9mF8FrgzvsXqPZlqhhIRaaczYeG835cDDznnVnVyv5NSRkoCh2obcM7F3lhEZIDo\nzEV/lZktAq4A/mlmaRwLkH4nIzmBhiZHjW7MExFp0Zmb6z4JzAa2OueOmtkQ4NPxLVbvyUxJAOBQ\nTSOpif3y3kMRkeMWs2bhnGsCxgKf9xaldGa/k1VGSjAgdGOeiMgxnZnu47fARUBoFtojwB/iWaje\nlJEcrFlo+KyIyDGdaWc5xzk3y8xWADjnys0sMc7l6jXHmqEUFiIiIZ1pTmowMx9ep7aZ5QD9dg7v\njFBYqBlKRKRF1LAIm1n2d8ATQK6Z/QB4E/ivHihbr8hIDp521VGFhYhISEfNUO8Bs5xzD5nZMuBi\ngvNDXeecW9sjpesFx2oWjb1cEhGRvqOjsAhNHIhzbh2wLv7F6X0Jfh+piX71WYiIhOkoLHLN7GvR\nVjrnfhWH8vQJmqZcRKS1jsLCD6QRVsMYKDK9KT9ERCSoo7DY65y7o8dK0odkpAQ4VKM+CxGRkI6G\nzg64GkVIRrJqFiIi4ToKiy4/s8LMLjWzTWa21cxui7D+fDNbbmaNZnZtm3U3m9kW7+fmrpbleOgB\nSCIirUUNC+dceVcObGZ+gvdoXAZMAW4wsyltNtsNfAL4a5t9BwPfB84EzgC+b2bZXSnP8eipZ1ps\n2lfNgt+9RcWR+rh/lohIV8RzQsAzCM5Uu905Vw88AiwI38A5t9M5t5r2d4RfArzgnCt3zlUALwCX\nxrGsrWQkB6iua6S5Ob4zsb+7vYxVeyp5e1tZu3U7S4/omRoi0mfEMyxGAHvC3hd5y+K9b5dlpCTg\nHFTXxbeTu7iyBoClu1pX4tYWV3HhL17l9S2lHe7/wvr9FFUcjVv5RERCTuqpxs3sFjNbamZLDx48\n2G3HzejmyQSdczQ0tZ9OKxQWy3dVtFr+2ubguWzadyjqMRubmvn8/y3j/jd3dEsZRUQ6Es+wKAZG\nhr0v8JZ1277OuXudc3Occ3Nyc3NPuKBthaYp764RUX9ZvJuzf/pyu8Ao8cJiXckhauqPPZnvra3B\nGsWusui1hv3VdTQ2u5ZjiIjEUzzDYgkwwczGeFOaXw8s7OS+zwPzzSzb69ie7y3rEaEHIHXXiKhX\nNx2k9HAdxRWtL+zFFTUMTU+isdmxqqgSgNqGJpZ6NY3d5dHDYl9V8Fh7q2q7pYwiIh2JW1g45xqB\nLxK8yG8AHnPOrTOzO8zsKgAzO93MioDrgHvMbJ23bznwQ4KBswS4o6ujs45H+KNVu8o5x8o9wSDY\nFXbxr2ts4kB1HZdPzQNgmRcQy3dVUN/YTHZqQodhUVJZ2+q3iEg8xfUh0865RcCiNstuD3u9hGAT\nU6R9HwAeiGf5ounOZqiiihpKD9cBsKvsCBBsLttfFVw2JT+D8UPTWsLirW2l+H3GVdPz+cvi3TQ2\nNRPwt8/0vV7NovRwHXWNTSQF/F0uq4hINCd1B3e8dGcHd6hWAa37IIoqg69HZKUwe1Q2y3dX0Nzs\neGtrGdMLMpmSn+H1SUSuOYQvDwWPiEi8KCwiSE8KYNY9YbFidyVJAR/jcge1CovQxX5EVgqzC7Op\nPNrAqqJKVhdVcu74IYwaPAiAXeVHIh43VLMAKKlSJ7eIxJfCIgKfz0hPCnTLA5BW7qlgWkEm43LT\nvGaooNAopuGZycweHbw5/e5Xt9Hs4JxxQyjMSQWij4jaV1VLQXYK0Do4RETiQWERRXfMD1Xf2Mza\nkkPMGJlFYU4qu8uPttwVXlJZw5C0JJIT/IwdMois1AReWL+fpICPWYVZDM9IJtHvY0+UTu6Sqlpm\njcr2jqVObhGJL4VFFN0xP9SGvYeob2xmxshsRuUMoq6xmf3VwQt7cWUNI7KSATAzZnsX/tNHDyYp\n4MfnMwoGp0SsWdQ3NlN6uI6xucGQUc1CROJNYRFFd0xTHurcnjkqi9FtmpWKK2sY4TUjAS1NUeeM\nz2lZVjg4tdVw25D9h2pxDvIzU8jLTGFvN9Ys3t1exr/9eSmNEe44F5GBS2ERRUZKoMvNUCt2VzA0\nPYm8zGQKQx3WZcEJAksqa8jPPBYWF00aSnpSgPlThrUsK8wZxO6y9hMKhm7Ey8tKJj8zmZJuvDHv\nzuc28vy6/WwvjdyxLiIDk8IiimAzVNc6uFfuqWTmqCzMjPysZAI+Y1fZUcqP1FPb0Ex+1rGwOCUv\ngzU/uITxQ9Nblo0anMqR+ibK20xhHmp2ystMJi8ruduaoZbtqmD57mBtaMPe6PNSicjAo7CIoqvN\nUBVH6tlZdpQZI4PNSwG/j4LsFHaVHz02bDasGSqSUYO9pqs2TVGh/fO8ZqjKow2t5pY6Ufe/uZ2M\n5AABn7FpX3WXjyci/YfCIoqMlASO1jdFnC02XEllDZv2VbdrKgr1V8wYmdWyrDBnELvKjrTMNjsi\nq+OwCA2f3d2mk3tvVQ0ZyQEGJQXI9zrJu3qvxZ7yozy3dh83nlnI+KFpbFRYiEiYuE73cTLLDLuL\nOyctqd36PeVH+e3LW3l8eRFNzY6C7BTmTxlOYU4qG/cd4t3t5fgMphVktuxTmJPK8t0VLWGRHyMs\nRg6OfK9FSWVty755Xr/H3spaxuWmneDZwh/f2onPjE+cM5q9VTUs3VkReycRGTAUFlGEZp49VNvY\nLizue2M7P/vnRnw+46azCpk8PJ0XN+zn/xbvor6xmazUBE4ZnsHHzipkUNKxf+LCnEFU1zayrriK\nlAQ/2akJHZYhOcHP8IzkdhMK7jtUQ15msEYR6iTvSs2iqqaBR5fs5srp+QzPTGby8AyeXllCVU1D\nS2iKyMCmsIgiNJlgpBFRD7y5g+kjs/jdjbMY7l20rz9jFEfqGjlc18jQ9CTMrN1+hV5N4Z3tZeRn\nJUfcpq1Rg1PZ3WbKj72VtUwdEWzeGpYZDLKuPNfiyeVFHKlv4tNzxwAweXiwk33TvmrOGDP4hI8r\nIv2H+iyiyIwymeCh2gZKqmqZd8rQlqAIGZQUYFhG9BAI9UHsraqN2QQVMiontVUzVG1DE2VH6sn3\nPjsp4GdIWlKX7rVYuaeSEVkpnDYi2GQ2OS8UFhoRJSJBCosoWmaebTMiasv+wwBMGpbebp9YRg5O\nJZQjsTq3QwoHp3Kguq5ltNO+lnssju2fn5XcpWaoLfsPM3HYsf6O4RnJZCQH2KBObhHxKCyiaHmm\nRZt7LTbvD15AJ55AWIT6IKDzYTHKq43sqQjWLkKhkB9Wq8nLTD7hJ+Y1NTu2HTzMhLDzMTMm52Vo\n+KyItFBYRBFqhmrbZ7FpXzWDEv2dvti3FWqK6nQzlNfPsdO7ozpUsxjeKixS2FtZ0274bmfsKT9K\nXWMz44e2Hkk1eXg6m/ZVt0x8KCIDm8IiiuQEHwl+a9cMtXl/NROGpePzxe6cjiQ07Udnw2LCsHTS\nkwI8umQPEDbVR2brZqgj9U0xp1T//tNrufXhFa2WRaspTR6eweG6xpZhviIysCksoghO0ZHC1gOH\nWy3fvL+6Vfv+8SocEqwpFMS4ezskLSnAF943npc2HuDNLaWUVNaQnZpASuKxx6i23GvRQb/FSxv2\n8+A7u3hu7V5qG47d7b3FO792NQuvk1s354kIKCw6dNaYHN7dXkaT1xRTeriO0sP1J9RfEXLt7AK+\nf+WUTocFwCfOGU1Bdgo/+sd6iipqWtUqgJa7uKONiKo62sC3/76G1EQ/DU2ONcVVLeu27K9mRFYK\naUmtR1GHznFjN8wR5Zzj589v5K2tpV0+loj0DoVFB84Zn0N1bSNrvYtrqMlm0vATD4uh6cl88twx\nnbrHIiQ5wc9tl01m475q3thysCUcQvJi3Jj3w3+sp/RwPb//2GwgOGFgyJYDh9vVKiBYoxk1OJWN\n+7tes3hieTG/e2UbD769s8vHEpHeobDowNnjgs+WeHtbGQCbvSaZExk221UfmJrH7MJsmh3t7u8Y\nmp6EzyLXLF7ZdIDHlxXxuQvGcsHEXMYOGdQSFk3Njq0HDjMhQlhAMBS7WrM4WF3HD59dD8DqoqoY\nW4tIX6Ww6MDQ9GQmDkvj7W3B5pNN+w+TlZpAbnr7uaLizcz43hVTgGOd5CEBv48xQwaxaO1ejtYf\n6+Q+WF3HNx9fzYShaXxp3gQAZhVms3xXBc45iiqCI6GiNaudMjydHaVHWh3zeP3gmXXU1Ddx45mj\n2HeolgOH9AhYkZORwiKGc8YNYcnOcuoam7zO7fTjakLqTjNGZvHMF+dy45mj2q374YLT2FF6hNuf\nXgcEaw1ffXQlh2oa+J8bZ5IUCHaIzy7MpuxIPbvKjrbcYDg+Sof9OeOH0OzgtifWnNAQ2hfX7+fZ\n1Xu59X3j+dDMEYBqFyInK4VFDOeMy6G2oZkVuyvZvK+6V5qgwk0tyGw1OWHIOeOHcOtF43l8WRFP\nLCvi7le28ubWUu5YcCqTh2e0bDe7MPh8jWW7Kth8INisFq0Z6qyxOfzHJZNYuKqEO5/fdFzlXF9y\niG/9fQ2ThqXzbxeM49T8THwGq4sqj+s4ItI3aCLBGM4cm4PPgpPtVdc1MrELndvx9qV5E3h3Rznf\neWoN9Y3NfHDmCD48Z2SrbcbnppGeHGDZ7gpq65vIy0wmPTn6zLL/fuE4Sipr+MNr2xiekcSHZhfg\nN6PZOfZW1bK77Cj7q2uZNSqbycODta7n1+3jq4+uJD05wF03zCQxEPybZOKwdFapZiFyUlJYxJCZ\nksDUgiyeWlkC9E7ndmcF/D7uun4ml9/1BvlZCfzo6tPaNZn5fMasUcF+i4DfWk3zEYmZ8YOrTmX/\noVr+85n1/Ocz66NuOy53ENMKsvj7imKmF2Ry78fnMCzjWGf8tIJMXtxwAOdcrzXliciJUVh0wjnj\ncljlPfmuKzfk9YThmck89+XzSErwR2yugmBT1K9f3EyC38dNZ+XEPGbA7+N/bpjFwlXFVNc20tTs\ncATnpBo1OJXBgxJ5fUspz64q4emVxVw9I5+fXTON5AR/q+NMLcjisaVFFFXUtDzYKZpdZUcoPVzH\n7EJNkS7SFygsOuHccUP4/avbGJqeRFZqYm8XJ6ahGckdrp9dmI1zUN/YHLW/oq2URD8fOb19x3rI\nTTmDuOmsQhqamknwR+4Km+49NXB1UVXMsPjlvzbz9rYyln734k6VT0TiSx3cnTBndDaJfl+Xbsbr\nS6aPzCI0tVWsZqjjFS0oIHjfRqLfx+ri2J3cu8qPUnq4jooj9d1ZPBE5QQqLTkhO8PPtyye3PEnu\nZJeWFGCSN0Iq0t3b8ZIU8DM5L53Ve2J3chd7U7JvPXg4xpYi0hPUDNVJnzi3fwRFyEWTcmludj3+\njO1pBZk8vaKE5mYXdebemvomSg8HaxRbDxzm9NHqtxDpbapZDFBfnz+JZ780t8c/d9qILKrrGtlR\ndiTqNsWVxx4j23bWXxHpHQqLAcrvsw77F+Jl2shQJ3f0fos9FcEJEQM+U1iI9BEKC+lR43PTSEnw\n88rGg1Gf7FfkhcXswmyFhUgfobCQHhXw+7j+jJEsXFXC1/+2irrGpnbbFFUcJdHv4+xxORRX1lBT\n334bEelZCgvpcbdfMYWvvX8iTy4v5qb736PyaOvhsUUVNeRnJbfMhrtNI6JEel1cw8LMLjWzTWa2\n1cxui7A+ycwe9dYvNrPR3vLRZlZjZiu9nz/Es5zSs8yML82bwG+un8HK3ZXc9sSaVuuLKmooyE5t\nGdarsBDpfXELCzPzA78DLgOmADeY2ZQ2m30aqHDOjQd+DfxX2LptzrkZ3s/n4lVO6T0LZoxgwYx8\n3ttZ3qr/oriihoLsFApzUvGZRkSJ9AXxrFmcAWx1zm13ztUDjwAL2myzAHjQe/04MM80w9yAMrUg\nk/Ij9ZRUBR+KVNvQROnhOgqyU0gK+CnMGaSwEOkD4hkWI4A9Ye+LvGURt3HONQJVQGhmuzFmtsLM\nXjOz8+JYTulFU0cEh9Ku8aYuD42EKsgOzh01LjdNYSHSB/TVDu69wCjn3Ezga8BfzSyj7UZmdouZ\nLTWzpQcPHuzxQkrXnZKXgd9nrPHmiyrypvkoyE4BgtOR7Cw7QmNTc6+VUUTiGxbFQPiTdwq8ZRG3\nMbMAkAmUOefqnHNlAM65ZcA2YGLbD3DO3eucm+Ocm5ObmxuHU5B4S07wM2FoGmuKDwHtaxbjh6bR\n0OTYVX406jFEJP7iGRZLgAlmNsbMEoHrgYVttlkI3Oy9vhZ42TnnzCzX6yDHzMYCE4DtcSyr9KJp\nBZmsLa7COUdRRQ0JfmNoehJwbKJDNUWJ9K64hYXXB/FF4HlgA/CYc26dmd1hZld5m90P5JjZVoLN\nTaHhtecDq81sJcGO788558rjVVbpXVNHBDu5iytrKKo4yoislJZJBsflDgI0fFakt8V11lnn3CJg\nUZtlt4e9rgWui7DfE8AT8Syb9B2neZ3ca4urWu6xCElPTmB4RrJqFiK9rK92cMsAckpeBgGfsaYl\nLFJarR83dBDbFBYivUphIb0uOcHPhGHpLNlZ0XKPRbgJQ9PZvP8wR+sbO3W8xqZm3t5aSnNz5IkK\nReT4KSykT5g6IoMlO4PdUuHNUACXnTacmoYmFq3Z126/SDPX3vXSFm68bzFPLC+KT2FFBiCFhfQJ\nUwuyCF33R7SpWZwxZjCjc1J5bOmeVsufXlnMGT95ieW7K1qWrS85xN2vbgPgoXd2RZ0GXUSOj8JC\n+oTQndxAu2YoM+O6OSN5b0c5O0qDT9irbWjip4s2crC6jk/+cQmb9lXT0NTMfzy+iqzURL72/oms\nKa5i5Z7oD1kSkc5TWEifMHl4OgGfefdYJLdbf+3sAnxGS+3ioXd2su9QLXdeO43kBB833b+YHz67\nnnUlh/jR1afyqbljSEsK8Od3dvXwmXSPw3WN3Pi/7/L3FV1rSiuqOMpnHlzKx+5bTJP6cDqt4kh9\nu6nzBzqFhfQJyQl+Jg5LJz8rBb+v/VySwzKSuXDSUJ5YVkTFkXrufnUb50/M5cNzRvLQp86krrGZ\nh97ZxQem5nHpaXmkJQW4ZtYInl29l9LDdb1wRl3zl3d38fa2Mr7xt9W8tGH/ce/f2NTMfW9s5/2/\nep1XNx3gza2lPLOqJA4l7brmZkdDN0/n0tjUzN2vbuVg9fF/91U1DVz52zf5zINLu7VMJzuFhfQZ\nX5o3ni9eND7q+g/PGcmB6jo+89BSKo828M1LJgEwaXg6D37qDK6cns8PFpzasv1NZ4+mvqmZR5fs\niXbIPqm2oYn73tzBGWMGMyUvgy/8dTnLdlXE3tHjnOPWh1fwo39s4OxxObzyjQs5JS+DX72wudsv\nyhC8MN90/2Je2XTghPb/3tNrWfDbt7q1f+md7WXc+dwm7n9zx3Ht55zj20+uoaiihqW7Kthdpmlm\nQhQW0mdceloe180ZGXX9vFOGMiQtkWW7KvjAtLyWm/kAZozM4n9umMmQtKSWZeOHpnHu+Bz+8u6u\nPjUR4TOrSjjnpy/x43+s58Ch2nbrH19WxMHqOr4ybwJ//OTpDM9I5tMPBvtlOuPplSX8c+0+vjF/\nIvffPIeRg1P5j0smsrv8aLtBAt1hVVEVb2wp5e/L2079Fltzs+P5dftYv/cQb28r67YyvbQhGFzP\nrCo5riEhr/1QAAATaElEQVTUjyzZwz/W7OXjZxcG91/dN2tjvUFhISeNBL+Pa2ePJOAzvv7+dvNK\nRvTxs0dTUlUb83/65mYX9S/b2oYmHl9WxLW/f5uvPbqyS38BP76siC8/sgK/37j/zR3MvfMVbn96\nLeVHgu3jjU3N3PP6NmaMzOLscTkMSUvioU+dSYLfxzW/f5sX13fcJHWgupbvL1zHrFFZfP7C8YQe\nD3PRpKHMLszmrpe2UNvQvc80f2trKQBLdx7/jDwb91VTejh47t3Vv+Sc44X1+0lLClBcWdNqtFxH\ntuyv5gfPrOO8CUP4zytPZXZhdp9tuusNCgs5qXzl4gm8+LULGJub1qntLz5lGNMKMvnJoo0cqm2I\nuM3WA4c552cvc+EvXuWOZ9bz9tZS3ttRzoNv7+T/Pb6as376Et/42yp2lx/lyRXFPPzesb/OnXP8\n/PmNXP6bN3jkvd0dXoj/ung33/jbKs4dP4R/feUCXv76hXxo5gj+ung383/9Gs+t3cuzq/eyp7yG\nL1x07EI/KieVp79wLqOHpPLZPy/lf17aEjGwnHN89+9rqWlo4ufXTW/V92Nm/Mclk9h/qK7bO/3f\n9MKipKqW4sqa49r3jS3BRwtcPSOfFzbsZ19V+5rW8dq0v5riyhq+cvEEkhN8PL0y9gW/udnx1cdW\nkpYU4Jcfno7PZ1w5LY+N+6rZvL9zNbr+TmEhJ5XkBD+jhwzq9PZ+n/Gjq0+j9HAdv/rX5nbrtx08\nzA3/+y6NzY4xQwbxf4t3ceN9i/nwPe/w/YXreG7dPs4dN4SHP3sW735rHnPHD+GHz65vGcJ77+vb\n+d0r2yg7UsdtT65h7n+9wn+/uJm1xVUttZXF28v43J+X8e2/r2He5KH878fnkJIYPI+fXTONZ26d\ny7CMZD73f8v51pNrmDgsjXmTh7YqZ35WCo9/7hwWTM/nly9sZu5/vcKXHl7Bg2/vZNGavTy9sphf\nvbCZf63fzzfmT2RchDA9a2wO50/M5a6XtrSrBazcU8lPFm3gtc0Hj6vJ7khdIyt2V3DehCEALNlx\nfLWLN7eWMnFYGl99/0SaneORJbsjbvfUimJufuA9qo5GDvxwodrXVdPzmXfKMBat2dvSV1NV08BH\n7nmHx9r0Y/1jzV7WFh/iux+Y0jIa7wPT8vEZLOxE2PSmZbvKeyTQ4jqRoEhfMK0gi4+dWchD7+zk\nujkFnJof7OvYfvAwN9z7Ls45Hv7sWUwYls6Rukbe2VaG32eckpfBsIwkwp/0+4vrpnPJf7/OVx9d\nyY1njuKn/9zIFdPy+M31M3l3exn3vL6d/35xC//94haGpCUxeFACm/cfJis1gS9eNJ4vzZtAYqD1\n32in5GXw1BfO5Q+vbuO3r2zla++f2DLrbrjkBD+//sgMzpuQy0sb97N4RxkL2zSTnDFmMJ+eOzbq\nv8VPPzSVj923mI/et5i7PzqL900eyp/e3slPFm2goclx7+vbyU1P4uoZ+dxy/jhy05OiHgvgvZ3l\nNDQ5Pj13DCt2V7JkZzlXz2z7QMzIahuaWLyjnJvOKqQwZxAXTMzl4fd284WLxpPgP/Zv9NA7O7n9\n6XUA/Ogf6/n5ddM7PO6LGw4wfWQWQzOSWTA9n3+s3stbW0u5YGIutz2xmsU7yllVVMlZY3MYlZNK\nY1Mzv35hM5OGpXPV9PyW4+SmJ3HOuCEsXFXC1+dPpLue+NzQ1MwrGw/w2uaDzBiZxeVT8xiUdOKX\n4v9cuJ6Gpmb++eXzuq2MkSgsZED4xvxJ/HPtXr771FpuOW8sL288wL/W7yfgM/7qBQXAoKQAF08Z\nFvU4wzOT+dHVp3HrwytYuaeSc8bl8MsPB5t8zh0/hHPHD+FAdS1vbC7l1c0HKams4ScfnMoHZ44g\nJdEf9bgJfh+3zpvAv180PuLQ4RAz45rZBVwzuwDnHHurajlU20CC30ei30deZnKH+4/ISuFvnzub\nT/5xCbf8eRlzCrNZvKOceZOH8tMPTWX57gr+vqKYP761k0fe28Ot88bziXPGtAu4kLe2lJIY8HHW\n2BxmFWazdGfnR229t6Oc+sZm5nq1ko+dWchnHlrKSxv2c+lpeQDc/epW7nxuExefMozROanc9+YO\nrpiezwUTIz/s7EB1LSv3VLb0aV0wKZeM5AALV5Wwq+wo/1y7j8/MHcMjS/bwnafW8NCnzuDJ5cVs\nLz3CPTfNbhfSV03P55tPrGZVURUzRmZ1+twi2V12lD+9vZOnVxZTdqSexICPvyzeze1Pr2P+qcPI\nSkngUG0jh+saueX8sZw+enDMY64uqmRNcRV3LDg1rkEBCgsZIDJTE/j25afwtcdW8fm/LCcjOcD5\nE3P58rwJLUHRWVdOz+e9HeVs2l/NPTfNJinQOgSGpie3XNCPV0cX+rbMjPysFPJJib1xmCFpSTx8\ny1n825+X8u72cm67bDK3nDcWn8+49LTgfSrbDh7mx//YwE8WbeSvi3dz3ZyRXHLq8JaHUYW8ta2M\nOYXZJCf4Ob0wm1++sJmqow1kpia02q652fGtJ9cwOC2R/3fpZCDYBJXo93HmmOBF8aLJQxmRlcId\nz6znD69t52B1HcWVNSyYkc8vrptOU7Pj1c0H+dYTq3n+q+eTntz6MwBe2RgcBRUK/KSAn8tOy2Ph\nqhKeXbWX900eyrcvP4XCnFS+9/Q6Hlmyh9++vJXpBZnMj/BHwiWnDee7T63l969u5QPT8kn0+8jP\nSmbqiMxOX5zXFlfxh9e2sWjNXvw+4/1ThnHNrALOn5jLqj2VPLG8iOfW7sMBGckJVB6tZ8v+ap7/\n6vnt/ttq66+Ld5OS4O90ba4rFBYyYHzQ+x9qRFYKswuzCfhPvMvuh1ef1l3F6hVpSQEe+tSZlB2u\nY2hG+zvmx+Wm8cAnTufVTQe466Ut/Pz5Tfz8+U1MHJbGL6+bwdSCTEoP17Fh7yH+w7vf5XTvor90\nVznzTml94f3fN7bzqDdsd3xuGtfMLuD1zQeZXZhNamLwMuT3GV99/0Tuf3MH6ckBxg4ZxJT8DD55\n7hj8PiPBD3deO41rfv82P/vnRn78wantyv3C+gOMyEph8vBjfwBcNSOfR5fuIS8zmV9eF+y8/uiZ\nhTy9soRv/30NzgWb5yJd/DNTEph/6jCeXb2X59cdG4k2cnAKV07L57LT8jglLz3if0vVtQ386NkN\nPLp0D2lJAT57/lg+de4YhoX9e88ZPZg5owfz0w9Na1n2+uaDfPyB97jvjR18Iey+o71VNaQnJ5Dm\nNVlV1zawcFUJV07PIyNCcHY3hYUMGGbGh2Yd/1/7/ZXfZxGDItyFk4Zy4aSh7Kuq5V/r93HPa9v5\n5J/e48nPn8vKouC8W3PHB5uRphdkkeA3luysaBUWy3dX8PPnN3HpqcMpP1rPd59aS15WMhv3VfPN\nSye1+rxrZxdwbQc1slmjsvn0uWO4780d5KQl8ZV5E1qajg7VNvDm1oN8ZM7IVhf+s8bmcOv7xnPJ\nqcPJHpQIgM9n/OyaqVz+mzeZMSqrpYM+kl9/ZAbfvGQy9U1N1DU2s77kEM+s3ss9r2/n7le3kZLg\nZ2pBJjNGZjElL4PJeensP1THt55Yzb5DtfzbBWP59wvHk5nSuQv6+RNzufTU4fz25a18cOYI8rNS\neHH9fr748HIKBw/i8c+fTXpyAk+tLOFofRM3nlnYqeN2lfWXWTnnzJnjli7V7fki8bTt4GGu+f3b\nZKcmMmFoGu9uL2PF7fNbms8+ePdb+M14/PPnAMHRRx+46w0A/vGl86htaOLy37zBkfpGahuaefbW\nua1uruyMusYmvvP3tTy+rIj5U4bxiw9P57k1+/j5vzZxsLqOxz93NnM60d4PsHHfIfIyUto1m3VG\n2eE63txayordlazYU8mGkkPUh40kG5s7iF9cN51Zo7KP+9hFFUeZ98vXuHjKMM4Zl8P3nlrLuNw0\ndpQe4dzxQ7j/5jlc+du38Bk8e+vcLvVXmNky59ycmNspLETkeCzdWc5H71tMXWMzl546nD/cNLtl\n3U8WbeBPb+1k9X/O50hdI197bBVvbS3lb587m5neRfP1zQe5+Y/vkZ2ayNLvXBxx5Fcszjn++NZO\nfrxoAwGfUdfYzKxRWXzviiktn9PTGpqa2VF6hA17D3G4rpFrZhWQnNBxn0NH7nppC796ITjc+32T\nh/LbG2eycGUJtz25hvMmDOGNLaX8+IOn8dEu1iw6GxZqhhKR4zJn9GB+c/1M/v0vy5h/auu+idNH\nD+be17fz00UbvGaSRn6w4NRWF/DzJ+bywwWn4TM7oaCAYJPip+aOYeKwdO5+dSvXnzGKK6flxX1E\nUEcS/D4mDktn4nEOmIjmlvPH8trmg5yan8HtV0wh4Pdx/Rmj2FF2hHte286gRD8LZsS/YztENQsR\nOSGVR+vJTElodYEuP1LPrB++AMCZYwbz4w+exvih3XPxlKDmZsePF21gRFYKn5o7psvHU81CROIq\nKzWx3bLBgxL57gdOYUhaEgtm5PfqX/r9lc9nfO+KKT3+uQoLEelWnzkv+h3kcvLS3FAiIhKTwkJE\nRGJSWIiISEwKCxERiUlhISIiMSksREQkJoWFiIjEpLAQEZGY+s10H2Z2EOjKk+iHAKXdVJyTxUA8\nZxiY5z0QzxkG5nkf7zkXOuciP3owTL8Ji64ys6WdmR+lPxmI5wwD87wH4jnDwDzveJ2zmqFERCQm\nhYWIiMSksDjm3t4uQC8YiOcMA/O8B+I5w8A877ics/osREQkJtUsREQkpgEfFmZ2qZltMrOtZnZb\nb5cnXsxspJm9YmbrzWydmX3ZWz7YzF4wsy3e7955gHEcmZnfzFaY2bPe+zFmttj7zh81s/ZP8TnJ\nmVmWmT1uZhvNbIOZnd3fv2sz+6r33/ZaM3vYzJL743dtZg+Y2QEzWxu2LOJ3a0F3eee/2sxmnejn\nDuiwMDM/8DvgMmAKcIOZ9fwjqHpGI/B159wU4CzgC9653ga85JybALzkve9vvgxsCHv/X8CvnXPj\ngQrg071Sqvj6DfCcc24yMJ3g+ffb79rMRgBfAuY4504D/MD19M/v+k/ApW2WRftuLwMmeD+3AL8/\n0Q8d0GEBnAFsdc5td87VA48AC3q5THHhnNvrnFvuva4mePEYQfB8H/Q2exC4undKGB9mVgB8ALjP\ne2/A+4DHvU364zlnAucD9wM45+qdc5X08++a4JM/U8wsAKQCe+mH37Vz7nWgvM3iaN/tAuAhF/Qu\nkGVmeSfyuQM9LEYAe8LeF3nL+jUzGw3MBBYDw5xze71V+4BhvVSsePlv4JtAs/c+B6h0zjV67/vj\ndz4GOAj80Wt+u8/MBtGPv2vnXDHwC2A3wZCoApbR/7/rkGjfbbdd4wZ6WAw4ZpYGPAF8xTl3KHyd\nCw6N6zfD48zsCuCAc25Zb5elhwWAWcDvnXMzgSO0aXLqh991NsG/oscA+cAg2jfVDAjx+m4HelgU\nAyPD3hd4y/olM0sgGBR/cc496S3eH6qWer8P9Fb54uBc4Coz20mwifF9BNvys7ymCuif33kRUOSc\nW+y9f5xgePTn7/piYIdz7qBzrgF4kuD339+/65Bo3223XeMGelgsASZ4IyYSCXaILezlMsWF11Z/\nP7DBOfersFULgZu91zcDT/d02eLFOfct51yBc240we/2ZefcR4FXgGu9zfrVOQM45/YBe8xskrdo\nHrCefvxdE2x+OsvMUr3/1kPn3K+/6zDRvtuFwMe9UVFnAVVhzVXHZcDflGdmlxNs1/YDDzjnftzL\nRYoLM5sLvAGs4Vj7/bcJ9ls8BowiOGvvh51zbTvPTnpmdiHwDefcFWY2lmBNYzCwAviYc66uN8vX\n3cxsBsFO/URgO/BJgn8c9tvv2sx+AHyE4Mi/FcBnCLbP96vv2sweBi4kOLvsfuD7wFNE+G694Pwt\nwSa5o8AnnXNLT+hzB3pYiIhIbAO9GUpERDpBYSEiIjEpLEREJCaFhYiIxKSwEBGRmBQWIifAzL7j\nzXC62sxWmtmZZvYVM0vt7bKJxIOGzoocJzM7G/gVcKFzrs7MhhC8n+FtgrOelvZqAUXiQDULkeOX\nB5SGbu7ywuFagnMSvWJmrwCY2Xwze8fMlpvZ37x5uTCznWZ2p5mtMbP3zGy8t/w671kMq8zs9d45\nNZHIVLMQOU7eRf9NgtNgvwg86px7zZuDao5zrtSrbTwJXOacO2Jm/w9Ics7d4W33v865H5vZxwne\nbXuFma0BLnXOFZtZljetuEifoJqFyHFyzh0GZhN8mMxB4FEz+0Sbzc4i+ECtt8xsJcH5egrD1j8c\n9vts7/VbwJ/M7LMEp58R6TMCsTcRkbacc03Aq8CrXo3g5jabGPCCc+6GaIdo+9o59zkzO5Pgw5qW\nmdls51xZ95Zc5MSoZiFynMxskplNCFs0g+DkbdVAurfsXeDcsP6IQWY2MWyfj4T9fsfbZpxzbrFz\n7naCNZbwqaVFepVqFiLHLw34HzPLIjjD6VaCTVI3AM+ZWYlz7iKvaephM0vy9vsusNl7nW1mq4E6\nbz+An3shZASfo7yqR85GpBPUwS3Sw8I7wnu7LCKdpWYoERGJSTULERGJSTULERGJSWEhIiIxKSxE\nRCQmhYWIiMSksBARkZgUFiIiEtP/B1/5ZVzGrAl3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12c8b49b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model._add_tensorboard(logPath, sess)\n",
    "sess.run(tf.global_variables_initializer())\n",
    "\n",
    "# training iterations\n",
    "trainLosses = []\n",
    "valLosses = []\n",
    "for epochLoop in range(numEpochs):\n",
    "    sess.run(training_init_op)\n",
    "    sess.run(validation_init_op)\n",
    "\n",
    "    for iterLoop in range(100):#range(int(trainExam/batchSize)):\n",
    "        imagesBatch, labelsBatch = sess.run(next_batchTrain)\n",
    "        model.trainStep.run(feed_dict={images:imagesBatch,labels:labelsBatch})\n",
    "        trainLoss, trainSummary = sess.run([model.loss, model.merged],\n",
    "                                           feed_dict={images: imagesBatch, labels: labelsBatch})\n",
    "        model.train_writer.add_summary(trainSummary, iterLoop)\n",
    "        trainLosses.append(trainLoss)\n",
    "\n",
    "        #test loss\n",
    "        imagesBatch, labelsBatch = sess.run(next_batchVal)\n",
    "        valLoss, valSummary = sess.run([model.loss, model.merged],\n",
    "                                       feed_dict={images: imagesBatch, labels: labelsBatch})\n",
    "        model.test_writer.add_summary(valSummary, iterLoop)\n",
    "        valLosses.append(trainLoss)\n",
    "\n",
    "        if iterLoop %10 ==0:\n",
    "            print('Step %d, Train Loss %g, Val Loss %g' % (iterLoop,trainLoss,valLoss))\n",
    "\n",
    "save_path = model.saver.save(sess, logPath+\"model.ckpt\")\n",
    "plt.figure(1)\n",
    "plt.xlabel('Steps')\n",
    "plt.ylabel('Train MSE')\n",
    "plt.plot(trainLosses)\n",
    "plt.figure(2)\n",
    "plt.xlabel('Steps')\n",
    "plt.ylabel('Test MSE')\n",
    "plt.plot(valLosses)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to test the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True: [ 73.  19.  23.  49.], Predicted: [ 54.  14.  27.  59.], Error: [ 19.   5.  -4. -10.]\n",
      "True: [ 56.  19.  65.  58.], Predicted: [ 49.   8.  56.  33.], Error: [  7.  11.   9.  25.]\n",
      "True: [ 39.  11.  76.  76.], Predicted: [ 53.  14.  48.  30.], Error: [-14.  -3.  28.  46.]\n",
      "True: [ 36.  25.  59.  98.], Predicted: [ 50.  16.  43.  67.], Error: [-14.   9.  16.  31.]\n",
      "True: [ 37.  16.  60.  82.], Predicted: [ 48.   4.  45.  64.], Error: [-11.  12.  15.  18.]\n",
      "True: [ 48.   9.  41.  28.], Predicted: [ 49.  15.  64.  45.], Error: [ -1.  -6. -23. -17.]\n",
      "True: [ 43.  25.  83.  99.], Predicted: [ 36.  17.  75.  76.], Error: [  7.   8.   8.  23.]\n",
      "True: [ 26.  21.  37.  59.], Predicted: [ 41.  34.  47.  55.], Error: [-15. -13. -10.   4.]\n",
      "True: [  32.   14.   93.  108.], Predicted: [ 35.  24.  68.  76.], Error: [ -3. -10.  25.  32.]\n",
      "True: [ 31.  18.  69.  78.], Predicted: [ 48.   7.  57.  62.], Error: [-17.  11.  12.  16.]\n"
     ]
    }
   ],
   "source": [
    "restoreMode = True\n",
    "\n",
    "imagesBatch, labelsBatch = sess.run(next_batchVal)\n",
    "labelsPred = model.labels.eval(feed_dict={images: imagesBatch, labels: labelsBatch})\n",
    "\n",
    "error = np.around(labelsBatch[:,:4] - labelsPred*128)\n",
    "for line in range(10):\n",
    "    print('True: %s, Predicted: %s, Error: %s' % \n",
    "          (labelsBatch[line,:4], np.around(labelsPred[line,:4]*128), error[line,:]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's no consistent pattern in accuracy from looking at a sample like this, suggesting the error for most of the coordinates is around the same. The test MSE corresponds to an arror of around +/-22 pixels which covers around a third of the image.\n",
    "\n",
    "With the model having converged at this level quite quickly, it appears it does not have the depth to complete this task to any greater accuracy. Although typically, models like these would cycle over the entire dataset many times (versus about 3% of it here), so perhaps given more time and with optimisation of hyperparameters slow gains would be made.\n",
    "\n",
    "The conclusion from this data though is that the a more complex model is likely to be required. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}