Navigation_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Navigation\n",
    "\n",
    "---\n",
    "\n",
    "In this notebook, you will learn how to use the Unity ML-Agents environment for the first project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893).\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing some necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "from dqn_agent import Agent\n",
    "import numpy as np\n",
    "import random\n",
    "from collections import namedtuple, deque\n",
    "from model import QNetwork\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Banana.app\"`\n",
    "- **Windows** (x86): `\"path/to/Banana_Windows_x86/Banana.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Banana_Windows_x86_64/Banana.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Banana_Linux/Banana.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Banana_Linux/Banana.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Banana.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Banana.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env = UnityEnvironment(file_name=r\"C:\\Users\\jason\\Desktop\\navigation project DRL\\Banana_Windows_x86_64\\Banana_Windows_x86_64\\Banana.exe\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "The simulation contains a single agent that navigates a large environment.  At each time step, it has four actions at its disposal:\n",
    "- `0` - walk forward \n",
    "- `1` - walk backward\n",
    "- `2` - turn left\n",
    "- `3` - turn right\n",
    "\n",
    "The state space has `37` dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction.  A reward of `+1` is provided for collecting a yellow banana, and a reward of `-1` is provided for collecting a blue banana. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions: 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length: 37\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents in the environment\n",
    "print('Number of agents:', len(env_info.agents))\n",
    "\n",
    "# number of actions\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Number of actions:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "state_size = len(state)\n",
    "print('States have length:', state_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agent's performance, if it selects an action (uniformly) at random with each time step.  A window should pop up that allows you to observe the agent, as it moves through the environment.  \n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agent is able to use its experience to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 0.0\n"
     ]
    }
   ],
   "source": [
    "env_info = env.reset(train_mode=False)[brain_name] # reset the environment\n",
    "state = env_info.vector_observations[0]            # get the current state\n",
    "score = 0                                          # initialize the score\n",
    "while True:\n",
    "    action = np.random.randint(action_size)        # select an action\n",
    "    env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "    next_state = env_info.vector_observations[0]   # get the next state\n",
    "    reward = env_info.rewards[0]                   # get the reward\n",
    "    done = env_info.local_done[0]                  # see if episode has finished\n",
    "    score += reward                                # update the score\n",
    "    state = next_state                             # roll over the state to next time step\n",
    "    if done:                                       # exit loop if episode finished\n",
    "        break\n",
    "    \n",
    "print(\"Score: {}\".format(score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Paramters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "number_of_inputs = len(env_info.vector_observations[0])\n",
    "number_of_outputs = brain.vector_action_space_size\n",
    "hidden_layer = 64\n",
    "buffer_size  = 100000\n",
    "batch_size = 64\n",
    "gamma = 0.99\n",
    "learning_rate = 0.02"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize the model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = Agent(number_of_inputs,number_of_outputs,seed=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100\tAverage Score: 0.70\n",
      "Episode 200\tAverage Score: 4.09\n",
      "Episode 300\tAverage Score: 6.99\n",
      "Episode 400\tAverage Score: 10.18\n",
      "Episode 488\tAverage Score: 13.04\n",
      "Environment solved in 488 episodes!\t Average score: 13.04\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEGCAYAAACD7ClEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd7gkVZn/v29Vhxsn3QkMYRwyDEgcsuIgQQyIWXFV1p8ruiqGdVVkjasr7ArGZVVQ1oxZYBmCMBKVNMAwERgGhsnhTr6xu6vO74+qU33q1Knq6r7dfUO/n+e5T9+urnCq+t73PW88JIQAwzAM05pYoz0AhmEYZvRgJcAwDNPCsBJgGIZpYVgJMAzDtDCsBBiGYVqYzGgPIA3Tp08Xc+fOHe1hMAzDjCueeOKJXiHEjKR9xoUSmDt3LhYvXjzaw2AYhhlXENFLlfZhdxDDMEwLw0qAYRimhWElwDAM08KwEmAYhmlhWAkwDMO0MA1TAkR0EBHdS0SriGgFEX3C3/4VItpIREv8n9c1agwMwzBMMo1MES0B+LQQ4kki6gbwBBHd7X/2bSHENQ28NsMwDJOChlkCQojNQogn/d/3AVgF4IBGXY9hmImN6wr8bvF6FB13tIcyoWhKTICI5gI4EcCj/qaPEdFSIrqRiKbGHHMZES0mosXbt29vxjAZhhnD3LxkIz77h6X44X1rRnsoE4qGKwEi6gLwRwCfFELsBfADAIcCOAHAZgDXmo4TQlwvhJgvhJg/Y0Zi1TPDMC3AnsEiAGBHf2GURzKxaKgSIKIsPAXwKyHEnwBACLFVCOEIIVwANwA4tZFjYBhmYsGrIdaXRmYHEYCfAFglhPiWsn22stubASxv1BgYhpk40GgPYILSyOygswC8F8AyIlrib7sSwCVEdAIAAWAtgA81cAwMwzBMAg1TAkKIh2BW3rc36poMw0x82BlUX7himGGYcYHnYQY4JFBfWAkwDDMu8HUABNsCdYWVAMMwTAvDSoBhGKaFYSXAMMy4QGaZcEygvrASYBhmfEBcKdAIWAkwDDOuYEOgvrASYBhmXMDuoMbASoBhmHEBe4MaAysBhmGYFoaVAMMw4wz2B9UTVgIMw4wLCNw2ohGwEmAYZlzAMYHGwEqAYRimhWElwDDMuILdQfWFlQDDMOOCoE6AA8N1hZUAwzDjAo4JNAZWAgzDjCvYHVRfWAkwDDMuCFJER3kcEw1WAgzDMC0MKwGGYZgWhpUAwzDjA7nGMPuD6gorAYZhxgWNShEVQmDr3qG6nlOlkeeuB6wEGIYZF1CDckR//vBLOO0bi/Dc1n11P/fDa3bgtG8swm1LN9X93PWClQDDMOOLOruDHlzdCwB4acdAfU8MYNXmvQCAxWt31f3c9YKVAMMw4wLRoGCAPK/VAEPD9k/qjuFABisBhmHGBY0So26gBOqvBaRicVxWAgzDMCNCztjrLU6lfG5EyMEKLIH6n7teNEwJENFBRHQvEa0iohVE9Al/+zQiupuIVvuvUxs1BoZhJg7So1Jvt5A8W2MsAbkQztjVAo20BEoAPi2EOBrA6QA+SkTzAFwBYJEQ4nAAi/z3DMMwiQjttW7nZXdQYxBCbBZCPOn/vg/AKgAHALgYwM/83X4G4E2NGgPDMGOTpRt2Y/7X78HugULqYxo1mZYxgYa4g6h6d9C//99KfOb3T9d/MDE0JSZARHMBnAjgUQCzhBCbAU9RAJgZc8xlRLSYiBZv3769GcNkGKZJfP+vz6O3bxiPvLAz9TGNyrBx3cadv6wE0p/7xr+9iN8/saHuY4mj4UqAiLoA/BHAJ4UQe9MeJ4S4XggxXwgxf8aMGY0bIMMwo0Y1s+/AHVRnWS0FdCNcNpYVvsZYpKFKgIiy8BTAr4QQf/I3byWi2f7nswFsa+QYGIYZe9QkExtVJ+C/NkQJ1OAOajaNzA4iAD8BsEoI8S3lo1sBXOr/fimAWxo1BoZhxiq+H76KI6QgbVRguNRIJTCGtUCmgec+C8B7ASwjoiX+tisBXA3gd0T0AQDrALy9gWNgGGYMIif11fQDCuoE6mwRSPncCEE9HiqGG6YEhBAPIV7Rn9uo6zIMM36oxhJodMVwIywBeX9jWQlwxTDDME1nDIUEAkugXjGBm5/aiBd7+wGo8Ybqz7N+5wC+ePNyrG5Ad1MVVgIMwzQdUUNuvtugthGiztlBn/ztErz2uw8AUMZcgwbbtm8Yv3jkJWza09j1CFgJMAzTdKRIrKlAaxykiA4VXf/c3nunBiVQbmxXt2EZYSXAMMy4oGHuIN9VU4+YgB5clhZALaeW57IbtJiOhJUAwzBNJ8gOqiI03KjgauC3r8P59XOMxB3kBC4zVgIMw0wwApFYS8VwA9YYBgCnluithu5SklZGLa4mqTfsBvuDWAkwDDNqVJUiKsKv9SKICdThvBElELiDarAEXI4JMAwzQanFPRJnAQwUSti2z8ugWb9zANv2DmHfULGKsXivjpveEujtGzZeQ48ryHPXFBNokjuokRXDDMMwiVRXMRx+lbzlf/6OZ7bsw6NXnotX/te9AIBpnTk8+cXzU523lmKx+V+/x3gNPTAcWALsDmIYhilTyyph5eUlw8c8s8Urptq+bzjYtrO/+nUKqhXUpmvoiiRoScHuIIZhmCjVyMZK+9aa4lnPthFxMYFa4g3lOgG2BBiGmWDI2Xw1M+RKMrrWFNKiU79iMT1FdCRN71gJMAwzYaklYCoVR5w8rbULaNFPDa2LEnDq5w6Sx3JMgGGYCYeoQTgGcYSYzwul2vL8pRuoHkqgpGUYlVtSVH8ujgkwDDPhqSowHBxj/nyg4NQ0hqKvPOrSNiJSMey9jsQdxBXDDMPUhX/62WLcvmzzqI5h/c4BHPL5hXhmi7fceFXuoAr+9X/6+eKaxlR0a3cHnf1f9+Jnf18bvI/WCVQX+zjbT3H1jvVe2R3EMExduPfZbViyfveojuHO5VvgCmDXgFdoVYs7qN49hKQbqRYlsG7nAL5864rgfSkSE0jvanJdgXU7B4L37A5iGKZuuK6A44qIkGo2BUf3mac/NggM13E8JccNxtBYd1DlYweLYXcWZwcxDFM3pMtjtJc5LGpKoBpfeTnTpn7jGVaCyfVYYzhaLCbrBCqfW49pBEqA3UEMw4yUUh1z4UdCVAmkP7aWKuNKqBlFaS2BpOtH1xPwt6cY86CmBOSj4vUEGIYZMVIJNGIx9VrGIakqJlBDgVklVPdU2vMmKdKIJeDK3kGVzztQLGnX8Q7imADDMCNGCrt6uDzqMQ5JVcMJevzUcTw1WAJJ+0XbRsjX6i0BeR12BzHMBGLFpj24+amNTb+uLGKqx+pZALB84x78+akNVR+nu4OqaxsRbSB3Rw0pr/c+uw1X/nkZNu4exHCpLHj/7+lN2La3vKj735/vxb3PbMOfn9qAL9+yHHsGvYymJEsgtneQtn35xujfQdQd1JzAMLeSZpgm8vrvPQQAeNOJBzT1usVSfWMCb/i+dx9vPvHAmsYhqa6LqPeq3sI//+rJqq4PAN+5ZzWeXr8bR+/XjZNeNjX02b/+YSl+/v9OBQC8+8ePhj5bcORMnHPUzERFqj9feX+69SCfn/p3oAeGZU8jdgcxDDNiRlIQVddxjChF1H9NqTji9pO+9uGSG2k1MZRQdRz0GEpIs41rJZ2mpcWAliJa8q/H7iCGYUZMIMBGOUU0GhOo3h2UVnEUY4S1dK8Ml9xQiigAtOfsiuerLibgvU+jBAYL4cBwsUnuIFYCDNMCBCmio1wsNiJLoMqKYb2Zm47JEuhIUAKlFLUWcYHhguMag/LqNt0dJC2WcZsiSkQ3EtE2IlqubPsKEW0koiX+z+sadX2GYcoU6mgJjCRPX08Rra2xWrr99PhDsN0pz851JZBkCRRSNJrTFY96f7oVpG/TlYB8Vg3WAQ21BH4K4ELD9m8LIU7wf25v4PUZhvGpZ7FY33Cp8k4xRNxBVYxHlKvFUu1fjLEEpK+9UHIj40k6ddByOsGairaNSK8E4lJEx20DOSHEAwB2Nur8DNNshooOXuztH+1h1IS6cEq19/Hc1n2hGW016/fGjUOyeltfZBsAbNkzhD1+kzkAWLO9DwWn2piAWQnI7QXHCaWIAsBAIV7BFR0XO/qGsUVJI5XIZ6paOmu294UsDVNcYPnGPcq1YwLD49UdlMDHiGip7y6aGrcTEV1GRIuJaPH27dubOT6GMfKvv38a51xzX6KgGKtIwecKgc/+YSnOueY+9KeY0d//3HZc8O0H8PvF5ZoAVQlU687Rg7W/enQdvn7bysh+p1+1CGdevQgAsHnPIM699n7c9Ni64B7SXDuuWZ4cw3Ax6g5KWpOg6Aic/PV78I4fPRz57BO/eQrnXHNfqAncudfej589/FLwXg9CA8C7b3g0UAR6A7lyYDh2SHWh2UrgBwAOBXACgM0Aro3bUQhxvRBivhBi/owZM5o1PoaJ5YHnvMlIrStYjSZBZosj8Pc1OwCkc+us2dYHAFi5eW+wTb3/ar1Lptn5wy/sMO7b7wvkHX1hy0Ne05T9o7pOTO4XdQwFp6wEHvzsOTj+wMkRl0ylsUvuWrEVgKdY4oj7u9m4e9A7VrNKHEeAaIItKiOE2CqEcIQQLoAbAJzazOszzEiQIofQ4KlZAygpgeF8xvu3T6PMTDJeDYxWG2MwXbPa2HC5ACt6rk4lsBtnCcjxF5QU0SkdWczozidaAqUEJSCJUzzyeonHap8XXbfhriCgyUqAiGYrb98MYHncvgwz1pDCisZhYrV0LbhuWQno7ockVFlUS9M1iSmzptI59I/l+7jsH0mlmIBaJ5DLWGjPZRKfSRqlaXL5lD9Lft76+R1XNDw9FGhg2wgiugnAAgDTiWgDgC8DWEBEJ8CbYKwF8KFGXZ9h6k3ZFz3KA6kBdR3dnFQCKdbkNfnd1Rl2tZaASTBXep5Cs0fk9xCX/ZN0LXW7miKasy10ZO3EZ7IjRUA8SVHEfSafp65ASr47qNE0TAkIIS4xbP5Jo67HMI2mnv3shRAN9/WqqIVOUgnUvDC7IlyrbU1dNLmDKhyjP26pBCqtkhY3tpJaJ+C4yNkWiAjtOTsx6L/VkBWkkzTbj1MC0vqIuIMct+HpoQBXDDNMasr97Ed+rma38CkodQJld1DlwLBJUalKoNrW1AWD4NbdQbrvXf88cAdV8NEbFY4QgXIYdlwMF93geXTk7ER3kCk1VCfJEhiOGa9sF6HHExxXTLyYAMOMZ6rpDV/5XM3RAr99fB0O+fzCoDGa45otgbtWbMHBn1+I7y1aDcDLHJp7xcKgVfP//m0tfnDfGgCaO6jqFNGoINSfhSoMj/vKXfjh/WuM++vnmtyeDVkV7/7xo/iRf+xF338IX7h5WSijaLjooOA4wfPozGdQdAS+e89q49i37BmucHfJSuD9//s4Dv+327Hgm/eGtg8UHFx83d/wxEu7QtsXPbOtKX8nrAQYJi3VFawm0iwl8LXbVsEVwN6hci/8nB1VAis37YUQwDI/Z/2lHV4x2WJFMP3nnc8AGJklYIoh6K59Nc1y71ApSL8M9tdSRF9x2HRc9+6TsPDjr4ic+6o7vDEv27gHv3xkXWjsBcdFyRHI2N5s+y0neW2dVynpsCp7B4vG7aGxVwgeFx2BtTsGQtsGCg6eXr+7pvPVA15PgGFSIt1B9YkJjPgUqZBCtxBKEfXSKNUgqHSDSCGZNL6iW7slYErr1J9nUpolUP4e5Fjfd8bLcMEx+wFAxeRd1YoplFyUXIGM5SnF2ZPbcdyBk2P9+pXGJc9ZLdVkaTUCtgQYJiWmRU1qpVmWgLxOJXeQDIimEWKqr73a7CCjJaBtqjQGqUekEsja6cWYKsiHSy5KjhtYAoCXJZRG2Kc5v0TGHOJICkY3Y/0HVgIMkxLT8oa1n2vEp0h5He9CcrbpuAIZP+NE7V8vFYIUwMmN1FR3UHXjMWXs6M+zkgtEX61LFeImVJeVHHtb1lIsAUUJZKwRVYSbrIikzqRA7Vla9YKVAMOkxB2HloCcSQ4Wyw3k5LVV4SNdQ3Imm6ToiiMIDJt2159npaKqICZQqmwJZG0KuVtkgVlXPoNCyfWVYvn4kSoB07FtmWQlkKZeo5GwEmCYKqk2GGpCNKn9kBzqoOIOkjJcXc5QKgQZlI2T7cMlJxRcdao1BQxEYgKV3EFBsZj3mk2wBLK2FVJ2ssCsI5dBwXFRdEQoFz+fsUYUjDUqgWyymB1Ja+56wEqAYUaBkVoCa3v7g/THnz+8NpTRsmT9bvz28XWh/Yd8ge8KEQjuwURLwMyTL+3Gdfc+H7wfgfs8QNWpQghc85dnU+2fxhLIWBS6T6nAOvMZOK7AcMkJKZFcxsYzW/ZFnl9aTAqkUsziwdW9NV2rXqRWAkT0CiJ6v//7DCI6uHHDYpixy1ioE3jfjY/hqjuewc7+Ar50ywq89rsPBp+96bq/4XN/XBbaXyqBkitCFbOSgWK6wPAlNzwSdgdVYRXF7as+ixd7+/G3581dRfX9pX9fdedc9w8n4YxDeoL3WdsK7g0oZwd15csZUqolINNn9eeXFpMSsC3C8QdNMe5/+Myumq5TT1IpASL6MoDPAfi8vykL4JeNGhTDjGXGQsWw9JunXdtADQxLIaoK5cAdVCpbDGmoRpnFVfiq7rU0mT7BymC+QM9lykL8lYfPwE2XnR68z9gUcgdJS6cj52XH9xeckBLJV3DdVML0fRARbnjvyZHtX3/Tsbj1Y9HahmaT9o7fDOCNAPoBQAixCUB3owbFMGOZevUOGgkyzXPfUHVKwHXLbRNUAT4UKAE32C8N1VgCcb18RIp9TNeUykAV4joZywq5g8qWgKcEBgqlSIpoLST1Y7Ko/LlK1qaK8YJmkHYEBeH91QoAIKLOxg2JYcY2Y8ESkMJqT4oqVqAs5EuuCISoqgQGtCZmaRvDVZMdFNePXz1Fmp79xUAJ+IHhhDz8rF2OCeQyVnD+Tt8dNKC5g+Jy+isph3xCZ1aLKEYJWE1tIhhHWiXwOyL6EYApRPRBAPfAWxSGYVqOelgCI40JyKrf3QPplMBgKDAc7w4qOC6EELGWgN7UsppMqbgirEqLsevoVdDZhE6bGdsKFFw+Uy4E65SWwHApUidgopKbSH4fJuVpkVmJZGq0OupNqrYRQohriOh8AHsBHAngS0KIuxs6MoYZo9Qjw3+kSkAKqz2DXo/7ShPKIaVOQAoqGd91XIFCyQ2qZYuOiLUEsnY4hbIqd1BM62f1WVRqDy2vKYQIZvXJdQJWUBSXz9jB+Tv9mMBA0QkJ4zhLoC1rJ7rekqqCicgo8HMVityaRUVVREQ2Ed0jhLhbCPEZIcS/sgJgWplKAnzlpr1BgHXN9j7sG4rO1mvRAXsGiljb249te4eCxd6lJZAxzIbV/vdlS6AsuKVFI4OZUzqyALwZdpxw12e0jiuC+90zUMSLvf2x448NDAvg6fW7IYSo2B5asnnPENbt9NbmTaoYztqEB57zUjDzGSuUIgp430MaS6A9m1zwlWQpxBkqSbGMZlLREhBCOEQ0QESThRB7mjEohhnLJNVHbdo9iNd970G8+7Q5+MabX45zr70fLz9gMv7v8nAWSC2WwOu+92CwKLlktx8TMC0+cto3FhnPI/3+5WpiT0FMas9i277hoJLWRDZjAUpH5Q27BvHuHz+Kd582B/c+sw2b9wxh7dWvNx5rWhhejufi6/6Gr7/pWBw6I13K5JlX/7U8pgRLYPdAEUs3eO2w23N2EE+QMQEg7JaJ8/13VGj9kFQVHLcmgCmWcf68Wbh75VbD3o0jbRfRIQDLiOhu+BlCACCE+HhDRsUwY5iklgo7+rwZ+tIN5dbAsj2zSi2BYV0BAMDuAe961axFqysB+SoXaR8uObHuIN3i2N7naYQl63Zj857kRVcqzfJXbd6LOdM6AAAzuvPYvq9y/36Lkl0xO5UlIS0qWz9tysxevad8zIx/Rncez2zZF3sd1YKY0pENxWpilYD2LH/5gdNw1mE9eOj5Xrz3J4/FXqvepFUCC/0fhml50jRXq2Tq16t3kBQ2VhXLEMrgqF4vIBudJVoC2kxZ+tjTeDYq+fuLjhs8v2kduVRKoD1rJ2bYyJYMsyblvZYZ8l4VYa9aUXFWxczutsRxqIpo9uT2kBKIG55uCfR05UBEmNyeTbxWvUkbGP4ZEeUAHOFvelYIkS4tgWEmGEkCPOhsaVFiFlE9MoyA5JhAHNISkHJeCkZZQFUoubGpn7rPXLagIKWTf9z6yZUyf0qOQMFv8Ja2aKs9l24eO7O7Df3DpeCeVSWgto2I+15mTsonnl+1LA6Y0hZq4xFnCejfmXy2zVhSUiVtxfACAKsBXAfgfwA8R0RnN3BcDDNmSVxwRRYw2ZSYOVOvVtLlmED6IOOw5g4qabPj4ZIb2xhOl0/Sx67KszhhX6kGoKBYApV68Esq+eolM7rzcJTUV7W9s2oJxH1nM7uTlYCqHGdOClsNsZaAb3XIz3Pa+2aR1h10LYALhBDPAgARHQHgJgDRWmiGmeAkWgKygMm2Egup6ucO8mMCVSSaFLTWEG5gCZSVQJzrRheSegsH7/xukDevEhcYLn/uBkrUdLyJNBW3OdvClPZsqGWGepzquouLhVTjDtIziWJjAv6XlrEIRUcE1o8pyN9I0v7pZKUCAAAhxHPw+gcxTMuRNIsvxwQoMYuoDh2YAajuoOpX19ItgQ4lJhCnpHTlIAX7LiUAqzehk0qmWOGmS44IzpfWEkjjOpnWmYNlEYQoVzirSkZ1y8Td96wK7iD1fPrY44YoU1ul0M/b3jnGpDsIwGIi+gkRLfB/bgDwRCMHxjA6Dzy3HXOvWIjNe7wsmX//v5U47it3NX0cSf58KcQyKSyBXzzyEuZesdDYufNrt63E3CsWBj9/fmqD8TwyvTPN7FHKlriYQJuvBC654RFc85fnjOfQM3yka2nXQFkJ7OgvYO4VC3HzUxvx4wdfwCFX3o6bHlsXWpbSREG1BOrYU2daZw4WyeZ53raQO0iJCcQFZXu6KigBZbx63CROqEv3z+EzvTZsUik02RBIrQT+GcAKAB8H8AkAKwF8uFGDYhgTv37U6/G+ZJ2Xfnnj317E3pQN1OpJklNDzpQzVnJMQAjgmrs849q0qMhPHnox9P7aGKEsiQsM/9Mryh3f5Qw1WCFNSxFVZ7NxmTnSarjxH+cDKHcdVauI12zrAwD88P41WOkHSJdu2F2xH1HJKbe5TusOiuOBz5wT/N6WtWBbBEeIQIGHAsOKFXXRcfvj2rcfHzlfxiL89P2n4PJXHwYgOrtX6wRyGQu3f/yVOP7AyQDKQv32j78S33nnCeVz+kL/f99/Cm543/yggK3Z/YTSKoEMgO8KId4ihHgzgO8BGNm3xDDjlKR+OYE7yLYS9wt18CxGm47p7K3QKC4uRfQtJx0Y/N6m+aqlpVJy07tgio6LA6a0B7NXuRKZiqoQZEO13r5CxTqBcExgZJbAnJ4OzH/ZVACelWQRwVVSRNtiUkQti/DWkw+ETsYmLDhyJg6Y0g4A0J+2OvvP2Rbm7T8JJ87xri8tgXn7T8JJ/jagHBOY3pXH+fNmlccwRpXAIgDtyvt2eE3kGKblSJrQSldL1p95xp+j/NlgGiVQweKJswRUt4cuWIPAsJDB7MrCp+QIWFZZcJrWA5bbiMrr++7sL1QODLvVxwSSkMLUtgi2RXBF2R3kKQbv9zTptfJ+pbLVBbU6XllwJs+rzuzV0E02Jo4zVt1BbUKIPvnG/72jMUNimOqoV8596uslOITkLDhNiqiUDfVYaDxu9qi6PXRLQFoqakZTJRxXwCIKhOJQgiVAKHcn3dE3XDFFtOS4wT5xlbvVIGWstAQct5wialFZsJuau+mPU1ZkWxSjBJSYQF5m/dgy77+8nxrAz2bM31m90ofTklYJ9BPRSfINEc0HEK1hVyCiG4loGxEtV7ZNI6K7iWi1/zo16RwMk4ZqOlnWgySdIy0B27IqxASibZzrge6CUlMh9f420lKR44xrnqZSdF1YRIEQNFkCBYM7aEcKd5CXHVTOrhopqsC2iOCKstUjFUPctfQ2HFJ4B/pCO0Sd1cvnmDFYDarrKS6jK65Go1GkVQKfBPB7InqQiB4A8BsAH6twzE8BXKhtuwLAIiHE4fBcTFdUMVaGMZJ2AZR6kZTjL9Mvc1UUi5mWJKw2V1xeS0/DVAOsek693NUR6S0BIcKz6CRLACjf277hEvqGk5Vd0XFRdAVythV7/9U8l7A7yHtG8l5VRWY6px5jsW3dEogfl1QCpvOGW1SY7yVlI9W6kVgsRkSnAFgvhHiciI4C8CEAbwFwJ4AXk44VQjxARHO1zRcDWOD//jMA98Fbu5iZANy9cisGCiVcfMIBdT3v7xevx6xJ8cU6BceNuDrS8pcVWzBQcFB0XMya1Iazj5hR8ZjFa3fhxd5+vO+MuZHPZKDUtqyIslBn/6GYgD9bXrh0M25eshEvm9aBfMaqykJ4dus+3LJkI849elZouypodBdLuXdQdcFYiyiYKZtjAt75ViqtE4Bwa2sTL/T247EXdyJjU6x7K1tBuarIU9hEsPyYgHzsROWZukkY65aAlN12TExAlffyOcrzqu7DwAVlUWwWUKnJlkCliuEfATjP//0MAFcCuBzACQCuB/C2Kq83SwixGQCEEJuJaGbcjkR0GYDLAGDOnDlVXoYZDT7488UAUHcl8Jk/LAUAXHjMfsbP0yxEEsdlvwiXu8S1QVb57qLVAGBUAgVHBkWjbir1rSqQpLC/6bF1eOj5cu/7Shw+swtTO3J4bO1OAMAnfrMES750fmgfdeYZyQ6qISYgzylnxmZLIKwYpnfl0ds3jB1KQVkcT7y0C5PaMolVtvKaR+3XjWvfEU3nlASzdpkdpLSNsImQz1rYN2xuuSEVxCWnzsFw0QliK3ExAdVykD2YTjukBy8/YDLOUxRzWfHEP+sjZnXjvKNnougIvP642bH71YtKSsAWQuz0f38ngOuFEH8E8EciWtLIgQkhroenaDB//vwmh0qY8UTahUiagdqmWbcE1PfqR/r6viENru4AACAASURBVIAnLCr10JzakcPvPnwG/uW3S/CnpzZ659CehTrbbKuYHZROCZBiCZjSW/W00eldOfT2DRsX14nuV0AuY8W2wVDH+LP/d2qihSgVoO2PV3cHTW7PorevYFyURgr10w+ZFprUSOGvH6E+5xndOQDAKXOnRdaRKAej491aWdvCjy89JfbzelPpW7eJSCqKcwH8Vfksbd8hla1ENBsA/NdtNZyDaXH0GcFYVAJCiIhvV7UMwu4gz2+uzqDTtIaWbgZ13ySrSHcH6W0j0qSIAl5wVE6eVSUgj9eznXq6PKHYVyHNVaazZqz4BdjVIG57heZxqgvHcweVA+eWRZjSkYucUz9Wn/EHlhWZ9weAns746mI7hSXQbCqN5CYA9xPRLfCygR4EACI6DEAtq4zdCuBS//dLAdxSwzkYJkSl/PNmEnToVBZ0l4iQO0hd3jFadZsmO0aeT903SSFGLQHvNcgOSimYwjGB8vUmt3tCVa97kEJx31ApMbDbkfXmldkMxe6nCs+OCnEgUtxBcrwlVwQCW7aISArgRpUAjNvV90nrAchxpFW4zSBxNi+E+A8iWgRgNoC/iHJky4IXG4iFiG6CFwSeTkQbAHwZwNUAfkdEHwCwDsDbRzZ8phXR/30q5Z83E7UvTzQmELYEhvx95cxZdeWkyYIJ/hlDSiBeIcbVCVSTIgr47iArqgSmdmTR2zccUQKyMd2+oaLnlomps5Az+6xlxRZMyTHmbMuY368SWAJU/r3kimDsUlibnlk5cyi8PVAsCZZAkhVnWeQHpceOJZBmjeFHDNuSG5l4+1wS89G5KcbFMKmptFhJo3BdEfmHH3YUd5AWE1DfewuoePsGlkDR9XrcKIIqCTknUy2BpMIzNUVU9tIByu6gtErAJk8YEoWtG7lQvT4GOUneN+xbAjFDlMoia1sVF2Kp5AoCVEFuBd9T0XEDQS6VwB5DSw75/HW3lJzJ69ur6feTsSj1s24GY2ckzLim2qrdp9btCvzJjivw+NqdFY4AntvmrfGqZwiu3zmAl3b044XtfaE0xP7hEpZtiHothRC4a8WW0DrAtWCqT5BBUXUpw+C6iq7qV5rGDSiWgFznt5r+MarCuG3Zptj92rTVtPTlJdP6qQPhqo1xUpuvBGLaYOwbKiW6uaQSSE4RtUL7phqn0uZi5aa9EXeQSQnE3WPZTWTePw0WUV2K4eoFKwGmLgxXaBOssnH3IN78P3/HF272ism/u2g13v7Dh/HES7sSj3thez+AqL/9w798Eq/65n149bX347RvLAq2X37TU7jovx+KdOlcumEPPvSLJ/DG//5b6jGblJwpX1365NXq1GB/5X2/UiA2VCpnB3X5nSTTBLvl2VRB9aP7X4jsd/TsSQAQyqTJ2la5WKxaJSCzbjRB1paz0Z61IwHgc4+aFTnWhFwqMp+JdwfJdXlTWQLKOOX5Hn1xZ/C8XnWkVxNyytxo44KgrXPEHSRfzXUCLz9gcsVxZSwaU4HhWjJ8GCZCNf1v9vgLoSzf6M3Sn93iFRVtq1BMJHFdYezBr/PkOk+pFEsuoCRsbEl5HRWjwHddtGvNdGWhj2uwBMKdQ8vjD1bnKrnBMobDRRfH7D8JKzaFC65U5OlsLcj46w+ehnff8Gjw/s8fORN7h4ro3VfO089n7MACKbeSTmsJeK9Z2wop/3zGwrTOHLb3lZNb33P6HJx79MzAzaXOgL/7rhPwmmP2w1FfvBNAOdA7rTMf6w7L+feazhKQr2HLQv5+0pypeOZrFxoLDePcPnZMTIAIeP4/XltxTICnlMZSYHjsqCNmXDOQohOmRArDWvumOymVgEQX3zv6KhctRa5psgQMAUWZoqkWJknU92pqpRTCwyUHHb4lMOy4kWUK49BdC7Mnt4fet2VtzOxuCwnWfKbc26haS6Cc5hi+bj5jYXpXDjuVorD2rA0iCgR8uG2CFV6W0Rfs07tyCSmivjsoW3n+qncRDbYnFNDp+8S5gwi6JUDIpAhWy3OMJUtg7IyEGdcMGvrfxCHlqfxfVEv50+AIgWEnvdLRZ+Q7+iqVYUUxVfKbYgJymyOiikPdXVUCJVeg5LhwBdCVLy/xWMnPHBeF0QVXsF35b5eBSaG41tIGK4PGa5ogy2fsyApcshpXCviQMKbwREDu09OVi08RrcIdRIoloF+3EnrX0ODYmJhANRMa27ISi8WaDSsBpi5U0+dGrdoMk+4fw3GFcTGTOPReLGnaF0SuabAETD1epBJwDXUCcQvJlBwRZDh15soz3IpyRcvuAfxVtGIEjNoeQdYEOK6oulhMCjy9riDnu4PC1/ReOwxKQBec0rqb1JaNTxGtyh1UtgTUa6XJvAqEvSYhrRg3UTVx3rEWExg7I2HGNaoSqJQpJIOegSVQ5bUcV1SVFqpX0dakBEyzfoM7SDZjEwYl4ITcQUpMwHUDASgDw0D6jpmqW6ojl4nNPFEtBDnrd4Soum2E1BX6bDZnW0F1sH5NGfRVx6ZbLPJvqCNnV3QHpbEEbNUdRPHKx3hsTFFYOVZgvlYaxpo7iAPDTF1QA8OOKxLNXTmLl/+M1bqDXCFSLcmojkelNndQuuwgqRiMKaIGd1AuY6HkiCDA2qkogbTuINVKac/ascdZBneQ65bHXE3FMBBVGvmMhcnZcLWsnFHLmbsV8s2Hzytdiu25TKxNKDN9UlkCiiBXh1qNO0hXxPKc+iOuJrzFgWFmQqJaApX6+8tOmxG/asprOa7Ap34b37/wiH+7I/T+s39cirlXLAze70ywBOZesRDfuH1VZPu/37Yysk11B33+T0txxBfuUNxBYffP3CsW4tePrQvey2rhzpyNkhLoVpVAJcEiT68qm2MPmBRrCWQM7qC/rNyCb9/j1X6m6Vek7qdfx+gOorASyCS4g2ZN9lJYZ09uiyiI8kGeoutui2/NoJ/ftqpft1feo35UXDsJ2YcoDe1ZO3XQvxmwJcDUBbX5WUUlUApbAtU6hBxXJK7Lq7uKHnsxXIhWyYq4/oEXcOXrjg5t274vaj2o93nTY+u9bY6aIqqfd01kDB25jBfjUJSCJK3gkkpgwZEzcO07Tohd9MZkCfz072tTXSN0HgqfQz2n7qaRwlRaDWpcQt7frR87C5Paspg9pQ0nHjQFZx02Hbcs2Wi8dlvGxo8vPQWHzuxMPU7bClcgp6lrlMpL39XUSvqH7zkZpx/SU/mkPt98+3GJ/YWaDVsCTF1QUzYr9fKRAk/PDkqLKwSKJYHzjo5djsJ4XhmrqKXhXMFxMWdaeFltU0ygqKSIpskOas/ZKDrlmEBHyB2UPCbZRVQqgQvm7YeufCbWP50xLIFYqbOnCTvBEtBdRHo6qcktc9yBUzB3eifyGRsXHjvb/8x8D+05C2cc2oOZ3fEtpCUyjdPWsoPSfPtSeekuPVMT0QuPNa9zEcdxB07By3oqK7FmwUqAqQvq7LuSJTAcsQTiMVfq+i0W8vGGrOk4ed1aWk+XHDcy8zXdp7RQHGGOI0ikEujM2b4l4L2XKaJA5cCwvEU5DimUY5dmNASG9WrqNMjvzZQiqseCbG1f1RJICqbGKYGOXHrnhVSSXuvr2iwB/TusNn41HmAlwNQFNWWz0vJ/Bc0SkJiUgulUrhAYLjqJSqDoiMg/qoxbFB236t4tRUdEAqemBcH3+n1oTNlBKjI7SLqDzDGBdEpAXieunYNEdQfl7dotASmgTSmi+jY5Frld1RFJ9xcXo67Glx7UoyhtI/xPKh4rxx1nzVUbYxjLsBJg6oJqCVSaaQcxAS3sZvJlx2XgFBw3lE6ZNB6JXPS85IiqA3NFx0VeW6i96ITdMYBWJ5Aw5QwsgbyNouuW6wSqcgeFrxlYAincQfJe+qoo8pMkpYjqylV3HWVCMYH4a8QpiDRZQRL5tciVxfTtScS5g4KFfFgJMEwYtYdMJUsgiAn4f31yb5P7xKQYSq6LYgVBbmorofbtb6tCmACeEohaAt7Ydg1Es40cN9o2IjQWNTDslIvfuqpIEZUE6+YGBU4pUkRtWTGc6hLh88SliGatoKK3fE1/34x0B1HkMxNxiqwaJQCUn0vYHZTCEvB3jywR6v9ZTSAdwEpgNHnkhR2hjJGxwMKlm/G7xeurPk7NDvrabasS/9HK7iBZJyBbLaRTAmqOfZrxSP7ld09juOSg5IpQb30T+jMoucIYE3h2yz589g9LI8cPFBxc8adlseffsGsQgCfUiq65YrhinYDWCrqSi8sUE6gGXclEegfZFrJabmewkpbBVZU03LgU0fZqYgKK6yaUHZTi2MAdpM0lpCVQa9+rsQiniI4i77reW6/nsrMPHeWRlPnor58EALxj/kFVHafOvO9ZtRXb9w1jZswi4LJOINJq2WgJRI+Xs+ikrpcmS2DZxj2495ltcNzK7qDP/mFp6BmUHBG5Xslx8Z6fPGpMH11qWMfARNa2QjGBtqzXRtkV3mzzmrcfj3U7B/C9RasBAJecOgdvOG42bn5qIy47+xAAwJcumod81sI5R5WzpS47+xAs8FslS1QBXIsSyPidQNUuoiq5jIVsRncHhfdNahuhEudRrM4SkGOg0N9aGuvnK288Bh25DM4+Ynpo+5GzuvG2kw/Eh84+BOt3DWDJupGtSTEWYCXA1AVd6O4ZLMYrAZmlU/L+GwN3UMqYwGDBO96kBF55+HQ8uLo3tsuoVCDVxgQKMdlB1XQzNZGxCSUlRTSftZGxLRRK3ipjbzv5QOwZKAZK4MsXzUNb1sZZh5WF0/5T2vHdd50YOq9e5wCEhW7OlovXpPORA+XW0bbBvw94SkDfpjebS2oboRKXZpymXYREDQyr6bxp3EEHTu3A9y45MbI9Y1u45u3HAwAOn9WNVytrJYxX2B3E1AVdGPYmtGsOUjWDPjvedtP/vekfdrDoBTNzGSsizOX74ZJrnPHJtQziWgjHUTLEBEqGDKRqyViEkpIimrPLGTZBAzRbFd71+ZeVCi2fSf8cZCCYKMYdlLETsoOi8Yokd1cxRjNVWlxeRbpuvGdc/uOqJQ4ykWElwATUkj8v0VcW29Ef35+nEJOvb0q5NLqDCtIdZKO7LWzMyuyauJXOtuz1xlXNjNIbqykmMPK1jW3L8pWAd65cxlKErbdPJmUwtRoCJVAhNqIiZ/mxHT4z0RbJgdVgUF5JCjTOEqiqTkDJDgpZAqnP0BqwEmACdtXQXVOiWwJJC7fIffWKW9P/vdEdpASGdSUgfcaFkmsUMnIN4rYqZsCAnx2kKYFKWVAmdJ921qZQV9S8UnUbuFIasB6tvJdqnkNQ9SsLqbQpdXLFcDQbKckSMFVjA17FcFqCMxA0S4DVgAorASYgyYVTCT0vP6lTp5z1ymNM3TAlRndQoew66dIaiUlLIK7V9OY9XlaOagmkcbF4KaLaUpKOqNq1MKNbX3SFQusj5GwryKQp976pvxKQ8ZRqAsR6L3393rN2tDumrbmO1O8ztkkcyq7C6LirjwkQwtXdNejuCQ0rgQnKw2t2oH+4hFuWbMTitTsrH4CoC2fbvqFgHeBK6CmZvf0FLFm/Gzv7C9g3VMTj/hiGig5uX74ZQHS2Z2zXbJCyT/uZN7mMhUkJloCJrb47SMYEMhbFuiV++/g6rNsx4OX8i6jr5IXefuzxK4TT0qN12ZSz/IFCCVnby2fXc+obkY6YD2IC8SJA/0yOR77qX1fGii6vqDeQU62HpMBwnJWVdh1koBwT0M8n2CEUgpXAGCCpqKgWNuwawCU3PIKLvv8QPvGbJXjbDx9OddzugbBAu+DbD+AN338o1bGFkhtydezqL+BN1/0Nb/zvh/DBny/G23/4MAYLDu5ZtTWYocmYgJ7vrpL0aPIZC+88JZzKKsdgqhMAyovZywByLmPFuiU+98dluPrOVcE4dYvhh/eHazwq1R4AwJH7dQe/z5qUD4TmQMEJzp9poPCXyGvlsxYOm9kVUU4AIg3zygu3e6/ye5P7debtWEtAxgrU7zPp/k5+2VQAwCWnet/vW046AAAwqZrum0GfHwrO5407/SlaAVYCY4Ck9gK1IIX5C739VR2nBzp1pZCEpwTKs3LZp2fDrkE88oJnBRRdN5iJv+aYWZFunqYU0SQFmctYeMNx++PFq14XbJNj0C2Bz114FC6YNwv9Bdm9szwT1mXRBfPKaX/9w07gSqjkOqkUtFz06Vfhq288Nnj/6JXnKZaAg3xgnSQHYOuBGhO4+1Nn44kvnh/Z5x3zD8ITXzgveB+0UQ4sAe+5fPK8w7H26tcjn7GjxWLSErCilkDS/R2z/2S8eNXrcNVbjsPaq1+Pb73jBLx41euqyuqSVyL/fL/8wGmh7YwHK4ExQC0BxiSqWXpRJS4Yl4ZhzRIYNKw57DgCO/qGkbEIsya1GbKD0lUMS6QgU2eUce6gXMYKhCwQtgR0WaTeh9e2Or4uQaVS7cGsSW2RIK/qDgosAZlO2UBLIKNYAnEzctui0MLx5UXWw7N6dZx69lK5bUQ0jlDp/vRxVWsZSUtFHhbEVlgLhGAlMAaotxKoVZjHtYBO467S3UGmFsVF18WOvgJ6unLI2laQBhjUCRjbRsRf0xQklLNxvU5A73DZpioBTbiorQmGS24QpKxsCSQrAVP8wVbdQZmwO6iRSiAoTksItEaWVtTcQa4mZI3nCDKcTJZAY1svBJaATLWVwWnWAiFYCYwB6u0O0mfBabNL4pRAGstiuBTu729SAo4rsKN/GD2deWRtK+IOcowLt8c/G9PMvDNvtgTyGSskxKUSyGfsiCWgzugLJTcYZ6UsIqkE4taPzdpRhSMFfv9wKRhfOSCceLkRIWMmSdaN/mcTFK9p2UFJwlx6h0yB4Wa135HdanULhvEYlbYRRLQWwD4ADoCSEGL+aIxjrFDvwPCA1h7YcQWEEBXNaSdG2A+X3Iq+2ILjhoTnvqFoPKHkCPQGlgBF0gDTpohKTDPzwB3kRJWAKvACd5BtRRaSVfvfDJfcwGKpZAnItNOcbaHoRN1hJmUciglISyDovd84KSlTUpO+V921I/9+ghTRFG2VpQVQThEtf9aI1FcVfQEY+ay5TiDMaFoC5wghTmh1BQDU3x1kWn+30mpfSfuk6Y+jF2ftMyxWUnIFXtrRj+ldniUghFcZKmeHqjIcLjnY0TecaIWYZuZtStsI9bnmbE0J5OJjAqrwLZScIHaRsaNBZBXpiqom9166KIzuoAYKyeEUcQ5duMt3QYpoirbKegO5amICI0X/a5bjZhUQhhvIjQHq7Q4yBmVdAX3S94Wbl2FXfxHX/cNJAMJK4Igv3BH8/tk/PI2OXCbYz8RwycEkpXDLpFC+ffdz2DVQxMzufDBjP+zfytdRn8Nrvv0A1u4YwIFT22OvaWp5kLM9t4/XMtoN7ZszWQIZC8ceMBl/X7Mj+EwN3q7Z3o/zvvWAf26vJXHc96UqlkocNM27L7nc4kChhNm213BPWgKNlJHTOr3v6pAZ8WvdysfQkbODbC91+9zp3rHTu/L6ocq+eopouuygelCOWYTrG9gQCDNaSkAA+AsRCQA/EkJcr+9ARJcBuAwA5syZ0+ThNZc6tKAJMWBQAiah/ML2/lB7B3XmrM7+n1y3G/vFdAQNzu8IzJ3egV//02l4YHVvkEM/b/Yk9HTl8ODqXqzcvBcA8IFXHoycbeHrC1eFzqFO+mW/ffmqc/17TzamZNoWIWsRHCe8vOOUjpwxMJzPWLjuH07Cio17sWXvIPaf3B6ktOpkLK/Ns/p0j9qvG//x5pdjZnc+6PSpKoHPvOZIfPOuZ0Pn+dNHzsTL/Nz6rOoOUgrYgPBM+Y5PvDLSImMkvOaY/XDjP87HgiNmxu4jhed9/7oAW/cO43N/9NZNkBbKpy84AqcfMg2nHjwt9hyRthGG8zcMpWIYaEz7jYnAaCmBs4QQm4hoJoC7iegZIcQD6g6+YrgeAObPnz+hdXfdLQGTO8jgVimUXAwUy26buAZyewaLmNSevKh7yRXIWBbOPGw6VmzaG3z27tPmYGZ3Hg+u7sXWPUM4de40zOz2FMopc6fi8bW7gn3lzE2eL4kFR4aFl2yJnLEs2H5nTjXwPL0zH7Ic5Kw9n7EwqS2LMw7tCT57PKbCOpux/CBj+bznHDUzKETqUGICADCpLYMFR86IKIGT5pQLl2xFCQTr8GptIwDg6NmT4h5FTRBRxTbI0i02c1JbqC24urKY/j3oBLUFhn5DzZLJkRRRJsSoxASEEJv8120A/gzg1NEYx1jBlBUzEvTAMGC2BIZLbsh1lBSbMLmYJEH2jD8DVvvy2BYFs8B9wyX0dJUrU/VVouT19awhE7rLRXU7yL73Kj1duiUQ3zvHjmlqkzWkeKrnbA9iAmW3UJKrBAgvzCL983qjttEirrdPNbJUV2jqn1jDA8Oa95+VgJmmKwEi6iSibvk7gAsALG/2OMYS9bYEVHeQnJ2aBHyh5Ib2TRK+JhdT+Tg/cOr/k3VoSkD955umtCfQe8OXlUD1/jF1QXPbIgxr1lBHzg6EM1COCZjy5OPcBllDiwlViZQtgXJu/NSOaDsG07jVc+mN2kaLuMBtNQFdvf+RCKWINik7COGYABNmNNxBswD82f8DyAD4tRDizlEYx5ih7tlBISWQwUDBMQrWguNisOgE6aOmfv7BOZX9dGRxmpzVqkogY1FIqKoVqHpPf/kc5Pm68xnsM9QbmFCbm2UswpDWO4iIjHUCpgyjuKycjEWRWXDepATkjD5DFYPE6rOR5yrPnse/Eii7g7z3zXQHRVNEuSzKRNOVgBDiBQDHN/u6Y5mk1gi1UI0lIAQwVHTRnrMT/fDqfpHz+ApGujFUN49tUaiz5PSQO0hTAv5zkOeb1J5NrQTKvfe9TpYyD15FFdj5jBfkNQnpWEvAUOyVNykWqQRSCB312QTFYprgHC3iZs7VzKj1ILc6z2h8iqifHSSvxzrACD+WMYDjCqzYtAdfuXUFhBDYsmcIn/7d03hy3S586ZblwbZ/+d0SDBmCvgBw40Mv4o5lm7F+5wBufXpTsF3tpXPln5fh+W19wWeyalTGECq1m7j8pifxqd8uCa0V0DdcwuU3eYvTS4GmFo3p7qCezrIloLuDlqzbjXdd/zD+9OQGAKgqGyartAQwWQJAWOBnbG89XHNMIEkJxJ+zXDEc7gGURMgdJBWCv6mRdQJp0C8v772WmIA8Vn1+jVYC0g3ElkAyXCcwBnBcgQdWb8dP/74Wnzr/CHxt4UosXLoZT63bhRd6+/GZ1xyJr9y6Aneu2IIL5s3ChcfOjpzj329bCQD47rtOCG2XrRxWbt6LXz+6Dk++tAt3fvJsAOWCoYGCgx5ULii7Z9U2AJ5w/veLvW6YS9fvDlIqpfBTWyrbFHYHdSmCXe+1I1NIt+3zlIypbfD7z5qLI2d1R7b/+oOn409PbsDk9ixsi7Cr36tYPu7AyfjoOYcBQKRO4CPnHIpXHj49cq44JdCZtxNjAmcdNh2XnHoQOnMZ3Pfs9uB5fO+SEzFoCNYDQJfSakM+Gz2jpp58/5ITE4P8kktOPSiSPfT/zjoYtz69Cacf0hNzVPkal9/0FICyIjt6v0n4xzPn4tIz5+Kca+7zP6vhBqrgqxcfg6md2eA+RjvQPlZhJTAGcIUIMoQGC07wxyqFYckpp0zGZa5I5Gy+LWthqOgmLrIit8mU0qSYgIrqvepVlqTMKgFRiecOivq9gWh2kGRg2BvPZE0JXHLqQfjyRccYjzl69iT82+vnede3reDZfer8I3COn8aoXtu2CJ887wjjueKUwPSufMQSUAPLsya14aq3HIdv3vVMMA4AeOPx+xvPB4QD5TJe0kh30EUJY1G56i3HRba99eQD8daTD0x1jS/eshy7B4rle7EIX3lj+LtrtCUgvw+JncIya0XYPhol1CwJxxUo+kJ+oFAKhJBswlZ03EBAV1oJsZypEw7SSj+7pWRpyG0yhpC2+6iaeqe6huTMN6f03snYFFIKOc0vb0Le96Q23RJI90+csQi9/rjUxVLSrkoVN2NsyxosAcMXIo/PpRA6UzvK9zjdH6t8XKMdGK4HSXOWZt8fF4uZYSUwSqiuF8cVgZAfKDiRmWjRLVsClf5xioHF4O3X6c+2ZU98+U9ZVNbHDWICKbOUVEtArTiW/2Sq0LcobAmoQjPuVqQS0C2BtKjXU7OR9DWC0xyvo8uRrEGxEEWfQ/y1yvvIsZZTRCsePmaRfyNJLphmy+SJoFQbASuBUcLRlIAUwINFJzJjKZaUJmsVMomksJdKoD1YbjFsCaiN2aSPWF9ZTKKPR9UV6rrE2SA1UgnAWlbo+DRLMEqSqpSTCKWkKpZAOQ8/+fjk7Bf9WUS/j6DlcpVSThbSNTIm0GyS7qHZdRBsCZhhJTBKqHn7jhCBK2ag4ESyQkquG3xeqaNnyQ0rARkYli4f+Y+nnqeSO2j/KfFN3HoVS0CmRGaV8evZQWln40DUEkgrM+T1uvKZUKtk6Q5Sg7HG4xMvFH5GpjbgcnJfbXbP9E5pCXjvJ4AOGFP3MNrZVmMVVgJ14L5nt+EPT2yI7VMuhMCdy7dEZv8S1y2/HyyUopaA0gxNzui37BnCEy95fXfuf257aF8gWr0rXT4E4M7lW0L9hQaLDjbvGYztmRNtHlce+05DYFhthaC2jQCqa7McjQmkQ15PbVGh0l3hvNXM4E0utKBrZZUyR1o+5era6o4fS3DP/vEDK4E68I//+zj+9fdPY81288Ludy7fgg//8glc/8ALwTa1RYMjRGAZDBadiAlddNygkEoqgXOvvQ9v/cHfsX7nAC698bHQvkDZDJdKQC6wvmT9bnz4l0/gBmUsgwUH53/rAeyKWVh+wVEzQu9Vr5EaGM4Y8uN1SyCvpVQmUWtMQF5vinb8NF8pfOScQxOPN8UEPvCKgwGUBfOJc6YAjipRSgAAFy9JREFUAI7ZP9rY7SC/Q+ihM7pSjfeSUw8CUBb+pmZrzeKtJ1XO/knDhxd4z9jU6XW0kd8l4zH2vqFxhjqjjyvkkumKG3cPGI9z3fJMf6DgRP75i44I3A7SjSOFut7Tp+SIkCUh/wkHtMrbLXuGgt8HCo5xOUgA+MiCQ/GRBYfhkOmd+PAvvaKwcHZQ2RKQQV915q+3jVAtgSNmdWPt1a8HAFz83w/h6Q17Qtc21QmkQV4vrxWjTWrLBtdLQgrhOdM68MBnzzHuc8G8/fDnj5xl/OyNx++Pc4+aWXHNYclVbzkulMpoarHQLK59x/G49h0jL+j/yILD8JEFh9VhRPUlzfffarASGCFqx864xmdBtaQSVFSDsKHAcMGJtDwoOm7wuR4T0H2uRcdFxqZAgMg1d3VlobZVjitkUlEzXaRsGio6obYOmRh3kKk1go7JX9uVz4BIbQSWDjnWtCmhcccn+bMreYw6K8QdkghW7mKPCtME2B00QtTqy7gUSynE1Bl0SXMHlZScfX1JxZIhJmA6D+BZDVnbCpRAYAloSkAV1EkdQuXZ1UIbuU2NB6jntLXAsGoJxGVomIKxuYwVakGROjBsyzz92v68k2IC8t4bmblDo+gOYloPVgIjJNSKOcVavJKS5g4qKe4gfbafZAnobpyi4/pKwHtfjgmE91NdVwMxbiygrMCyBktAdQUB5cCwiu4OiksLNFkCWZtSu1RCx/nnqiYIrRL0uzF8JgOejcw0kadmHcA0A1YCIySkBGIsAZM7SI0JlJSYwGChFJntFx03ENoFJyyw+4bDwVxZZyACS8D2z6sfV1YKSb1kpPWizo7ltl6lRgAIFz5JLC0wHIfJEsjYVijFMy32CN1Beg98E43MNix33GQtwDQeVgLwZ+I1LGRSctzQDNtkCRQdN5jRlVwXrutlAqnxg0LJDbKFBgpO0N1TMlh0AkGtxwv2DYVn+P3DJWRtK7im7G2zZzCsLPqU40wrkQX45wllzFRpCaQpCjIpilotgUydLAETzXAHycvXe7EhhjHBgWEAH/31k7hj+ZaqMweO+MIdoeCdXnE7UCjhtG8swssPmAwAuOmx9di+r4B7Vm3FxSeUG3l9+vdP4+Dpnd4xxbI7aHJ7FnsGi/jEb5YE++rxAl0J9A2XkLUJR8yajHuf3R4EKDcr2UAAsHfIUwpd+UyqmEAmZAl47NQsAVP//LTCstzmwg4yn7KWFdtkLgmpsGpVApkEd5CkoZbABA8MT+3IxqYjM82HlQCAO5Zvqek4/Z+0oAVpd/QVsG+ohGUby6mP96zaCgC4Zcmm0L4v9no1BoN+YPiQGZ34ykXH4H1KDQAQjQnoSmCg4CBrW/j+u0/C89v6IrnyEqkU9p/SlugOki4JNTtIBiyHNKvE1EcnTU99oNxOYuaktuBZZDNWZM2BNAQpooalI9MQWAKGoctgcyNbHki34UR1B939L68KGvwxow+7gxRG+k+nu5RkVa4uqJMYKJQwXHRxcE8nDp8VLTaKBobDM6r+4RIytoWufAYnHDQltn2uHNP+U9rTZQeploC/cbjkhGbEpsyftNW3Motp1qRyw7eMFXYHUcokURkTaIQ7SKbWNsMdNFGzg6Z35XHUftEiO2Z0YCWgYFqNqhr0OoEk4RqHtATyWcvYhVIPGpvcQWoL46QlDqd2ZNHdlg21kNAJsoOUczpKppI6284mtFWuhGx0p/bXz9pWaAnK9L2DvNdaU0RJe1Vp8++3xlOnQrqDJqYKYMYarAQUqhHaJquhqLmDEgOuCWMolFzkbMsoxCopAWkJSJJmtT1deXRk7VTjVM8jlV2h5IZm26ZrpV3ST7p99EKzWgLD8qup1RKQ36LJ5SOzlRrpDuLsIKaZtHxMoGRoqZwGk9Wgu4Pi2kgkMVBwUHI94Wryp1cKDPcXnNCsPal9bk9nDu05O1WKqCqcS0rhWiVBm3Y1JynsdaVRS+8ZaanUmiKaVKHc1gR3kHzUE9UdxIwtWl4JqFkK1VgCpn2jlkAN7qCiAyEEchnL6F4Z1hTLvqFwTKBQckPHJRU1Te/KoyNnY7DoLWTjGGaeweIgMZZAJUGb3h3k/Snqgq+9Jksg3Em1VkxDl5ZAU+oEWAcwTaCl3EH/c9/z+Oivn8TVdzwTbFMXRRksOnh6/W589g9P487lm3HxdX/Dvc9uM57LNHv+j9tX4SO/eiIw42tRAgOFUuBrN+XdFxw31KbXFHQ2KQ8T0zpz6MjZoVbVOlIgqQL1wdW9eGh1L4adFJZA6sBweBnMYLuSHZT2XPJeql3URSKFv+k5yhhIIy2BNCmqDFMvWsoS+K87nw1+/9yFR4KIQkJ0oFDC5b9+Cjv6C9i4exBPr9+Ne1ZuDRYqV5ECPmsTPnrOYfjOPasBALcv24K+t5YwqS1rVBQHTGlHPmvhhZi200NFF7ZFyGWsiN85Y3njVVtOmLp/6jPgz114FPYOFfGD+9aEtvd05XDu0bNwzV+eAwB05zP44kXz8Nk/LAUAvOf0OfjEeYd759QE4nt+8ijOnzcLOdvC7z98BpZpHUD1sfznW1+OWZF1CcrIGX/JcfHby07H8k17AQAXHrsf+oZLGCo6sQvD67gjbO1w+MwufOhVh+A9p70s8pnMDmrkYinvOnUOXuztx8f9Z88wjaSllIBKf8GJFEoNKm2cn9/WByBaFSuRwdTr3zsf5xw1M1ACQDmN02QJ/PT9p+DhF3bgS7esiB2b4wpjUHj2lDbs6BsOpYmalIA+g/1nv7f7g6u3Y/nGvcH2nq48jp49CafOnYbH1u7E5ecehpP8PvkA8PU3vTz43TSrHvbdQafMnYZT5k4z3osUxO88ZY7xc4m0BIqOwGmH9OC0Q3oAAIfP6sbnX3d04rE6gSVQo6QmInz+teZryuwgPUBfT9qyNr568bENOz/DqLSUO0hFLoaitlFWhfbWvd7nO/rNRS1ylm/yWUsBYWrR3J6zU6Uu5g1r8c6e3I5dA8XQOPWYAGBu3wCUO45KF850Px2zu82bC+RsK9bNYfKvF0pOzQVZOmUlMHLhKk/RCJeNDAw3UgkwTDNpXSXgt0HWLQHdBbOjP84S8JWAoaI1yRJoz9pGAQ+EBa1JUew/2XOnqAvC6MFowNzIDSj722UVcU+XV5jVJZVAxo71oxszlVJkB6VFKpN6KIGRuoOSkIFhPUDPMOOVUVECRHQhET1LRM8T0RWjMQbp5lEF9UChFFkbNdYd5AsBUx57oASKTmRW3pHLxC62rq6kZcq6me0v+L5pz6DxeElcYFhaAlM6pBIIWwJFx02wBMw1C/VSAnLMcYvdV4NUAo0o6JLfC1sCzESh6UqAiGwA1wF4LYB5AC4honmNvm5UuEt3ULivviqEJrVlsGewGGnV4B3nuXpM7qBC4A5yMLO7LTS7bstasYJTCmPAXOgkLYHNuyspAbMgLwaWgCf8e3x3UFfeUwr7horBWNs0a8VkIQwUnJpz8XWkpVEfd5BvCTTAHSQtllpqQBhmLDIageFTATwvhHgBAIjoNwAuBrCy3he679ltWLl5L954/P6Y2R3OTLnhwRdgW4TfPL4u2Pbwmh2h5RKP3K8bj6/dhRsefAGzJ7ehb7gUZBM9tW4XAHMx03Nb9+GB1duxYtMedOZtTO3IBQ2ziChWcKpKwORrnz3ZswTuXJHc8C7OEpACtjNvI2MRJrVlQ9fdN1QKBKhUFEm82NuP4w6cXHG/NEj3l96ErxbKlkDjYgKsBJiJwmgogQMArFfebwBwmr4TEV0G4DIAmDMnObMkjkWrtuEXj7yEHX0FfOr8cHrhmu39+IyfCgl46ZEPru5Vrg+cceh0PL52F75517MwMXtyW0hwS77/19VYu8NbVP6i4/eH6wosXLY5EP5xloAUyuo+c3s6gnPNnd6J6V05PPLCTuQyVsRC6enMYUd/ITKLl1z+6sPx5VtX4MxDp2O45AY+89ccMwvfvOtZvObY/YLePf9yQbp0zLgg96VnvAy/eXy98TMTc3o6AAD/9IqDUx8TxzvmH4Tbl23BiXOmjvhcOucePRNX3fEMLjx2dt3PzTCjwWgoAeOqfZENQlwP4HoAmD9/fk3Twy9fNA9/fWYbtu8rp1V+5aJ5mLf/ZLzjRw+H9p3WlcO+4RJefdRM/OTS+XBcgSXrd+N7i1aH9rvlo2fh6NleB8SMRcbg46bdQzhqv27c+rFXIGt7i6pcp3yuKoG1V78eZ161CJv2DIXdQb5wve8z52DuFQsBeL78R688D44rYJHn7ii5IrjGyV+727uXzjxMXHrmXFx65lwAwAfPPiTYftjM7tBaCpXWVVh79evx6d89jT8+uSE2yP3Vi4+tKs1xUlu26vUc4lhw5My6nUtHf1YMM94ZjcDwBgAHKe8PBLApZt8RkbEt7De5DTv6h4PVuvJZGzO6o0JS+sdnTcqDiJCxrSB7RmW/yW3IZTy/flz2ScFxMaM7byz4AqKzZ6kUuhVLwCRccxkrKCTL2N711WvIIPf0rsqunJEiWz6nrU5mGGZsMhr/wY8DOJyIDiaiHIB3Abi1URfr6cxhR18hsARythVkxahIN0iPMos27Te1I52A7emM3y8u6BpyBxmEa6X6AtkSuifGEqgnUkHqC8swDDO+aLoSEEKUAHwMwF0AVgH4nRAivnx2hPR0eX7yQAlkLHTno14wOaNV+9mb9kubEhnnkgEQSRGVWSyVsoPS9sc3Ka96M62znFHEMMz4ZVTaRgghbgdwezOu1dOZx87+QjBjjXPRSFQlMJKe8UmCWBfwwfq6eRsWed0jTUogbfFTkhVSL7r9tFJT2wqGYcYPE96h29OV84O8Xkpnpbz2tGviViLJL6+Podyp0wpSTkfSjmFqM5SAklbKMMz4ZcIrgdl+gdUX/YZtphl2xiIce8Bkf//22HNVk3ee5JfXxyCtho6cHRSfqYrigCnxY1I5eHongOYEa/f3x3T07O6GX4thmMYx4buIvvqoWfjRe0/Gh37xBICycH30ynNRKLlwhUB7zkZPZx5nHtoTyS2X+wkBtOXiheujV56LwYKDBdfcB6A6d9DVbz0OS9fvxoIjZ+IH96+J7LPw46+I7WGk8qd/PjPVfrXywGfOCZTUQdM6cNvlr8BhM7sadj2GYRrPhFcCuYyF1xyzX/BeullMve1NxUVJPfD1/YQQgU9/uiG9VKJ35DxgSnsw25cN6VRLYEpHDlNSZCVN7cw11BUkC7ok0npiGGb8MuHdQTr1anhmgoiC8ydZAkkBZznTbuQ4GYZhJBPeEtBJm2Y50vPXskC6d5wdOg/DMEwjaT0l0OAZdi5jhyp/q6U9m4FF8WsCMAzD1JOWkzT1an2cdP6RtG3oyNl1W62LYRimEi2jBPbzA7yNnmHns1ao4KxaOnI2xwMYhmkaLeMOuumy07Fw6SZMMrR+rieXv/owzOiqnFH0nXeeYGxk985TDuKsG4ZhmgbpK26NRebPny8WL1482sNgGIYZVxDRE0KI+Un7sN+BYRimhWElwDAM08KwEmAYhmlhWAkwDMO0MKwEGIZhWhhWAgzDMC0MKwGGYZgWhpUAwzBMCzMuisWIaDuAl2o8fDqA3joOZzzRqvfeqvcNtO69t+p9A8n3/jIhxIykg8eFEhgJRLS4UsXcRKVV771V7xto3Xtv1fsGRn7v7A5iGIZpYVgJMAzDtDCtoASuH+0BjCKteu+tet9A6957q943MMJ7n/AxAYZhGCaeVrAEGIZhmBhYCTAMw7QwE1oJENGFRPQsET1PRFeM9njqCRHdSETbiGi5sm0aEd1NRKv916n+diKi7/nPYSkRnTR6Ix8ZRHQQEd1LRKuIaAURfcLf3gr33kZEjxHR0/69f9XffjARPerf+2+JKOdvz/vvn/c/nzua4x8pRGQT0VNEdJv/vlXuey0RLSOiJUS02N9Wt7/3CasEiMgGcB2A1wKYB+ASIpo3uqOqKz8FcKG27QoAi4QQhwNY5L8HvGdwuP9zGYAfNGmMjaAE4NNCiKMBnA7go/732gr3Pgzg1UKI4wGcAOBCIjodwH8C+LZ/77sAfMDf/wMAdgkhDgPwbX+/8cwnAKxS3rfKfQPAOUKIE5R6gPr9vQshJuQPgDMA3KW8/zyAz4/2uOp8j3MBLFfePwtgtv/7bADP+r//CMAlpv3G+w+AWwCc32r3DqADwJMAToNXLZrxtwd/9wDuAnCG/3vG349Ge+w13u+BvrB7NYDbAFAr3Ld/D2sBTNe21e3vfcJaAgAOALBeeb/B3zaRmSWE2AwA/utMf/uEfBa+mX8igEfRIvfuu0SWANgG4G4AawDsFkKU/F3U+wvu3f98D4Ce5o64bnwHwGcBuP77HrTGfQOAAPAXInqCiC7zt9Xt7z1T58GOJciwrVXzYSfcsyCiLgB/BPBJIcReItMtersato3bexdCOABOIKIpAP4M4GjTbv7rhLh3InoDgG1CiCeIaIHcbNh1Qt23wllCiE1ENBPA3UT0TMK+Vd/7RLYENgA4SHl/IIBNozSWZrGViGYDgP+6zd8+oZ4FEWXhKYBfCSH+5G9uiXuXCCF2A7gPXlxkChHJCZ16f8G9+59PBrCzuSOtC2cBeCMRrQXwG3guoe9g4t83AEAIscl/3QZP8Z+KOv69T2Ql8DiAw/0MghyAdwG4dZTH1GhuBXCp//ul8Pzlcvv7/MyB0wHskabkeIO8Kf9PAKwSQnxL+agV7n2GbwGAiNoBnAcvUHovgLf5u+n3Lp/J2wD8VfiO4vGEEOLzQogDhRBz4f0f/1UI8Q+Y4PcNAETUSUTd8ncAFwBYjnr+vY920KPBAZXXAXgOnt/030Z7PHW+t5sAbAZQhKf9PwDP77kIwGr/dZq/L8HLlFoDYBmA+aM9/hHc9yvgmbdLASzxf17XIvd+HICn/HtfDuBL/vZDADwG4HkAvweQ97e3+e+f9z8/ZLTvoQ7PYAGA21rlvv17fNr/WSHlWD3/3rltBMMwTAszkd1BDMMwTAVYCTAMw7QwrAQYhmFaGFYCDMMwLQwrAYZhmBaGlQDTEhCR43dhlD+JXWWJ6MNE9L46XHctEU1Pue+HiOgfiegEIvrhSK/NMGmYyG0jGEZlUAhxQtqdhRCjIYRfCeDLAN4A4MFRuD7TgrASYFoavxXBbwGc4296txDieSL6CoA+IcQ1RPRxAB+G18Z6pRDiXUQ0DcCN8Ip5BgBcJoRYSkQ98Ar5ZsArVCLlWu8B8HEAOXhN7z4ihHCI6FPwqj4PB3AsvO6wm4jolUKIDzf0ATAtD7uDmFahXXMHvVP5bK8Q4lQA/w2vJ43OFQBOFEIcB08ZAMBXATzlb7sSwM/97V8G8JAQ4kR4JfxzAICIjgbwTnjNwE4A4AD4BwAQQnwbXjvsRf5nq4UQ81gBMM2ALQGmVUhyB92kvH7b8PlSAL8iopsB3OxvewWAtwKAEOKvRNRDRJMBnA3gLf72hUS0y9//XAAnA3jc73jajnLTLwA4CcDTfp+YXWCYJsFKgGHCrXZNfVReD0+4vxHAF4noGCS37DWdgwD8TAjx+dBGrz3wX+D1gx8CcAmAbn/NgLcKIdZUcyMMUy3sDmIYz00jXx9WPyAiC8BBQoh74S1qMgVAF4AH4Ltz/B73vUKIvdr21wKY6p9qEYC3+UJfrhH7MiHENt9CeRJei+BfAni/8JYSZAXANBy2BJhWod2fXUvuFELINNE8ET0Kb1J0iXacDeCXvquH4K1pu9sPHP8vES2FFxiWbX2/CuAmInoSwP0A1gGAEGIlEX0B3gpRFrzurx8F8JK/HnaPEKKXiM4EoLbIZpiGwl1EmZbGzw6aL4ToHe2xMMxowO4ghmGYFoYtAYZhmBaGLQGGYZgWhpUAwzBMC8NKgGEYpoVhJcAwDNPCsBJgGIZpYf4/UKb8L63BMJIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def dqn(number_of_episodes = 2000, max_t = 1000, eps_start = 1.0, eps_end = 0.01, eps_decay = 0.995):\n",
    "    \"\"\"Deep Q-Learning.\n",
    "    Params\n",
    "    ======\n",
    "    n_episodes (int): maximum number of training episodes\n",
    "    max_t (int): maximum number of time steps per episode\n",
    "    eps_start (float): starting value of epsilon, for epsilon-greedy action selection\n",
    "    eps_end (float) : minimum value of epsilon\n",
    "    eps_decay (float) : multiplicative factor (per episode) for decreasing epsilon\n",
    "    \"\"\"\n",
    "    scores = []                     #list containing scores from each episode\n",
    "    scores_window = deque(maxlen = 100) # last hundred scores\n",
    "    eps = eps_start  #Initialize epsilon\n",
    "    for i_episode in range(1, number_of_episodes+1):\n",
    "        env_info = env.reset(train_mode=True)[brain_name]\n",
    "        state = env_info.vector_observations[0]\n",
    "        score = 0\n",
    "        for t in range(max_t):\n",
    "            action = agent.act(state, eps).astype(np.int32)\n",
    "            env_info = env.step(action)[brain_name]\n",
    "            next_state = env_info.vector_observations[0]\n",
    "            reward = env_info.rewards[0]\n",
    "            done = env_info.local_done[0]\n",
    "            agent.step(state, action, reward, next_state, done)\n",
    "            state = next_state\n",
    "            score += reward\n",
    "            if done:\n",
    "                break\n",
    "        scores_window.append(score)\n",
    "        scores.append(score)\n",
    "        eps = max(eps_end, eps_decay*eps)\n",
    "        print(\"\\rEpisode {}\\tAverage Score: {:.2f}\".format(i_episode, np.mean(scores_window)), end =\"\")\n",
    "        if i_episode % 100 == 0:\n",
    "            print(\"\\rEpisode {}\\tAverage Score: {:.2f}\".format(i_episode, np.mean(scores_window)))\n",
    "        if np.mean(scores_window) >= 13.0:\n",
    "            print(\"\\nEnvironment solved in {:d} episodes!\\t Average score: {:.2f}\".format(i_episode, np.mean(scores_window)))\n",
    "            torch.save(agent.qnetwork_local.state_dict(), 'checkpoint.pth')\n",
    "            break\n",
    "    return scores\n",
    "\n",
    "scores = dqn()\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores)\n",
    "plt.ylabel(\"Score\")\n",
    "plt.xlabel(\"Episode#\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}