practice_vi (1).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Markov decision process\n",
    "\n",
    "This week's methods are all built to solve __M__arkov __D__ecision __P__rocesses. In the broadest sense, an MDP is defined by how it changes states and how rewards are computed.\n",
    "\n",
    "State transition is defined by $P(s' |s,a)$ - how likely areare you to end at state $s'$ if you take action $a$ from state $s$. Now there's more than one way to define rewards, but we'll use $r(s,a,s')$ function for convenience."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For starters, let's define a simple MDP from this picture:\n",
    "<img src='https://s17.postimg.org/mawroys8f/750px-_Markov_Decision_Process_example.png' width=300px>\n",
    "_img by MistWiz (Own work) [Public domain], via Wikimedia Commons_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "transition_probs = {\n",
    "  's0':{\n",
    "    'a0': {'s0': 0.5, 's2': 0.5},\n",
    "    'a1': {'s2': 1}\n",
    "  },\n",
    "  's1':{\n",
    "    'a0': {'s0': 0.7, 's1': 0.1, 's2': 0.2},\n",
    "    'a1': {'s1': 0.95, 's2': 0.05}\n",
    "  },\n",
    "  's2':{\n",
    "    'a0': {'s0': 0.4, 's1': 0.6},\n",
    "    'a1': {'s0': 0.3, 's1': 0.3, 's2':0.4}\n",
    "  }\n",
    "}\n",
    "rewards = {\n",
    "  's1': {'a0': {'s0': +5}},\n",
    "  's2': {'a1': {'s0': -1}}\n",
    "}\n",
    "\n",
    "from mdp import MDP\n",
    "mdp = MDP(transition_probs, rewards, initial_state='s0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use MDP just as any other gym environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initial state = s0\n",
      "next_state = s2, reward = 0.0, done = False\n"
     ]
    }
   ],
   "source": [
    "print('initial state =', mdp.reset())\n",
    "next_state, reward, done, info = mdp.step('a1')\n",
    "print('next_state = %s, reward = %s, done = %s' % (next_state, reward, done))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "but it also has other methods that you'll need for Value Iteration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mdp.get_all_states = ('s1', 's0', 's2')\n",
      "mdp.get_possible_actions('s1') =  ('a0', 'a1')\n",
      "mdp.get_next_states('s1', 'a0') =  {'s1': 0.1, 's0': 0.7, 's2': 0.2}\n",
      "mdp.get_reward('s1', 'a0', 's0') =  5\n",
      "mdp.get_transition_prob('s1', 'a0', 's0') =  0.7\n"
     ]
    }
   ],
   "source": [
    "print(\"mdp.get_all_states =\", mdp.get_all_states())\n",
    "print(\"mdp.get_possible_actions('s1') = \", mdp.get_possible_actions('s1'))\n",
    "print(\"mdp.get_next_states('s1', 'a0') = \", mdp.get_next_states('s1', 'a0'))\n",
    "print(\"mdp.get_reward('s1', 'a0', 's0') = \", mdp.get_reward('s1', 'a0', 's0'))\n",
    "print(\"mdp.get_transition_prob('s1', 'a0', 's0') = \", mdp.get_transition_prob('s1', 'a0', 's0'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Value Iteration\n",
    "\n",
    "Now let's build something to solve this MDP. The simplest algorithm so far is __V__alue __I__teration\n",
    "\n",
    "Here's the pseudo-code for VI:\n",
    "\n",
    "---\n",
    "\n",
    "`1.` Initialize $V^{(0)}(s)=0$, for all $s$\n",
    "\n",
    "`2.` For $i=0, 1, 2, \\dots$\n",
    " \n",
    "`3.` $ \\quad V_{(i+1)}(s) = \\max_a \\sum_{s'} P(s' | s,a) \\cdot [ r(s,a,s') + \\gamma V_{i}(s')]$, for all $s$\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's write a function to compute the state-action value function $Q^{\\pi}$, defined as follows\n",
    "\n",
    "$$Q_i(s, a) = \\sum_{s'} P(s' | s,a) \\cdot [ r(s,a,s') + \\gamma V_{i}(s')]$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_action_value(mdp, state_values, state, action, gamma):\n",
    "    \"\"\" Computes Q(s,a) as in formula above \"\"\"\n",
    "    #<YOUR CODE>\n",
    "    next_states = mdp.get_next_states(state, action)\n",
    "    Q = 0\n",
    "    for s1 in next_states:\n",
    "        Q+=mdp.get_transition_prob(state, action,s1)*(mdp.get_reward(state, action,s1)+gamma*(state_values[s1]))\n",
    "    return Q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6900000000000001\n",
      "4.49\n",
      "{'s1': 0, 's0': 1, 's2': 2}\n"
     ]
    }
   ],
   "source": [
    "print((get_action_value(mdp, test_Vs, 's2', 'a1',0.9)))\n",
    "print((get_action_value(mdp, test_Vs, 's1', 'a0',0.9)))\n",
    "test_Vs = {s : i for i, s in enumerate(mdp.get_all_states())}\n",
    "print (test_Vs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "ename": "AssertionError",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0mTraceback (most recent call last)",
      "\u001b[0;32m<ipython-input-9-8663407ac9d0>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mtest_Vs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0ms\u001b[0m \u001b[0;34m:\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmdp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_all_states\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mallclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mget_action_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmdp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_Vs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m's2'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'a1'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.9\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.69\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mallclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mget_action_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmdp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_Vs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m's1'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'a0'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0.9\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3.95\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0m: "
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "test_Vs = {s : i for i, s in enumerate(mdp.get_all_states())}\n",
    "assert np.allclose(get_action_value(mdp, test_Vs, 's2', 'a1', 0.9), 0.69)\n",
    "assert np.allclose(get_action_value(mdp, test_Vs, 's1', 'a0', 0.9), 3.95)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using $Q(s,a)$ we can now define the \"next\" V(s) for value iteration.\n",
    " $$V_{(i+1)}(s) = \\max_a \\sum_{s'} P(s' | s,a) \\cdot [ r(s,a,s') + \\gamma V_{i}(s')] = \\max_a Q_i(s,a)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_new_state_value(mdp, state_values, state, gamma):\n",
    "    \"\"\" Computes next V(s) as per formula above. Please do not change state_values in process. \"\"\"\n",
    "    if mdp.is_terminal(state): return 0\n",
    "    \n",
    "    max_=float('-inf')\n",
    "    actions = mdp.get_possible_actions(state)\n",
    "    for a in actions:\n",
    "        max_=max(max_,get_action_value(mdp, state_values, state, a, gamma))\n",
    "    \n",
    "    return  max_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_Vs_vopy = dict(test_Vs)\n",
    "assert np.allclose(get_new_state_value(mdp, test_Vs, 's0', 0.9), 1.8)\n",
    "assert np.allclose(get_new_state_value(mdp, test_Vs, 's2', 0.9), 0.69)\n",
    "assert test_Vs == test_Vs_vopy, \"please do not change state_values in get_new_state_value\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's combine everything we wrote into a working value iteration algo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 3.50000   |   V(s1) = 0.000   V(s0) = 0.000   V(s2) = 0.000\n",
      "\n",
      "iter    1   |   diff: 1.89000   |   V(s1) = 3.500   V(s0) = 0.000   V(s2) = 0.000\n",
      "\n",
      "iter    2   |   diff: 1.70100   |   V(s1) = 3.815   V(s0) = 0.000   V(s2) = 1.890\n",
      "\n",
      "iter    3   |   diff: 1.13542   |   V(s1) = 4.184   V(s0) = 1.701   V(s2) = 2.060\n",
      "\n",
      "iter    4   |   diff: 0.73024   |   V(s1) = 5.319   V(s0) = 1.854   V(s2) = 2.871\n",
      "\n",
      "iter    5   |   diff: 0.61135   |   V(s1) = 5.664   V(s0) = 2.584   V(s2) = 3.540\n",
      "\n",
      "iter    6   |   diff: 0.54664   |   V(s1) = 6.275   V(s0) = 3.186   V(s2) = 3.989\n",
      "\n",
      "iter    7   |   diff: 0.49198   |   V(s1) = 6.790   V(s0) = 3.590   V(s2) = 4.535\n",
      "\n",
      "iter    8   |   diff: 0.42210   |   V(s1) = 7.189   V(s0) = 4.082   V(s2) = 4.959\n",
      "\n",
      "iter    9   |   diff: 0.36513   |   V(s1) = 7.611   V(s0) = 4.463   V(s2) = 5.352\n",
      "\n",
      "iter   10   |   diff: 0.32862   |   V(s1) = 7.960   V(s0) = 4.816   V(s2) = 5.717\n",
      "\n",
      "iter   11   |   diff: 0.29262   |   V(s1) = 8.280   V(s0) = 5.145   V(s2) = 6.032\n",
      "\n",
      "iter   12   |   diff: 0.26189   |   V(s1) = 8.572   V(s0) = 5.429   V(s2) = 6.323\n",
      "\n",
      "iter   13   |   diff: 0.23503   |   V(s1) = 8.830   V(s0) = 5.691   V(s2) = 6.584\n",
      "\n",
      "iter   14   |   diff: 0.21124   |   V(s1) = 9.065   V(s0) = 5.925   V(s2) = 6.817\n",
      "\n",
      "iter   15   |   diff: 0.19012   |   V(s1) = 9.276   V(s0) = 6.135   V(s2) = 7.028\n",
      "\n",
      "iter   16   |   diff: 0.17091   |   V(s1) = 9.465   V(s0) = 6.325   V(s2) = 7.218\n",
      "\n",
      "iter   17   |   diff: 0.15366   |   V(s1) = 9.636   V(s0) = 6.496   V(s2) = 7.388\n",
      "\n",
      "iter   18   |   diff: 0.13830   |   V(s1) = 9.790   V(s0) = 6.649   V(s2) = 7.542\n",
      "\n",
      "iter   19   |   diff: 0.12445   |   V(s1) = 9.928   V(s0) = 6.788   V(s2) = 7.680\n",
      "\n",
      "iter   20   |   diff: 0.11200   |   V(s1) = 10.052   V(s0) = 6.912   V(s2) = 7.805\n",
      "\n",
      "iter   21   |   diff: 0.10079   |   V(s1) = 10.164   V(s0) = 7.024   V(s2) = 7.917\n",
      "\n",
      "iter   22   |   diff: 0.09071   |   V(s1) = 10.265   V(s0) = 7.125   V(s2) = 8.017\n",
      "\n",
      "iter   23   |   diff: 0.08164   |   V(s1) = 10.356   V(s0) = 7.216   V(s2) = 8.108\n",
      "\n",
      "iter   24   |   diff: 0.07347   |   V(s1) = 10.437   V(s0) = 7.297   V(s2) = 8.190\n",
      "\n",
      "iter   25   |   diff: 0.06612   |   V(s1) = 10.511   V(s0) = 7.371   V(s2) = 8.263\n",
      "\n",
      "iter   26   |   diff: 0.05951   |   V(s1) = 10.577   V(s0) = 7.437   V(s2) = 8.329\n",
      "\n",
      "iter   27   |   diff: 0.05356   |   V(s1) = 10.636   V(s0) = 7.496   V(s2) = 8.389\n",
      "\n",
      "iter   28   |   diff: 0.04820   |   V(s1) = 10.690   V(s0) = 7.550   V(s2) = 8.442\n",
      "\n",
      "iter   29   |   diff: 0.04338   |   V(s1) = 10.738   V(s0) = 7.598   V(s2) = 8.491\n",
      "\n",
      "iter   30   |   diff: 0.03904   |   V(s1) = 10.782   V(s0) = 7.641   V(s2) = 8.534\n",
      "\n",
      "iter   31   |   diff: 0.03514   |   V(s1) = 10.821   V(s0) = 7.681   V(s2) = 8.573\n",
      "\n",
      "iter   32   |   diff: 0.03163   |   V(s1) = 10.856   V(s0) = 7.716   V(s2) = 8.608\n",
      "\n",
      "iter   33   |   diff: 0.02846   |   V(s1) = 10.887   V(s0) = 7.747   V(s2) = 8.640\n",
      "\n",
      "iter   34   |   diff: 0.02562   |   V(s1) = 10.916   V(s0) = 7.776   V(s2) = 8.668\n",
      "\n",
      "iter   35   |   diff: 0.02306   |   V(s1) = 10.941   V(s0) = 7.801   V(s2) = 8.694\n",
      "\n",
      "iter   36   |   diff: 0.02075   |   V(s1) = 10.964   V(s0) = 7.824   V(s2) = 8.717\n",
      "\n",
      "iter   37   |   diff: 0.01867   |   V(s1) = 10.985   V(s0) = 7.845   V(s2) = 8.738\n",
      "\n",
      "iter   38   |   diff: 0.01681   |   V(s1) = 11.004   V(s0) = 7.864   V(s2) = 8.756\n",
      "\n",
      "iter   39   |   diff: 0.01513   |   V(s1) = 11.021   V(s0) = 7.881   V(s2) = 8.773\n",
      "\n",
      "iter   40   |   diff: 0.01361   |   V(s1) = 11.036   V(s0) = 7.896   V(s2) = 8.788\n",
      "\n",
      "iter   41   |   diff: 0.01225   |   V(s1) = 11.049   V(s0) = 7.909   V(s2) = 8.802\n",
      "\n",
      "iter   42   |   diff: 0.01103   |   V(s1) = 11.062   V(s0) = 7.922   V(s2) = 8.814\n",
      "\n",
      "iter   43   |   diff: 0.00992   |   V(s1) = 11.073   V(s0) = 7.933   V(s2) = 8.825\n",
      "\n",
      "iter   44   |   diff: 0.00893   |   V(s1) = 11.083   V(s0) = 7.943   V(s2) = 8.835\n",
      "\n",
      "iter   45   |   diff: 0.00804   |   V(s1) = 11.092   V(s0) = 7.952   V(s2) = 8.844\n",
      "\n",
      "iter   46   |   diff: 0.00724   |   V(s1) = 11.100   V(s0) = 7.960   V(s2) = 8.852\n",
      "\n",
      "iter   47   |   diff: 0.00651   |   V(s1) = 11.107   V(s0) = 7.967   V(s2) = 8.859\n",
      "\n",
      "iter   48   |   diff: 0.00586   |   V(s1) = 11.113   V(s0) = 7.973   V(s2) = 8.866\n",
      "\n",
      "iter   49   |   diff: 0.00527   |   V(s1) = 11.119   V(s0) = 7.979   V(s2) = 8.872\n",
      "\n",
      "iter   50   |   diff: 0.00475   |   V(s1) = 11.125   V(s0) = 7.984   V(s2) = 8.877\n",
      "\n",
      "iter   51   |   diff: 0.00427   |   V(s1) = 11.129   V(s0) = 7.989   V(s2) = 8.882\n",
      "\n",
      "iter   52   |   diff: 0.00384   |   V(s1) = 11.134   V(s0) = 7.993   V(s2) = 8.886\n",
      "\n",
      "iter   53   |   diff: 0.00346   |   V(s1) = 11.137   V(s0) = 7.997   V(s2) = 8.890\n",
      "\n",
      "iter   54   |   diff: 0.00311   |   V(s1) = 11.141   V(s0) = 8.001   V(s2) = 8.893\n",
      "\n",
      "iter   55   |   diff: 0.00280   |   V(s1) = 11.144   V(s0) = 8.004   V(s2) = 8.896\n",
      "\n",
      "iter   56   |   diff: 0.00252   |   V(s1) = 11.147   V(s0) = 8.007   V(s2) = 8.899\n",
      "\n",
      "iter   57   |   diff: 0.00227   |   V(s1) = 11.149   V(s0) = 8.009   V(s2) = 8.902\n",
      "\n",
      "iter   58   |   diff: 0.00204   |   V(s1) = 11.152   V(s0) = 8.011   V(s2) = 8.904\n",
      "\n",
      "iter   59   |   diff: 0.00184   |   V(s1) = 11.154   V(s0) = 8.014   V(s2) = 8.906\n",
      "\n",
      "iter   60   |   diff: 0.00166   |   V(s1) = 11.155   V(s0) = 8.015   V(s2) = 8.908\n",
      "\n",
      "iter   61   |   diff: 0.00149   |   V(s1) = 11.157   V(s0) = 8.017   V(s2) = 8.909\n",
      "\n",
      "iter   62   |   diff: 0.00134   |   V(s1) = 11.159   V(s0) = 8.019   V(s2) = 8.911\n",
      "\n",
      "iter   63   |   diff: 0.00121   |   V(s1) = 11.160   V(s0) = 8.020   V(s2) = 8.912\n",
      "\n",
      "iter   64   |   diff: 0.00109   |   V(s1) = 11.161   V(s0) = 8.021   V(s2) = 8.913\n",
      "\n",
      "iter   65   |   diff: 0.00098   |   V(s1) = 11.162   V(s0) = 8.022   V(s2) = 8.915\n",
      "\n",
      "Terminated\n"
     ]
    }
   ],
   "source": [
    "# parameters\n",
    "gamma = 0.9            # discount for MDP\n",
    "num_iter = 100         # maximum iterations, excluding initialization\n",
    "min_difference = 0.001 # stop VI if new values are this close to old values (or closer)\n",
    "\n",
    "# initialize V(s)\n",
    "state_values = {s : 0 for s in mdp.get_all_states()}\n",
    "\n",
    "\n",
    "for i in range(num_iter):\n",
    "    \n",
    "    # Compute new state values using the functions you defined above. It must be a dict {state : new_V(state)}\n",
    "    #<YOUR CODE>\n",
    "    new_state_values = {}\n",
    "    \n",
    "    all_states = mdp.get_all_states()\n",
    "    \n",
    "    for state in all_states:\n",
    "        new_state_values[state] = get_new_state_value(mdp, state_values, state, gamma)\n",
    "        \n",
    "    \n",
    "    assert isinstance(new_state_values, dict)\n",
    "    \n",
    "    # Compute difference\n",
    "    diff = max(abs(new_state_values[s] - state_values[s]) for s in mdp.get_all_states())\n",
    "    print(\"iter %4i   |   diff: %6.5f   |   \"%(i, diff), end=\"\")\n",
    "    print('   '.join(\"V(%s) = %.3f\"%(s, v) for s,v in state_values.items()), end='\\n\\n')\n",
    "    state_values = new_state_values\n",
    "    \n",
    "    if diff < min_difference:\n",
    "        print(\"Terminated\"); break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final state values: {'s1': 11.163174814980803, 's0': 8.023123818663871, 's2': 8.915559364985523}\n"
     ]
    }
   ],
   "source": [
    "print(\"Final state values:\", state_values)\n",
    "\n",
    "assert abs(state_values['s0'] - 8.032)  < 0.01\n",
    "assert abs(state_values['s1'] - 11.169) < 0.01\n",
    "assert abs(state_values['s2'] - 8.921)  < 0.01"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's use those $V^{*}(s)$ to find optimal actions in each state\n",
    "\n",
    " $$\\pi^*(s) = argmax_a \\sum_{s'} P(s' | s,a) \\cdot [ r(s,a,s') + \\gamma V_{i}(s')] = argmax_a Q_i(s,a)$$\n",
    " \n",
    "The only difference vs V(s) is that here we take not max but argmax: find action such with maximum Q(s,a)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_optimal_action(mdp, state_values, state, gamma=0.9):\n",
    "    \"\"\" Finds optimal action using formula above. \"\"\"\n",
    "    if mdp.is_terminal(state): return None\n",
    "    \n",
    "    possible_actions = mdp.get_possible_actions(state)\n",
    "    best_action=''\n",
    "    best_value=''\n",
    "    max_value= float('-inf')\n",
    "    \n",
    "    for action in possible_actions:\n",
    "        value = get_action_value(mdp, state_values, state, action, gamma)\n",
    "        if value>max_value:\n",
    "            max_value=value\n",
    "            best_action=action\n",
    "    return best_action"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert get_optimal_action(mdp, state_values, 's0', gamma) == 'a1'\n",
    "assert get_optimal_action(mdp, state_values, 's2', gamma) == 'a0'\n",
    "assert get_optimal_action(mdp, state_values, 's1', gamma) == 'a0'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "average reward:  0.9205\n"
     ]
    }
   ],
   "source": [
    "# Measure agent's average reward\n",
    "\n",
    "s = mdp.reset()\n",
    "rewards = []\n",
    "for _ in range(10000):\n",
    "    s, r, done, _ = mdp.step(get_optimal_action(mdp, state_values, s, gamma))\n",
    "    rewards.append(r)\n",
    "    \n",
    "print(\"average reward: \", np.mean(rewards))\n",
    "\n",
    "assert(0.85 < np.mean(rewards) < 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Frozen lake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*FFF\n",
      "FHFH\n",
      "FFFH\n",
      "HFFG\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from mdp import FrozenLakeEnv\n",
    "mdp = FrozenLakeEnv(slip_chance=0)\n",
    "\n",
    "mdp.render()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def value_iteration(mdp, state_values=None, gamma = 0.9, num_iter = 1000, min_difference = 1e-5):\n",
    "    \"\"\" performs num_iter value iteration steps starting from state_values. Same as before but in a function \"\"\"\n",
    "    state_values = state_values or {s : 0 for s in mdp.get_all_states()}\n",
    "    for i in range(num_iter):\n",
    "\n",
    "        # Compute new state values using the functions you defined above. It must be a dict {state : new_V(state)}\n",
    "#         new_state_values = <YOUR CODE>\n",
    "        \n",
    "        new_state_values = {}\n",
    "    \n",
    "        all_states = mdp.get_all_states()\n",
    "    \n",
    "        for state in all_states:\n",
    "            new_state_values[state] = get_new_state_value(mdp, state_values, state, gamma)\n",
    "        \n",
    "        assert isinstance(new_state_values, dict)\n",
    "\n",
    "        # Compute difference\n",
    "        diff = max(abs(new_state_values[s] - state_values[s]) for s in mdp.get_all_states())\n",
    "        print(\"iter %4i   |   diff: %6.5f   |   V(start): %.3f \"%(i, diff, new_state_values[mdp._initial_state]))\n",
    "        \n",
    "        state_values = new_state_values\n",
    "        if diff < min_difference:\n",
    "            print(\"Terminated\"); break\n",
    "            \n",
    "    return state_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 1.00000   |   V(start): 0.000 \n",
      "iter    1   |   diff: 0.90000   |   V(start): 0.000 \n",
      "iter    2   |   diff: 0.81000   |   V(start): 0.000 \n",
      "iter    3   |   diff: 0.72900   |   V(start): 0.000 \n",
      "iter    4   |   diff: 0.65610   |   V(start): 0.000 \n",
      "iter    5   |   diff: 0.59049   |   V(start): 0.590 \n",
      "iter    6   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n"
     ]
    }
   ],
   "source": [
    "state_values = value_iteration(mdp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*FFF\n",
      "FHFH\n",
      "FFFH\n",
      "HFFG\n",
      "\n",
      "down\n",
      "\n",
      "SFFF\n",
      "*HFH\n",
      "FFFH\n",
      "HFFG\n",
      "\n",
      "down\n",
      "\n",
      "SFFF\n",
      "FHFH\n",
      "*FFH\n",
      "HFFG\n",
      "\n",
      "right\n",
      "\n",
      "SFFF\n",
      "FHFH\n",
      "F*FH\n",
      "HFFG\n",
      "\n",
      "down\n",
      "\n",
      "SFFF\n",
      "FHFH\n",
      "FFFH\n",
      "H*FG\n",
      "\n",
      "right\n",
      "\n",
      "SFFF\n",
      "FHFH\n",
      "FFFH\n",
      "HF*G\n",
      "\n",
      "right\n",
      "\n",
      "SFFF\n",
      "FHFH\n",
      "FFFH\n",
      "HFF*\n",
      "\n"
     ]
    }
   ],
   "source": [
    "s = mdp.reset()\n",
    "mdp.render()\n",
    "for t in range(100):\n",
    "    a = get_optimal_action(mdp, state_values, s, gamma)\n",
    "    print(a, end='\\n\\n')\n",
    "    s, r, done, _ = mdp.step(a)\n",
    "    mdp.render()\n",
    "    if done: break\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's visualize!\n",
    "\n",
    "It's usually interesting to see what your algorithm actually learned under the hood. To do so, we'll plot state value functions and optimal actions at each VI step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def draw_policy(mdp, state_values):\n",
    "    plt.figure(figsize=(3,3))\n",
    "    h,w = mdp.desc.shape\n",
    "    states = sorted(mdp.get_all_states())\n",
    "    V = np.array([state_values[s] for s in states])\n",
    "    Pi = {s: get_optimal_action(mdp, state_values, s, gamma) for s in states}\n",
    "    plt.imshow(V.reshape(w,h), cmap='gray', interpolation='none', clim=(0,1))\n",
    "    ax = plt.gca()\n",
    "    ax.set_xticks(np.arange(h)-.5)\n",
    "    ax.set_yticks(np.arange(w)-.5)\n",
    "    ax.set_xticklabels([])\n",
    "    ax.set_yticklabels([])\n",
    "    Y, X = np.mgrid[0:4, 0:4]\n",
    "    a2uv = {'left': (-1, 0), 'down':(0, -1), 'right':(1,0), 'up':(-1, 0)}\n",
    "    for y in range(h):\n",
    "        for x in range(w):\n",
    "            plt.text(x, y, str(mdp.desc[y,x].item()),\n",
    "                     color='g', size=12,  verticalalignment='center',\n",
    "                     horizontalalignment='center', fontweight='bold')\n",
    "            a = Pi[y, x]\n",
    "            if a is None: continue\n",
    "            u, v = a2uv[a]\n",
    "            plt.arrow(x, y,u*.3, -v*.3, color='m', head_width=0.1, head_length=0.1) \n",
    "    plt.grid(color='b', lw=2, ls='-')\n",
    "    plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 0\n",
      "iter    0   |   diff: 1.00000   |   V(start): 0.000 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAACv1JREFUeJzt3WtsVHUexvHvqb2EIrdIqWujgERW0VUuFWV13ZjdGDdxxWzWKCaajUQX35j1FtfLC5ImKptgSAyKpUA0RPoCI6BbISZbNsELWbBqV62kRSobXMECdnc7bWc6sy9mOra2nWntmfP/99fn00za0w4zD6dPT8+ZzpxfkEqlELGqyHUAkUJSwcU0FVxMU8HFNBVcTFPBxTQVXExTwcU0FVxMK853hSAI7gfuTy9NXQaXFjiSyGgcJpVKBfmuFYzlT/VBUJ2Cw+OKVTj9/4+8/2eHlDFMoym4dlHENBVcTFPBxTQVXExTwcU0FVxMU8HFNBVcTFPBxTQVXExTwcU0FVxMU8HFNBVcTMv7fPCCmAXcBFwElAFdwEngr8AZJ4kG+xMwc5jPbwL+HXGWkfie0ZN8bgp+B3A+cBToAKYDc4Fp+FHwfl8wOM//XAXJwfeMjvNFX/AppMsdA14d8Plz8G+HqQlocR0iD98zOs4XfcF7MpcpwBrgS6AdaAPikafJbQkwb8DyXkc5cvE9o+N80Rc8CeyB4t8Wkzg/kd6arwD+C7wGnIg80ch++oNl38oD/md0nM/NPvinsPGLjZyde5bH5z4OS4FzgV8CO5wkGlZNfQ3FLcU8wROuo4yopr6GnS07+ZiPXUcZVlV9FdtbtnMjNzq5/+j3eouAi6AkUcLytuXwN+BA5mulkacR46LfghcD98JTp55i0deL0vvdl2W+djTyNGJc9AVPAO9DybwSDl5yEEqATuAfwLuRpxHj3Bxk7oO1rGU+853tm+W0If3ueq7nAz5wm2UkAzLuZKfbLMPJ5JvGNKcxfHvkWSRUKriYpoKLaSq4mKaCi2kquJgWecFnM5vneI75zAdgHeu4nMujjpFTNdWsYx0A13Itz/AM05nuONVgt3N7NuNDPMRjPEbg2SmPH+ERHuVRIP19voM7Is8QecFnMIOruTq7vJSlVFIZdYycqqhiCUuyy9dwDedyrsNEQ13BFVRTDcBc5rKCFRR59Au5iCJWsIIFLADSG40rudJBjoi10cZhDtNHHwCnOc1+9kcdI6cGGuiiC4AECRpp5IRXT3OEOuqIZ55f3EUXddRl16kPkiTZzObseuyll81sjjyHkx/5WmqJEydGjJd5mSRJFzFGFCfONrbRTTd99LGVra4jDXGc4xzkIEmS9NDDPva5jjTEO7xDjBhJkjTRxDGORZ7BScFbaaWZZjrp9G7r3a+BBnro4QAHvNt696ujLvvep613v/6teEBALbVOMjib0VNOOcUU00lnKLdXiNkys5hFjBjddId0i+FnnMMcvuXbEH8LhpuxiCIqqOAbvgnl9gbSECrvKGOYNIRKJr0f8XTZ0W/x3fA9HyhjGEb3GybvFjwIgvuDIDgUBMEhODXuWCJR0j54pJQxTNoHl0lPBRfTVHAxTQUX01RwMU0FF9NUcDFNBRfTVHAxTQUX01RwMU0FF9NUcDEt+tMnezI/cUT9+er5fjrYPOAPQDfwnJNUQ2k9joqbGT3gfH6iGVqPObkruO/zHScKrcec3BV8Is139OusbYNpPebkruATbb6jr7Qec3JW8Jvqb+Lmlpt5mIddRchtuIMjD9XU11DXUkc77a6jDM/xenT2MOFCFg46waX8eDOHfThFQI+Di3EquNiWSqVGfYFlKdLnFRj3ZStbU400hnZ7kMpcwrq9QlzCz9hIY+oqrvI6Y6Euo+mstuBimgoupqngYpoKLqap4GKaCi6mqeBiWuQFv4RL2M3u7CDYPezhV/wq6hgT3tM8zR72ALCe9bzCKxQ7fO6cryIv+GlOM4Up2eVSSjnJyahjTHinOEUZZQCcwzkkMm8yWOQF76CDt3mbXnpJkqSNNpppjjrGhLeDHdnJajFivMRLjhP5yck++Ku8SooUvfSyiU0uIkx4nXSyi1300cfXfM0hDrmO5CUnBe+ggzd5k8/4TFvvcdjBDs5ylhd50XUUb2lGT6SUMUya0SOTngoupmkQrBPKOH4aBCuig8xoKWOYdJApk54KLqap4GKaCi6mqeBimgoupqngYpoKLqap4GKaCi6mqeBimgoupqngYpoGwQ5HGcdPg2Dxf4CpMk54GgSbizJOeBoEm4syjp8GwWb49o0BZQzDZB0Eu6p+FXe23MlKVrqKkFdNfQ1nWs7wPM+7jjKimvoaXmt5jc/53HWU4U3WQbDncR7TvR4Cn1ZBhesIeZVS6jqCt/Q4uJimgottGgSbe8jqszzrfUYNgtUgWJmkVHAxTQUX01RwMU0FF9NUcDHNScErqcz+FbOSSu/mO5ZQQiWVAMxgBnOY4zjRUFOZms1YQQWzmOU4kZ8iP33yYhaznvUUZX62+uhjIxt5gzfGdbvph0YhjNP+3sM93M3dg37w7uM+Wmkd5y2Hl3EDG7iMyyillAQJ4sS5lVtDmJUZXsZC8/L0yS20ECOWXU6Q4CAHo46R07u8O6goZzjDl3zpMNFQjTRmMwYENNGkQbDDiLzg3XSzne3EiJEgwQEOcIITUcfIqX84bR99dNFFHXX00ec61iANNBAnDkCcOJvZ7DiRn5zsg+9iF8nM21a2uoiQVy21JEnSQw/72Oc6zhBx4mxjG0mSfMiHHOOY60hecnJ01003L/ACVVR5t/Xu10oru9nNJ3zi3da7XwMNLGMZW9jiOoq3NKMnUsoYJi8PMkWipIKLaRoE64Qyjp8GwYroIDNa6YxjWOWRC7Krz+f1mKaDTJn0VHAxTQUX01RwMU0FF9NUcDFNBRfTVHAxTQUX01RwMU0FF9NUcDFNBRfTon1NpifDQfPyfcgqMG/DPNq/ax/y+aY/NrH4/MUOEg2jCrgeuBCYAnQBJ4FDENVIIb9OKeWbCTBk9ZaFt7Bg1oLsckW5JzOFFgG/J72PcAo4ApSRLv3PUMG9MMYhq7OZzQ3cwF720kVXwWINtHrJam679LZRX/+7978j1hZjzp1zKCou0B5qCXAL6XI3A28AyczXAmB2Ye52OG4K7ng46KiNccjqdVzHAzzAalZTTz2v83rBi76laQv7j+3PLm+4eUPO63/17Fecfvs0Rx8/ysXrLi5M0S8EyjMf/53vyw3p13xE+MIwNwV3PBx01H6Qs3FvY95/kiBBOeXcxV2sYhVP8iQf8VGBAsJbR94atHzbb0a3Ne890cuRNUc4+uejVB+uprQyxFGEUwd8fDbz/tek98f7rQ3v7nJxU3DHw0FHq6K+gpqWmuwcynznJ5zPfJKZzVWSJB100ElnQTNuu3Abi55f9P0nLs99/a5PM79RAiAJ5ZeWE5SG/PK0gccq04HTwFfAJ8CV4d5VPtoHz+EUp1jDmlFffyUreZAHOc5xNrGJ93ivgOnSZv58JsvvXT7q6zevbKbjrQ5m3jiTBX9ZwLSl08IPdZz0IyblwC+A3aQPMjtRwSeyveylnfaC7pKM18KXFtK7tpdpSwpQ7H5xoAH4HenjmJ8A/wJmFO4uR6KCh6iHHq/LDVB2QRllF5QV/o7+SXqLfR3pg87FpHddWoFPC3/3/XTaiEjptBFh0mkjZNJTwcU0FVxMU8HFNBVcTFPBxTQVXExTwcU0FVxMU8HFNBVcTFPBxTQVXExTwcU0FVxMG+PzwYP/kD5biK9mA9+6DpGHMoZjbiqVynsSmLG+oueLVCpV/SMDFVwQBId8zgfKGDXtoohpKriYNtaC1xYkRXh8zwfKGKkxHWSKTDTaRRHTVHAxTQUX01RwMU0FF9P+D4hT7rpeckcbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 1\n",
      "iter    0   |   diff: 0.90000   |   V(start): 0.000 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAC5pJREFUeJzt3X1sFHUex/H3tNs2LZaClFZplCKCCKiAFUVrLkZDNNFgjJ5U/7iL5jzvv/MpnKe5EJt44kVD4vlAqXAapDUhOUAORc3B5TCAByHa02sJrTxoUUpbrNjt4+79Mdstpe3uFnbnN/vr50U27WyX3U+HD9OZ7e58nXA4jIitMkwHEEklFVyspoKL1VRwsZoKLlZTwcVqKrhYTQUXq6ngYrVAvBs4jvMY8Ji7NOF6mJPiSCKJOEA4HHbi3coZy6/qHacsDAcuKFbqDHwfcb9ng5QxmRIpuHZRxGoquFhNBRerqeBiNRVcrKaCi9VUcLGaCi5WU8HFaiq4WE0FF6up4GI1FVyspoKL1eK+HjwlJgNLgcuBHKATOAn8A2g3kmio3wOTRrj+LeB7j7OMxu8ZfZLPTMEfBC4BmoBWYCIwHcjHHwUf0MDQPD+bChKD3zMazud9wXNxyx0E3j3r+kz8t8N0EKg3HSIOv2c0nM/7gndHLrnA48A3wFGgEej1PE1sC4HSs5Y/MpQjFr9nNJzP+4KHgK0QuCdA3yV97tZ8CXAG2Ag0e55odFeds+y38oD/MxrOZ2Yf/Ct4veF1Tk8/zYrpK2ARcBHwC6DGSKIRVdZWEqgP8CzPmo4yqsraSjbVb+ILvjAdZUQltSVsqN/Abdxm5PG93+vNAC6HrL4sFjcuhn8CuyNfy/Y8jVjO+y14AHgEnmt5jrkn5rr73VdHvtbkeRqxnPcF7wP2QFZpFvtm7YMsoAP4D/CZ52nEcmYOMnfASlYygxnG9s1iWu1+KKecvew1m2U0Z2XcxCazWUYSyZdPvtEYfnvmWSSpVHCxmgouVlPBxWoquFhNBRereV7wQgp5iZeYwQwAVrGKeczzOkZMZZSxilUA3MRNvMiLTGSi4VRDPcAD0YxP8ATP8AyOz055/BRP8TRPA+6/84M86HkGzwteQAE3cEN0eRGLKKbY6xgxlVDCQhZGl2/kRi7iIoOJhpvPfMooA2A601nCEjJ89AM5gwyWsISZzATcjca1XGsgh8caaeQAB+inH4A22tjFLq9jxLSd7XTSCUAffexkJ82+epkjVFNNb+T1xZ10Uk11dJ36QYgQa1kbXY899LCWtZ7nMPJfvooqeuklSJA1rCFEyESMUfXSy3rW00UX/fSzjnWmIw1znOPsYx8hQnTTzQ52mI40zCd8QpAgIUIc5CBHOOJ5BiMFP8xh6qijgw7fbb0HbGc73XSzm92+23oPqKY6+tFPW+8BA1txB4cqqoxkMDajJ488AgTooCMp95eK2TKTmUyQIF10Jekek5+xiCJOcSqJPwWTmzGDDKYylR/4ISn3dzYNofIdZUwmDaGSce88Xi6b+BbfDL/nA2VMhsR+wsTdgjuO85jjOPsdx9kPLRccS8RL2gf3lDImk/bBZdxTwcVqKrhYTQUXq6ngYjUVXKymgovVVHCxmgouVlPBxWoquFhNBRerqeBiNe9Pn+yT+YmjGshXy+B0sFLg10AX8JKRVMNpPSbEzIweMD4/0RpajzGZK7jf5zumC63HmMwVPJ3mO/rrrG1DaT3GZK7g6Tbf0a+0HmMyVvCltUu5s/5OnuRJUxFiG+ngyIcqayuprq/mKEdNRxmZ4fVo7GnC2cwecoJLOX+TRnw6RUDPg4vlVHCxWzgcTvgC14dxzytwwZd1rAvvZGfS7g/CkUuy7i8Vl+Rn3MnO8HVc5+uMqbok0lltwcVqKrhYTQUXq6ngYjUVXKymgovVVHCxmucFn8UstrAlOgh2K1u5ndu9jpH2nud5trIVgFd4hXd4h4DB1875lecFb6ONXHKjy9lkc5KTXsdIey20kEMOAJlk0hf5I0N5XvBWWvmQD+mhhxAhGmmkjjqvY6S9Gmqik9WCBHmTNw0n8icj++Dv8i5hwvTQw1u8ZSJC2uugg81spp9+TnCC/ew3HcmXjBS8lVY+4AO+5mttvS9ADTWc5jRv8IbpKL6lGT2eUsZk0oweGfdUcLGaBsEaoYwXToNgRXSQ6S03Y3PzCcM5Rjdt2qWRz/y8Hl06yJRxTwUXq6ngYjUVXKymgovVVHCxmgouVlPBxWoquFhNBRerqeBiNRVcrKaCi9U0CHYkaZBx8cbFfHvm22HXf3zfx8wvnG8g0Tk0CBb/DzBNg4x3XH4HpRNLo8tTcqeYC+NDGgQbSxpkrJhTwV2ld5mO4VsaBBtLGmSsqa9hT/Oe6PILN79gMM0INAg2woflSYeMnx77dMiy7wo+XgfBVtRWsLx+OctYZipCXJW1lbTXt/Mqr5qOMqrK2koe+stD5F6fG//GJozXQbBTmMJEXw+Bd01lqukIcYW7/f4OeHP0PLhYTQUXq3m/D77a/bCIRZ4/dMIiGcspZy97zWYZxecPfQ5Aw8oGw0lGsXqE644AK72NoS24WE0FF6up4GI1FVyspoKL1VRwsZqRghdTHP0tZjHFvpvvmEUWxRQDUEABRRQZTjRcf0c/vcd7Aeg90Utfi0YIjsTzgi9gARvZyBTc1y2/x3vcwz1ex4ipggo2sAGAq7ma93mfK7nScKqhvvvVdzTd2gTA9098T9NNTYR79Cv7c3le8HrqCRKMLvfRxz72eR0jps/4bMhQ1Xba+YZvDCYaLn9ZPk4gcnrsEOSV5+Fk+/+c3l7zvOBddLGBDQQJ0kcfu9lNM81ex4hpYDhtP/100kk11fTTbzrWEAUVBTg5bqGdbIepf/T/i8JMMLIPvpnNhCJ/1rHORIS4qqgiRIhuutnBDtNxhsnIyaDwmULIgLxb88i5Ksd0JF8ycnTXRRev8RollPhu6z3gMIfZwha+5Evfbb0HFFQU0PnvTgpXFJqO4lua0eMpzehJJs3okXFPBReraRCsAYO7AX7m9/WoQbAi9h1kpsMB3BhWueec6IZRB5kivqeCi9VUcLGaCi5WU8HFaiq4WE0FF6up4GI1FVyspoKL1VRwsZoKLlZTwcVq3r4n0yfDQePx/ZBVoHR1KUd/PDrs+oO/PciCSxYYSDSCEqAcuAzIBTqBk8B+4H/eRPDXKaV8Jh2GrN49+25mTp4ZXZ6a55PTR8wF7sfdR2gBDgE5uKW/BhXcD8Y6ZLX3RC8/bfuJguUFZOZnpjDZoEcXPsq9c+5N+PY/7vmRYGOQouVFZARStIeaBdyNW+464O9AKPI1B/DwJABmCm54OGiixjpk9cyOM7S80MKpl09x8e8uZvJvJqe86G8ffJtdR3ZFl1ffOdLskEHH/nyMtg/baFrRxBWrrkhN0S8D8iKf/4vBcoP7vhQP3xhmpuCGh4Mm6twhqw/f/3D8v5QJ4Z/DtP21jbbX2yh5p4QJ5RNSlBC2Hdo2ZPneuxLbmvc093Do8UM0/aGJsgNlZBdnJy/U2d/u6cjHO3D3xwesTN7DxWKm4IaHgyZqzbw1XFN5zeAcyjj/MXsaetwfy/1AJgSKAmRenNot+PrL1jP31bmDV8yLffvOrzrdTxzccxrOScE5DX8+6/OJQBtwDPgSuDa5DxWP9sFjCJQEKP2oNOHbt/+tnZPPnSRrRhZFfypiwtIJOE5q39s46eZJLH5kccK3r1tWR+u2VibdNomZL88kf1F+8kMdx33GJA+4FdiCe5DZgQqezgp+WUDOrBxyb85NebHP1+w3Z9Ozsof8hSko9oBeYDtwH+7x1qXAt0BB6h5yNCp4EmXkZZB3S178GxqUMy2HnGkenKjzv7hb7FtwDzoX4O66HAa+Sv3DD9BpIzyk00Ykl04bIeOeCi5WU8HFaiq4WE0FF6up4GI1FVyspoKL1VRwsZoKLlZTwcVqKrhYTQUXq6ngYjUVXKw2xteDOz8BDamLc8EKgVOmQ8ShjMkxPRwOxz0JzFjf0dMQDofLzjNQyjmOs9/P+UAZvaZdFLGaCi5WG2vBq1KSInn8ng+U0VNjOsgUSTfaRRGrqeBiNRVcrKaCi9VUcLHa/wHwyRCM9ZfW7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 2\n",
      "iter    0   |   diff: 0.81000   |   V(start): 0.000 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADA1JREFUeJzt3VtwlOUdx/Hvu9lkSUIIkRCUiAQzHMrBIRgREG1RhxGHVsaRKnrRqlO0F50WlaK102FMR0VnOlzIFAKCdTzkgpl6oCjqILV2kAEmtRQbGEgF5CAhIQZIQg67vdhkJSTZJLD7Ps8+/D4Mk32XTfaX5Zc3756evxeJRBBxVcB0AJFkUsHFaSq4OE0FF6ep4OI0FVycpoKL01RwcZoKLk4L9nUBz/MWA4ujW9k3woQkRxLpj91EIhGvr0t5A3mq3vNKI7D7smIlT+f30ef3bJAyJlJ/Cq5DFHGaCi5OU8HFaSq4OE0FF6ep4OI0FVycpoKL01RwcZoKLk5TwcVpKrg4TQUXp6ng4rQ+Xw+eFHnAXOA6IAQ0AieBvwGnjSTq6jfA0B7OXw2c8DlLb2zPaEk+MwW/H7gaqAZqgSHAaCAHOwreaR9d85wzFSQO2zMazud/wTOJlrsJeP2C89Ow74CpEqgyHaIPtmc0nM//gp/v+JsJPA78DzgEHARafU8TXwlQdMH2h4ZyxGN7RsP5/C94GHgPgj8O0nZ1W3RvPhM4C7wFHPM9Ue/GX7RtW3nA/oyG85k5Bt8Lq/aton50PctGL4NpwGDgh8DbRhL1qKyijGBVkGd4xnSUXpVVlLGxaiNf8qXpKD0qrCjkjao3mMMcI9fv/1FvALgO0tvSmX5wOmwFPu/4twzf04jj/N+DB4FH4NmaZ5l4fGL0uPsHHf9W7XsacZz/BW8DtkN6UTo7xu6AdKAB2An80/c04jgzdzK3wHKWM4Yxxo7N4loZ/TCb2XzBF2az9OaCjBvZaDZLTzry5ZBjNIZtjzyLJJQKLk5TwcVpKrg4TQUXp6ng4jTfC55PPi/yImMYA8AKVjCJSX7HiKuUUlawAoAZzOB5nmcIQwyn6mohC2MZl7CEpSzFs2zJ4yd5kqd4Coj+P9/P/b5n8L3gueRyEzfFtqcxjRGM8DtGXIUUUkJJbPtmbmYwgw0m6m4ykymlFIDRjGYmMwlY9As5QICZzKSYYiC607iBGwzk8NlBDrKb3bTTDkAddWxjm98x4trMZhppBKCNNj7lU45Z9TJHWMc6WjteX9xII+tYF7tNbRAmzFrWxm7HFlpYy1rfcxj5kS+nnFZaaaKJNawhTNhEjF610soGNtBMM+20s571piN1c4Qj7GAHYcKc5zxb2GI6Ujcf8zFNNBEmTCWVfM3XvmcwUvADHGAPe2igwbq9d6fNbOY85/mcz63be3dax7rYR5v23p069+IeHuWUG8lgbEZPFlkECdJAQ0K+XjJmy+SRRxNNNNOcoK+Y+IwFFHCKUwn8LZjYjAECDGc43/JtQr7ehTSEyjrKmEgaQiVXvEt4uWz/9/hm2J4PlDER+vcbps89uOd5iz3P2+V53i6ouexYIn7SMbivohkrK/9lOEfvSkqmdpyy+XaM0jG4XPFUcHGaCi5OU8HFaSq4OE0FF6ep4OI0FVycpoKL01RwcZoKLk5TwcVpKrg4zf/lky2Zn9irznwVfD8drAj4OdAMvGgkVTd3f3Q3x5uOdzu/4kcVjM+9eDCOAZbcjmZm9IDx+YmuuG3EbVybfW1sOy8jz2Aa+5gruO3zHVPEgtELmHONhUMELGGu4Kk039GuVdu6eOfQO+w6tSu2vXTKUoNpemD4djRX8FSb72ipz779rMu2dQU3fDsaK/jcirncVXUXT/CEqQjx9XTnyEJlFWXM++M80orTTEfpmeHb0djDhOMY12WBS7l04dN2LX1nEz0OLk5TwcVpxpaNWM/6BM/JvDKXjagvqSd7bTbppekJ+XpaNkIkhajg4jQVXJymgovTVHBxmgouTlPBxWm+F3wsY3mXd2ODYN/jPe7gDr9jpLxzT5+j/rb66OnHztGwoIFIq+2L1vvP94LXUUcmmbHtDDI4yUm/Y6S8QEGA2GysMJCOydeGWsv3gtdSywd8QAsthAlzkIPsYY/fMVJe6OEQdL6AMBMyn8jE8+x/9tFvRo7BX+d1IkRooYXVrDYRIeUF8gKEFoYgAIHCAMEZ2n33xEjBa6nlfd7nK77S3vsyhB4O4eV5ZD6pvXdvjP3Yr2KVqat2RiAvQO4nuaZjWE0PE4rTVHBx2iW8HnxX3xcUSTovMa8H1yBYSWXODYJNhSGrx451X3LNFiNHXtNxyv5HZfSOHrniqeDiNBVcnKaCi9NUcHGaCi5OU8HFaSq4OE0FF6ep4OI0FVycpoKL01RwcZoGwfbA+iGrwPS3pvPN2W+6nf/RvR8xOX+ygUQX0SBYrB8EmwpDVu+87k6KhhTFtodlDjMXxkIpNQh2HOM4wAHC+DN0KRWGrC6asIh5RfNMx7BWygyCHcMY1rCGGmoop5ytbE160a0fsgq8XfU2249tj20/N+s5g2l6oEGwUS98+ELci49gBGHCDGc4S1jCYhbzMi+zk51Ji2j9kFXgk8OfdNm2ruBX6iDYYRXD2Fi18ZI+N6Pjz0QmJrXgZRVl3D7pdrJ+n5W067hcZRVlPPjyg2TemNn3hU0wPAjWWMFrqR3QhLViilnNatppZyc7WctaDnM4iQmjwifsH7IaOa9VZXuTMgvaHeUob/ImW9nqS7HFDSlT8GaaeY3XTMeQFKNlI+KoL6kneEuQwa8MTsjXS8ayEftG7mPUxlFkzUrM/QQtGyGSQlRwcZoKLk5TwcVpKrg4TQUXp6ngPYicj9B+rD16uj5C+Lh9z2a2N7TTeqQVgNbjrbTVtBlOZCcVvAfNG5o585MzALTvbafh7gbaquwq0NGfHaX61moATiw5QfWMaiItesr+Yip4D9LnpHd5jtfL80grTuv9EwzIuScHL9jxPEcYsmZn4WXY/+SM31TwHgTHBwmWBKO3TiYM+tUgvHS7ypO7KBcvFM3kZXgM/91ww4nspIL3YtCvB0EAvEEeGfMzTMfpJhAKkL80HwKQdWsWofEh05GslDIvtvJbcEKQjIUZBKcFrdt7d8pdlEvjPxrJX5ZvOoq1VPA4sn5r7xsdILoXL3y10HQMq+kQRZymgovTNAhWUpQGwYq4946eVBiyOoCb3HdebJ9o5yNHF9I7euSKp4KL01RwcZoKLk5TwcVpKrg4TQUXp6ng4jQVXJymgovTVHBxmgouTlPBxWn+vmXNkuGgfbF+yCpQtLKIQ98d6nZ+5WOVTL16qoFEPSgEZgOjgEygEThJ9C0F//Ungt6TGUcqDFmdP24+xXnFse3hWZYsHzERuI/oMUINsB8IES39FFRwGwx0yGrr8VbObDpD7gO5pOX4s1DQoyWPsmDCgn5f/rvt39F0sImCBwoIBJN0hJoOzCda7j3AXyE20tQDfFwEwEzBDQ8H7a+BDlk9u+UsNc/VcOqlU1z1y6vI+0Ve0ov+auWrbPt6W2x75V0r417+8AuHqfugjupl1Vy/4vrkFH0U0Lkgwd+hy7zeCL6+McxMwQ0PB+2vi4esPnTfQ31/UhpEzkWoe6WOulV1FP6lkOzZ2UlKCJv2b+qyvWBe//bmLcda2P/4fqqfrqZ0dykZIxK4uNGF3259x8c7iR6Pd1qeuKuLx0zBDQ8H7a81k9YwpWzK93Mo+/jBbNnXEv213A6kQbAgSNpVyd2Dbxi1gYl/mvj9GZPiX75xb2P0hEd0TcMJSVjT8NwFp4cAdcBh4N/ADYm9qr7oGDyOYGGQog+L+n3506+d5uSzJ0kfk07BHwrInpuN5yX3vY1DZw1l+iPT+335PffsoXZTLUPnDKX4pWJypuUkPtQRoo+YZAG3Au8SvZPZgAqeynJ/mktobIjMWZlJL/alGvfncbQsbyGnJAnF7tQKbAbuJXp/6xrgGyA3eVfZGxU8gQJZAbJusXu5t9DIEKGRPizU+R+ie+xbiN7pnEr00OUAsDf5V99Jy0b4SMtGJJaWjZArngouTlPBxWkquDhNBRenqeDiNBVcnKaCi9NUcHGaCi5OU8HFaSq4OE0FF6ep4OI0FVycNsDXg3tngH3Ji3PZ8oFTpkP0QRkTY3QkEulzEZiBvqNnXyQSKb3EQEnned4um/OBMvpNhyjiNBVcnDbQgpcnJUXi2J4PlNFXA7qTKZJqdIgiTlPBxWkquDhNBRenqeDitP8DPFRB4w7x6rAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 3\n",
      "iter    0   |   diff: 0.72900   |   V(start): 0.000 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADNtJREFUeJzt3X9s1PUdx/Hn9370aDs8KqVoq1CsIgOdFhCdqJMNGjA6iXEbuphNDWwkLNtcnNtcZmOXIJtZjBlB3ZQpiVZdMn9NsSPRzRiDg7HBmOBsp6ggxZZapEjbu+/++LaF0mvvCvf9fr734fUwxvt8+Z7f93159dvvfe++n7fjui4itoqYLkDETwq4WE0BF6sp4GI1BVyspoCL1RRwsZoCLlZTwMVqsWwrOI6zDFjmjUpnwTSfSxLJxWZc13WyreWM5qN6x5ntwuYTKss//a8j62s2yKuxqekvhusYXl3dgr5HYd6PnlwCrlMUsZoCLlZTwMVqCrhYTQEXqyngYjUFXKymgIvVFHCxmgIuVlPAxWoKuFhNARerKeBitazfB/dFGVAHTAISQBfQCvwZ2G+kosF+AIzLsPwB4KOAaxnGTZtvYu/hvUOWr7lgDTWlNQYqOkZI9qGZgH8DOA1oAdqAU4DJwFjCEfB+Oxlcz0FThQzv4rKLqRxTOTBOxpIGq8nA8D4MPuDFeOE+BDx21PIo4Tth2gLsyH318ziP5SznUR7lTd70rayjLaxYyNzxcwPZVjXVLGMZ/+JfPMmTuT1plPsw34IP+OG+f4uB7wL/A94DmoGewKsZWS1QfdR4/cir11DDVKZyF3fRSitrWON70Ne3rmdr59aB8fIpy/O+jf5gz2QmceJEieYe8FHuw3wLPuBp4DmIXROj97Re72j+ReBT4HFgd+AVDe/cwcNX1r+S9Slp0pRQQjXVrGIVK1lJE00+FQgb928cNM53wCcykbWsJU2aSN+v2DnM4RVG3hdLWMJe9g7Zh/YHHGA7rN65mo7JHdwx+Q6YCXwO+BLwhJGKMoo1xjh7x9kD4+WMHJ4FLOAarqGbblxcnuZpXud1X2tsaGzgklsvwb3An2mw97GPlaxkKUsp6funmWbu5d4Rn9dGGwBVjVV07ehiv6E3V8EHPAKcAfFdceY0z/FOTbqAhUBR4NWMqJdedoziBHICE1jEIp7maZ7iKQ6G8V3pKKVJ00QTG9jAfOazlKW8xVs575cP+dDnCkcWfMBjwC1w5747mb5nunfe/fm+P2sJvJq8eo3XeJ3XSZM2XUreHR30Qnp9wQe8F3gD4tVxNp6zEeJAJ/B38Pm3eSAK6S//eBTa6zPzJvNlqKeeKUxhHvMCLyGr+0wXkN26WesAiNfH6aXXcDUZhGQfhu3Ks0heKeBiNQVcrKaAi9UUcLGaAi5WCzzg5ZRzD/cwhSkArGIVM5gRdBkFL/LHCNGfRgGI3h8lcm+EArtEHYjAA54kyUVcNDCeyUwmMjHoMgqes93B+Yc3PbbzvkNkowKeSeABb6aZzWwmRQqAdtp5lVeDLqPgpW5OeZ8CA+4Yl9QtKVNfnQs1I+fgD/EQPfRwiEM8yIMF9/FvKEwC9yIX13EhAe4Cf75NWOiMBPwd3mEb2+ikU0fvE5C62fstqKP38IztlnrqiRHT0ftETILedb0w3nQh4WUs4F10mdq0XSpMFxBuug4uVjuONoKbfCxHJFdOftoIOo6zzHGcTY7jbIJ9+alNJCBqBBsor8YtW/5puI7h1dZe2PcozPvRo0awctJTwMVqCrhYTQEXqyngYjUFXKymgIvVFHCxmgIuVlPAxWoKuFhNARerKeBiteDv6AlJ/8Rh9dfXyJHuYNXAt4HPgHuMVDXEVU1XsefQniHLG69s5NzksY1xDAjJfjR3q2oB9KAsBFdMvIIzSs8YGJcVlRmsJnzMBdxw/0RbLJ68mHmnh7CJQEiYC7jh/olZHV3fKQbryOKZ955h08dHbiO8/fzbDVaTgeH9aC7ghvsnZhWC09hc/G3v3waNQxdww/vRWMDrGutYuGMht3GbqRJGlunNUQg1NDaw6JeLiNZETZeSmeH9aOwy4VSmUkutqc1bJb1fkycNR9fBxWoKuFjN2LQRj/BInvtknpzTRnTUdlD6u1Lis+N5+f9p2giRAqKAi9UUcLGaAi5WU8DFagq4WE0BF6sFHvBzOIdneXagEexzPMdX+ErQZRS8gz85SMcVHd7j7xykc3Enbo86rR0r8IC3004xxQPjIopopTXoMgpepCLi3RkDXgPYOOq0lkHgAW+jjZd4iW66SZOmmWa2sS3oMgpe4uYE9H+BsBiKbyvGccL/6WPQjJyDP8ZjuLh0080DPGCihIIXKYuQ+FoCIhCpihC7RIfvTIwEvI02nud5/sN/dPQ+AYmbEzhlDsU/0tF7OMZ+7Fez2tSmrREpi5DckDRdRqjpMqFYTQEXq6kRrBQoNYIVGf0RvKlppY/lHL+6ugVAYTRZ3b176JRrYVFZeXrfo/BfldEdPXLSU8DFagq4WE0BF6sp4GI1BVyspoCL1RRwsZoCLlZTwMVqCrhYTQEXqyngYrXAb1m7afNN7D28d8jyNResoaa0JuhyMgp9k1VgzuNz+ODTD4Ysb7quifPKzzNQ0TFO9kawF5ddTOWYyoFxMha+ewsLocnq/EnzqT6lemA8vni8uWJCyFjAF1YsZO74ubk/wQX+C9RwZD4QnxVCk9Ubpt3AoupFpssILWMBX9+6nq2dWwfGy6csH/kJ70J8RRx3vEvq1hTuPNf3oIe+ySrwxI4neGP3GwPjuy+922A1GZysjWA37t84aLzi9ytGXN9pdXAdF6fNIXp/FB6G1G0p3Iv8m48v9E1WgQ27Ngwahy7gJ2sj2LvK7+LKFVce35O7gR5w3nJ8DXhDYwNfnvFlSn5e4ts2TlRDYwM3/vpGimcVZ1/ZBMONYM3N9zUeepp6cl+/GWIrYhAFd5Z3msIk/8rrl/4o/E1W3cOaVXY4hTOhXSWkl6RJz0sHEmyxQ+EEvBjS3wr/0VTCJfCAr5u1LuhNjtqLdS8C0FHfAaO4khmkN298E4Cd9TsNVzKM+zIsexeoD7YMfVQvVlPAxWoKuFhNARerKeBiNQVcrKaAZ+AedkntTnmPO1zSe8J3/T3VmaLnfe+T4J49PfTu6zVcUTgp4Bl8tvYzDnz1AACp7Sk6r+qkd0e4AvThtz6k5fIWAD764Ue0XNKC262P7I+lgGcQnxcf9BGYU+YQrQnoS+g5GnvtWJxY3/TYaSi5rASnKPxzegdNAc8gdm6MWG3M2zvFMOZ7Y3Di4QpP8oYkTsKrySlymPCzCYYrCicFfBhjvj8GIuCMcSi6ush0OUNEEhHKby+HCJRcXkLi3ITpkkKpcL5sFbDYtBhFXysiNjMWuqN3v+QNSbpe66L8jnLTpYSWAj6Ckh+H90YH8I7iVQ9XmS4j1HSKIlZTwMVqagQrBUqNYEWO5wi+2cdyToT3OgqhyeoodnngnIFjYjivHB1NjWDlpKeAi9UUcLGaAi5WU8DFagq4WE0BF6sp4GI1BVyspoCL1RRwsZoCLlZTwMVqwd6yFpLmoNmEvskqUH1fNe998t6Q5Vu+s4ULT7vQQEUZVAGXAWcCxUAX0Ip3S8FbwZSgezJHUAhNVq+eejU1ZUc6RE8oCcn0EdOB6/HOEfYBbwMJvNCfjwIeBqNtstqzp4cDLxwguSRJdGwwEwXdWnsri6ctznn9T974hEPNh6hYUkEk5tMZahy4Gi/c24A/Af2z3zlAgJMAmAm44eaguRptk9VPX/6UfXfv4+Nffcypy0+lbGmZ70F/eMvDvPruqwPj+xZm6h1yxK6Vu2h/qZ2WO1o4a9VZ/gT9TKB/QoK/ciTc4N2XEuCNYWYCbrg5aK6ObbL6zeu/mf1JUXAPurT/tp321e1UPVpF6WWlPlUIL7z9wqDx4kW5Hc27d3fz9nffpuUnLczePJuiiXmc3Ojol9vR99/5eOfj/erzt7mRmAm44eaguXpwxoOc33D+kT6UWX4wu3d2e7+WU0AUYhUxoqf6ewRfe+Zapv9m+pEFM0Zev2t7l/fAwZvTcJoPcxoePOrxKUA7sAvYCnwhv5vKRufgI4hVxaheX53z+vv/sJ/WO1uJT4lT8YsKSutKcRx/720cd+k45twyJ+f1t127jbYX2hg3bxw1v6ph7Myx+S/qfbwrJiXA5cCzeG8yO1HAC1ny60kS5yQovrTY92Afr6lrptJd383YWh+C3a8HeBG4Du/91unAB0DSv00ORwHPo0hJhJK54Z7uLVGZIFEZwESd/8Y7Ys/Fe9N5Id6pyzvAdv8330/TRgRI00bkl6aNkJOeAi5WU8DFagq4WE0BF6sp4GI1BVyspoCL1RRwsZoCLlZTwMVqCrhYTQEXqyngYjUFXKw2yu+DOweAnf6Vc8LKgY9NF5GFasyPya7rZp0EZrR39Ox0XXf2cRbkO8dxNoW5PlCNQdMpilhNARerjTbgD/lSRf6EvT5QjYEa1ZtMkUKjUxSxmgIuVlPAxWoKuFhNARer/R8RmFSXyAwc8gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 4\n",
      "iter    0   |   diff: 0.65610   |   V(start): 0.000 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADVpJREFUeJzt3X9wF3V+x/Hn7vf7zTc/CN+ExIhBIVxOpXBWAUELinBVBqi9YxyvFa1DldaTGZy72ipXbHsZ0xn12uk4d3UYnXPsHR3JHzdz1471OM7pcHe9cbiCdKB4YiUqKEiAEBPy85vvd/vHkoSQb/JNSHY/+/3weswwZJf9zr6zvPLJfve7+3k7nuchYivXdAEiQVLAxWoKuFhNARerKeBiNQVcrKaAi9UUcLGaAi5Wi+fbwHGcx4DH/KWyxTAv4JJE8ps27QgdHR1Ovu2ciXxU7zi3erB/UoUFx/8+du5sMlzH6DZseACA3bt/briS0a1efQ8Q7eMIsG3bNpqbm/MGXKcoYjUFXKymgIvVFHCxmgIuVlPAxWoKuFhNARerKeBiNQVcrKaAi9UUcLGaAi5WU8DFannvBw9EJbAamA0kgS6gBfgP4JyRioZ54ugTnOk/M2L9c3Oeo664LvyCcnh4/8Oc6j01Yv32m7dTX1ZvoKLhonIMzQT8j4GZQDNwFpgOzAHKiUTABywqW0RNUc3g8vTYdIPV5HZb5W3UFtcOLqfiKYPVjGT6GIYf8BL8cHcDP7xofYzInTCtTK1kSfmScW+feC9B+b+Wc/7+8/Td3Ad5b8efvDU1a1hetTz4HQHx43Gm7ZxG37w+ur7SNa7XTPQYTrXwA9574U8J8DjwIfAxcBRIh17NmPZ8vod3u98dXN5Ys3HM7ePH4iQ+TFDxYgWZ6gwdf9IReNB3teziYPvBweXNczdP+T7ix+NMe30aycNJ//8ow7gDPtFjONXCD3gW+HeI/2Gc/pn9/mj+e8B54HXgROgVjeqdznegc2h56ze25n2N53i4vS7upy4zXphB2+Nt9NzVE1iNe8/tHbY81QF3T7tUP12Nh4dz4Se1+GAxMzfMHPN1sW/GoGLkMbQ/4ACH4aUjL9E2p42tc7bCImAacBew00hFOT1Z8yTLzi4bXD7beHbM7Yt/VUzpf5aSdbPgQOe6TnqX9AZaY2NTI7dvuh3v5mCmwc5WZWl7vI3ypnKcbge31yV9XZr2x9rHfl0mC8DT7tMsvmox2YpsIPXlE37AXeBaSBxLsPToUv/UpAtYAxSFXs3Y4pD+4vjPm9xWl5JflNC5rpOuP+jCK7Ng7nUXeu7qoeeOHor/q5jypnLSX0znPS7eUQ/6oX9mP9lyM+EGEwGPA4/CM6efYf7J+f453e9c+Lfm0KuZUr1Le2lZ3OK/YbZNbCjoUbsYMJbwA94PvA2JugR7r98LCaAd+G/g16FXM/VsDPfFCuz7M/Mm82fQQANzmcsqVoVeQj7fq/+e6RLy2rF4BwCJhgT99BuuZqSoHMMC+mUjMnEKuFhNARerKeBiNQVcrKaAi9VCD3g11TzP88xlLgAv8AILWBB2GQXP/ZFL7K/9i9Kx78Zw/9H1L8HKMKEHPEWKJQzdPrmIRVzN1WGXUfCcww7OO/7NT85xB3evAp5L6AE/ylH2s58MGQBaaWUPe8Iuo+BlHsn4nwIDXrFH5tGMqVvnIs3IOfgrvEKaNN108zIvk9XQM3GzwVvi4TkeJMG7x4IbuwJgJOAf8AGHOEQ77Rq9JyHziP9bUKP36IwdlgYaiBPX6D0Zs6F/Rz9UmS4kuowFvIvxPfIkedTk3+RKpuvgYrXLaCO4L8ByRMbLwfO8ybcRdBznMcdx9jmOsw9OT01tIiGZ8Ai+c+dfBVjO5RtoshrKZCSXzT/WBw78j+E6Rrdw4S0XvorycfRNyQguUsgUcLGaAi5WU8DFagq4WE0BF6sp4GI1BVyspoCL1RRwsZoCLlZTwMVqCrhYLfQneqLSP3FU3wQqgCbgvQvr6oA/BXqA541UNcK63es42X1yxPqmlU3cmLrRQEWXiMhxNPbImun+ibZYcfUKri27dnC5sqjSYDXRYyzgpvsn2mL9nPWsuiZ6TQSiwljATfdPzGsh/q9U8DsxR9RPPv4J+84MPUb41E1PGawmB8PH0VjATfdPzCsCp7Hj8ctTvxy2HLmAGz6OxgK+7YNtrHhnBef+NkLN6S+W681RBDU2NbL279cSq49odyjDx9HYZcLEhwmS7yZN7d4q2XOaPGk0ug4uVlPAxWqhn4MP9E+s+n5EJ9R7Mce6j4CGcMvI583VbwLQ1tBmuJJRROQ4agQXqyngYjUFXKymgIvVFHCxmgIuVlPAxWqhBzz+YZyaP6sh8YnfA69mUw3Fvy4Ou4yC1/mtTtpW+NfAO7/eSfv6dry0Oq1dKvSAZ1NZnN6haZ2dtEOmKhN2GQXPrXH9J2PAbwCbQJ3Wcgg/4DOydN/VjRf38PBIz0mTnpcOu4yCl3wkCQM3EJZAyZMlOE70J60Pm5Fz8PP3nfcbCBRBx0MdJkooeG6lS/JrSXDBneUSv13Ddy5GAp6dkaXr97vou75Po/ckJB9J4lQ6lPylRu/RGPux79iokXuy3EqX1Fsp02VEmi4TitUUcLGaGsFKgVIjWJGJj+C7dz8XYDmXb/Xqe4DCaLJ64sTIKdeiorb2mgtfRf+qjBrByhVPARerKeBiNQVcrKaAi9UUcLGaAi5WU8DFagq4WE0BF6sp4GI1BVyspoCL1UJ/ZO3h/Q9zqvfUiPXbb95OfVl92OXkFPkmq8DS15fyyflPRqzffd9uvlT9JQMVXeJKbwR7W+Vt1BbXDi6n4tF7trAQmqzePftu6qbXDS5XlUS0sYAhxgK+pmYNy6uWj/8FHvB/QD1D84EErBCarG6Yt4G1dWtNlxFZxgK+q2UXB9sPDi5vnrt57Bd8BIktCbwqj8ymDN4qL/CgR77JKrDzvZ28feLtweVnlz1rsJocrtRGsHvP7R22vOX7W8bc3mlx8BwP56xD7LsxeBUyT2bwlgQ3H1/km6wCbx17a9hy5AJ+pTaC/Xb1t1m5ZeXlvbgPSIPzWyfQgDc2NfLlBV+m9G9KA9vHZDU2NfLgPzxIyeIS06XkZrgRrLn5vqogvXsCs1odhfiWOMTAW+yfpjA7uPIGZD+LfpNVr1ezyo6mcCa0q4XsA1myq7KhBFvsUDgBL4HsxuiPphItoQd8x+IdYe9ywoY1WZ3Alcww/ebB3wBwpOGI4UpGoUawIsFTwMVqCrhYTQEXqyngYjUFXKymgOfg9XpkTvitDb02j+zJ6F1/z7RnSB/3PwlOn0zTf7rfcEXRpIDn0PNaDx1f8XsIZQ5naF/XTv970QrQpxs/pfnOZgA++4vPaL69Ga9PH9lfSgHPIbEqMewjMKfSIVYf0k3o41T+1XKc+IXpsbNQekcpTlH05/QOmwKeQ/zGOPGFcf/olEDxE8U4iWiFJ7UhhZP0a3KKHK7adpXhiqJJAR9F8TeKwQWn2KHo3iLT5YzgJl2qn6oGF0rvLCV5Y9J0SZFUODdbhSw+L07R14qIL4pHbvQekNqQoutXXVRvrTZdSmQp4GMofTq6DzqAP4rPenWW6TIiTacoYjUFXKymRrBSoNQIVuRyRvD9AZYzGf73UQhNVidwyEPnDI6J0bxydDE1gpUrngIuVlPAxWoKuFhNARerKeBiNQVcrKaAi9UUcLGaAi5WU8DFagq4WE0BF6uF+8haRJqD5hP5JqtA3Yt1fPz5xyPWH/j6AW6ZeYuBinKYBdwBXAeUAF1AC/4jBb8NpwQ9kzmGQmiyeu8N91JfOdQh+qrSiEwfMR+4H/8c4TTwPpDED/1NKOBRMNEmq+mTaTre6CD1QIpYeTgTBW1auIn189aPe/vP3/6c7qPd1DxQgxsP6Aw1AdyLH+5DwI+BgdnvHCDESQDMBNxwc9DxmmiT1fM/O8/pZ09z5jtnmLF5BpV/Xhl40F898Cp7PtozuPzimly9Q4Yce+4YrT9tpXlrM1944QvBBP06YGBCgl8wFG7wn0sJ8cEwMwE33Bx0vC5tsvrQ/Q/lf1EMvE6P1n9upfWlVmb9YBZld5QFVCG88f4bw5bXrx3faN53oo/3H3+f5m81c+v+Wym6egonN7r422278Pfd+OfjAxqmbndjMRNww81Bx+vlBS9zU+NNQ30o8/xg9h3p838tZ4AYxGvixGYEO4K/dt1rzP+n+UMrFoy9fdfhLv8LB39Ow3kBzGnYedHX04FW4BhwEPjdqd1VPjoHH0N8Vpy6XXXj3v7cv5yj5ZkWEnMT1PxdDWWry3CcYJ9trFhWwdJHl457+0NfPcTZN85SsaqC+u/UU76ofOqLOo5/xaQUuBP4N/w3me0o4IUs9UcpktcnKVlWEniwL9cN22+gr6GP8oUBBHtAGngTuA///dY1wCdAKrhdjkYBn0JuqUvp8mhP95asTZKsDWGizv/FH7GX47/pvAX/1OUD4HDwux+gaSNCpGkjppamjZArngIuVlPAxWoKuFhNARerKeBiNQVcrKaAi9UUcLGaAi5WU8DFagq4WE0BF6sp4GI1BVysNsH7wZ0O4Ehw5UxaNXDGdBF5qMapMcfzvLyTwEz0iZ4jnufdepkFBc5xnH1Rrg9UY9h0iiJWU8DFahMN+CuBVDF1ol4fqMZQTehNpkih0SmKWE0BF6sp4GI1BVyspoCL1f4fmu6KxEARLygAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 5\n",
      "iter    0   |   diff: 0.59049   |   V(start): 0.590 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADcpJREFUeJzt3X1wXNV5x/Hvubur3dXbSpa8AdkGObKxAqFghA1jg18CeGyXIS7jZGyoxyVOiJUxQ0oLpO5LPKgTmqTpMKEdD5kQ2nEHayY0TWcY13VJQ5IyjDMGM6bGNmDZ+BVLtiTrXdqX2z+uJSFrpdVKuvfcPX4+/2jPzt3ZR3d+Orp7997zKNu2EcJUlu4ChHCTBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwWjDbBkqpx4HHnVFRHdS6XJIQ2RUXH6Ozs1Nl207l8lW9UnfaO3d+fUqFuaW+fisAu3c3aq5kbBs3bgBg377/1lzJ2FategDw934E2L59O01NTVkDLocowmgScGE0CbgwmgRcGE0CLowmARdGk4ALo0nAhdEk4MJoEnBhNAm4MJoEXBhNAi6MJgEXRst6PbgbWpIt/KLrF3w88DF9dh/FVjFVwSo2lGxgZnCmjpJGeOL4E1xMXhz1/PM3Pk91pNr7gjLY9M4mLvRfGPX8ztt2UlNUo6GikfyyD7UE/KXLL3E2eZYFBQuIB+K0p9r5KPERl9OXmYn+gA+6o+gO4gXxoXFpoFRjNZndVX4XVZGqoXEsGNNYzWi696HnAe9Od3M2eZaoivJk2ZMo5VyznrAT2PhrncQVsRUsKlk04e1DR0OU/GsJXeu7GLhtALJejj91q+OrWVqx1P03AoKngxTvLmagdoCeh3om9Jpc9+F08zzgERUhrML02r18r/V7LChYwLzQPL4Q/gJhFfa6nHG9eflNPuj9YGi8Ob553O2Dp4KEToQoe6GMVGWKzj/udD3oe5v3cqjj0NC4fm79tL9H8HSQ4leLCR8OQwJIMeGA57oPp5vnAQ+oAJtKN/Fq26ucSZ7hTPIMv+JXlFql1JfVUx2q9rqkMb3b/S50D4+fffLZrK+xlY3Vb2GdtZjx/Rm0b22nb3mfazXub9s/YjzdAbdaLCqfqcTGRl35S40cinDdxuvGfV3g2wEoG70PjQ84QF2kjod+9hBHwkf49dZf81bvW3SkO9jTtYdvlX9LR0kZPRV/iiWXlgyNLzVcGnf7yO8iFP5PIWkrDQq613bTv6jf1RobGhu4e8vd2Le5c3iXrkjTvrWdksYSVK/C6rdIzEnQ8XjH+K9LpQF4xnqGupl1pMvSrtSXjecBT9kpTiROMDc1l8XHF1NaXEqxKua1rtfot90NQ86CkJiXmPDmVqtF9DdRutd20/OHPdhF/vpMMSkW9C3vo++ePiL/G6GksYTEvETW/WIftyEJyeuSpEv0hBs0BDxhJ/hR24/4+Vd/zvzT8+nr6OO9vvcAqA3n95IU/Yv7aa5rhoDuSlwQGA56Pn174nnAQyrEfYX3cfLiSfbP309fbx/lgXKWR5azqnCV1+VMPxPD/Vl59vtp+ZC5vmQ9c/9tLuHzYY7uPOp1CVm9WPOi7hKy2lW3C4DQjhBJkpqrGc0v+zCP/tkIkTsJuDCaBFwYTQIujCYBF0aTgAujeR7wYFuQ2S/OJnzeubBq9ouziRyPeF1G3rNeswj8hXNSOvDjANbfW6DvC0Pf8jzgga4ARUeKhsZFx4oItYa8LiPvqcMK9a5z8ZM6rbD2S8Az8Tzg/XP66a7txlbOdRrJkiSddZ1el5H3Uo+l4Mq8YEdsUl9Labp0zt+0HIO3/FELdtAmXZCm+eFm+SQwGTeAvch2Joow2A8YcGGXC7REq39OPz3zekgVpWT2noLUYynnp8zeY9K2W8594xwqpWT2noobILkrCRW6C/EvbQFPR+UT0bSIZ9/kWibzpzBazm0E4YCL5QgxUQrbtqfeRlAp9bhS6oBS6gC0TE9tQngk5xl89+4/d7GcyRtssurJYiST5uzrgwff01zH2BYuvP3KIz/vR8e0zOBC5DMJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGE0CLowmARdG8/yOHr/0TxzTt4EyoBEYXNm5GvgToA/4Oy1VjbJ231rO954f9XzjikYWxBZoqOgqPtmP2m5Z090/0RTLPreM2UWzh8blBeUaq/EfbQHX3T/RFOtuXMfK61fqLsO3tAVcd//ErBbi/EsF8PE/l19+8ksOXBy+jfDpW5/WWE0GmvejtoDr7p+YlQ8OYyfitxd+O2Lsu4Br3o/aAr794+0se3cZbX/dpquE8WX6cORDDY0NrPnbNQRqfNodSvN+1HaaMHQiRPgDf7XuzlfpNlljZixyHlwYTQIujOb5Mfhg/8SKn/p0Qb0XMjx3EtjhbRnZ7Fm1B4D2He2aKxmDT/ajzODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujed8I9kSQ+NfjhM44PfDiW+JE3pJGsLnq/k437cucc+Dd3+ymY10HdkI6rV3N84CnY2lU//CyziqhSFWkvC4j71lxy7kzBpwGsCGk01oG3gd8Rpre5b3YQRsbm8SNCRK1Ca/LyHvhx8IweAFhFKJPRVHK/4vWe03LMXjXw11OA4EC6HxU+mROhlVuEf5KGCywZlkE75bpOxMtAU/PSNNzXw8D8wdk9p6C8GNhVLki+mcye49F259952aZuafKKreIvRHTXYavyWlCYTQJuDCaNIIVeUoawQqR+wy+b9/zLpYzeatWPQDkR5PVc+dGL7nmF1VV11955P+zMtIIVlzzJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGM3zW9Y2vbOJC/0XRj2/87ad1BTVeF1ORr5vsgosfnUxZ7rOjHp+38P7+GLlFzVUdJVrvRHsXeV3URWpGhrHgv67tzAfmqzef8P9VJdWD40roj5tLKCJtoCvjq9macXSib/ABj4CahheD8Rl+dBkdWPtRtZUr9Fdhm9pC/je5r0c6jg0NK6fWz/+C05CaFsIu8ImtSWFvdJ2Pei+b7IK7D66m7fPvT00fm7JcxqryeBabQS7v23/iPG2n24bd3vVrLCVjbqkCPw4AC9D6qkU9iL31uPzfZNV4I1Tb4wY+y7g12oj2O9WfpcV21ZM7sUDQALUEeVqwBsaG/jSLV+i8K8KXXuPqWpobOCRHz5CtC6qu5TMNDeC1bfeVwUk9uWwqtVxCG4LQgDsOucwhRvcK29Q+lP/N1m1+2VV2bHkz4J2VZDekCa9Mu1JsIUZ8ifgUUhv9v9sKvzF84Dvqtvl9VvmbEST1RzOZHrp94/8HoBjO45prmQM0ghWCPdJwIXRJODCaBJwYTQJuDCaBFwYTQKegd1vkzrntDa0223S5/13/j3VkSJx2vkmOHE+QbIlqbkif5KAZ9D3Sh+dDzk9hFKHU3Ss7SB51F8BOrv5LE33NgHw6Z9+StPdTdgD8pX91STgGYRWhkZ8BabKFYEajy5Cn6CSL5eggleWx05D4T2FqAL/r+ntNQl4BsEFQYILg87eiULkiQgq5K/wxDbGUGGnJlWgmLl9puaK/EkCPobIkxGwQEUUBQ8W6C5nFCtsUfl0JVhQeG8h4QVh3SX5Uv5cbOWxYG2Qgq8UELwj6LvZe1BsY4ye3/VQ+Wyl7lJ8SwI+jsJn/HujAziz+KyXZ+kuw9fkEEUYTQIujCaNYEWekkawQkxmBn/HxXKmwvk98qHJag673HNqaE7055mjz5JGsOKaJwEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmje3rLmk+ag2fi+ySpQ/UI1n1z+ZNTzB795kNuvu11DRRnMAu4B5gBRoAdoxrml4Ig3Jcg9mePIhyarD970IDXlwx2iZxb6ZPmIm4H1OMcILcCHQBgn9LciAfeDXJusJs4n6Hy9k9iGGIESbxYK2rJwC+tq1014+8tvX6b3eC/xDXGsoEtHqCHgQZxwvw/8OzC4+p0CPFwEQE/ANTcHnahcm6x2/VcXLc+1cPEHF5lRP4Pyb5S7HvSXD77MmyffHBq/sDpT75Bhp54/Ret/ttL0bBOf//7n3Qn6HGBwQYLfMBxucO5L8fDGMD0B19wcdKKubrL66PpHs78oAHa3Tes/ttL6T63M+pdZFN1T5FKF8PqHr48Yr1szsdl84NwAH279kKbvNHHnO3dS8LlpXNzos79u+5Wf9+Mcjw/aMX1vNx49AdfcHHSiXrrlJW5tuHW4D2WWP8yBYwPOv+UUEIBgPEhghrsz+CtzXuHmf7h5+Ilbxt++53CP80DhrGlY68Kaht2feVwKtAKngEPAH0zvW2Ujx+DjCM4KUr23esLbt/1zG81/2Uxoboj438QpWlWEUu7e21i2pIzFX1s84e3f//L7XHr9EmUry6j5QQ0ld5RMf1Gncc6YFAL3Av+B8yGzAwl4Pot9NUZ4fpjokqjrwZ6sm3bexMCOAUoWuhDsQQlgD/Awzuet64EzQMy9txyLBHwaWYUWhUv9vdxbuCpMuMqDhTr/D2fGXorzofN2nEOXj4HD7r/9IFk2wkOybMT0kmUjxDVPAi6MJgEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGG0HK8HV53AMffKmbJK4KLuIrKQGqfHjbZtZ10EJtc7eo7Ztn3nJAtynVLqgJ/rA6nRa3KIIowmARdGyzXgP3Gliunj9/pAavRUTh8yhcg3cogijCYBF0aTgAujScCF0STgwmj/D6950SHWwvgbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 6\n",
      "iter    0   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADcpJREFUeJzt3X1wXNV5x/Hvubur3dXbSpa8AdkGObKxAqFghA1jg18CeGyXIS7jZGyoxyVOiJUxQ0oLpO5LPKgTmqTpMKEdD5kQ2nEHayY0TWcY13VJQ5IyjDMGM6bGNmDZ+BVLtiTrXdqX2z+uJSFrpdVKuvfcPX4+/2jPzt3ZR3d+Orp7997zKNu2EcJUlu4ChHCTBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwWjDbBkqpx4HHnVFRHdS6XJIQ2RUXH6Ozs1Nl207l8lW9UnfaO3d+fUqFuaW+fisAu3c3aq5kbBs3bgBg377/1lzJ2FategDw934E2L59O01NTVkDLocowmgScGE0CbgwmgRcGE0CLowmARdGk4ALo0nAhdEk4MJoEnBhNAm4MJoEXBhNAi6MJgEXRst6PbgbWpIt/KLrF3w88DF9dh/FVjFVwSo2lGxgZnCmjpJGeOL4E1xMXhz1/PM3Pk91pNr7gjLY9M4mLvRfGPX8ztt2UlNUo6GikfyyD7UE/KXLL3E2eZYFBQuIB+K0p9r5KPERl9OXmYn+gA+6o+gO4gXxoXFpoFRjNZndVX4XVZGqoXEsGNNYzWi696HnAe9Od3M2eZaoivJk2ZMo5VyznrAT2PhrncQVsRUsKlk04e1DR0OU/GsJXeu7GLhtALJejj91q+OrWVqx1P03AoKngxTvLmagdoCeh3om9Jpc9+F08zzgERUhrML02r18r/V7LChYwLzQPL4Q/gJhFfa6nHG9eflNPuj9YGi8Ob553O2Dp4KEToQoe6GMVGWKzj/udD3oe5v3cqjj0NC4fm79tL9H8HSQ4leLCR8OQwJIMeGA57oPp5vnAQ+oAJtKN/Fq26ucSZ7hTPIMv+JXlFql1JfVUx2q9rqkMb3b/S50D4+fffLZrK+xlY3Vb2GdtZjx/Rm0b22nb3mfazXub9s/YjzdAbdaLCqfqcTGRl35S40cinDdxuvGfV3g2wEoG70PjQ84QF2kjod+9hBHwkf49dZf81bvW3SkO9jTtYdvlX9LR0kZPRV/iiWXlgyNLzVcGnf7yO8iFP5PIWkrDQq613bTv6jf1RobGhu4e8vd2Le5c3iXrkjTvrWdksYSVK/C6rdIzEnQ8XjH+K9LpQF4xnqGupl1pMvSrtSXjecBT9kpTiROMDc1l8XHF1NaXEqxKua1rtfot90NQ86CkJiXmPDmVqtF9DdRutd20/OHPdhF/vpMMSkW9C3vo++ePiL/G6GksYTEvETW/WIftyEJyeuSpEv0hBs0BDxhJ/hR24/4+Vd/zvzT8+nr6OO9vvcAqA3n95IU/Yv7aa5rhoDuSlwQGA56Pn174nnAQyrEfYX3cfLiSfbP309fbx/lgXKWR5azqnCV1+VMPxPD/Vl59vtp+ZC5vmQ9c/9tLuHzYY7uPOp1CVm9WPOi7hKy2lW3C4DQjhBJkpqrGc0v+zCP/tkIkTsJuDCaBFwYTQIujCYBF0aTgAujeR7wYFuQ2S/OJnzeubBq9ouziRyPeF1G3rNeswj8hXNSOvDjANbfW6DvC0Pf8jzgga4ARUeKhsZFx4oItYa8LiPvqcMK9a5z8ZM6rbD2S8Az8Tzg/XP66a7txlbOdRrJkiSddZ1el5H3Uo+l4Mq8YEdsUl9Labp0zt+0HIO3/FELdtAmXZCm+eFm+SQwGTeAvch2Joow2A8YcGGXC7REq39OPz3zekgVpWT2noLUYynnp8zeY9K2W8594xwqpWT2noobILkrCRW6C/EvbQFPR+UT0bSIZ9/kWibzpzBazm0E4YCL5QgxUQrbtqfeRlAp9bhS6oBS6gC0TE9tQngk5xl89+4/d7GcyRtssurJYiST5uzrgwff01zH2BYuvP3KIz/vR8e0zOBC5DMJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGE0CLowmARdG8/yOHr/0TxzTt4EyoBEYXNm5GvgToA/4Oy1VjbJ231rO954f9XzjikYWxBZoqOgqPtmP2m5Z090/0RTLPreM2UWzh8blBeUaq/EfbQHX3T/RFOtuXMfK61fqLsO3tAVcd//ErBbi/EsF8PE/l19+8ksOXBy+jfDpW5/WWE0GmvejtoDr7p+YlQ8OYyfitxd+O2Lsu4Br3o/aAr794+0se3cZbX/dpquE8WX6cORDDY0NrPnbNQRqfNodSvN+1HaaMHQiRPgDf7XuzlfpNlljZixyHlwYTQIujOb5Mfhg/8SKn/p0Qb0XMjx3EtjhbRnZ7Fm1B4D2He2aKxmDT/ajzODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujed8I9kSQ+NfjhM44PfDiW+JE3pJGsLnq/k437cucc+Dd3+ymY10HdkI6rV3N84CnY2lU//CyziqhSFWkvC4j71lxy7kzBpwGsCGk01oG3gd8Rpre5b3YQRsbm8SNCRK1Ca/LyHvhx8IweAFhFKJPRVHK/4vWe03LMXjXw11OA4EC6HxU+mROhlVuEf5KGCywZlkE75bpOxMtAU/PSNNzXw8D8wdk9p6C8GNhVLki+mcye49F259952aZuafKKreIvRHTXYavyWlCYTQJuDCaNIIVeUoawQqR+wy+b9/zLpYzeatWPQDkR5PVc+dGL7nmF1VV11955P+zMtIIVlzzJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGM3zW9Y2vbOJC/0XRj2/87ad1BTVeF1ORr5vsgosfnUxZ7rOjHp+38P7+GLlFzVUdJVrvRHsXeV3URWpGhrHgv67tzAfmqzef8P9VJdWD40roj5tLKCJtoCvjq9macXSib/ABj4CahheD8Rl+dBkdWPtRtZUr9Fdhm9pC/je5r0c6jg0NK6fWz/+C05CaFsIu8ImtSWFvdJ2Pei+b7IK7D66m7fPvT00fm7JcxqryeBabQS7v23/iPG2n24bd3vVrLCVjbqkCPw4AC9D6qkU9iL31uPzfZNV4I1Tb4wY+y7g12oj2O9WfpcV21ZM7sUDQALUEeVqwBsaG/jSLV+i8K8KXXuPqWpobOCRHz5CtC6qu5TMNDeC1bfeVwUk9uWwqtVxCG4LQgDsOucwhRvcK29Q+lP/N1m1+2VV2bHkz4J2VZDekCa9Mu1JsIUZ8ifgUUhv9v9sKvzF84Dvqtvl9VvmbEST1RzOZHrp94/8HoBjO45prmQM0ghWCPdJwIXRJODCaBJwYTQJuDCaBFwYTQKegd1vkzrntDa0223S5/13/j3VkSJx2vkmOHE+QbIlqbkif5KAZ9D3Sh+dDzk9hFKHU3Ss7SB51F8BOrv5LE33NgHw6Z9+StPdTdgD8pX91STgGYRWhkZ8BabKFYEajy5Cn6CSL5eggleWx05D4T2FqAL/r+ntNQl4BsEFQYILg87eiULkiQgq5K/wxDbGUGGnJlWgmLl9puaK/EkCPobIkxGwQEUUBQ8W6C5nFCtsUfl0JVhQeG8h4QVh3SX5Uv5cbOWxYG2Qgq8UELwj6LvZe1BsY4ye3/VQ+Wyl7lJ8SwI+jsJn/HujAziz+KyXZ+kuw9fkEEUYTQIujCaNYEWekkawQkxmBn/HxXKmwvk98qHJag673HNqaE7055mjz5JGsOKaJwEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmje3rLmk+ag2fi+ySpQ/UI1n1z+ZNTzB795kNuvu11DRRnMAu4B5gBRoAdoxrml4Ig3Jcg9mePIhyarD970IDXlwx2iZxb6ZPmIm4H1OMcILcCHQBgn9LciAfeDXJusJs4n6Hy9k9iGGIESbxYK2rJwC+tq1014+8tvX6b3eC/xDXGsoEtHqCHgQZxwvw/8OzC4+p0CPFwEQE/ANTcHnahcm6x2/VcXLc+1cPEHF5lRP4Pyb5S7HvSXD77MmyffHBq/sDpT75Bhp54/Ret/ttL0bBOf//7n3Qn6HGBwQYLfMBxucO5L8fDGMD0B19wcdKKubrL66PpHs78oAHa3Tes/ttL6T63M+pdZFN1T5FKF8PqHr48Yr1szsdl84NwAH279kKbvNHHnO3dS8LlpXNzos79u+5Wf9+Mcjw/aMX1vNx49AdfcHHSiXrrlJW5tuHW4D2WWP8yBYwPOv+UUEIBgPEhghrsz+CtzXuHmf7h5+Ilbxt++53CP80DhrGlY68Kaht2feVwKtAKngEPAH0zvW2Ujx+DjCM4KUr23esLbt/1zG81/2Uxoboj438QpWlWEUu7e21i2pIzFX1s84e3f//L7XHr9EmUry6j5QQ0ld5RMf1Gncc6YFAL3Av+B8yGzAwl4Pot9NUZ4fpjokqjrwZ6sm3bexMCOAUoWuhDsQQlgD/Awzuet64EzQMy9txyLBHwaWYUWhUv9vdxbuCpMuMqDhTr/D2fGXorzofN2nEOXj4HD7r/9IFk2wkOybMT0kmUjxDVPAi6MJgEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGG0HK8HV53AMffKmbJK4KLuIrKQGqfHjbZtZ10EJtc7eo7Ztn3nJAtynVLqgJ/rA6nRa3KIIowmARdGyzXgP3Gliunj9/pAavRUTh8yhcg3cogijCYBF0aTgAujScCF0STgwmj/D6950SHWwvgbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 7\n",
      "iter    0   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADcpJREFUeJzt3X1wXNV5x/Hvubur3dXbSpa8AdkGObKxAqFghA1jg18CeGyXIS7jZGyoxyVOiJUxQ0oLpO5LPKgTmqTpMKEdD5kQ2nEHayY0TWcY13VJQ5IyjDMGM6bGNmDZ+BVLtiTrXdqX2z+uJSFrpdVKuvfcPX4+/2jPzt3ZR3d+Orp7997zKNu2EcJUlu4ChHCTBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwWjDbBkqpx4HHnVFRHdS6XJIQ2RUXH6Ozs1Nl207l8lW9UnfaO3d+fUqFuaW+fisAu3c3aq5kbBs3bgBg377/1lzJ2FategDw934E2L59O01NTVkDLocowmgScGE0CbgwmgRcGE0CLowmARdGk4ALo0nAhdEk4MJoEnBhNAm4MJoEXBhNAi6MJgEXRst6PbgbWpIt/KLrF3w88DF9dh/FVjFVwSo2lGxgZnCmjpJGeOL4E1xMXhz1/PM3Pk91pNr7gjLY9M4mLvRfGPX8ztt2UlNUo6GikfyyD7UE/KXLL3E2eZYFBQuIB+K0p9r5KPERl9OXmYn+gA+6o+gO4gXxoXFpoFRjNZndVX4XVZGqoXEsGNNYzWi696HnAe9Od3M2eZaoivJk2ZMo5VyznrAT2PhrncQVsRUsKlk04e1DR0OU/GsJXeu7GLhtALJejj91q+OrWVqx1P03AoKngxTvLmagdoCeh3om9Jpc9+F08zzgERUhrML02r18r/V7LChYwLzQPL4Q/gJhFfa6nHG9eflNPuj9YGi8Ob553O2Dp4KEToQoe6GMVGWKzj/udD3oe5v3cqjj0NC4fm79tL9H8HSQ4leLCR8OQwJIMeGA57oPp5vnAQ+oAJtKN/Fq26ucSZ7hTPIMv+JXlFql1JfVUx2q9rqkMb3b/S50D4+fffLZrK+xlY3Vb2GdtZjx/Rm0b22nb3mfazXub9s/YjzdAbdaLCqfqcTGRl35S40cinDdxuvGfV3g2wEoG70PjQ84QF2kjod+9hBHwkf49dZf81bvW3SkO9jTtYdvlX9LR0kZPRV/iiWXlgyNLzVcGnf7yO8iFP5PIWkrDQq613bTv6jf1RobGhu4e8vd2Le5c3iXrkjTvrWdksYSVK/C6rdIzEnQ8XjH+K9LpQF4xnqGupl1pMvSrtSXjecBT9kpTiROMDc1l8XHF1NaXEqxKua1rtfot90NQ86CkJiXmPDmVqtF9DdRutd20/OHPdhF/vpMMSkW9C3vo++ePiL/G6GksYTEvETW/WIftyEJyeuSpEv0hBs0BDxhJ/hR24/4+Vd/zvzT8+nr6OO9vvcAqA3n95IU/Yv7aa5rhoDuSlwQGA56Pn174nnAQyrEfYX3cfLiSfbP309fbx/lgXKWR5azqnCV1+VMPxPD/Vl59vtp+ZC5vmQ9c/9tLuHzYY7uPOp1CVm9WPOi7hKy2lW3C4DQjhBJkpqrGc0v+zCP/tkIkTsJuDCaBFwYTQIujCYBF0aTgAujeR7wYFuQ2S/OJnzeubBq9ouziRyPeF1G3rNeswj8hXNSOvDjANbfW6DvC0Pf8jzgga4ARUeKhsZFx4oItYa8LiPvqcMK9a5z8ZM6rbD2S8Az8Tzg/XP66a7txlbOdRrJkiSddZ1el5H3Uo+l4Mq8YEdsUl9Labp0zt+0HIO3/FELdtAmXZCm+eFm+SQwGTeAvch2Joow2A8YcGGXC7REq39OPz3zekgVpWT2noLUYynnp8zeY9K2W8594xwqpWT2noobILkrCRW6C/EvbQFPR+UT0bSIZ9/kWibzpzBazm0E4YCL5QgxUQrbtqfeRlAp9bhS6oBS6gC0TE9tQngk5xl89+4/d7GcyRtssurJYiST5uzrgwff01zH2BYuvP3KIz/vR8e0zOBC5DMJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGE0CLowmARdG8/yOHr/0TxzTt4EyoBEYXNm5GvgToA/4Oy1VjbJ231rO954f9XzjikYWxBZoqOgqPtmP2m5Z090/0RTLPreM2UWzh8blBeUaq/EfbQHX3T/RFOtuXMfK61fqLsO3tAVcd//ErBbi/EsF8PE/l19+8ksOXBy+jfDpW5/WWE0GmvejtoDr7p+YlQ8OYyfitxd+O2Lsu4Br3o/aAr794+0se3cZbX/dpquE8WX6cORDDY0NrPnbNQRqfNodSvN+1HaaMHQiRPgDf7XuzlfpNlljZixyHlwYTQIujOb5Mfhg/8SKn/p0Qb0XMjx3EtjhbRnZ7Fm1B4D2He2aKxmDT/ajzODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujed8I9kSQ+NfjhM44PfDiW+JE3pJGsLnq/k437cucc+Dd3+ymY10HdkI6rV3N84CnY2lU//CyziqhSFWkvC4j71lxy7kzBpwGsCGk01oG3gd8Rpre5b3YQRsbm8SNCRK1Ca/LyHvhx8IweAFhFKJPRVHK/4vWe03LMXjXw11OA4EC6HxU+mROhlVuEf5KGCywZlkE75bpOxMtAU/PSNNzXw8D8wdk9p6C8GNhVLki+mcye49F259952aZuafKKreIvRHTXYavyWlCYTQJuDCaNIIVeUoawQqR+wy+b9/zLpYzeatWPQDkR5PVc+dGL7nmF1VV11955P+zMtIIVlzzJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGM3zW9Y2vbOJC/0XRj2/87ad1BTVeF1ORr5vsgosfnUxZ7rOjHp+38P7+GLlFzVUdJVrvRHsXeV3URWpGhrHgv67tzAfmqzef8P9VJdWD40roj5tLKCJtoCvjq9macXSib/ABj4CahheD8Rl+dBkdWPtRtZUr9Fdhm9pC/je5r0c6jg0NK6fWz/+C05CaFsIu8ImtSWFvdJ2Pei+b7IK7D66m7fPvT00fm7JcxqryeBabQS7v23/iPG2n24bd3vVrLCVjbqkCPw4AC9D6qkU9iL31uPzfZNV4I1Tb4wY+y7g12oj2O9WfpcV21ZM7sUDQALUEeVqwBsaG/jSLV+i8K8KXXuPqWpobOCRHz5CtC6qu5TMNDeC1bfeVwUk9uWwqtVxCG4LQgDsOucwhRvcK29Q+lP/N1m1+2VV2bHkz4J2VZDekCa9Mu1JsIUZ8ifgUUhv9v9sKvzF84Dvqtvl9VvmbEST1RzOZHrp94/8HoBjO45prmQM0ghWCPdJwIXRJODCaBJwYTQJuDCaBFwYTQKegd1vkzrntDa0223S5/13/j3VkSJx2vkmOHE+QbIlqbkif5KAZ9D3Sh+dDzk9hFKHU3Ss7SB51F8BOrv5LE33NgHw6Z9+StPdTdgD8pX91STgGYRWhkZ8BabKFYEajy5Cn6CSL5eggleWx05D4T2FqAL/r+ntNQl4BsEFQYILg87eiULkiQgq5K/wxDbGUGGnJlWgmLl9puaK/EkCPobIkxGwQEUUBQ8W6C5nFCtsUfl0JVhQeG8h4QVh3SX5Uv5cbOWxYG2Qgq8UELwj6LvZe1BsY4ye3/VQ+Wyl7lJ8SwI+jsJn/HujAziz+KyXZ+kuw9fkEEUYTQIujCaNYEWekkawQkxmBn/HxXKmwvk98qHJag673HNqaE7055mjz5JGsOKaJwEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmje3rLmk+ag2fi+ySpQ/UI1n1z+ZNTzB795kNuvu11DRRnMAu4B5gBRoAdoxrml4Ig3Jcg9mePIhyarD970IDXlwx2iZxb6ZPmIm4H1OMcILcCHQBgn9LciAfeDXJusJs4n6Hy9k9iGGIESbxYK2rJwC+tq1014+8tvX6b3eC/xDXGsoEtHqCHgQZxwvw/8OzC4+p0CPFwEQE/ANTcHnahcm6x2/VcXLc+1cPEHF5lRP4Pyb5S7HvSXD77MmyffHBq/sDpT75Bhp54/Ret/ttL0bBOf//7n3Qn6HGBwQYLfMBxucO5L8fDGMD0B19wcdKKubrL66PpHs78oAHa3Tes/ttL6T63M+pdZFN1T5FKF8PqHr48Yr1szsdl84NwAH279kKbvNHHnO3dS8LlpXNzos79u+5Wf9+Mcjw/aMX1vNx49AdfcHHSiXrrlJW5tuHW4D2WWP8yBYwPOv+UUEIBgPEhghrsz+CtzXuHmf7h5+Ilbxt++53CP80DhrGlY68Kaht2feVwKtAKngEPAH0zvW2Ujx+DjCM4KUr23esLbt/1zG81/2Uxoboj438QpWlWEUu7e21i2pIzFX1s84e3f//L7XHr9EmUry6j5QQ0ld5RMf1Gncc6YFAL3Av+B8yGzAwl4Pot9NUZ4fpjokqjrwZ6sm3bexMCOAUoWuhDsQQlgD/Awzuet64EzQMy9txyLBHwaWYUWhUv9vdxbuCpMuMqDhTr/D2fGXorzofN2nEOXj4HD7r/9IFk2wkOybMT0kmUjxDVPAi6MJgEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGG0HK8HV53AMffKmbJK4KLuIrKQGqfHjbZtZ10EJtc7eo7Ztn3nJAtynVLqgJ/rA6nRa3KIIowmARdGyzXgP3Gliunj9/pAavRUTh8yhcg3cogijCYBF0aTgAujScCF0STgwmj/D6950SHWwvgbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 8\n",
      "iter    0   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADcpJREFUeJzt3X1wXNV5x/Hvubur3dXbSpa8AdkGObKxAqFghA1jg18CeGyXIS7jZGyoxyVOiJUxQ0oLpO5LPKgTmqTpMKEdD5kQ2nEHayY0TWcY13VJQ5IyjDMGM6bGNmDZ+BVLtiTrXdqX2z+uJSFrpdVKuvfcPX4+/2jPzt3ZR3d+Orp7997zKNu2EcJUlu4ChHCTBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwWjDbBkqpx4HHnVFRHdS6XJIQ2RUXH6Ozs1Nl207l8lW9UnfaO3d+fUqFuaW+fisAu3c3aq5kbBs3bgBg377/1lzJ2FategDw934E2L59O01NTVkDLocowmgScGE0CbgwmgRcGE0CLowmARdGk4ALo0nAhdEk4MJoEnBhNAm4MJoEXBhNAi6MJgEXRst6PbgbWpIt/KLrF3w88DF9dh/FVjFVwSo2lGxgZnCmjpJGeOL4E1xMXhz1/PM3Pk91pNr7gjLY9M4mLvRfGPX8ztt2UlNUo6GikfyyD7UE/KXLL3E2eZYFBQuIB+K0p9r5KPERl9OXmYn+gA+6o+gO4gXxoXFpoFRjNZndVX4XVZGqoXEsGNNYzWi696HnAe9Od3M2eZaoivJk2ZMo5VyznrAT2PhrncQVsRUsKlk04e1DR0OU/GsJXeu7GLhtALJejj91q+OrWVqx1P03AoKngxTvLmagdoCeh3om9Jpc9+F08zzgERUhrML02r18r/V7LChYwLzQPL4Q/gJhFfa6nHG9eflNPuj9YGi8Ob553O2Dp4KEToQoe6GMVGWKzj/udD3oe5v3cqjj0NC4fm79tL9H8HSQ4leLCR8OQwJIMeGA57oPp5vnAQ+oAJtKN/Fq26ucSZ7hTPIMv+JXlFql1JfVUx2q9rqkMb3b/S50D4+fffLZrK+xlY3Vb2GdtZjx/Rm0b22nb3mfazXub9s/YjzdAbdaLCqfqcTGRl35S40cinDdxuvGfV3g2wEoG70PjQ84QF2kjod+9hBHwkf49dZf81bvW3SkO9jTtYdvlX9LR0kZPRV/iiWXlgyNLzVcGnf7yO8iFP5PIWkrDQq613bTv6jf1RobGhu4e8vd2Le5c3iXrkjTvrWdksYSVK/C6rdIzEnQ8XjH+K9LpQF4xnqGupl1pMvSrtSXjecBT9kpTiROMDc1l8XHF1NaXEqxKua1rtfot90NQ86CkJiXmPDmVqtF9DdRutd20/OHPdhF/vpMMSkW9C3vo++ePiL/G6GksYTEvETW/WIftyEJyeuSpEv0hBs0BDxhJ/hR24/4+Vd/zvzT8+nr6OO9vvcAqA3n95IU/Yv7aa5rhoDuSlwQGA56Pn174nnAQyrEfYX3cfLiSfbP309fbx/lgXKWR5azqnCV1+VMPxPD/Vl59vtp+ZC5vmQ9c/9tLuHzYY7uPOp1CVm9WPOi7hKy2lW3C4DQjhBJkpqrGc0v+zCP/tkIkTsJuDCaBFwYTQIujCYBF0aTgAujeR7wYFuQ2S/OJnzeubBq9ouziRyPeF1G3rNeswj8hXNSOvDjANbfW6DvC0Pf8jzgga4ARUeKhsZFx4oItYa8LiPvqcMK9a5z8ZM6rbD2S8Az8Tzg/XP66a7txlbOdRrJkiSddZ1el5H3Uo+l4Mq8YEdsUl9Labp0zt+0HIO3/FELdtAmXZCm+eFm+SQwGTeAvch2Joow2A8YcGGXC7REq39OPz3zekgVpWT2noLUYynnp8zeY9K2W8594xwqpWT2noobILkrCRW6C/EvbQFPR+UT0bSIZ9/kWibzpzBazm0E4YCL5QgxUQrbtqfeRlAp9bhS6oBS6gC0TE9tQngk5xl89+4/d7GcyRtssurJYiST5uzrgwff01zH2BYuvP3KIz/vR8e0zOBC5DMJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGE0CLowmARdG8/yOHr/0TxzTt4EyoBEYXNm5GvgToA/4Oy1VjbJ231rO954f9XzjikYWxBZoqOgqPtmP2m5Z090/0RTLPreM2UWzh8blBeUaq/EfbQHX3T/RFOtuXMfK61fqLsO3tAVcd//ErBbi/EsF8PE/l19+8ksOXBy+jfDpW5/WWE0GmvejtoDr7p+YlQ8OYyfitxd+O2Lsu4Br3o/aAr794+0se3cZbX/dpquE8WX6cORDDY0NrPnbNQRqfNodSvN+1HaaMHQiRPgDf7XuzlfpNlljZixyHlwYTQIujOb5Mfhg/8SKn/p0Qb0XMjx3EtjhbRnZ7Fm1B4D2He2aKxmDT/ajzODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujed8I9kSQ+NfjhM44PfDiW+JE3pJGsLnq/k437cucc+Dd3+ymY10HdkI6rV3N84CnY2lU//CyziqhSFWkvC4j71lxy7kzBpwGsCGk01oG3gd8Rpre5b3YQRsbm8SNCRK1Ca/LyHvhx8IweAFhFKJPRVHK/4vWe03LMXjXw11OA4EC6HxU+mROhlVuEf5KGCywZlkE75bpOxMtAU/PSNNzXw8D8wdk9p6C8GNhVLki+mcye49F259952aZuafKKreIvRHTXYavyWlCYTQJuDCaNIIVeUoawQqR+wy+b9/zLpYzeatWPQDkR5PVc+dGL7nmF1VV11955P+zMtIIVlzzJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGM3zW9Y2vbOJC/0XRj2/87ad1BTVeF1ORr5vsgosfnUxZ7rOjHp+38P7+GLlFzVUdJVrvRHsXeV3URWpGhrHgv67tzAfmqzef8P9VJdWD40roj5tLKCJtoCvjq9macXSib/ABj4CahheD8Rl+dBkdWPtRtZUr9Fdhm9pC/je5r0c6jg0NK6fWz/+C05CaFsIu8ImtSWFvdJ2Pei+b7IK7D66m7fPvT00fm7JcxqryeBabQS7v23/iPG2n24bd3vVrLCVjbqkCPw4AC9D6qkU9iL31uPzfZNV4I1Tb4wY+y7g12oj2O9WfpcV21ZM7sUDQALUEeVqwBsaG/jSLV+i8K8KXXuPqWpobOCRHz5CtC6qu5TMNDeC1bfeVwUk9uWwqtVxCG4LQgDsOucwhRvcK29Q+lP/N1m1+2VV2bHkz4J2VZDekCa9Mu1JsIUZ8ifgUUhv9v9sKvzF84Dvqtvl9VvmbEST1RzOZHrp94/8HoBjO45prmQM0ghWCPdJwIXRJODCaBJwYTQJuDCaBFwYTQKegd1vkzrntDa0223S5/13/j3VkSJx2vkmOHE+QbIlqbkif5KAZ9D3Sh+dDzk9hFKHU3Ss7SB51F8BOrv5LE33NgHw6Z9+StPdTdgD8pX91STgGYRWhkZ8BabKFYEajy5Cn6CSL5eggleWx05D4T2FqAL/r+ntNQl4BsEFQYILg87eiULkiQgq5K/wxDbGUGGnJlWgmLl9puaK/EkCPobIkxGwQEUUBQ8W6C5nFCtsUfl0JVhQeG8h4QVh3SX5Uv5cbOWxYG2Qgq8UELwj6LvZe1BsY4ye3/VQ+Wyl7lJ8SwI+jsJn/HujAziz+KyXZ+kuw9fkEEUYTQIujCaNYEWekkawQkxmBn/HxXKmwvk98qHJag673HNqaE7055mjz5JGsOKaJwEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmje3rLmk+ag2fi+ySpQ/UI1n1z+ZNTzB795kNuvu11DRRnMAu4B5gBRoAdoxrml4Ig3Jcg9mePIhyarD970IDXlwx2iZxb6ZPmIm4H1OMcILcCHQBgn9LciAfeDXJusJs4n6Hy9k9iGGIESbxYK2rJwC+tq1014+8tvX6b3eC/xDXGsoEtHqCHgQZxwvw/8OzC4+p0CPFwEQE/ANTcHnahcm6x2/VcXLc+1cPEHF5lRP4Pyb5S7HvSXD77MmyffHBq/sDpT75Bhp54/Ret/ttL0bBOf//7n3Qn6HGBwQYLfMBxucO5L8fDGMD0B19wcdKKubrL66PpHs78oAHa3Tes/ttL6T63M+pdZFN1T5FKF8PqHr48Yr1szsdl84NwAH279kKbvNHHnO3dS8LlpXNzos79u+5Wf9+Mcjw/aMX1vNx49AdfcHHSiXrrlJW5tuHW4D2WWP8yBYwPOv+UUEIBgPEhghrsz+CtzXuHmf7h5+Ilbxt++53CP80DhrGlY68Kaht2feVwKtAKngEPAH0zvW2Ujx+DjCM4KUr23esLbt/1zG81/2Uxoboj438QpWlWEUu7e21i2pIzFX1s84e3f//L7XHr9EmUry6j5QQ0ld5RMf1Gncc6YFAL3Av+B8yGzAwl4Pot9NUZ4fpjokqjrwZ6sm3bexMCOAUoWuhDsQQlgD/Awzuet64EzQMy9txyLBHwaWYUWhUv9vdxbuCpMuMqDhTr/D2fGXorzofN2nEOXj4HD7r/9IFk2wkOybMT0kmUjxDVPAi6MJgEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGG0HK8HV53AMffKmbJK4KLuIrKQGqfHjbZtZ10EJtc7eo7Ztn3nJAtynVLqgJ/rA6nRa3KIIowmARdGyzXgP3Gliunj9/pAavRUTh8yhcg3cogijCYBF0aTgAujScCF0STgwmj/D6950SHWwvgbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 9\n",
      "iter    0   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADcpJREFUeJzt3X1wXNV5x/Hvubur3dXbSpa8AdkGObKxAqFghA1jg18CeGyXIS7jZGyoxyVOiJUxQ0oLpO5LPKgTmqTpMKEdD5kQ2nEHayY0TWcY13VJQ5IyjDMGM6bGNmDZ+BVLtiTrXdqX2z+uJSFrpdVKuvfcPX4+/2jPzt3ZR3d+Orp7997zKNu2EcJUlu4ChHCTBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwWjDbBkqpx4HHnVFRHdS6XJIQ2RUXH6Ozs1Nl207l8lW9UnfaO3d+fUqFuaW+fisAu3c3aq5kbBs3bgBg377/1lzJ2FategDw934E2L59O01NTVkDLocowmgScGE0CbgwmgRcGE0CLowmARdGk4ALo0nAhdEk4MJoEnBhNAm4MJoEXBhNAi6MJgEXRst6PbgbWpIt/KLrF3w88DF9dh/FVjFVwSo2lGxgZnCmjpJGeOL4E1xMXhz1/PM3Pk91pNr7gjLY9M4mLvRfGPX8ztt2UlNUo6GikfyyD7UE/KXLL3E2eZYFBQuIB+K0p9r5KPERl9OXmYn+gA+6o+gO4gXxoXFpoFRjNZndVX4XVZGqoXEsGNNYzWi696HnAe9Od3M2eZaoivJk2ZMo5VyznrAT2PhrncQVsRUsKlk04e1DR0OU/GsJXeu7GLhtALJejj91q+OrWVqx1P03AoKngxTvLmagdoCeh3om9Jpc9+F08zzgERUhrML02r18r/V7LChYwLzQPL4Q/gJhFfa6nHG9eflNPuj9YGi8Ob553O2Dp4KEToQoe6GMVGWKzj/udD3oe5v3cqjj0NC4fm79tL9H8HSQ4leLCR8OQwJIMeGA57oPp5vnAQ+oAJtKN/Fq26ucSZ7hTPIMv+JXlFql1JfVUx2q9rqkMb3b/S50D4+fffLZrK+xlY3Vb2GdtZjx/Rm0b22nb3mfazXub9s/YjzdAbdaLCqfqcTGRl35S40cinDdxuvGfV3g2wEoG70PjQ84QF2kjod+9hBHwkf49dZf81bvW3SkO9jTtYdvlX9LR0kZPRV/iiWXlgyNLzVcGnf7yO8iFP5PIWkrDQq613bTv6jf1RobGhu4e8vd2Le5c3iXrkjTvrWdksYSVK/C6rdIzEnQ8XjH+K9LpQF4xnqGupl1pMvSrtSXjecBT9kpTiROMDc1l8XHF1NaXEqxKua1rtfot90NQ86CkJiXmPDmVqtF9DdRutd20/OHPdhF/vpMMSkW9C3vo++ePiL/G6GksYTEvETW/WIftyEJyeuSpEv0hBs0BDxhJ/hR24/4+Vd/zvzT8+nr6OO9vvcAqA3n95IU/Yv7aa5rhoDuSlwQGA56Pn174nnAQyrEfYX3cfLiSfbP309fbx/lgXKWR5azqnCV1+VMPxPD/Vl59vtp+ZC5vmQ9c/9tLuHzYY7uPOp1CVm9WPOi7hKy2lW3C4DQjhBJkpqrGc0v+zCP/tkIkTsJuDCaBFwYTQIujCYBF0aTgAujeR7wYFuQ2S/OJnzeubBq9ouziRyPeF1G3rNeswj8hXNSOvDjANbfW6DvC0Pf8jzgga4ARUeKhsZFx4oItYa8LiPvqcMK9a5z8ZM6rbD2S8Az8Tzg/XP66a7txlbOdRrJkiSddZ1el5H3Uo+l4Mq8YEdsUl9Labp0zt+0HIO3/FELdtAmXZCm+eFm+SQwGTeAvch2Joow2A8YcGGXC7REq39OPz3zekgVpWT2noLUYynnp8zeY9K2W8594xwqpWT2noobILkrCRW6C/EvbQFPR+UT0bSIZ9/kWibzpzBazm0E4YCL5QgxUQrbtqfeRlAp9bhS6oBS6gC0TE9tQngk5xl89+4/d7GcyRtssurJYiST5uzrgwff01zH2BYuvP3KIz/vR8e0zOBC5DMJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGE0CLowmARdG8/yOHr/0TxzTt4EyoBEYXNm5GvgToA/4Oy1VjbJ231rO954f9XzjikYWxBZoqOgqPtmP2m5Z090/0RTLPreM2UWzh8blBeUaq/EfbQHX3T/RFOtuXMfK61fqLsO3tAVcd//ErBbi/EsF8PE/l19+8ksOXBy+jfDpW5/WWE0GmvejtoDr7p+YlQ8OYyfitxd+O2Lsu4Br3o/aAr794+0se3cZbX/dpquE8WX6cORDDY0NrPnbNQRqfNodSvN+1HaaMHQiRPgDf7XuzlfpNlljZixyHlwYTQIujOb5Mfhg/8SKn/p0Qb0XMjx3EtjhbRnZ7Fm1B4D2He2aKxmDT/ajzODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujed8I9kSQ+NfjhM44PfDiW+JE3pJGsLnq/k437cucc+Dd3+ymY10HdkI6rV3N84CnY2lU//CyziqhSFWkvC4j71lxy7kzBpwGsCGk01oG3gd8Rpre5b3YQRsbm8SNCRK1Ca/LyHvhx8IweAFhFKJPRVHK/4vWe03LMXjXw11OA4EC6HxU+mROhlVuEf5KGCywZlkE75bpOxMtAU/PSNNzXw8D8wdk9p6C8GNhVLki+mcye49F259952aZuafKKreIvRHTXYavyWlCYTQJuDCaNIIVeUoawQqR+wy+b9/zLpYzeatWPQDkR5PVc+dGL7nmF1VV11955P+zMtIIVlzzJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGE0CbgwmgRcGM3zW9Y2vbOJC/0XRj2/87ad1BTVeF1ORr5vsgosfnUxZ7rOjHp+38P7+GLlFzVUdJVrvRHsXeV3URWpGhrHgv67tzAfmqzef8P9VJdWD40roj5tLKCJtoCvjq9macXSib/ABj4CahheD8Rl+dBkdWPtRtZUr9Fdhm9pC/je5r0c6jg0NK6fWz/+C05CaFsIu8ImtSWFvdJ2Pei+b7IK7D66m7fPvT00fm7JcxqryeBabQS7v23/iPG2n24bd3vVrLCVjbqkCPw4AC9D6qkU9iL31uPzfZNV4I1Tb4wY+y7g12oj2O9WfpcV21ZM7sUDQALUEeVqwBsaG/jSLV+i8K8KXXuPqWpobOCRHz5CtC6qu5TMNDeC1bfeVwUk9uWwqtVxCG4LQgDsOucwhRvcK29Q+lP/N1m1+2VV2bHkz4J2VZDekCa9Mu1JsIUZ8ifgUUhv9v9sKvzF84Dvqtvl9VvmbEST1RzOZHrp94/8HoBjO45prmQM0ghWCPdJwIXRJODCaBJwYTQJuDCaBFwYTQKegd1vkzrntDa0223S5/13/j3VkSJx2vkmOHE+QbIlqbkif5KAZ9D3Sh+dDzk9hFKHU3Ss7SB51F8BOrv5LE33NgHw6Z9+StPdTdgD8pX91STgGYRWhkZ8BabKFYEajy5Cn6CSL5eggleWx05D4T2FqAL/r+ntNQl4BsEFQYILg87eiULkiQgq5K/wxDbGUGGnJlWgmLl9puaK/EkCPobIkxGwQEUUBQ8W6C5nFCtsUfl0JVhQeG8h4QVh3SX5Uv5cbOWxYG2Qgq8UELwj6LvZe1BsY4ye3/VQ+Wyl7lJ8SwI+jsJn/HujAziz+KyXZ+kuw9fkEEUYTQIujCaNYEWekkawQkxmBn/HxXKmwvk98qHJag673HNqaE7055mjz5JGsOKaJwEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmje3rLmk+ag2fi+ySpQ/UI1n1z+ZNTzB795kNuvu11DRRnMAu4B5gBRoAdoxrml4Ig3Jcg9mePIhyarD970IDXlwx2iZxb6ZPmIm4H1OMcILcCHQBgn9LciAfeDXJusJs4n6Hy9k9iGGIESbxYK2rJwC+tq1014+8tvX6b3eC/xDXGsoEtHqCHgQZxwvw/8OzC4+p0CPFwEQE/ANTcHnahcm6x2/VcXLc+1cPEHF5lRP4Pyb5S7HvSXD77MmyffHBq/sDpT75Bhp54/Ret/ttL0bBOf//7n3Qn6HGBwQYLfMBxucO5L8fDGMD0B19wcdKKubrL66PpHs78oAHa3Tes/ttL6T63M+pdZFN1T5FKF8PqHr48Yr1szsdl84NwAH279kKbvNHHnO3dS8LlpXNzos79u+5Wf9+Mcjw/aMX1vNx49AdfcHHSiXrrlJW5tuHW4D2WWP8yBYwPOv+UUEIBgPEhghrsz+CtzXuHmf7h5+Ilbxt++53CP80DhrGlY68Kaht2feVwKtAKngEPAH0zvW2Ujx+DjCM4KUr23esLbt/1zG81/2Uxoboj438QpWlWEUu7e21i2pIzFX1s84e3f//L7XHr9EmUry6j5QQ0ld5RMf1Gncc6YFAL3Av+B8yGzAwl4Pot9NUZ4fpjokqjrwZ6sm3bexMCOAUoWuhDsQQlgD/Awzuet64EzQMy9txyLBHwaWYUWhUv9vdxbuCpMuMqDhTr/D2fGXorzofN2nEOXj4HD7r/9IFk2wkOybMT0kmUjxDVPAi6MJgEXRpOAC6NJwIXRJODCaBJwYTQJuDCaBFwYTQIujCYBF0aTgAujScCF0STgwmgScGG0HK8HV53AMffKmbJK4KLuIrKQGqfHjbZtZ10EJtc7eo7Ztn3nJAtynVLqgJ/rA6nRa3KIIowmARdGyzXgP3Gliunj9/pAavRUTh8yhcg3cogijCYBF0aTgAujScCF0STgwmj/D6950SHWwvgbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "state_values = {s : 0 for s in mdp.get_all_states()}\n",
    "\n",
    "for i in range(10):\n",
    "    print(\"after iteration %i\"%i)\n",
    "    state_values = value_iteration(mdp, state_values, num_iter=1)\n",
    "    draw_policy(mdp, state_values)\n",
    "# please ignore iter 0 at each step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after iteration 29\n",
      "iter    0   |   diff: 0.00000   |   V(start): 0.198 \n",
      "Terminated\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC4CAYAAABQMybHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXl8VNXd/9+zZWay7xsEkhCWsBN2iAQExAVw37U/d0sXtfrz6aPPU2uhL221au3jY7WupbVirT/rghYtKIgIhD0BkpCwJJB93yaZzPL745oZhpkkc2fO6DC579fLl/fce+eTO9zvPXPuOedzviq73Y6CQqii/r4vQEEhkCgBrhDSKAGuENIoAa4Q0igBrhDSKAGuENIoAa4Q0igBrhDSKAGuENJohzpBpVLdA9wjlSJmwoQAX5KCwtAYDIcxmUyqoc5TyRmqV6lm2fPyxAzt79u3F4B58+YL0du58xsACgoWC9ED2Lr1SwBWrLhYiN6mTf8C4IorrhSiB/DPf74PwI033iRE7+23/wbAXXfdLUTv1VdfAeD++x8QotfP22+/TV1d3ZABrjRRFEIaJcAVQholwBVCmiFfMr2lN7yX05NO0xXfhVVrRWvWYuwwknEoA323XrbevoJ9mI1mt/1Tvp5CREeET9e4c95Oeo29bvtnFs4ksjNStt7WGVvpMfS47Z9/cD7R3dGy9TZN3IQpzOS2f3HpYmJNsbL1AD7M/pAuXZfb/otPXkxcb5xsvQ2pG+jUdrrtv7LuShL6Eny6xtejX6dD0+G2/6b2m0iyJvmk2Y+wAD8++zimGBNRDVHou/SYDWY6EzrpM/T5FOD9xNbHYug2OMo6s87va41vjMdoMgrTTGpOwtjr1AuzhPmll9KWQoTZ+RDrLb7/+/WT3plOpNn5EOut/mlmmDKItjgfYoPVMMjZ3pFlziLGFuMoG23GQc72DiEBbtFZMMWY0Jg15HyTgwrp5damtsGQ77mDk3w6mfj6eAFX6SStJo3ExkRheiPqR5DSkiJMb3TzaNLb0oXpAYxpG8PIzpHC9MZ3jSezJ1OYHsAk8yTG9I0RqikkwDUWDWqLGmuYlZKCEqIao4hsiiSqIQqNVeM4T2VXYVfJ62asH1lPe3y7o5xZkumXHkBNWg2tsa2Ock55jrTRLyXzoTyTfIbmmGZHOfdkrvOgXb7eqfhTNEY2OspTz0z1Sw+gIqaCOmOdozyzYaZfmqURpdToaxzl+W3+d/ceDjvMae1pR7nAVOC3ppAAV9lVjD4wmspplZhiTJhiTNSPqSfMFMas7bOIbY5lWss0FjYs5LO0z9iZuNNr7dbkVpfy3H1zATBYDNxecjtH447yWcZnsq63ObHZpbzgwAIAVlSsILY3ls+yP6M0odRrvYb4BpfyoqJFAKS1pbGsZBl7Ru1hV9Yur/XqYupcykuOLHFsX7njSpqimtg+eTsNsQ3nfnRAqiOrXcrLji0DILc8l+zKbAqnFVKWVea1XpWxyqUsIsBPhJ1wKQdNgAPEVccRUxtDqi6VRGMiG/M20hLZAqPg4a8fdpx35ekrqTHWsNtL3UmFk3hh4wsDHp/aNBWNXcOnMq71rk/u4ubdN3s8ZsfO9Yev54n8J7zWW7J3CY999NiAxxecWECnvpMPvdSbc3wOv13/2wGPG3uNXLT3It5a+pbX13jLrlu489M7BzxesLvA7UEdjJuKbuLazdey8ZqNXn9mKNZtWEfnwk56It1f3H1FSIDbVXa64rqIbI6kvree+t56Ik5G0DK5hUMJh3go7yEmtE1gWe0yPkn/hONRx73W7tP08ci8R9z2h1nDuOPoHRTHF7MzdSfs/qnXmt9kfMMx4zG3/Rcev5AEUwKbszZj0Vi81muJaOE3K37jtj+5PZmLD1/MzuydlCWXQYmXgip4/ornPR66fMflNMQ0sHfsXq+vD+DUyFP86cY/ue3POZHD+OPj2T19N81xzR4+6ZmY1hhiW33r2RmMsJ6w4Atwm9pGWX4Zhg4DxjYjaqua1lSpaRHdIL1pl8SUUBLj7R0eGrPGzEuTXxKmB7Ale4tQvfroetbPXy9U84MFHwjVK88qpzyrXKhmMCEkwNU2NckVyXQkdtCe3I5NYyOsJ4ykk0mklIvrXVBQkIuwl8yRh8V1QQHkbc0Tqgcwb+c8oXoF+/1/CTqbFUdWCNUDWH18tVC9G2pvAGDGmRnCNO9ovwOA/JJ8tswU+yuqDNUrhDRKgCvIxthlZGzJWACm7J0iRHP++1I348L/t9A5HiEAJcAVZNNr6KXH0INVbaUpqUmIZmtyK3aVndakVr9Hv89GCXAF2dg0NvbO20tdWh3VGdVDf8ALymeW05bYxuH8w0L0+pH9kqlWi30mgl0PQKsVNh4GgMHg/8SkcwkPDxeqFxk5+OzKxumNNE5vJBLvZmFGRw8xuzIa9v1w37ebQ8/E1Gg0Q54DPljWYI/X5ysoBIq0tHSqq6v9t6ypVKp7VCrVHpVKtQe8H8pVUAgGZNfgs2aJeQPYs6cQgAULFgrR27HjawCWLLlQiB7AF19IfbKXXbZSiN7GjR8DcMMNNwrRA9iw4W0A7rzzLiF6r732KgAPPPAzIXq///1zAPziFwPP1fGFV199VUwNrqBwPqMEuEJIowS4QkgjrP/r0IWHMIe7m4QnbptIeLv8Lqy9F+z1aBCe9s00n03H38z9xqNJeNaeWUR1RcnW2zJ1Cya9u0k4vzifGFOMh08MzodjPqRb1+22f8WJFT4ZhAHeSXvHo0n4itorfDIJvxb9Gh1qDwbhjptItib7dI1/UP+BNlWb2/67rXeTSqpPmv2I7eAFYupi0Hc5Da1as39/Iq4hzsV07K8eQEJTgovpOKzPP5Nwcmsy4T3Oh9hfk7BogzB4MAnb/OuLz+pzNQiH2/zvhx9rH0uc3fkgh+O/pvAAT6xMJK7Ot9rGE8mnk0lo8G05goFIq0kjqcm/5QjOJqMhg9RW/2qas8luzRZqEAYY1zWOTFOmML1J5knk9OUI0wOYbpvOBMFrXwoP8MZRjXQkOH/CRh0ZBUB6dzrTmqexLWWbx3U6BuJc03FWaRYAGpuGZZXLOBp/lMroSlnXeK7peGyFNHFoUu0kwi3h7Evbh1Vj9VqvKqmKpijnnIxJVZMAiDJFMfPETPZn7qct3P0neCCOxx6nPrzeUc6rd04dzivOoymuiVPpp2TN2SiLKKNWX+soz2uVpg4n1SSRVp3G0clH6dP3ea13rkF4sWmx9xczAAfUBzhlP+Uor7D7P31YeIC3pbjeyHfWv+NSXlS3iBcmvOD1eGhLUotL+W9v/M2lvLB6IbtTd7NDxjU2JbhOEPrLq39xbNuxU3CygOcWPOe1Xn1svUv5jZfecCnnnchj44yNeOtePNcg/PIfXnYpW1VWapJq2Hih937Ic03CLzzj6nOdWDSRj6/62Gu9EzpXg7CIAD+mOuby0K6wBmGAT9o5iZv23oTaLnXQ9DvoJ7dOJtISyeHYw7SGtQ4m4cL4/eNZdXAVSSapSbE7RbIrGywGpjZNpU3fxtH4o1A7mIoriwoXsXq/0wiwN13yN86snolVZWV/6n5sKpvXenNK53DNrmscC/4cHHUQgNTWVFLaUzgTf4b66PrBJFzIP51PweEC0uultVGOjDniODaxYiJmnVmWAx7gsqrLWP31alR2KYKOjZc8qVnlWWitWk5lnaLX4P5SPxA/3fFTrvrsKj54QJyFbt2GdegW6ehK8f4XfiiEB3iPpof3R7/vtv99+/tEm6Np1Xsf3P3sSPdcP2/q2USbvk322ijNxmY2jnev/f495t+o7WpMOveekcGwqq1szd3qfsAOMaYYWc2TfipGV1AxusJt/95Je+nR92DTeP8AAvQYe9i90H0tg8L5heh79Jgi5H3n8wXhAT4QNpXNp+AejFaDWL1erfc1mFeo8Cm4B6M73L0b0R9sGlvIBjcoAz0KIY6wGnzqlqlDnySDmV/NHPokmczfJSabRD8XHhI3sQtgdYVYgzDA9TXXC9W7s11aPGha9TRhmvfZ7gMk0/GuRd6vAOYNSg2uIBu1RU3GkQwAEqrEjFGk7U8DYOQesf3/SoAryCasJwyNTXLUJJwRE+ApxdL6OYmliYrpWOH7pSeyh1MTT2HRWjg+w/tl+AajYqnUY1RxYYVQ0/F31ouiEFoUFxRTOrcUi977NRwHoyO9g50/3IkpQWyPjuLJVDgvGTkyg6qqKsWTqTC8kV2DT5/u/SSkwThwYD8As2bNFqIn2uMJTp+nuEah9G/9gx/8H0F6sH79nwFYs+ZHQvT++McXAXjssV8K0Vu79lcAPPPMs0L0+nnuuefE1OAKCuczSoArhDRKgCuENMK6CQ9fdJi+cPcJ8+O+GEd4m3zrkWiPJwj2eT4AxAIbcKYmyQRuA3oA94wmQ/JexnsezSArT68k3uxbKsW/Jv7VY5LVa5uuJdEiP5Xi86rnPfon77Hd47N/8tedv6bF3uK2/8HwBxmhGeGTZj/C+8Gja6MJ63J6HLW9/v0J0R5PCIzPUyQju0YSZXGaoEUkWR3dO1qoJ3OsfSzxOB86Ef7JiZqJJKidI6ORKvnZp89F+J2NPxVPbI245ESiPZ4QGJ+nSHI6chjVPUqoZq4pl6zeLGF6M+wzhPsn5+jmMEUnZr3xfoQHePPoZjoTncsUjCySJs+o7WrizHE06eWtJz2Qx9MfBvJ5+sQMpKYJ4MWiqF5RHlXukrR1dpOzKzWiM4IeY48szyjAUeNRzujOOMr5nfmANHHKYDLQHSVvnvl+1X5OIdY/ubtvNxVWp8njCsMVfmsKD/D21HaX8s8//jkAk9smY7AZKIou4t2Md73WO9fjKSLAz/V5+hXg4/28GA+cjjjtUn7wgwcd2znHc+gN66Uwr5Bj49xTIQ7EKf0pOGv1iX7NzLJM6XjOKfYt2ue13jGV698WEeBHrEfgrOc2KAP8gi8uYO3WtQMen9Axgag+7xfZuWjLRTyy7REenvXw0Cd7yboN6yhTl1ER624Jk42nl0w/WVyzmDXvrSG50fNCOlqLlqxTWbIC/JYjt3Dn3wdOBDvi5AiK5xZ7rffj4h9zzT+u4YvHvvD6M0OxbsM64q6IozdJnLNKeIBXhVfxs+nuK5OmmdKY2jqVr5K+olvr/c+hiBes71JXCCrYeKlnx/zkosm0xLdwJv2Mx+MD0ZjeyN/X/N1tf0JNAsnVyRybcgxLmPcTp3TdOll/31tUFoFTCfkOZxPWGGuoMdZ8V38uZCme4n0t6w1NaU00pYnJsxOMKAM9CiGNsBp80meTREkBTo9nTq245cH6fZ75Jfn8bfzfhjh7CH7vYd9J4HHfJa+uutr3Dw/ALY23CNW7334/2GFc7Thhmv8d+d8A5JXkcWTlkSHOlkfQ1+CxZqlPPdwiJslShFkasYzplb/6q4ITQ5v0DqPp8S4Z1FDo2qQ2fViLfwuhnktQB7jRYuSC+gsAWF0pxnF+dblUS1528jK0tuAawTxfMLQYSKiQBsoyt2UK0cx+IxuArL9kgbw1jQYlqAPcpDVRHlWODRuFiYVCNPek7MGOneL4YixqMXar4UZPbA+dSZ3YVXZqp8pYM28QGuc1YsdO88xmoVEZ9FXYBxkfMK1lGhVRAvqsgaPxR9mevt2xxqGCD6igdFUpsSdj6UoVs45g0+wmDLUG6i6sG/pkGSieTIXzkszMLE6cOKF4MhWGN997nsz8/AuE6G3f/hUAy5YtF6IH8O9/fw7A6tWXC9H78ENpqeHbb79DiB7AG2+8/u2WWN/os896vz76YDz4oDSq/eabfxai18/jjz8upgZXUDifUQJcIaRRAlwhpFECXCGkCdpEsIULCz0ahKfvnE5kp29eve2ztntMBDt3/1yfEsF+Pulzj4lgC44W+JQI9t0R73pM2rq6erVPSVsDYYxe17HOo0H4oYiHfDYIP1T1EE1W9xmNv0r7FaP1o33S7Oe8SAR7dtJWXZ//85ATmxOFaqa0pRDR63Tl9yej8pWM7gxX07GfBuFAMFHrahCOUPmWffpsphmnkax1mjyiNf57AIM+EWxqdapwg3B6bTrJzb6lnfbEqMZRpLWlCdMb2zmW0Sb/aq5AM1c3V7hBeFHkImZGiM3s8Z0lgs1tzWV59XI+Hfkpx6K8t1rVptfSFuf0ZWaXSZNy9BY9txXfxpGEI+xM3ynrGqtTq2mJcf7Mjj8hGSuXHFtCQncCX+R8QVOE9yaAysRKl0Swk09PBiCpLYmL9l9E4dhCjqV7/52PRR6j1uCc4zG3Za7Xn/VIAIzRu/p2UW4td5SvNFwJgLZdy5hXxtA0u4mm+fKMFNs6t1HSU+Io35xws9/X+Z0ngr2n7B5eGveSrESwLTiD8a3X33I5nt6ZzojOEchxBjbGN7qU17+y3rFtx864hnE8teQpr/XqYlznT7z24msu5cv2XsYXvV/gbUbJqnDXpK1+B3gAjNFHLK7zttc+6urDNZ4xYqwzIoeDpoMu5aAM8LG7x5Jdme0o/2ay9CYzpWUKBXUFbErfxMnIk17r5R7MJfN0pmNq67MzpVVKDVYDtxXfRkl8CVtGbYGmH3utmVecR3a18xr/d8H/ArC8bDlxpjg2j90sa6bh7PLZTKhxrhHy+lJpdDGlNYWlB5eyb8w+jmQcAS/dZpeeupSH33iY9bevH/pkbwiAMfp24+3kdeahskqDiYcfPQxI87qzX8+mJa+F2uW18I33mus2rCP75mz6MrxPKT4UwgPcprLRZHD/afoy7Uu+TP3SpxHlDr370mMAT8590ic9i9pCa7h7js13p327nIVcTRW0RbgvZ9YW0UZZepnQlBzBRF+ceyCaE80U/booaL7zdztdVvSXDna9QGkGO0H0nZWBHoWQJmgTwc7+Wkzmh7PJ35MvVG/5YXEzFwGuPXMtAJHt/i86CQTEGP2LqF/4/uEBeCbjGQBGl4ymmmqh2koNHmzY4YJt0hTiBV8t+J4v5rsj/g1ppdrEVxKHjydzWKIC7FJ3pcoeRI3ZQGOTvrMdu9A2vBLgQcje2XuxaC0cmHHg+76U74y2K9qwG+y03tCqJIINdepT63n71re/78v4TrEmWKl6qWroE2WimI4VzktycsZy7NgxxXSsMLyRXYNPmyZmsZyDB6X25XBMBPvQQ/9XkB4888zvAFi7dp0Qvccek7oBX3rpZSF6P/zhvQB89NHHQvT6+dnPfiamBldQOJ9RAlwhpFECXCGkEdZNeOSiI/RFeEgEu2UcxjZ584JheCaCfSXiFdrV7W77b+26lWSbbw6kZ23P0or7zMk1qjWkqeS7kB6tf5RmW7Pb/v9K+C8ydBk+XeOdRXdSb6532/987vNkh2d7+IT3iE8EW6MkgvWXbEs2sTZnrlGjXX4FcS7jGOeSuDUC/zyUU/RTSNIkOcqRav/nz8yOmU2a3vnQxWj9X8M9IIlgY2rELS4/HBPBTu6bzFjLWKGaM1UzyVXlCtNbaFzIdMN0YXoAyxOXMz92vlDNgCeCHVEkLSUQY44hpzOH/XH7sanOmU1jh9y2XFp1rdREuCaq8jURbFxPHBntGRxKOuTWyxfsiWCLdcWc1jhzZS7pXeLYTi1LpTOhk84E9+UlBmOvfS8n7Ccc5UvVlwJgaDIQWRNJ48RGWW9kX5u+psxc5ihfF32dtGGB6N3RdE3uwhotL1nt542fU9zhtD3dnXG3rM97Qnwi2DTXNuS7f3VN+np5zeW8MOYFR1lv1XPfkftI7nVtY96QfwN14XU+JYJdfmo5C6sXorPruL78eqfmnBuoM9YFfSLY49rjLuVnnnjGpWzVWDmTe4ZDKw55rVlGmUv5yceedCn3xPRw8C5XT+RgFPUWuZR/8R+u02htOhv117q3qwejsM01yUFQBvj0bdN55OtH0FuldnObVgrQGIvUbOnSdGFROQeL7NhpD2t3BHi7rt2xH+A/3/1PKs2VlMW43qDB6NJ1YVfZwQ7d2m6sKquL5g8+/QEV6grpHH8JgN9xdfdqVm5dSUax9NLWE+FcrMjQJb079EbIS5Z6e/PtXLf+OtQWqZo2R0ov8GGd0vtSX0Qfdo33/x5rItdw6fpLMZyUrscSLd1TbbsUUnatHWuEvBp83YZ15P04D1umuPmywgO8NayVJyY84bbfaDUywjSC8ohylyaDWWPm5fEvM7JrJJ3aTlr1rQ4dgHcz35XdBt+RvoN9yftI60rjRIzzZ7lNLz1su1J3BXUbHBWU5pdSml/qdijuTBzdsd2yA7wnoYfCB93TwOg6dRibjLSPdu+9GQy71s6ZH3lIRmuD8KPhmMaasIfZ4RVZssL5zroPTBoT5ZHlAx4/Nz+7v/Roe1yCO1RoGeG+bJo/9EX20RcpzsWOGroneZ/JOtAoAz0KIY2wGnziZxNFSQHiPZ7gTAQrhAD4He/u8v+l6lweVD8oVO+JZPfmp7+8NkVaKCny8Ui6EVv7KzW4QlCgKZISymr2iUks248S4ApBQdibUm9O2DthiulYIfQw3yx1W5qvMw+vRLAKwwPrDCumR01YZ8jrOx8KxZOpcF6SmzuRI0eOKJ5MheGN7Bp8wgQxuclLSo72qwrR6/c7il35UdK8994fClF7+eWXAHjkkUeF6AE8+aTUbff8838Qonf//fd9uyX2vuzZs1eQnsStt94qpgZXUDifUQJcIaRRAlwhpBHWTVi+qhxLhPuaKZn/ysTQKjMNXgD8joHQfCvpLY95La9uuJpES6JsvRfDXqRN5Z4p4g7zHaTYU+RfIPCrtl959FA+HPUwI7Uj5YkF4r4AqzavosZU47b/rQveYnyMfxPuhfeDR5yJcMwxBtD0ih16DUZG9Ywi2uq08xht/nkoc6w5xNqdnsxwu28m67OZpJtEotr50InwUIrmguQLGBnhfOjiwvy3KgoP8NjjsUSdkZ81+HxmQvcEsnr9cAWdwzTbNMbZxgnTA5gXNo+pYeInsInk8lGXszh1sVBNnwNca9N6zETWmt1Kd7JzRljKft9+WoGA+B0DoVkSXkK13pmZYGG7c/k4tUWNTStvcsVB9UFOqU45ysutzkwSaosam8Ymuxdvp3kn5RbnfPyrwq+SJ3A2gbgvwAeVH7C3ydmd+NCkh/zWlB3gcX1xXN5wOXPb5/JR4kfUhEltpypbFRYsdI3oogtnX7lfAR4Av2MgNCsNlS7l6w9IPtCE+gRyD+ZSMaGCotlFnj7qkXKNqzHk5sPOfJFTP5hKZ2InpctKaRntvfnhcN9hl7JfAR6I+wJ8Vf+VS/l7CfBZ7bOY0z4HgFWNqxz7t1i30EUX6zasY3PvZmr1tQNJeE8A/I6B0Lz6xNX85M8/GfD4mJIxNCV5n/X3avPVrHlqDRqL5/eXqIYoxm8ez847vM/w/MsPf8nYzLE0FcjLPuyRQNwXJE/miqdXoE4T17knO8A/j/+c0/rTLGlZwoaUDTSHSW/oTTrpH+6lES8NuzZ4R0wHG+7d4LY/qiWKGTtnUDKthPr0evjKw4c9oYLND2/2eGjyh5PpSOmgKk/eYvHabi3hleE0ISDAA4i9S4AR/Czkt8FVcDTyKEcjjw597jCnI66DbZdsE6pZvNrLdMkKgDLQoxDiCOsmzPkoR5RUQPyOgdC8ueHmoU+SwY/MPxKqB/DLmF8CMLVkKi2z/HTkB+K+AB8t/QiAjsc9p2z3B6UGHw5820upsgRvWkJ737dtb/cFhf1CCfBhQPK/pFXDYg/EEtYYNsTZ3w/da7od/7fbxL1oKgE+DOga14VNY6Mvqg9znOAqUhDa+VpQg2a2BpVa3C+N4skcBnTldNGa10r75HYI0qlBYTeEYd1rxfATmRPzhkAJ8GHC6VvELo0nGlWEivAX/Z9U5qarmI4VzkemT5/B/v37FdOxwvBGdg0+fry8zAIDUVraP5lBrLn1kksuFaQHn376CQDXXXf9EGd6x9///s63W+KN0f/4x3tC1K655upvt8Tel7Y2ecszD0VBQYGYGlxB4XxGCXCFkEYJcIWQRlg3YcWqCiyR7g6f0Z+ODg7TMfDl9C8x6U1u+xcWLSS6W7415eOcj+kOc1/PennFcuJ6ffATBuB7ryldQ0Ofe+fA02OeJsso02YXoPsy5bUpVLZXuu3/6uavmJrsn81uWJqOk1qSCO9x9rmG9fk3fJ3WkUak2Wni7U/AFUzMjJpJaliqoxytFeg1E8TFWReTFet86BLD5a9McC7CAzymIiboDQ8ZDRmktPhhpTuH7NZsRnSMEKYXCJbGLWVO9Jzv+zIG5dbJt7IyZ6VQTeEB3jamDVOKsxmQvM+3HOtAwMytVUlVNEU5nS0TK6X0K9NPTSeyN5LCrEJ6da5ZzGLbY8k7msfBcQdpinN1xRyPPU59uDMn5Iy6Gf5dYAC+9+aWzRzucvoyb0+73XexAN2XvxT/he2ntzvKv1nsY5vnLIQH+LmmY78CPEDm1oY41zbpmy+/6di2Y2fWyVn8z9L/cewbWTuSldtWokLF+FPOi9r6wFa6w7qpiXJdtMbvAA/A997b4br4pV8BHqD78q8T/3IpB2WAr9uwjvST6Tw1+in/xQJkbl10cBEFRwsc5S0TtgCwoHwBADvG7MCqdi7E3hTbRElmCbknc6lKqaIyVXoh6q/lb9x1IyqjCovW/SXbJwLwvddtWMfs5Nl03idgoC6ApuO7X78b4yT/Fk46m4BMtlIJHakTT6exkz1Z7nNq9o/ajwoVFo1roJoMJr6c8yU7pu/ArDM7BvnMOmnqaWV6ZdC3wYcrymzCs7BqBk+fYQ4LzrnUCgOjDPQohDTCavAxH40BIL80nxMGP1NoB8jcuvjAYv8EzmFludgurUB87z+O/yMACSUJ9CbLy2/vRoDuS9Gd0qpf+x7f55+QB4TW4Nnd2QBk9WQRZQnuvvDhhG6vDgD9Nj24D+QGBbW/l1ZCq/lNDXJmuA6F0AA/e03scJt4d4aCb2hqpNFku8qOyhScHQCdO6Tene793c50SwIQGuCFUYW0aFvYH7mfurA6kdIKftCzvAdbhI3eZb3Y48UujSaKEY+PABWMWDsieE3HdpWdtZlrsagE9QcriEGAGfszAAAFN0lEQVQPrS+0YtcHZ3ADGCcamVI6BW2S2I49xZOpcJ6iwm63K55MheGN7Bp86tQ+IX/40KGDAMyaNVuI3p49hQDk518gRA9g+/b+9Y7F+hM/+uhjQXqwapXUVVlUJGbV2SlTJgPiPJQxMf2zscS/3AqpwRUUzmeUAFcIaZQAVwhphPXJHF1xlL4I9/b52M1jMbbJn/546MJDmMPdJzdN3DaR8HbfBpEKFxbSa3Qfrp6+czqRnTLzRgbAn3hn0Z3Um+vd9j+f+zzZ4dnyBYEVn66gurvabf+7S99lQuwE2XoB8U+OAPKBDMAIdAP1SB12fiYSET6bMKomCn2X05Oo7fXvT8TUxbjqmf2/5LiGOIwm50On69P5rSmS2TGzSdOnOcox2hi/NQtSC8iIzHCU4/T+JVkV5p+cCFyD1JZoAMoAPVLQTyH4Ajz+ZDwxNf7fkH4SKxOJq/M/4+3ZpFanktCQIFRTJMsTlzM/dr5QzSszr2TpiKXC9IT4J3XASqTgLgLex7FYPyrAf8+x+ABvzmymK8lpWUs/lA6AzqYjpSeF0+HyVjltHNVIR4IztcWoI6Mc2yO7RlJrrPWYkFZn1ZFoSqQm0j0Hem16LW1xzpzw2WW+/fwDAfEnft74OcUdzm6/uzPudmyrT6qxJdpAZovq/ZPvs6fROUj382k/B8DWYcNWbUM7Xl4oDOSftNvtdO/pJnx6OCrdEL14GUB/a3MrzuAGqUdVwLCL8ADvSOugA2dAPv3e0wBkmjIBOBF+gjdHvem1XltKm0u5X89gNZDck0y3ppv3Rr/nMr6aV5vHJScuwWA1UBNR45g6UDK7hF5jLy1JLbTgzFfjV4AHwJ9Y2FboUr7v9/c5tjVlGux6O+YbzfRd7f2YxNbarS7le397LwCWIunfRjdPR9RT3s8APdc/edva2wDoq+mjr7oPbZKWUS+OYtCnPuKs7dZv/78MqT3ez+NeX5JHhAf4vC/m8eTWJwc8ntqTisHm/UJA43eNZ+2WtST3ejYvh9nCSOtOc9mX2pWK1iZ9tbQu5zGNXZpV98A/H+BA+AGPNb9sAuBPfDTrUZa8sARt4QC3xwrqY/I6wH4X/jtm/sdMR9mC63e3lFiwm7wf9Fu/ZD1Z12ZhqZd0unFdAMnaaqW3bIj5511nbUcDzUAlcAjwb70fB8IDvMZQw/1T73fbn9CbQG5HLrvid9Gn9r7msagtPD3labf9apua+Q3zKY0ppdHQCLW/dhz7ZMwn7EndQ2ZbJoVphdhV0o1r1UvVxKasTUHdBkcNPY/1eDyk/VyLbbQN2zibx+MDoZ2qJbHIvVHbV9aH5ZAFw+WGoZsUZ6GJ1DD1mHsU2kw2Gl5vIHZVLPpRevivQUSqkHpMwoELgA+QXjLbCd4AH4gmfRPb9duHPtFLbGobX6d8PeDx+oh66iPcu9zOdyzLxc7U1I3ToRsnrhdJbVST8mMvF1XqAz4BrkJ6l0kDTgPi+igU07HC90wxUo29EOmlczpS06UcODzI57xEWIDnbsoVJQXA1C2CfqPOYvbXYiZ2AQHxJ7425TXfPzwAmy7ZJFSv3z8plMpv/wsAylC9QkijBLhCSKMEuEJIowS4QkijBLhCSCPTsqbqAEoF/v1EoDGI9QKhORyvMRDfebTdbk8a6iS53YSldrt9lo8X5IZKpdoTzHqB0ByO1xiI7+wtShNFIaRRAlwhpJEb4H8S/PeDXS8QmsPxGgPxnb1C1kumgsL5htJEUQhplABXCGmUAFcIaZQAVwhplABXCGn+P94mjRGy6KGCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import clear_output\n",
    "from time import sleep\n",
    "mdp = FrozenLakeEnv(map_name='8x8',slip_chance=0.1)\n",
    "state_values = {s : 0 for s in mdp.get_all_states()}\n",
    "\n",
    "for i in range(30):\n",
    "    clear_output(True)\n",
    "    print(\"after iteration %i\"%i)\n",
    "    state_values = value_iteration(mdp, state_values, num_iter=1)\n",
    "    draw_policy(mdp, state_values)\n",
    "    sleep(0.5)\n",
    "# please ignore iter 0 at each step"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Massive tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 1.00000   |   V(start): 0.000 \n",
      "iter    1   |   diff: 0.90000   |   V(start): 0.000 \n",
      "iter    2   |   diff: 0.81000   |   V(start): 0.000 \n",
      "iter    3   |   diff: 0.72900   |   V(start): 0.000 \n",
      "iter    4   |   diff: 0.65610   |   V(start): 0.000 \n",
      "iter    5   |   diff: 0.59049   |   V(start): 0.590 \n",
      "iter    6   |   diff: 0.00000   |   V(start): 0.590 \n",
      "Terminated\n",
      "average reward:  1.0\n",
      "Well done!\n"
     ]
    }
   ],
   "source": [
    "mdp = FrozenLakeEnv(slip_chance=0)\n",
    "state_values = value_iteration(mdp)\n",
    "\n",
    "total_rewards = []\n",
    "for game_i in range(1000):\n",
    "    s = mdp.reset()\n",
    "    rewards = []\n",
    "    for t in range(100):\n",
    "        s, r, done, _ = mdp.step(get_optimal_action(mdp, state_values, s, gamma))\n",
    "        rewards.append(r)\n",
    "        if done: break\n",
    "    total_rewards.append(np.sum(rewards))\n",
    "    \n",
    "print(\"average reward: \", np.mean(total_rewards))\n",
    "assert(1.0 <= np.mean(total_rewards) <= 1.0)\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 0.90000   |   V(start): 0.000 \n",
      "iter    1   |   diff: 0.72900   |   V(start): 0.000 \n",
      "iter    2   |   diff: 0.62330   |   V(start): 0.000 \n",
      "iter    3   |   diff: 0.50487   |   V(start): 0.000 \n",
      "iter    4   |   diff: 0.40894   |   V(start): 0.000 \n",
      "iter    5   |   diff: 0.34868   |   V(start): 0.349 \n",
      "iter    6   |   diff: 0.06529   |   V(start): 0.410 \n",
      "iter    7   |   diff: 0.05832   |   V(start): 0.468 \n",
      "iter    8   |   diff: 0.01139   |   V(start): 0.480 \n",
      "iter    9   |   diff: 0.00764   |   V(start): 0.487 \n",
      "iter   10   |   diff: 0.00164   |   V(start): 0.489 \n",
      "iter   11   |   diff: 0.00094   |   V(start): 0.490 \n",
      "iter   12   |   diff: 0.00022   |   V(start): 0.490 \n",
      "iter   13   |   diff: 0.00011   |   V(start): 0.490 \n",
      "iter   14   |   diff: 0.00003   |   V(start): 0.490 \n",
      "iter   15   |   diff: 0.00001   |   V(start): 0.490 \n",
      "iter   16   |   diff: 0.00000   |   V(start): 0.490 \n",
      "Terminated\n",
      "average reward:  0.869\n",
      "Well done!\n"
     ]
    }
   ],
   "source": [
    "# Measure agent's average reward\n",
    "mdp = FrozenLakeEnv(slip_chance=0.1)\n",
    "state_values = value_iteration(mdp)\n",
    "\n",
    "total_rewards = []\n",
    "for game_i in range(1000):\n",
    "    s = mdp.reset()\n",
    "    rewards = []\n",
    "    for t in range(100):\n",
    "        s, r, done, _ = mdp.step(get_optimal_action(mdp, state_values, s, gamma))\n",
    "        rewards.append(r)\n",
    "        if done: break\n",
    "    total_rewards.append(np.sum(rewards))\n",
    "    \n",
    "print(\"average reward: \", np.mean(total_rewards))\n",
    "assert(0.8 <= np.mean(total_rewards) <= 0.95)\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 0.75000   |   V(start): 0.000 \n",
      "iter    1   |   diff: 0.50625   |   V(start): 0.000 \n",
      "iter    2   |   diff: 0.39867   |   V(start): 0.000 \n",
      "iter    3   |   diff: 0.26910   |   V(start): 0.000 \n",
      "iter    4   |   diff: 0.18164   |   V(start): 0.000 \n",
      "iter    5   |   diff: 0.14013   |   V(start): 0.140 \n",
      "iter    6   |   diff: 0.07028   |   V(start): 0.199 \n",
      "iter    7   |   diff: 0.06030   |   V(start): 0.260 \n",
      "iter    8   |   diff: 0.02594   |   V(start): 0.285 \n",
      "iter    9   |   diff: 0.01918   |   V(start): 0.305 \n",
      "iter   10   |   diff: 0.00858   |   V(start): 0.313 \n",
      "iter   11   |   diff: 0.00560   |   V(start): 0.319 \n",
      "iter   12   |   diff: 0.00260   |   V(start): 0.321 \n",
      "iter   13   |   diff: 0.00159   |   V(start): 0.323 \n",
      "iter   14   |   diff: 0.00076   |   V(start): 0.324 \n",
      "iter   15   |   diff: 0.00045   |   V(start): 0.324 \n",
      "iter   16   |   diff: 0.00022   |   V(start): 0.324 \n",
      "iter   17   |   diff: 0.00012   |   V(start): 0.325 \n",
      "iter   18   |   diff: 0.00006   |   V(start): 0.325 \n",
      "iter   19   |   diff: 0.00003   |   V(start): 0.325 \n",
      "iter   20   |   diff: 0.00002   |   V(start): 0.325 \n",
      "iter   21   |   diff: 0.00001   |   V(start): 0.325 \n",
      "Terminated\n",
      "average reward:  0.663\n",
      "Well done!\n"
     ]
    }
   ],
   "source": [
    "# Measure agent's average reward\n",
    "mdp = FrozenLakeEnv(slip_chance=0.25)\n",
    "state_values = value_iteration(mdp)\n",
    "\n",
    "total_rewards = []\n",
    "for game_i in range(1000):\n",
    "    s = mdp.reset()\n",
    "    rewards = []\n",
    "    for t in range(100):\n",
    "        s, r, done, _ = mdp.step(get_optimal_action(mdp, state_values, s, gamma))\n",
    "        rewards.append(r)\n",
    "        if done: break\n",
    "    total_rewards.append(np.sum(rewards))\n",
    "    \n",
    "print(\"average reward: \", np.mean(total_rewards))\n",
    "assert(0.6 <= np.mean(total_rewards) <= 0.7)\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter    0   |   diff: 0.80000   |   V(start): 0.000 \n",
      "iter    1   |   diff: 0.57600   |   V(start): 0.000 \n",
      "iter    2   |   diff: 0.41472   |   V(start): 0.000 \n",
      "iter    3   |   diff: 0.29860   |   V(start): 0.000 \n",
      "iter    4   |   diff: 0.24186   |   V(start): 0.000 \n",
      "iter    5   |   diff: 0.19349   |   V(start): 0.000 \n",
      "iter    6   |   diff: 0.15325   |   V(start): 0.000 \n",
      "iter    7   |   diff: 0.12288   |   V(start): 0.000 \n",
      "iter    8   |   diff: 0.09930   |   V(start): 0.000 \n",
      "iter    9   |   diff: 0.08037   |   V(start): 0.000 \n",
      "iter   10   |   diff: 0.06426   |   V(start): 0.000 \n",
      "iter   11   |   diff: 0.05129   |   V(start): 0.000 \n",
      "iter   12   |   diff: 0.04330   |   V(start): 0.000 \n",
      "iter   13   |   diff: 0.03802   |   V(start): 0.033 \n",
      "iter   14   |   diff: 0.03332   |   V(start): 0.058 \n",
      "iter   15   |   diff: 0.02910   |   V(start): 0.087 \n",
      "iter   16   |   diff: 0.01855   |   V(start): 0.106 \n",
      "iter   17   |   diff: 0.01403   |   V(start): 0.120 \n",
      "iter   18   |   diff: 0.00810   |   V(start): 0.128 \n",
      "iter   19   |   diff: 0.00555   |   V(start): 0.133 \n",
      "iter   20   |   diff: 0.00321   |   V(start): 0.137 \n",
      "iter   21   |   diff: 0.00247   |   V(start): 0.138 \n",
      "iter   22   |   diff: 0.00147   |   V(start): 0.139 \n",
      "iter   23   |   diff: 0.00104   |   V(start): 0.140 \n",
      "iter   24   |   diff: 0.00058   |   V(start): 0.140 \n",
      "iter   25   |   diff: 0.00036   |   V(start): 0.141 \n",
      "iter   26   |   diff: 0.00024   |   V(start): 0.141 \n",
      "iter   27   |   diff: 0.00018   |   V(start): 0.141 \n",
      "iter   28   |   diff: 0.00012   |   V(start): 0.141 \n",
      "iter   29   |   diff: 0.00007   |   V(start): 0.141 \n",
      "iter   30   |   diff: 0.00004   |   V(start): 0.141 \n",
      "iter   31   |   diff: 0.00003   |   V(start): 0.141 \n",
      "iter   32   |   diff: 0.00001   |   V(start): 0.141 \n",
      "iter   33   |   diff: 0.00001   |   V(start): 0.141 \n",
      "Terminated\n",
      "average reward:  0.77\n",
      "Well done!\n"
     ]
    }
   ],
   "source": [
    "# Measure agent's average reward\n",
    "mdp = FrozenLakeEnv(slip_chance=0.2, map_name='8x8')\n",
    "state_values = value_iteration(mdp)\n",
    "\n",
    "total_rewards = []\n",
    "for game_i in range(1000):\n",
    "    s = mdp.reset()\n",
    "    rewards = []\n",
    "    for t in range(100):\n",
    "        s, r, done, _ = mdp.step(get_optimal_action(mdp, state_values, s, gamma))\n",
    "        rewards.append(r)\n",
    "        if done: break\n",
    "    total_rewards.append(np.sum(rewards))\n",
    "    \n",
    "print(\"average reward: \", np.mean(total_rewards))\n",
    "assert(0.6 <= np.mean(total_rewards) <= 0.8)\n",
    "print(\"Well done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submit to coursera"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from submit import submit_assigment\n",
    "submit_assigment(\n",
    "    get_action_value, \n",
    "    get_new_state_value, \n",
    "    get_optimal_action, \n",
    "    value_iteration, \n",
    "    \"mehdi.mashayekhi.65@gmail.com\", \n",
    "    \"L0x7jqRPMbsnN2Gw\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}