The_Sliding_window.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dec56cbb-6414-4671-99c3-849dfde8b093",
   "metadata": {},
   "source": [
    "# The Sliding Window"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5e80a4d6-ccd4-437e-b833-12a9b2d71dd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from itertools import product\n",
    "from skimage import draw, transform\n",
    "\n",
    "def circle(size, val=None, r_shrink=0):\n",
    "    circle = np.zeros([size[0]+1, size[1]+1])\n",
    "    rr, cc = draw.circle_perimeter(\n",
    "        size[0]//2, size[1]//2,\n",
    "        radius=size[0]//2 - r_shrink,\n",
    "        shape=[size[0]+1, size[1]+1],\n",
    "    )\n",
    "    if val is None:\n",
    "        circle[rr, cc] = np.random.uniform(size=circle.shape)[rr, cc]\n",
    "    else:\n",
    "        circle[rr, cc] = val\n",
    "    circle = transform.resize(circle, size, order=0)\n",
    "    return circle\n",
    "\n",
    "def show_kernel(kernel, label=True, digits=None, text_size=28):\n",
    "    # Format kernel\n",
    "    kernel = np.array(kernel)\n",
    "    if digits is not None:\n",
    "        kernel = kernel.round(digits)\n",
    "\n",
    "    # Plot kernel\n",
    "    cmap = plt.get_cmap('Blues_r')\n",
    "    plt.imshow(kernel, cmap=cmap)\n",
    "    rows, cols = kernel.shape\n",
    "    thresh = (kernel.max()+kernel.min())/2\n",
    "    # Optionally, add value labels\n",
    "    if label:\n",
    "        for i, j in product(range(rows), range(cols)):\n",
    "            val = kernel[i, j]\n",
    "            color = cmap(0) if val > thresh else cmap(255)\n",
    "            plt.text(j, i, val, \n",
    "                     color=color, size=text_size,\n",
    "                     horizontalalignment='center', verticalalignment='center')\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "\n",
    "def show_extraction(image,\n",
    "                    kernel,\n",
    "                    conv_stride=1,\n",
    "                    conv_padding='valid',\n",
    "                    activation='relu',\n",
    "                    pool_size=2,\n",
    "                    pool_stride=2,\n",
    "                    pool_padding='same',\n",
    "                    figsize=(10, 10),\n",
    "                    subplot_shape=(2, 2),\n",
    "                    ops=['Input', 'Filter', 'Detect', 'Condense'],\n",
    "                    gamma=1.0):\n",
    "    # Create Layers\n",
    "    model = tf.keras.Sequential([\n",
    "                    tf.keras.layers.Conv2D(\n",
    "                        filters=1,\n",
    "                        kernel_size=kernel.shape,\n",
    "                        strides=conv_stride,\n",
    "                        padding=conv_padding,\n",
    "                        use_bias=False,\n",
    "                        input_shape=image.shape,\n",
    "                    ),\n",
    "                    tf.keras.layers.Activation(activation),\n",
    "                    tf.keras.layers.MaxPool2D(\n",
    "                        pool_size=pool_size,\n",
    "                        strides=pool_stride,\n",
    "                        padding=pool_padding,\n",
    "                    ),\n",
    "                   ])\n",
    "\n",
    "    layer_filter, layer_detect, layer_condense = model.layers\n",
    "    kernel = tf.reshape(kernel, [*kernel.shape, 1, 1])\n",
    "    layer_filter.set_weights([kernel])\n",
    "\n",
    "    # Format for TF\n",
    "    image = tf.expand_dims(image, axis=0)\n",
    "    image = tf.image.convert_image_dtype(image, dtype=tf.float32) \n",
    "    \n",
    "    # Extract Feature\n",
    "    image_filter = layer_filter(image)\n",
    "    image_detect = layer_detect(image_filter)\n",
    "    image_condense = layer_condense(image_detect)\n",
    "    \n",
    "    images = {}\n",
    "    if 'Input' in ops:\n",
    "        images.update({'Input': (image, 1.0)})\n",
    "    if 'Filter' in ops:\n",
    "        images.update({'Filter': (image_filter, 1.0)})\n",
    "    if 'Detect' in ops:\n",
    "        images.update({'Detect': (image_detect, gamma)})\n",
    "    if 'Condense' in ops:\n",
    "        images.update({'Condense': (image_condense, gamma)})\n",
    "    \n",
    "    # Plot\n",
    "    plt.figure(figsize=figsize)\n",
    "    for i, title in enumerate(ops):\n",
    "        image, gamma = images[title]\n",
    "        plt.subplot(*subplot_shape, i+1)\n",
    "        plt.imshow(tf.image.adjust_gamma(tf.squeeze(image), gamma))\n",
    "        plt.axis('off')\n",
    "        plt.title(title)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677c1815-2135-4be7-9ac3-4d5b834aec7e",
   "metadata": {},
   "source": [
    "Introduction\n",
    "\n",
    "In the previous two lessons, we learned about the three operations that carry out feature extraction from an image:\n",
    "\n",
    "   1. filter with a convolution layer\n",
    "   2. detect with ReLU activation\n",
    "   3. condense with a maximum pooling layer\n",
    "\n",
    "The convolution and pooling operations share a common feature: they are both performed over a sliding window. With convolution, this \"window\" is given by the dimensions of the kernel, the parameter kernel_size. With pooling, it is the pooling window, given by pool_size.\n",
    "A 2D sliding window. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "759a3682-ce95-4be1-901b-70127d284aeb",
   "metadata": {},
   "source": [
    "<img src=\"refimages/cnn015.gif\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd5ab9d6-b455-4376-ae6d-d5914220fde5",
   "metadata": {},
   "source": [
    "There are two additional parameters affecting both convolution and pooling layers -- these are the strides of the window and whether to use padding at the image edges. The strides parameter says how far the window should move at each step, and the padding parameter describes how we handle the pixels at the edges of the input."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbdf99a9-e42a-4c1a-a7ad-1741faef8e41",
   "metadata": {},
   "source": [
    "With these two parameters, defining the two layers becomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1a63e15f-6617-4ffa-b78f-f103a0809d7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-08-06 17:41:20.057179: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.\n",
      "2023-08-06 17:41:20.131910: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.\n",
      "2023-08-06 17:41:20.132959: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2023-08-06 17:41:21.581140: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n"
     ]
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "model = keras.Sequential([\n",
    "    layers.Conv2D(filters=64,\n",
    "                  kernel_size=3,\n",
    "                  strides=1,\n",
    "                  padding='same',\n",
    "                  activation='relu'),\n",
    "    layers.MaxPool2D(pool_size=2,\n",
    "                     strides=1,\n",
    "                     padding='same')\n",
    "    # More layers follow\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0a64427-4406-40ea-b21e-2c9c1123b99f",
   "metadata": {},
   "source": [
    "\n",
    "# Stride\n",
    "\n",
    "The distance the window moves at each step is called the stride. We need to specify the stride in both dimensions of the image: one for moving left to right and one for moving top to bottom. This animation shows strides=(2, 2), a movement of 2 pixels each step."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8e3d52e-b384-45b8-ba3d-d2f27702e3c4",
   "metadata": {},
   "source": [
    "<img src=\"refimages/cnn016.gif\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a88cc66-6b64-49f6-9099-2ccc477968b1",
   "metadata": {},
   "source": [
    "What effect does the stride have? Whenever the stride in either direction is greater than 1, the sliding window will skip over some of the pixels in the input at each step.\n",
    "\n",
    "Because we want high-quality features to use for classification, convolutional layers will most often have strides=(1, 1). Increasing the stride means that we miss out on potentially valuble information in our summary. Maximum pooling layers, however, will almost always have stride values greater than 1, like (2, 2) or (3, 3), but not larger than the window itself.\n",
    "\n",
    "Finally, note that when the value of the strides is the same number in both directions, you only need to set that number; for instance, instead of strides=(2, 2), you could use strides=2 for the parameter setting."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a71ecb55-f8a1-46c7-bebf-c0f748474935",
   "metadata": {},
   "source": [
    "# Padding\n",
    "\n",
    "When performing the sliding window computation, there is a question as to what to do at the boundaries of the input. Staying entirely inside the input image means the window will never sit squarely over these boundary pixels like it does for every other pixel in the input. Since we aren't treating all the pixels exactly the same, could there be a problem?\n",
    "\n",
    "What the convolution does with these boundary values is determined by its padding parameter. In TensorFlow, you have two choices: either padding='same' or padding='valid'. There are trade-offs with each.\n",
    "\n",
    "When we set padding='valid', the convolution window will stay entirely inside the input. The drawback is that the output shrinks (loses pixels), and shrinks more for larger kernels. This will limit the number of layers the network can contain, especially when inputs are small in size.\n",
    "\n",
    "The alternative is to use padding='same'. The trick here is to pad the input with 0's around its borders, using just enough 0's to make the size of the output the same as the size of the input. This can have the effect however of diluting the influence of pixels at the borders. The animation below shows a sliding window with 'same' padding."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bafff71-1952-48b4-a66d-c11ef2989b9d",
   "metadata": {},
   "source": [
    "<img src=\"refimages/cnn017.gif\">\n",
    "\n",
    "he VGG model we've been looking at uses same padding for all of its convolutional layers. Most modern convnets will use some combination of the two. (Another parameter to tune!)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c3365f7-77fd-4630-bfc3-9cd5c319e10a",
   "metadata": {},
   "source": [
    "# Example - Exploring Sliding Windows¶\n",
    "\n",
    "To better understand the effect of the sliding window parameters, it can help to observe a feature extraction on a low-resolution image so that we can see the individual pixels. Let's just look at a simple circle.\n",
    "\n",
    "This next hidden cell will create an image and kernel for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "750ce25a-0fbf-4a19-8b6f-f88f0c3aa9b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdUAAAHWCAYAAAAhLRNZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAWnElEQVR4nO3de5SddXno8WeuSSZzSULuQyDInYDKvZUilCNYa3tE4CB4FE6tXSqgBz1dtrVLvK1q23UqKpfqoue0YCtYFehpbT2gHFRQuSsISIgQSDKEJCSZSy4zmdn7/MFiJxMyZHby7L1nz3w+f+V9551Zz5r1y/vdl3e/01AsFosBAOy3xloPAACThagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJCkeTwHFQqF6OnpiY6OjmhoaKj0TAAwoRSLxejv74/FixdHY+PYz0fHFdWenp5YsmRJ2nAAUI9WrVoVBx544JhfH1dUOzo6IiJiyftvjMbWtpzJICLW/viOWo/AJLTw9HNqPQKTTGFoa6z6u0tLPRzLuKL6yku+ja1t0ThNVMnT0NRa6xGYhJynqJS9vQXqQiUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkzbUeACjPrM62OOawxXHoknkxu3NmNDY2xKa+rbHqhY3x4GPPRt+W7bUeEaYsUa2AttamOLq7M16/ZFYct2RWHLukKw6c01b6+pqNW+Psv7y7dgNSVxoaGuI3jz80fu/MN8QZJx8Zxxy2eMxjC4VC/PCB5XH9N+6K7//kiSpOSb1yvsolqokuPf2QeOdJB8ahC9qjqbGh1uMwSTzw7U/GoQfNH9exjY2N8dunHhW/fepR8Z3/+2Bc+fmbY2DrYIUnpB45X1WGqCY66ZDZccSijlqPwSQzd3b7q/ateG5dPPT4ylj3Ul9sH9oR3QtmxxknHxndC2aXjjn/rSfFgrldccFHrovBoeFqjkwdcL6qDFGtsC2Dw/HEmt5Y1t0VbdP8utl3z63ZEF//l5/Gzd+9L3rWbX7V1xsbG+LSc0+Lv/joeTFjemtERPzWiYfHn3/o9+OqL99W5WmpR85X+89vLdHgcCEefX5zPLZqc/xydW88vro3fr1uIIrFiDv/9EyLlH2y6oWN8Ymrb41bvntfFArFMY8rFIrx97feE6vWboxbvvjBaGp6+eL+D7zrjPjbb9wVL6zvrdbI1AHnq8rwW0v0x9/4ea1HYBI685K/jpGRwriP//5Pnojv3PFgXPi2UyIiorWlOd725uPif3/nnkqNSB1yvqoMn1OFCa6coL7i1jseHrV9wrKlSdMAr0VUYRJ6ZtX6Udvz57ggBapBVGESap85bdT2juGRGk0CU4uowiS07LDuUdt7uloYyCeqMAld+LaTR23/6IHlNZoEphZRhUnmTSccFqefdERpu7d/a9z1M7cshGoQVZhE2qa3xpc+cfGofdf+012xZdtQjSaCqUVUYRL5mz99Vxx+8ILS9lPPro1rvv79Gk4EU4uowiRx2bvPiovefmppe/vgjvjAVTe67y9UkajCJHDuW46Pz37k3FH7PvqFW+IXv1pVm4FgihJVqHNnnHJkfPUzl5Tu9RsR8elrbo9bvntfDaeCqcm9f3exeNb0+N6fnDmuY3/nr+6Ons3bKzsQk8aShbPj4ds+Pa5jT3jnp2PV2k3jOvbEZQfH1//6j2Jaa0tp31duujO+fJP3USc756uJSVR31dAQzU3jfPLe4I/6UoaGhmhubhr3seNx9KGL4p+/dFl0zJxe2nfj7ffGp675l32ZkHrjfDUhefkX6tDS7rlx67VXxJxZM0v7br3zofjo52+p4VSAZ6q76Nm0LY75+L/XegwmoVUvbIzZJ1+R8rMWzeuK26+7IhbO7Srtu+OeX8YHPnljFItj/71VJhfnq4nJM1WoI3O6ZsZt1304Du6eW9r34weXxyV/8ncxvA9/Ig7IJapQJzpmTo9vf+XyOPKQhaV9Dzz2bFz8sa/6LCpMEKIKdWD6tJa4+YsfiOOPOai077Hlq+O//Pfr3YIQJhBRhQmuqakx/uELfxinnXB4ad/ylWvjvCuujd7+bTWcDNidqMIEd/2n3hNvPf3Y0vbKNRvi3MuuiQ2bBmo4FbAnrv5N9Fofxt7182Tdc9ri0S/8zh6Pe98N98eDz2ysxHjUoSULZ8eFbztlt31z4tH/89myfs7zL2yME8/7TOZo1Dnnq8oQ1UxlfBh7rON8RJtR9vCh/abxfuB/F+O+SQBTh/NVRfifBgBJPFNN5MPYZMu8aQTsyvmqMjxTBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgSXM5B5/zpoOjta29UrMwFZ1+ea0nANiroa0DccP1ez/OM1UASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASUQVAJKIKgAkEVUASCKqAJBEVAEgiagCQBJRBYAkogoASZprPcBkMaOlMQ6Z0xZd05tjenNT9A0Ox0tbh+K5jduiWOvhqFvWFZVgXVWOqO6nuTNb4nePnh/HzG+P5qZXP/Hv3bYjfvb85rjr6Q0xYrUyTtYVlWBdVZ6o7ofjuzvjgtcvimnNY7+K3jWjJd565LxYtqA9bnxwdWzaNlzFCalH1hWVYF1Vh/dU99GR82bGRW9cPGqBrh8YjJ+u3BQ/eHpD/HxNXwyNFEpfO3DWjHjfKUti2h4eHcIrrCsqwbqqHs9U90HHtKZ4z4nd0dTYEBERhWIx/u2JdfHjZzaOej9iZmtTXHJidxw6d2ZERCzqnB7nv35hfOORnhpMzURnXVEJ1lV1eRiyD/7T4XNjRktTafuOpzbEj3ZboBERW4ZG4ob7VsWL/YOlfW/s7oxFndOqNCn1xLqiEqyr6hLVMrW3NsWpB80qbW/YMhR3rdgw5vHDhWLc9tja0nZjQ0OcffjcSo5IHbKuqATrqvpEtUzLFnZEyy7vM/zsuU1R2MtVcite2hrrBnY++jtqQXu0NDVUakTqkHVFJVhX1SeqZVq2oH3U9qMv9I/r+37Rs/O41qbGOGLezNS5qG/WFZVgXVWfqJbpkAPaSv/u2z4cG7fuGNf3Pbdp66jt181pG+NIpiLrikqwrqpPVMvQMa151Bv+PX3bx/29a3pHH7ugw5v/vMy6ohKsq9oQ1TLMb28dtb1p2/ge9UVE9A+OxPAunwPb/WcxdVlXVIJ1VRuiWoau6aM/1ttb5t1GerfvPL5rekvKTNQ/64pKsK5qQ1TLsPvtvQaHR8r6/sHhnY/8mhobSh/GZmqzrqgE66o2RLUMrbvdsmt4b9em72b3490CjAjrisqwrmrDb6kMLcmL1Ge/iLCuqAzrqjZEtQzDhcKo7XJfDmne7fgdZS5yJifrikqwrmpDVMuw63sMEREt+7lIh3b7eUxN1hWVYF3VhqiWYXBk9KJqfY2/S7gnu144MFIolv1yDJOTdUUlWFe1Iapl6NvtkvRZM8q7zHzXS9x7t4//M2NMbtYVlWBd1YaolmHdwNCo7dllLNKOaU3RvMuFA7v/LKYu64pKsK5qQ1TL0Dc4HNt27Pys1+LO6eP+3u6u0cfu+lcgmNqsKyrBuqoNUS3Tsxt33mi6c3pzzGkb36O/pbNH35D6mZe2pc5FfbOuqATrqvpEtUyPrx0Ytf2GRZ3j+r7XL+4o/XvHSCGWrx94jaOZaqwrKsG6qj5RLdPja/tH3Wj61INnxd6uVD/sgLaY377zrzw8uW4ghkZcScdO1hWVYF1Vn6iWaWBoJO57fnNpe+7M1jjrsLljHt/c2BDnHrewtF0oFuP7yzdUckTqkHVFJVhX1Seq++AHT78U23e5AOCcI+fG6a+bE7s/AJzZ2hTvP3VJLNzlbxH+oqcvevq86c+rWVdUgnVVXQ3FYnGvz+v7+vqiq6sr/ugf74/WtvZqzDXhHT2/Pf7glAOjsWHn0lw/MBhPb9gaW3eMxNy21jhmYfuom1qv7R+Ma+5Z+ao7ncArrCsqwbraf0NbB+KG95wSvb290dk59nvTzWN+hdf05LqB+ObPX4jzj1tYulPJvPZpMW+X9yJ2taZ3e/zDA6stUF6TdUUlWFfVI6r74aHVvfH8pm3xu0fPj6MXtL/qXpkRL9+J5L7nNscPnt4Q3utnPKwrKsG6qg5R3U/rtwzFjQ+ujraWplg6Z0bMmt4S01oao3/7cGzcOhTPbtwW1iblsq6oBOuq8kQ1ydYdI/HEiz7LRS7rikqwrirH1b8AkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACRpLufgC49ZEDM7Ois1C1PQ8Utn1XoEJqFHVm6u9QhMMlv6++KGcRznmSoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAElEFQCSiCoAJBFVAEgiqgCQRFQBIImoAkASUQWAJKIKAEmaaz0AsGeberfEEyt64ter1semvi1RKBRjdmdbLFk0J0467pDoap9R6xGB3Yjqfti2bTCWP9sTv1qxJn61YnU8tWJ1rF2/ufT1BfNmxc3X/3HtBqSuFAqF+Okjv45/u/sX8cMHnoonf/3CmMc2NDTEGScfEZe9+6w4+7RlVZySeuV8VR2iug++9a/3xvf+38Px3Op1USgWaz0Ok8TJF3wunlm1flzHFovFuPv+p+Lu+5+K8845Mb70iYujY+b0Ck9IPXK+qi5R3QePPrkynl31Yq3HYJLZsGngVfsOPWhenLhsacw/oDOmt7bEmhc3xQ8feCp61m0uHXPrHQ/Fixt649tfuTymT2up4sTUA+er6hLVJDOmt8bhhyyO5c/0xPbBoVqPQx07aPEB8d53/GZc/PZTo3vB7Fd9fWSkEDfefm/8+dW3xvbBHRERce/DK+Iv/vZf43NXnlftcalDzleVI6r7oLWlOY467MA48tDuOOqw7jji0O44uHteNDY2xsWX/c/Yvt4ipXxLFs2JD110Zlz09lOjqWnsC/ObmhrjfeefHksWzomLPvbVKBRefknva9/8YXzo3WfF4vmzqjQx9cD5qrpEdR988qPvqvUITEJ33/TxaG5uGvfxZ5+2LM4/56T41vceiIiIHcMj8R8/eiz+8ILTKzUidcj5qrp8ThUmiHKC+orzzjlh1PbDj69MmgbYF6IKdex1S+aN2l63sb9GkwARogp1bWDL4Kjtln14tgvkEVWoY4+vWDNq20VKUFuiCnXsn//jgVHbbz75iBpNAkSIKtStex96Ou556OnSdmf7jDjrN46p4USAqEId2rJtMK78/M2j9l3xX8+K9rZpNZoIiBBVqEv/4y+/GSueX1faPmLpgvjwe99Sw4mACFGFunPdP/0gvvnv95e2p7U2x9c+e6n7/sIEIKpQR2678+G46iu3j9p39Z9dFG88+qDaDASMIqpQJ+6+71fxwU/dVLrXb0TEp654R1z8e79Rw6mAXYkq1IEHf7ky3vvxG2Jox3Bp34ff+5a48tKzazgVsDtRhQnuiRU9ceGV18fA1p13T7rkHW+Kz37k3NoNBeyRqMIE9uzq9XHeFdfGpt6tpX3vfMsJcfUnLqrhVMBYRBUmqJ51m+Pcy6+NF1/qK+07+7Rl8bXPXRqNjf7rwkTkfyZMQC9tHoh3Xn5NPN/zUmnfb514eNz0V+9303yYwEQVJpi+gW1xwUeui+UrXyztO+nYpXHzFz/os6gwwYkqTCDbtg/FxR/7Wvz8yVWlfcce3h3f+vJlbkEIdUBUYYIYHh6J//Zn/yt+8siK0r7DD14Qt157RczqbKvhZMB4iSpMAMViMS77zD/GHfc8Xtp38OID4vbrPxzz5nTUcDKgHA3FYrG4t4P6+vqiq6sr7nz4uZjZ0VmNuSa0tes3xXuuuHqPXysUCqO2x7pK82+u+oN4w7JD0merN8cvnVXrESaE51/YGG/4z1eN2tfY2BANDQ1l/ZyDFs2Jh2/7dOJk9emRlZtrPcKE4XyVY0t/X5x9wsHR29sbnZ1jd7C5ijNNHsVXL8axjHVcMfb6WIapZA+PbV++HWF562R4ZHzrkinE+aqqvPwLAEm8/EtNefmXSvDyL9nG+/KvZ6oAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASCJqAJAElEFgCSiCgBJRBUAkjSP56BisRgREVsG+is6DFNPX5/HdeTb0t9X6xGYZF7p3ys9HMu4otrf//IPO/fNx+7nWABQv/r7+6Orq2vMrzcU95bdiCgUCtHT0xMdHR3R0NCQOiAATHTFYjH6+/tj8eLF0dg49its44oqALB33tACgCSiCgBJRBUAkogqACQRVQBIIqoAkERUASDJ/wekYCunGg9ngwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.rc('figure', autolayout=True)\n",
    "plt.rc('axes', labelweight='bold', labelsize='large',\n",
    "       titleweight='bold', titlesize=18, titlepad=10)\n",
    "plt.rc('image', cmap='magma')\n",
    "\n",
    "image = circle([64, 64], val=1.0, r_shrink=3)\n",
    "image = tf.reshape(image, [*image.shape, 1])\n",
    "# Bottom sobel\n",
    "kernel = tf.constant(\n",
    "    [[-1, -2, -1],\n",
    "     [0, 0, 0],\n",
    "     [1, 2, 1]],\n",
    ")\n",
    "\n",
    "show_kernel(kernel)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "089cce4b-0405-4af6-9070-8fc85f96a6cd",
   "metadata": {},
   "source": [
    "The VGG architecture is fairly simple. It uses convolution with strides of 1 and maximum pooling with $2 \\times 2$ windows and strides of 2. We've included a function in the visiontools utility script that will show us all the steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e3cbb7f2-6d5d-4fa9-8ade-cd9e34cf2e04",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([3, 3])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kernel.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "cf180710-0e62-4d1c-b259-b543aa00399a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kernel.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8632cbf8-a015-4cea-9065-6272da283f85",
   "metadata": {},
   "source": [
    "The VGG architecture is fairly simple. It uses convolution with strides of 1 and maximum pooling with $2 \\times 2$ windows and strides of 2. We've included a function in the visiontools utility script that will show us all the steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6c8034e2-a0be-4237-b0d6-d1829ead73fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABW0AAAGACAYAAADWAkD8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA22klEQVR4nO3deZRV1ZU44F0yiENEJhERwThEjYkVIILtiqBglkNEImoGNY5U7E7UJNiKJr826bZt1JiYTrs6QYza0eDUSrRbOooDEUWwUDQ4IabAiAEh4JQOk7zfH1nSvLrnUe8VVbxbVd+3FmtZ+51z76lX95667jrv7JpCoVAIAAAAAAByYbtqDwAAAAAAgP8jaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5ImkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCOStgAAAAAAOSJpCwDtyKBBg6KmpmbTv+9///tFrz/++ONFr9fU1MTixYurMlYAAGhJixcvzjzrPv7449UeFjRL52oPgLZl5MiRMXPmzKJYoVCo0mi2nWnTpsX8+fM3fT1o0KA466yzqjYeoP1KzbNb0r1793jnnXdadAyLFy+OW265pSj2rW99K3bdddcWPQ9Aa0jNozU1NdG1a9fYYYcdYtddd40999wzDjjggDjyyCNj3Lhxsf3227fqmK6//vqiuXrkyJExcuTIVj3n1mj8B7+xY8dGbW1tVcYCtJxXX301pk6dGrNmzYqFCxfGqlWrYt26ddG9e/fYb7/9Yvjw4TFmzJgYMWJE1NTUVHu40OFJ2kIZpk2bFrfeeuumr0eMGCFpC7Rbixcvjh/84AdFsbPOOkvSFmizCoVCrF27NtauXRvvvPNOLF68OGbNmhVTpkyJiy66KC677LL49re/3WpJiuuvvz6WLFlSFMtz0rbx74BBgwZJ2kIbtmzZsvi7v/u7mDZtWnLR1cqVK2PlypUxe/bs+PGPfxzjxo2Le+65pwojBTYnaQsA7cisWbNiw4YNm76WaAXYspUrV8aECRPi8ccfj7vvvrvVV90CbEv19fVx/PHHx9tvv112n5UrV7biiIBySdoCQM41NDSUfG277Yq3p99zzz1bezgAbU5DQ0MUCoV477334pVXXon77rsv7rnnnvjwww83tXnggQfi7LPPjl/96ldVHClAy1myZEkyYbvHHnvEhRdeGEcccUT07t073nvvvfjd734XDz74YEybNq06gwUyFCKjRZ111llFG35/9LGvhx9+OE444YTo06dPbL/99rHPPvvEhAkTSu7DOHLkyKLjfLQVwdSpU2PkyJHRs2fP2GmnnaK2tjZ+9KMfxfr165PHabwBeeM9Grc05s3HsfnWCBERM2fOtLk5sM0MGjSo5L+99tor03ZLhci25JZbbomampo48sgjM6/tvffeyXl5c2vWrIkpU6bEmDFjYsCAAbHDDjvEzjvvHPvvv3+ce+65MXfu3JLnLjUX33XXXTFq1Kjo1atXxd8PwEcGDRoUe++9dxxyyCHxpS99Ke6444545JFHYpdddilqN3Xq1Pjv//7v5DGeeuqpqKuri09+8pPRvXv36Nq1a/Tr1y+OPfbYuOmmm5LPox/NaY23RvjBD36QeZZsqXM29sILL8S3vvWt+OxnPxu9e/eOrl27Rp8+feJTn/pU1NXVxYMPPhgRxcV7Gjv77LOLxjpo0KAmzwtU38UXX5xJ2B5xxBHx0ksvxaWXXhqHHXZY7LfffjFkyJA466yz4q677orXX389vvjFL2aOtX79+rjtttti3LhxMXDgwNhxxx1jhx12iAEDBsSYMWPipptuirVr1ybHUaoQ7h//+Me46KKLYp999olu3bpF7969Y8yYMTFnzpwtfl8vvfRSnHbaadGvX7/o1q1bfPzjH48LL7wwli9fXtH78+GHH8Zdd90Vp556auy9996x0047xQ477BB77713fPWrX42HH364ZN/vf//7yXnxmWeeiS9/+cvRr1+/2H777WPAgAFRV1cXS5cuLXmsRx55JM4444w44IAD4mMf+1h06dIlevfuHQceeGB84QtfiB/84Afxm9/8pmT/t99+O6688soYMWJE7LbbbtG1a9fo0aNHDB48OC699NJ48803K3pfyJECVGDEiBGFiCj6t7kzzzyz6LXPfe5zhQsuuCDT56N/n/zkJwvvv/9+k+c57bTTCl/60pdKHufwww8vvPfee5njNG538803Z9o0HvOIESO2+P2W+vfYY49t7dsL0OQ825SBAwcW9b3iiiuKXn/ssccyx29oaCgUCoXCzTffXPacd+aZZxYdd/bs2Zlzp/6df/75hXXr1mXGnZqLx48fn+nf+PsBaKySefS2225LPldu7t133y2ccsopTc5vBx10UOHVV18t6lvunNp4jFtzzo988MEHhXPOOafJYxxyyCGFQqFQaGhoKHusAwcOrPwHA2xTr732Wube7dWrV2HFihUVH2vBggWFAw88sMm5Ye+99y4888wzmf6p588pU6YUdtlll+RxunbtWvjNb36THMt9991X6Nq1a7Jf7969C/fdd19Z/6/+yiuvFD796U83+T198YtfTOYarrjiisy8ePXVVxe222675HH22GOPwptvvpk5zoQJE8qad7fffvvk+3HTTTcVdtxxxyb7TpkypYmfMnlkpS2tatasWfHTn/605OsvvvhiXH311U0e56677oo777yz5OtPPvlkjB8/vlljBGDrPPfcczF69OjMSrKUn/3sZ3H++ec32e7JJ5+MG2+8sSWGB1DSV77ylcwnFmbPnh2rV6+OiL+uLBszZkzcfffdTR7rpZdeiqOOOir++Mc/btWYWuKcGzZsiJNOOil+8YtfbNVYgLYr9amB8ePHR+/evSs6TkNDQxx55JHx8ssvl9V29OjRZbUdP358vPfee8nX1q1bF3V1dUVb2EREvPLKK/GVr3wl1q1bl+y3cuXKOO2005o89xtvvBEjR46MF154ocm29913X5x88smZsaSOeemll8bGjRuTr7/11lsxceLEolh9fX1cd911TY6hlMmTJ8e5554b//u//7vFdmvXro3zzjsvfvnLXzb7XFSHpC2tqlAoxE477RQ33HBDvPTSS3H77bdnPoZWzr5h69evj9133z1uv/32eOGFF+JXv/pV9OvXr6jNnXfeucWP3jbHHXfcEQ0NDTFu3Lii+LBhw6KhoaHo3/Dhw1v03AAfafxxss3/XX/99S12npNPPjkaGhpi6tSpmdeeeOKJojnvhz/8YUT8dZ4/99xz489//vOmtp/4xCfi9ttvjwULFkR9fX1cdtllRR+3/cUvfhGPPvroFsfyUTG1Cy+8MObMmRMvvvhi3H333XHooYe2xLcKEBF/3Rf8iCOOKIpt3Lgxnn/++YiIuOGGG2LmzJmbXuvSpUtcccUVMXfu3E3PtpsnfZcuXRqXXnrppq8/mjP79+9fdI6LLroo8yz5ka09Z8Rf/0D20EMPFcV23333+Nd//deYP39+vPrqq/Hggw9GXV1d7LDDDhHx1z3RG4/lI9dee23RWGfNmlXiHQXy4rnnnsvERo0aVfFxLrjgglixYkVRrK6uLp544ol4+umn41vf+lbRa++++2787d/+bZPHLRQK8dWvfjWeeeaZePLJJ2PEiBFFry9ZsiSeeuqpoth3v/vdWLNmTVHs9NNPj1mzZsVTTz1VVgIz4q/Pl8uWLdv0db9+/WLy5Mnx/PPPx/z58+Paa68tKkr50EMPZbZMTH0/nTp1iiuvvDIWLFgQ999/f+yxxx5Fbe65556ibW1++9vfFr0+ePDg+M1vfhOvvvpqvPjii/HQQw/FtddeG8cdd1x069atqO1bb72Vee+POeaYmD59erzyyivx+OOPx9ixY4tev+CCCzb9UZI2oroLfWlrKt0eISIKP/vZz4raXHvttZk2H3zwQZPnmTdvXlGbefPmZdp885vfLGrT+PVKt0eopA1AS6hkW5aIKPz4xz8u6r812yNU0uYjTzzxRFG7Ll26JD/6dfrppxe1GzduXNHrqd8fF198caVvH0DF28xceumlmfZ33XVXoVAoFPbZZ5+i+LXXXpvpP2PGjKI2nTp1KqxevbqoTVNz8+Za4pz77bdf0es9evQoOY83HmuhUN4zNJBvxx13XOZefvnllys6xpIlSzLHOP300zPtvv71r2favfTSS5teTz1bHnbYYYWNGzduavP2229n2vzbv/3bptffe++9QqdOnYpeP+qoozJjOeGEEzLH2Xx7hD/84Q+Z1+fOnZs5zve+972iNkOGDCl6vfH2CBFRmDhxYlGbu+++O9NmwYIFm16/+uqri16bNGlSyZ9F4y0a/umf/qmo76c+9anChx9+WNRmw4YNmd8/P/3pT0ueg/yx0pZWtfPOO2eK1RxwwAGZdk39tae2tjYGDx5cFBs8eHDU1tYWxZ5++ulmjROA5tl8NVjEXz8Zseeee2ZWBN92221F7RqvLGisS5cumY+QAbSGQqGQidXU1MTSpUvj9ddfL4r//d//fWZ+Gz16dFGbDz/8MLM6rFwtcc6lS5fGa6+9VvR6XV1dyeJhu+66a7PGCrR/qee1urq6TOzrX/96WX03941vfKPok1h9+vSJXr16FbXZPE8wb968zBYF55xzTua455577hbP2/jZNSLi0EMPzcyzV155ZVGb5557Lj744IMtHvub3/xm0ddN5T6GDBlS9No//MM/xIknnhj/7//9v/jlL38Z9fX1m7aC+NjHPrbF7+N3v/tddOrUqeh76Ny5c2b7sqZ+LuRL52oPgPZt0KBBRR8riIhNH8Ha3Ecfgy1l7733LhmfP3/+pq83/4gDQHuR+qjqR3r27LkNR5K1pUq4W7Jy5crYsGFDdO6cfhQZMGBA5sEdoDWk5rE+ffo0e36LiGbva9sS53zrrbcyrzVODADtX58+fTKxN998M5lILCU1n+yzzz6Z2Mc//vGy+m4uNY7GuYLN8wTLly/PtE/lCUrlDj7S3Hl248aNsXz58th5552Tr++8886ZrXCayn2MGjUqTjzxxPj1r38dEX/dy/f++++P+++/f1Obbt26xTHHHBPf/e53Y+jQoVv9fWztvutsW1ba0qpS/8PdqVOnKozkr1Kbh69cubIKIwEo36BBg0r+a7xPeFtRKBQye5JtrvEeYACtYePGjZlVR506dYpPf/rTW3Xcv/zlL1vVv62cE8ivz3zmM5nYI488UoWRpFWaK0h9KmJb29I829zcx7333hs333xzHHnkkZkFbxERa9asiWnTpsXhhx8es2fPrmzACX5XtC1W2tImlFpl1ji+++67F33dqVOnokRtalPyhQsXtsAIATqmxsnV7t27x7PPPhvbbdf034V32mmnkq9V8w98QMfxq1/9Kv7whz8UxYYPHx49evRI/vHo5z//eXz+859v8rjN/aRAS5wzdYx58+bFKaec0qwxAW3T8ccfnylUNWXKlLj44ovLnqNS88nrr7+eif/+97/PtGtcOHxr9e3bNxNraGiIv/mbv8nEtqTx2GtqauLZZ58ta6uYxitpW8J2220XZ511Vpx11lmxYcOGaGhoiNdffz0WLFgQkydP3rTdzbp16+Kaa66J++67b9P38fLLL286ztFHHx2TJ09u8nypxDD5ZaUtbcL8+fPj2WefLYo9++yzRVsjREQMGzas6OvGE+/mk1pExMMPP5zZ8yula9euRV/76xTQnjWe8yJKz3sjR44s+vrdd9+NOXPmbHF18LJly2L16tVF+5gBbGszZ86Mb3zjG5n4ZZddFhERe+65Z+Yjv9OmTYu99tqr5Py24447xrx58zJ7D5b7LNkS5+zfv3/st99+Rce48cYb44033kie85133snEunTpUtZ4gfzad9994+STTy6KrVy5Mk455ZR4//33S/b7wx/+ED/5yU8iIuKII47IvP7zn/+8rFiq79YYMmRI5o/6v/jFLzLtbrrppi0eZ8SIEUVfFwqFmD59+hafXf/85z/HG2+8kZkbt9Y777xTNL927tw59ttvvzjmmGPi4osvjmuuuaao/eb5jMbP4LNnz47169eX/B4GDBgQ8+bNk7RtYyRtaTO+8IUvxNSpU2PBggVxxx13xBe+8IVMmzPOOKPo68YfbZsyZUrccMMN8fLLL8fUqVMz7UtpvB/Q/Pnz4z//8z9j0aJFsXjx4njzzTcr/G4A8iu1B9r1118fL774YixevDgWL168aWuDww8/PA455JCituecc05cfPHFMXPmzHjttdfihRdeiGnTpsVll10Wn/zkJ+Owww6L559/fpt8LwAREYsXL46GhoZ4/vnn46677oovf/nLMWrUqHjvvfeK2n3lK1+J448/ftPXjZO606dPj6OPPjruvffeWLBgQbzyyivx+OOPx09+8pM47rjjYsCAAfHTn/40c/7G8+p9990XTzzxRDQ0NMTixYuLtutqiXNecMEFRV+vWrUqhg8fHjfccEO88MILsXDhwnj44YfjoosuKvp+S433l7/8ZTzzzDObfgekEr1A/vzwhz+M3XbbrSj22GOPxUEHHRTXXnttzJkzJxYtWhTz5s2LW2+9NU499dTYZ599Nq3m3GuvvTJzxO233x7nn39+PPnkkzF37tz4zne+k0najhgxIg466KAW/V522WWXGDNmTFHs0UcfjTPOOCOeeuqpmD17dowfPz4eeOCBLR5nwIABccIJJxTFvve978X48eNjxowZ8eqrr8aCBQviwQcfjH/8x3+Mz372s3HwwQfHo48+2qLfT0TErFmzol+/fnHaaafFjTfeGE888US88sorsXDhwpg+fXr8y7/8S1H7zffTPfvss4v2zP3ggw9i5MiR8eMf/zjmzJkTr732WtTX18dtt90WX//612PPPfeMk08+OfN7j5wrQAVGjBhRiIiif5s788wzi14bMWJE5hiPPfZY5hgNDQ1bPM+OO+6Y6dP43ymnnJI514033thkv5qamibH/MADD2zxGAMHDtyKdxXg/zQ1zzZl4MCBRX2vuOKKotfLmYM//PDDQp8+fbY47z322GOb2tfX1xd22mmnJufbzf/dfPPNRecs5/cHQDlS82g5/8aMGVP4y1/+UnSstWvXFo444oiKjpOavyZMmLDFPmeeeWaLnnPdunWFUaNGldX3kEMOyYx33LhxW+zT+HcLkF/PPPNMk891W5pTXn/99ULv3r3L7tu9e/fCiy++WDSGcp4/C4Wmn2NffPHFwvbbb7/F83fu3HmLz62FQqHQ0NBQ6Nu3b0XvSeOxXHHFFUWvp3ICDQ0NWxxLU3mGpsbw7//+7xX/rku97+SXlba0CSeffHKMHz++5OvDhw+PG2+8MRM/++yz48gjjyzZ78QTT4xTTz21yfMfc8wxMXjw4PIGC9DGbbfddps+HlyOIUOGxIwZM5qs1vuR7bffPrmaF6Aa+vTpEz/5yU/i17/+dXTr1q3ota5du8YDDzwQX/rSl8o+3oABAzKxb37zm2UXjmyJc3bp0iWmTZsWX/va18o+xuYuueSSFv8YMFAdQ4cOjeeffz7Gjh1b9tZUvXv33vTfH//4x+Oxxx6LAw44oMl+gwYNihkzZrT4KtuPHHTQQXH77beXnJ923nnnZF6gsUGDBsXMmTOjtra2rPN26tSpxffordRRRx0Vl156aVHs/PPPj5tuummLdSI217t376LVueSfQmS0CTU1NTF58uT4/Oc/HzfccEPMnz8/1q5dG/vtt1+cccYZceGFFyb3YOzUqVM8+OCDce2118bUqVPj97//fXTr1i1qa2tj/Pjxcdppp8VZZ53V5Pk7d+4cjzzySFx55ZXxX//1X7F48eJYu3ZtK3ynAPnw7W9/O3bbbbf4+c9/Hi+88EK89957W6zaO3z48Hj55ZfjzjvvjF//+tcxb968WLFiRaxbty522WWX2HvvvaO2tjaOOuqoOP7446N79+7b8LsBOrqampro3Llz7LjjjtGjR4/o379/HHjggTFq1Kg46aSTks+RH9lll13ijjvuiAkTJsQtt9wSTz75ZCxZsiTef//96NatW/Tr1y8OOuig+NznPhfHH398HHjggZljDBo0KJ5++um48sorY+bMmfH222/H+vXrW/WcO++8c9x6663x7W9/O26++eaYNWtWNDQ0xPvvvx/du3ePfv36xWGHHRZjx47N9D300EPjt7/9bVx99dUxe/bsWLlyZVFxX6Bt6devX9x3333xyiuvxNSpU2PWrFnx6quvxurVq2PdunWx6667xr777hvDhw+PMWPGZPZLPfjgg+OFF16IO++8M+69996or6+PFStWRKFQiF69esXgwYPjxBNPjNNPPz3zx6+WNm7cuHjuuefin//5n+PRRx+N1atXR9++fePzn/98fO973yv7OJ/4xCeivr4+7r///rjnnnti7ty5sWzZsvjLX/4SH/vYx2KvvfaKT3/60zFy5Mg44YQTMttMtITRo0fHjBkzYtasWTF79ux44403YsWKFbF69erYfvvtY4899ojPfOYzceqpp8a4ceOSSfdzzjknxowZEzfddFM8/PDD8eKLL8bq1aujUChEz549Y7/99ovPfvazcfTRR8eoUaP8Qa6NqSls6f/AoEpGjhwZM2fO3PT1mWeeGbfcckv1BgQAAAAA24jtEQAAAAAAckTSFgAAAAAgRyRtAQAAAAByRNIWAAAAACBHFCIDAAAAAMgRK20BAAAAAHJE0hYAAAAAIEckbQEAAAAAckTSFgAAAAAgRyRtAQAAAAByRNIWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEc6l9uwpqbspgAdWqGwoVn9pg+7vIVHAtA+HTvnqmb39UwLUJ7mPtOaZzu2sT0nlt327rdHt+JIqueU3WaU1W7aqkmtPBLyrql51kpbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByx2QwA5Fht/+WZ2O4T9k+2/XD4sNYeTovo9PScZHzZdQszsflL+7b2cACSezDefclb6cYXf62VR9NCfvgfyfAp1+yRidlXEQDyx0pbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHOlc7QEAQHtQ2395Mr77hP0zsQ+HDyv7uJ2enpOJLbtuYbLt/KXvln3caqrkvTq61d6rvmUfF8iPsT0nJuN3X/JWNnjx18o/8A//IxM65Zo9kk2nTRxd/nGrqLL3akb5B67kvVo1qfzjAi2i1L2fkpwPSqlkTi3TKbuVP/dUez6p6H19u9zfExX8PknMvaWUmpNTqv2+smVW2gIAAAAA5IikLQAAAABAjkjaAgAAAADkiKQtAAAAAECOKERGq1i/oYJiBm1Al85to+AE0PJSRbNSBbMiuif7pwphVVIwrH51t0xs8oo3k23ffOfWso9bTXvuOioZr1swMBMb2uOhso9b/s8q4phELP2zUrAMtoVUgZd0gZx00ZxU0ZW2UjCstZQqLtMlVUtnYvkFbsr/WUVEZAsXJX9WCuEAQIaVtgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5ImkLAAAAAJAjNYVCoVBWwxo1y9qr1iga1t4Kd3mPqEShsKFZ/aYPu7yFR0IplRSsSkkVrJqSKKIVETF5xZOZ2JvvPFL2uVJFu+r6HJ5sO7THmrKPW02p4moR2/a9Ou/gJZnY1l4DipZtO8fOuarZfT3TbjuVFazKUrCq7XMNtG3NfaY1z5YvdY+UUsm9U67UPVaKe6+6XCvtU1PzrJW2AAAAAAA5ImkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCN2CG/Hyi2epSBW01rjPfLzAQAAACBF0haAdqG2//JkfPcJ+yei3TORZdctTPafsmBgJjZ5xZuJlqlYRF2fwzOxofsPSbYt35qt7F9dQ3ukxz+0R+p9Kf+9ql/dLRObvOLJZNvJj2VjdYmf9XkHL0n2T11XxyTalbqu5i/tm4xD3pWqXp2uVJ2Nlao+rdJ0+5T6uXZJXEKVXVdfy0RcVwC0R7ZHAAAAAADIEUlbAAAAAIAckbQFAAAAAMgRe9q2QQpYtQ/l/nz8vAEAAAA6lppCoVAoq2GN/G5eSOJ1LH7ebU+hsKFZ/aYPu7yFR9J+pYqO9RmW/nW2Yk5NJpYuLpYuWJWSLC5WorgW+VRJ0bKU1DWQKlpWyXWpOFn5jp1zVbP7eqYtX6o4VN2+G5NtJy/KfoBPEShag+ty22nuM21Hn2fXT8oWy2sJpQruNeYapxKlCkE2li4MufW6TPyPVjluW9HUPGt7BAAAAACAHJG0BQAAAADIEUlbAAAAAIAc6dibzeRIufuWRti7tKNp6YJllRwTAAAAgG1P0haAXEgVF4uI2H3C/olo90zkf77zbrJ/3cJ5ieib2XaJwlIRpQqMKTrW1qV+rkN7DMnEUgXLItJFyyY/lmi3NHvMiIhjfpS9ho9JtFt23cJkf0XLaK5SBUfSBUaysY5eMITqSxVZmjY33TZdECobK1XgSUEnAKrJ9ggAAAAAADkiaQsAAAAAkCOStgAAAAAAOWJP2ypIFYxSGIqtVck15BoEAAAAyC8rbQEAAAAAcqSmUCgUympYY1FuS7HKkWpzDbauQmFDs/pNH3Z5C48kv2r7L8/E+gxL/zpaMacmE5uyYGAm9uyf1iX7D+7VNRMb2mNNU0OEitWv7paJVXJdnnfwkkyskvti/tK+TQ2x3Th2zlXN7tuRnmnH9pyYidXtuzHZdvKi7FqOaasmtfiYoNrcF+Vr7jNte51nHzz0khY/5nFzr2nxY0K1uVfK19Q8a6UtAAAAAECOSNoCAAAAAOSIpC0AAAAAQI60z81mqiS1T2iKvUOpttQ16PoFAAAAyAdJWwBaVblFxx6+d/dk/7qF8xLRN7Pt+hye7K/oGNtK+lrLFieLiJi84sls7LFEu6VDkv2PPmlZJlY7J3uvdaTiZB1ducWV2mshDyhXqpDYtLnptsliOouy91pHKk4GwLZjewQAAAAAgByRtAUAAAAAyBFJWwAAAACAHLGnbTOUKtikQBNtWbnXr+sfAAAAoHVZaQsAAAAAkCM1hUIhW8I71bDGotyPWGlIR+b6b1qhsKFZ/aYPu7yFR7Jt1fbPVq6PiOgzLPtr5uF7d8/EJi9K/x1xcK+umdjQHmsqHB3kX/3qbpnYs39al2xbt+/GTOzok5ZlYivm1CT7z1/at8LR5cuxc65qdt+2/kw7tme2cn1E+po4bu41rT0c6HAePPSSTKzUM8y0VZNaezitqrnPtG1pnk39PLeWuRfK19HvwabmWSttAQAAAAByRNIWAAAAACBH2s7nFqok9VFwHwOnIyt1/btXAAAAAFqGlbYAAAAAADlipS0AFUsVHUsVHIsov+hYquBYhKJjdBzpaz1bnCwiYvKiRIGyxL2WKk4WEVE7J3sPt/XiZO1RquhYquBYRNsqugFtWepeK1lIZ1H2Hm7rxckA2HastAUAAAAAyBFJWwAAAACAHLE9wmYUUoLmS90r7ikAAACAyllpCwAAAACQI1baAlBSquBYRLroWKrgWET5RccUHIOs0vdFtkBZucXJItIFyhQnq55UwbGIdNExBccgf0rdl8kCZYqTtbpSc2paurhjivkXWl6591XJgo8JlcwBeZ9/rbQFAAAAAMgRSVsAAAAAgBzpsNsjKJAErU9xMgAAAIDKWWkLAAAAAJAjkrYAAAAAADnSYbdHAKD5VsypycTqV2er2UdEDO6VjQ3tsaalhwQdSvoeyt6D9avT/WsT9zD5M3mR9RXQlrmHAdgafosAAAAAAOSIpC0AAAAAQI50iO0RVKuH/Ejde+5RAAAAgP9jpS0AAAAAQI7UFAqFQlkNa9ruolyr+CDf2ts9WihsaFa/6cMub+GRVKa2//Ky205ZMLDstoqOQfWUKhCYct7BS8puO39p3+YMp8UcO+eqZvet9jPt2J4Ty247bdWkVhwJUA1taQ5o7jNta8yzlbxvdftuLLvtcXOvac5wgBx78NBLym5bScHI1piTm5pnrbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHGm71cUAaLZU0bE+w7J1KR++d/dk/2f/tC4TG9yr69YPDNgmUvdwqrjY0SctS/avnZOdQ6pdnCyPUoVzUgVyFMKBjiNVyKZk0ZxF2Tmk2sXJANh2rLQFAAAAAMgRSVsAAAAAgBxpV9sjrN8wIxnv0nn0Nh4JUInUPZq6n93LAAAAQEdgpS0AAAAAQI5I2gIAAAAA5Ei72h4BgOZbMacmE6tf3S3ZdnCvbGxojzUtPSRgK5W+L7P3dv3qbKvaxLzA1pm8yJoJoJh5oWV5P6Fja09zQPv5TgAAAAAA2oE2u9JWkSJo38otTlaqLQAAAEBbZaUtAAAAAECOSNoCAAAAAORIm90eAYCm1fZfXnbbKQsGlt1W0TFo21L3cKrwYKl54byDl2Ripeab+Uv7Vji6tmdsz4llt522alIrjgRoi0rNC6m5pdR8Y24BaH+stAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAcUYgMAAAA2OYqKeRYLkXZoGOrZA6oZA4qt21LzkFW2gIAAAAA5IikLQAAAABAjrSJ7RHWb5iRiXXpPLoKIwGqqdR9b44AAAAA2hMrbQEAAAAAckTSFgAAAAAgR9rE9ggANK22//Ky205ZMLCsdkN7rGnucIA2JnW/16/ulmybmkPOO3hJsm1qbpq/tG+Fo8uPSqoMq2AObI3UHFJqDkrFzUEAbZuVtgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5ImkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCOdqz2AxtZvmJGJdek8ugojAdqK1BxhLgEAAADaKittAQAAAAByRNIWAAAAACBHcrc9AgBNq+2/vKx2UxYMLPuYQ3usae5wgHaq1LxQv7pbJlZqvjnv4CWZWGoOm7+0b4Wja31je04sq920VZNaeSQAf1VqvknNV6mY+Qqg7bDSFgAAAAAgRyRtAQAAAAByRNIWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEc6V3sAAAAAAKVMWzWp2kMA2qFK5paxPSe24kjSqpq0Xb9hRibWpfPoKowEaG9Sc4k5BwAAAGgLbI8AAAAAAJAjkrYAAAAAADkiaQsAAAAAkCMKkQG0QfOX9i2r3dAea1p5JEBHVMncUu58lUcK3wBthfkKoP2x0hYAAAAAIEckbQEAAAAAckTSFgAAAAAgRyRtAQAAAAByRCEyAAAAoEWM7Tmx2kMAqJqWnAOttAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMiRztUeAACl1fZfXnbbKQsGlt12aI81zRkOQEn1q7uV3fa8g5eU3Xb+0r7NGU5FKqnyO23VpFYcCUDLMbcBtG1W2gIAAAAA5IikLQAAAABAjkjaAgAAAADkiKQtAAAAAECOSNoCAAAAAORI5211ovUbZmRiXTqP3lanB0jOOeYmAAAAIG+stAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAc2WaFyACo3PylfctuO7THmlYcCcCWVTIHVTK3bQvTVk2q9hAAWpy5DaBts9IWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEckbQEAAAAAckQhMgAAAKBFKIAGtEetM7dducVXrbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBFJWwAAAACAHJG0BQAAAADIEUlbAAAAAIAckbQFAAAAAMiRztUeAAAAANDxjO05scWPOW3VpBY/JtA+5X0OstIWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEckbQEAAAAAckQhMoAcq1/drey25x28pOy285f2bc5wAEqq7b+87LZTFgwsu+3QHmuaM5xWU0nBCsVwgJZmDgLoOKy0BQAAAADIEUlbAAAAAIAckbQFAAAAAMgRSVsAAAAAgBzZZoXIunQenYmt3zCjrHYALcGcAwAAALQFVtoCAAAAAOSIpC0AAAAAQI5I2gIAAAAA5IikLQAAAABAjkjaAgAAAADkSOdqDwCA0ob2WFN22/lL+7biSAC2rJI5qJK5LW+mrZpU7SEAHZg5CKDjsNIWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEckbQEAAAAAckQhMgAAAGCbK7ew2tieE1t5JABbVo1CkFbaAgAAAADkiKQtAAAAAECOSNoCAAAAAOSIpC0AAAAAQI5I2gIAAAAA5Ejnag8AgMrVr+5WVrvzDl5S9jHnL+3b3OEAHUxt/+Vlt52yYGBZ7Yb2WNPc4VRdJVXNq1F5GGibzC0AHZuVtgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5UtU9bbt0Hp2Jrd8wo6x2AFtiLgEAAADaKoXIAAAAgHZBATfo2CqZA/LO9ggAAAAAADkiaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQI5K2AAAAAAA50rnaAwCgckN7rMnE6ld3y8SmLBiY7H/ewUsysdr+yzOx+Uv7NmN0QHuRmhdKKTXfpKTmsLasVPXxVPXiVEz1cujYKql0br4A6DistAUAAAAAyBFJWwAAAACAHMnd9ghdOo/OxNZvmFFWO6BjMkcAAAAA7YmVtgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5ImkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCOdqz0AAFrG0B5rMrH61d2SbacsGJiJnXfwkkystv/yZP/5S/tWODog70rd7ympOaSU1NzUUUxbNSkTG9tzYlmxUv2Btq3U/Z5iDgDo2Ky0BQAAAADIEUlbAAAAAIAcaRPbI3TpPDoTW79hRlntgPYjdd9HuPcBAACA9sVKWwAAAACAHJG0BQAAAADIkTaxPQIAAADQMU1bNanstmN7TmyVtpWMAWhZldyrlcj7fW2lLQAAAABAjkjaAgAAAADkiKQtAAAAAECOSNoCAAAAAOSIQmQA7djQHmuS8frV3TKxKQsGZmLnHbwk2b+2//JMbP7SvhWODqiW1D2ckpoXSik131AsVfCiVHGNVDzvBTOA/1Nu4Rz3NQApVtoCAAAAAOSIpC0AAAAAQI602e0RunQenYmt3zCj7LZAvqXuZ/cyAAAA0BFYaQsAAAAAkCNtdqUtAC3r2T+ty8RKFRc7+qRlmVjtnHRhIwXKoHpKFRzrM6yQiT187+6ZWGpeiIgY3Kvr1g2MIqWKED146CXZ4KJ0YSOFjKB6ShUcq9t3YyZ23NxrWns4HV4l82Fyni113LnNGQ3QElLzaSntaZ610hYAAAAAIEckbQEAAAAAcqRdbY9QqkiRgkaQb+5RAAAAgP9jpS0AAAAAQI5I2gIAAAAA5Ei72h4BgPIM7bEmEe2WidSvTvevnVPTsgMCtqkViXu4fnV2DhjcK90/PYfQ0iYvsr4C2jL3MABbw28RAAAAAIAckbQFAAAAAMgRSVsAAAAAgByRtAUAAAAAyBGFyACIiHRhoVRhooiIKQsGZmLnHbwk2ba2//JMbP7SvhWODmhK6l4rJXUPpyg4Vl3TVk3KxMb2nJhsm4qn+gNbp9Q9mOIezL9KisU9eOglZbU7bu41zR0OdDjl3lcdtbBjx/yuAQAAAAByqkOstO3SeXQmtn7DjLLaAS3LvQcAAACwZVbaAgAAAADkiKQtAAAAAECOdIjtEQBoWc/+aV0mVqq42NEnLcvEaucoTgZbI1V0rM+wQib28L27J/un7uHBvbpu/cBodaUKGyULeSxSnAy2RqroWN2+GzMxhacAaA1W2gIAAAAA5IikLQAAAABAjnTY7RFS1epVtYeW5Z4CAAAAqJyVtgAAAAAAOSJpCwAAAACQIx12ewQAmja0x5oSr3TLRCYvylajj4iIRPX6o09alonVzlme7D5/ad+S44P2rrZ/+r7oM6yQiT2cuNcmL0r/fX5wr66ZWOn7nbYgVb3+wUMvyTZcNDHZf9qqSS09JGgzxvZM3xd1+27MxFL3Gm1XJXNfXSTm1ITk3FuC64n2qJJ7oFwd9TnFSlsAAAAAgByx0nYzipNB87lXAAAAAFqGlbYAAAAAADkiaQsAAAAAkCO2RwCgYumCRdniZBElCpSVWZwsIl2gTHEy2qNU0bFUwbGI8ouOpQqORSg61lGUXZwsIlmgrKMW/aB9SxUdSxUci1AkCoDqstIWAAAAACBHrLRtQrnFyUq1hfbG9Q8AAADQuqy0BQAAAADIEUlbAAAAAIAcqSkUCukKF40b1thJ4SM+Hk5H5vpvWqGwoVn9pg+7vIVHkg/1q7MFyp79U7Y4WakiIMf8qHtZ51l23cJkXNEyqilVXCwiYvcJ+5fV/3++824yXm7RsfZacOzYOVc1u69n2rT1k75WVrtTrtkjGVe0jGpKFReLiLj7krfK6t9l4n+05HDajeY+03b0ebbc+bRSpebfxszHVKLU/NlYufNppTr6/NvUPGulLQAAAABAjnTsP4E1U6kVhaVWIJbbH6rJ9QsAAACQD1baAgAAAADkiKQtAAAAAECOSNoCAAAAAOSIPW0BaFXp6vXdMpHJi9Yl+9efvTYTO+/gJZlYn2GFZP/aOcszsflL+ybbwtao7Z+91kpdl8uuW5iJTVkwMBN79k/pv68P7tU1E0vfa1CeVPXmVEXpun03pg+wKNtWBXNaQyXX5SnX7JGJuS4BaCskbVtQuQWaUgWfFHdiW3INAgAAAOSX7REAAAAAAHJE0hYAAAAAIEckbQEAAAAAcqSmUCikK2Q0blhj+9uWYj9Rqs012LoKhQ3N6jd92OUtPJK2pX51tjhZRMTkFU+W1X/y/kOS8WN+1L2s/qnCUBGKlnV0qeJiERG7T9i/rP7/8513k/G6hfPK6l/X5/BkvKMXHTt2zlXN7uuZtnWsn/S1stqlCkNFKA7V0aWKi0VE3H3JW2X1TxXSY+s195nWPNs6yp1nK1FqTk4xT1dXqXkypdy5sxLm2dbR1DxrNq2CVHIslUSrpD+4hgAAAADaB9sjAAAAAADkiKQtAAAAAECOSNoCAAAAAOSIpC0AAAAAQI4oRAZALgztsaZEfEgmVr+6WyY2edG6ZP/6s9dmYucdvCQT233C/sn+xyRiy65bmInNX9o32Z98qu2/PBNLXwPdk/1T18CUBQMzsWf/lP77eF2fwzOx9D2Qvi8gb1JVpVOVrktXtM5WRU9VNVe9vG0p/xpIXxeuAQA6MknbnOjSeXTZbddvmNHixyS//LwBAAAAOhbbIwAAAAAA5IikLQAAAABAjkjaAgAAAADkSE2hUCiU1bDG9rd5YY/TjsXPu+0pFDY0q9/0YZe38Eg6llRxsoiIySueLKt/qjBURGVFy1IULdt2yi8ullZucbGIrb+uShXeozzHzrmq2X090+ZTZUXLshSs2nb8rDqO5j7TmmerK3WPllLJvRsXZwtGbq1Tdivv/3Ujqj9PVPS+vt0K/2/+w2zBz1JS82wp1X5fO7qm5lmzaRtUbnJOsi/f/HwAAAAASLE9AgAAAABAjkjaAgAAAADkiKQtAAAAAECO2NMWgHahVGGnoT2GZGKpomWlCktNfiwbq0sUp0oVLItIF8I6eviwZNuUTk/PycRSBbMi2k6Bs1TBsIj0e/VhK71XqQJjk1e8mWiZiqULjKWvQQXHoBypQihdEjVfShWCSRbTubj8AjepAi+lCrm0laItlb1XFRQYaofvFQDkkaRtO9bSBcta49xthfcIAAAAgG3F9ggAAAAAADkiaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQIzWFQqFQVsMaNcsoX2sU7qomRcOoRKGwoVn9pg+7vIVHQkuoX90tE5u84smy+9f1OTwTG9pjTdn9a/svz8R2n7B/su2Hw4eVfdxq6vT0nGR82XULM7H5S/uWfdxq/6zYdo6dc1Wz+3qmJWVsz4mZ2N2XvJVufPHXWnk0LeSH/5EMn3LNHpnYtFWTWns0tEHNfaY1z3Zsqfm0lLvfbp//n33KbuXlQ8y9NDXPWmkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCOStgAAAAAAOWKHcFqFwl1Ae5EqRDW0x5BMLFUEKyJdCOsfXnuk7PPvueuoTKxuwcBk26E9Hir7uNVU+r16MxN7851byz5u8r1KFBeLKFVgTNEx6KhSxWC6lKqlMzFd4AsAoCVZaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5UlMoFAplNaxRswygHIXChmb1mz7s8hYeCe1BqmhXqrhZRMSb75Rf4KyaUgXDItJFw9IFw+jojp1zVbP7eqYFKE9zn2nNswDlaWqetdIWAAAAACBHJG0BAAAAAHJE0hYAAAAAIEckbQEAAAAAckTSFgAAAAAgR5R1BIAcG9pjTSI2pETrUvG2Ivu9AgAAdERW2gIAAAAA5IikLQAAAABAjkjaAgAAAADkiKQtAAAAAECO1BQKhUK1BwEAAAAAwF9ZaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQI5K2AAAAAAA5ImkLAAAAAJAjkrYAAAAAADkiaQsAAAAAkCOStgAAAAAAOSJpCwAAAACQI/8fro4aNIGhLjsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1400x600 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_extraction(\n",
    "    image, kernel,\n",
    "\n",
    "    # Window parameters\n",
    "    conv_stride=1,\n",
    "    pool_size=2,\n",
    "    pool_stride=2,\n",
    "\n",
    "    subplot_shape=(1, 4),\n",
    "    figsize=(14, 6),\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96ee3ee-23d5-4cb8-8a54-4bb1f2ccd042",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "And that works pretty well! The kernel was designed to detect horizontal lines, and we can see that in the resulting feature map the more horizontal parts of the input end up with the greatest activation.\n",
    "\n",
    "What would happen if we changed the strides of the convolution to 3?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8fdc54a9-f76e-42b0-bec9-15550c20d9a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABW0AAAGACAYAAADWAkD8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAArCElEQVR4nO3de5SUxZ038N/IDKCiCIKIigwaEi+JEkCD8URA0GNQkQQvMWpEDegmXqMb8I0bY9bdoGSjruu7EcFLvKDoyqi7+CooElEEQZGAgsbMoGK8cEDRrMCg/f7hgdDTPdce6Jrh8zlnzqGfp+qp6p7pmuY79VSVZDKZTAAAAAAAkIQdit0BAAAAAAD+TmgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAK1IeXl5lJSUbP761a9+lXX+mWeeyTpfUlISVVVVRekrAAA0p6qqqpzPus8880yxuwVNUlrsDtCyDBo0KGbPnp11LJPJFKk3205FRUUsWrRo8+Py8vIYNWpU0foDtF75xtm6dOzYMT766KNm7UNVVVXceeedWccuvfTS2G233Zq1HYCtId84WlJSEm3bto0dd9wxdtttt9hnn33igAMOiMGDB8fIkSOjXbt2W7VPN954Y9ZYPWjQoBg0aNBWbbMQNf/gN2LEiOjTp09R+gI0n+XLl8eUKVNizpw58frrr8fq1atjw4YN0bFjx+jdu3cMGDAghg8fHgMHDoySkpJidxe2e0JbaICKioq46667Nj8eOHCg0BZotaqqquKaa67JOjZq1CihLdBiZTKZWL9+faxfvz4++uijqKqqijlz5sSkSZPikksuiSuvvDIuu+yyrRZS3HjjjbFixYqsYymHtjV/B5SXlwttoQV777334ic/+UlUVFTknXS1atWqWLVqVcydOzduuOGGGDlyZDz00ENF6CmwJaEtALQic+bMiY0bN25+LGgFqNuqVavi8ssvj2eeeSYefPDBrT7rFmBbWrBgQRx//PHxwQcfNLjOqlWrtmKPgIYS2gJA4iorK2s9t8MO2cvT77PPPlu7OwAtTmVlZWQymVi7dm0sW7Yspk2bFg899FB8/vnnm8s89thjcc4558R9991XxJ4CNJ8VK1bkDWz32muvuPjii+Ooo46KLl26xNq1a+NPf/pTTJ8+PSoqKorTWSCHjchoVqNGjcpa8HvTbV8zZsyIE088Mbp27Rrt2rWL/fffPy6//PJa12EcNGhQ1nU2LUUwZcqUGDRoUHTu3Dl23nnn6NOnT/zud7+L6urqvNepuQB5zTUa6+rzlv3YcmmEiIjZs2db3BzYZsrLy2v92nfffXPK1rURWV3uvPPOKCkpicGDB+ec69WrV95xeUvr1q2LSZMmxfDhw6NHjx6x4447RocOHeKrX/1qnHfeeTF//vxa265tLJ46dWoMGTIkdt9990Y/H4BNysvLo1evXnHooYfGaaedFvfff3889dRTseuuu2aVmzJlSvzP//xP3ms8//zzMWbMmDj44IOjY8eO0bZt2+jevXt897vfjcmTJ+f9PLppTKu5NMI111yT81myudqsafHixXHppZfGYYcdFl26dIm2bdtG165d4xvf+EaMGTMmpk+fHhHZm/fUdM4552T1tby8vN52geK74oorcgLbo446Kl599dUYO3ZsHHHEEdG7d+/o169fjBo1KqZOnRpvvvlmfO9738u5VnV1ddxzzz0xcuTI6NmzZ+y0006x4447Ro8ePWL48OExefLkWL9+fd5+1LYR7l//+te45JJLYv/994/27dtHly5dYvjw4TFv3rw6n9err74aZ5xxRnTv3j3at28f++23X1x88cXx/vvvN+r1+fzzz2Pq1Klx6qmnRq9evWLnnXeOHXfcMXr16hU//OEPY8aMGbXW/dWvfpV3XHzxxRfjBz/4QXTv3j3atWsXPXr0iDFjxsTKlStrvdZTTz0VZ511VhxwwAGxyy67RFlZWXTp0iUOPPDAOOGEE+Kaa66JJ554otb6H3zwQVx77bUxcODA2GOPPaJt27bRqVOn6Nu3b4wdOzbeeeedRr0uJCQDjTBw4MBMRGR9benss8/OOved73wnc9FFF+XU2fR18MEHZz755JN62znjjDMyp512Wq3XOfLIIzNr167NuU7NcnfccUdOmZp9HjhwYJ3Pt7avWbNmFfryAtQ7ztanZ8+eWXWvvvrqrPOzZs3KuX5lZWUmk8lk7rjjjgaPeWeffXbWdefOnZvTdr6vCy64ILNhw4acfucbi0ePHp1Tv+bzAaipMePoPffck/dz5ZY+/vjjzCmnnFLv+HbQQQdlli9fnlW3oWNqzT4W0uYmn376aebcc8+t9xqHHnpoJpPJZCorKxvc1549ezb+GwNsU2+88UbOe3f33XfPfPjhh42+1pIlSzIHHnhgvWNDr169Mi+++GJO/XyfPydNmpTZdddd816nbdu2mSeeeCJvX6ZNm5Zp27Zt3npdunTJTJs2rUH/V1+2bFnmkEMOqfc5fe9738ubNVx99dU54+J1112X2WGHHfJeZ6+99sq88847Ode5/PLLGzTutmvXLu/rMXny5MxOO+1Ub91JkybV810mRWbaslXNmTMnbr755lrPL126NK677rp6rzN16tR44IEHaj3/3HPPxejRo5vURwAK8/LLL8fQoUNzZpLl8/vf/z4uuOCCess999xzcdtttzVH9wBqdfrpp+fcsTB37txYs2ZNRHw5s2z48OHx4IMP1nutV199NY4++uj461//WlCfmqPNjRs3xve///24/fbbC+oL0HLlu2tg9OjR0aVLl0Zdp7KyMgYPHhyvvfZag8oOHTq0QWVHjx4da9euzXtuw4YNMWbMmKwlbCIili1bFqeffnps2LAhb71Vq1bFGWecUW/bb731VgwaNCgWL15cb9lp06bFySefnNOXfNccO3ZsfPHFF3nPv/vuuzFu3LisYwsWLIh/+7d/q7cPtZk4cWKcd9558b//+791llu/fn38+Mc/jrvvvrvJbVEcQlu2qkwmEzvvvHPccsst8eqrr8a9996bcxtaQ9YNq66ujj333DPuvffeWLx4cdx3333RvXv3rDIPPPBAnbfeNsX9998flZWVMXLkyKzj3/rWt6KysjLra8CAAc3aNsAmNW8n2/LrxhtvbLZ2Tj755KisrIwpU6bknHv22Wezxrzf/va3EfHlOH/eeefF3/72t81lv/a1r8W9994bS5YsiQULFsSVV16Zdbvt7bffHk8//XSdfdm0mdrFF18c8+bNi6VLl8aDDz4Yhx9+eHM8VYCI+HJd8KOOOirr2BdffBGvvPJKRETccsstMXv27M3nysrK4uqrr4758+dv/my7Zei7cuXKGDt27ObHm8bMvffeO6uNSy65JOez5CaFthnx5R/Innzyyaxje+65Z/z7v/97LFq0KJYvXx7Tp0+PMWPGxI477hgRX66JXrMvm0yYMCGrr3PmzKnlFQVS8fLLL+ccGzJkSKOvc9FFF8WHH36YdWzMmDHx7LPPxgsvvBCXXnpp1rmPP/44/uEf/qHe62YymfjhD38YL774Yjz33HMxcODArPMrVqyI559/PuvYL37xi1i3bl3WsTPPPDPmzJkTzz//fIMCzIgvP1++9957mx937949Jk6cGK+88kosWrQoJkyYkLUp5ZNPPpmzZGK+59OmTZu49tprY8mSJfHoo4/GXnvtlVXmoYceylrW5o9//GPW+b59+8YTTzwRy5cvj6VLl8aTTz4ZEyZMiGHDhkX79u2zyr777rs5r/1xxx0Xjz/+eCxbtiyeeeaZGDFiRNb5iy66aPMfJWkhijvRl5amscsjRETm97//fVaZCRMm5JT59NNP621n4cKFWWUWLlyYU+bCCy/MKlPzfGOXR2hMGYDm0JhlWSIic8MNN2TVL2R5hMaU2eTZZ5/NKldWVpb31q8zzzwzq9zIkSOzzuf7/XHFFVc09uUDaPQyM2PHjs0pP3Xq1Ewmk8nsv//+WccnTJiQU3/mzJlZZdq0aZNZs2ZNVpn6xuYtNUebvXv3zjrfqVOnWsfxmn3NZBr2GRpI27Bhw3Ley6+99lqjrrFixYqca5x55pk55c4///yccq+++urm8/k+Wx5xxBGZL774YnOZDz74IKfMf/zHf2w+v3bt2kybNm2yzh999NE5fTnxxBNzrrPl8ghvv/12zvn58+fnXOeqq67KKtOvX7+s8zWXR4iIzLhx47LKPPjggzlllixZsvn8ddddl3Vu/PjxtX4vai7R8M///M9Zdb/xjW9kPv/886wyGzduzPn9c/PNN9faBukx05atqkOHDjmb1RxwwAE55er7a0+fPn2ib9++Wcf69u0bffr0yTr2wgsvNKmfADTNlrPBIr68M2KfffbJmRF8zz33ZJWrObOgprKyspxbyAC2hkwmk3OspKQkVq5cGW+++WbW8X/8x3/MGd+GDh2aVebzzz/PmR3WUM3R5sqVK+ONN97IOj9mzJhaNw/bbbfdmtRXoPXL93ltzJgxOcfOP//8BtXd0k9/+tOsO7G6du0au+++e1aZLXOChQsX5ixRcO655+Zc97zzzquz3ZqfXSMiDj/88Jxx9tprr80q8/LLL8enn35a57UvvPDCrMf1ZR/9+vXLOvfLX/4yTjrppPinf/qnuPvuu2PBggWbl4LYZZdd6nwef/rTn6JNmzZZz6G0tDRn+bL6vi+kpbTYHaB1Ky8vz7qtICI234K1pU23wdamV69etR5ftGjR5sdb3uIA0Frku1V1k86dO2/DnuSqayfcuqxatSo2btwYpaX5P4r06NEj54M7wNaQbxzr2rVrk8e3iGjyurbN0ea7776bc65mMAC0fl27ds059s477+QNEmuTbzzZf//9c47tt99+Daq7pXz9qJkVbJkTvP/++znl8+UEtWUHmzR1nP3iiy/i/fffjw4dOuQ936FDh5ylcOrLPoYMGRInnXRSPPLIIxHx5Vq+jz76aDz66KOby7Rv3z6OO+64+MUvfhH9+/cv+HkUuu4625aZtmxV+f7D3aZNmyL05Ev5Fg9ftWpVEXoC0HDl5eW1ftVcJ7ylyGQyOWuSbanmGmAAW8MXX3yRM+uoTZs2ccghhxR03c8++6yg+i2lTSBd3/zmN3OOPfXUU0XoSX6NzQry3RWxrdU1zjY1+3j44YfjjjvuiMGDB+dMeIuIWLduXVRUVMSRRx4Zc+fObVyH8/C7omUx05YWobZZZjWP77nnnlmP27RpkxXU5luU/PXXX2+GHgJsn2qGqx07doyXXnopdtih/r8L77zzzrWeK+Yf+IDtx3333Rdvv/121rEBAwZEp06d8v7x6NZbb41jjz223us29U6B5mgz3zUWLlwYp5xySpP6BLRMxx9/fM5GVZMmTYorrriiwWNUvvHkzTffzDn+l7/8JadczY3DC9WtW7ecY5WVlfHtb38751hdava9pKQkXnrppQYtFVNzJm1z2GGHHWLUqFExatSo2LhxY1RWVsabb74ZS5YsiYkTJ25e7mbDhg1x/fXXx7Rp0zY/j9dee23zdY455piYOHFive3lC4ZJl5m2tAiLFi2Kl156KevYSy+9lLU0QkTEt771razHNQfeLQe1iIgZM2bkrPmVT9u2bbMe++sU0JrVHPMiah/3Bg0alPX4448/jnnz5tU5O/i9996LNWvWZK1jBrCtzZ49O37605/mHL/yyisjImKfffbJueW3oqIi9t1331rHt5122ikWLlyYs/ZgQz9LNkebe++9d/Tu3TvrGrfddlu89dZbedv86KOPco6VlZU1qL9Aur7yla/EySefnHVs1apVccopp8Qnn3xSa7233347brrppoiIOOqoo3LO33rrrQ06lq9uIfr165fzR/3bb789p9zkyZPrvM7AgQOzHmcymXj88cfr/Oz6t7/9Ld56662csbFQH330Udb4WlpaGr17947jjjsurrjiirj++uuzym+ZZ9T8DD537tyorq6u9Tn06NEjFi5cKLRtYYS2tBgnnHBCTJkyJZYsWRL3339/nHDCCTllzjrrrKzHNW9tmzRpUtxyyy3x2muvxZQpU3LK16bmekCLFi2K//qv/4o///nPUVVVFe+8804jnw1AuvKtgXbjjTfG0qVLo6qqKqqqqjYvbXDkkUfGoYcemlX23HPPjSuuuCJmz54db7zxRixevDgqKiriyiuvjIMPPjiOOOKIeOWVV7bJcwGIiKiqqorKysp45ZVXYurUqfGDH/wghgwZEmvXrs0qd/rpp8fxxx+/+XHNUPfxxx+PY445Jh5++OFYsmRJLFu2LJ555pm46aabYtiwYdGjR4+4+eabc9qvOa5OmzYtnn322aisrIyqqqqs5bqao82LLroo6/Hq1atjwIABccstt8TixYvj9ddfjxkzZsQll1yS9Xxr6+/dd98dL7744ubfAfmCXiA9v/3tb2OPPfbIOjZr1qw46KCDYsKECTFv3rz485//HAsXLoy77rorTj311Nh///03z+bcd999c8aIe++9Ny644IJ47rnnYv78+fGzn/0sJ7QdOHBgHHTQQc36XHbdddcYPnx41rGnn346zjrrrHj++edj7ty5MXr06HjsscfqvE6PHj3ixBNPzDp21VVXxejRo2PmzJmxfPnyWLJkSUyfPj1+/etfx2GHHRZf//rX4+mnn27W5xMRMWfOnOjevXucccYZcdttt8Wzzz4by5Yti9dffz0ef/zx+M1vfpNVfsv1dM8555ysNXM//fTTGDRoUNxwww0xb968eOONN2LBggVxzz33xPnnnx/77LNPnHzyyTm/90hcBhph4MCBmYjI+trS2WefnXVu4MCBOdeYNWtWzjUqKyvrbGennXbKqVPz65RTTslp67bbbqu3XklJSb19fuyxx+q8Rs+ePQt4VQH+rr5xtj49e/bMqnv11VdnnW/IGPz5559nunbtWue4N2vWrM3lFyxYkNl5553rHW+3/Lrjjjuy2mzI7w+Ahsg3jjbka/jw4ZnPPvss61rr16/PHHXUUY26Tr7x6/LLL6+zztlnn92sbW7YsCEzZMiQBtU99NBDc/o7cuTIOuvU/N0CpOvFF1+s93NdXWPKm2++menSpUuD63bs2DGzdOnSrD405PNnJlP/59ilS5dm2rVrV2f7paWldX5uzWQymcrKyky3bt0a9ZrU7MvVV1+ddT5fJlBZWVlnX+rLGerrw3/+5382+nddvteddJlpS4tw8sknx+jRo2s9P2DAgLjttttyjp9zzjkxePDgWuuddNJJceqpp9bb/nHHHRd9+/ZtWGcBWrgddthh8+3BDdGvX7+YOXNmvbv1btKuXbu8s3kBiqFr165x0003xSOPPBLt27fPOte2bdt47LHH4rTTTmvw9Xr06JFz7MILL2zwxpHN0WZZWVlUVFTEj370owZfY0s///nPm/02YKA4+vfvH6+88kqMGDGiwUtTdenSZfO/99tvv5g1a1YccMAB9dYrLy+PmTNnNvss200OOuiguPfee2sdnzp06JA3F6ipvLw8Zs+eHX369GlQu23atGn2NXob6+ijj46xY8dmHbvgggti8uTJde4TsaUuXbpkzc4lfTYio0UoKSmJiRMnxrHHHhu33HJLLFq0KNavXx+9e/eOs846Ky6++OK8azC2adMmpk+fHhMmTIgpU6bEX/7yl2jfvn306dMnRo8eHWeccUaMGjWq3vZLS0vjqaeeimuvvTb++7//O6qqqmL9+vVb4ZkCpOGyyy6LPfbYI2699dZYvHhxrF27ts5dewcMGBCvvfZaPPDAA/HII4/EwoUL48MPP4wNGzbErrvuGr169Yo+ffrE0UcfHccff3x07NhxGz4bYHtXUlISpaWlsdNOO0WnTp1i7733jgMPPDCGDBkS3//+9/N+jtxk1113jfvvvz8uv/zyuPPOO+O5556LFStWxCeffBLt27eP7t27x0EHHRTf+c534vjjj48DDzww5xrl5eXxwgsvxLXXXhuzZ8+ODz74IKqrq7dqmx06dIi77rorLrvssrjjjjtizpw5UVlZGZ988kl07NgxunfvHkcccUSMGDEip+7hhx8ef/zjH+O6666LuXPnxqpVq7I29wValu7du8e0adNi2bJlMWXKlJgzZ04sX7481qxZExs2bIjddtstvvKVr8SAAQNi+PDhOeulfv3rX4/FixfHAw88EA8//HAsWLAgPvzww8hkMrH77rtH375946STToozzzwz549fzW3kyJHx8ssvx7/8y7/E008/HWvWrIlu3brFscceG1dddVWDr/O1r30tFixYEI8++mg89NBDMX/+/Hjvvffis88+i1122SX23XffOOSQQ2LQoEFx4okn5iwz0RyGDh0aM2fOjDlz5sTcuXPjrbfeig8//DDWrFkT7dq1i7322iu++c1vxqmnnhojR47MG7qfe+65MXz48Jg8eXLMmDEjli5dGmvWrIlMJhOdO3eO3r17x2GHHRbHHHNMDBkyxB/kWpiSTF3/A4MiGTRoUMyePXvz47PPPjvuvPPO4nUIAAAAALYRyyMAAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJsREZAAAAAEBCzLQFAAAAAEiI0BYAAAAAICFCWwAAAACAhAhtAQAAAAASIrQFAAAAAEiI0BYAAAAAICFCWwAAAACAhAhtAQAAAAASUtrQgiUlDS4KsF3LZDY2qd7j3/o/zdwTgNbpu/P+tcl1faYFaJimfqY1zm471eN/VJR2y8b9oSjtFsuIzuOK3YVtqmL1+GJ3YbtR3zhrpi0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACSktdgdonao3zix2F5pVWenQYncBSNhxv+vY5Lr/72cfF9T2sPnXF1S/2H7d+5cF1f/x11c0U08ab9HKbkVrG7ZH1eN/1OS6ZeP+0Iw92f6M6DyuaG1XrB5ftLYBoJjMtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABJiIzK2yqZhrW3jLq8RAAAAANuKmbYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBCbETWijV08ywbYtVva7xGvj8AAAAA5GOmLQAAAABAQoS2AAAAAAAJEdoCAAAAACTEmrYAAAAATTD98J8Xre2ycdcXre3tScXq8UVpd0TncdtVuxHFe61TJbRtgWxg1To09Pvj+w0AAACwfRHaAtDiHfP99wqq32v8yoLqT/xZv4LqF6KYszuax7qCai9a2a3Jdce8vrCgtucPLqh6QX2HYih0vBk2v7AZYWXj/lBQfZqumDOfCp3xZdYWAC2VNW0BAAAAABIitAUAAAAASIjQFgAAAAAgIda0TURDN5uKsOHU9qa5NyxrzDUBAAAA2PbMtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABJiI7IiyLdhlI2hKFRjfob8DAIAAACky0xbAAAAAICECG0BAAAAABIitAUAAAAASIjQFgAAAAAgIUJbAAAAAICElBa7A61J9caZDSpXVjp0K/cE6pbvZ9DPLy3ZbyaXF1R/4lf3bJ6O0KJM/Gq/guovWllY+2NeX1hQ/fmD9ymo/qKV3Qqqz/Zn2Pzri90FtkMVq8cXtf0RnccVVL/Y/Qeg5TLTFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASIiNyJqgtg2bbNBES9bQn18//wAAAABbl5m2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQmxEVo98my7ZcIntWW0//94rAAAAAM3DTFsAAAAAgIQIbQEAAAAAEmJ5BAAAAKBFm374z4vS7rD51xelXVq/itXji92FbW5E53FFaTfV19pMWwAAAACAhJhpuwUbKUHT5XuveE8BAAAANJ7QFoAWr3+ndcXuAjTaxK/2K6j+T2YXdsPU/x341ybXXbSyW0FtA7QUhd4yW8itvqnergvAtmF5BAAAAACAhAhtAQAAAAASIrQFAAAAAEjIdrumrQ2SYOuzORkAAABA45lpCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQkNJid2BbsFs9pCPfe897FAAAAODvzLQFAAAAAEiI0BYAAAAAICFCWwAAAACAhAhtAQAAAAASIrQFAAAAAEhIabE7AEAajvn+e02u+5vJ5QW13b/TuoLqw/ZozFe+KKj+opXdmqknNMb0w3/e5LrD5l/fjD0BtoWK1eOL3QUAWigzbQEAAAAAEiK0BQAAAABIiNAWAAAAACAhrWpN2+qNM/MeLysduo17AjRGvvdovvez9zIAAACwPTDTFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASEiL3YjMJkXQujV0c7LaygIAAAC0VGbaAgAAAAAkpMXOtAUAAADSMqLzuKK0O2z++KK0C7C1mGkLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkxEZkAERExG8mlze5bv9O65qvI8A2sWBN+ybX/fHXVxTU9qKV3Qqq35INm399sbsAtBCFbuhVsdrGXAAtmZm2AAAAAAAJEdoCAAAAACREaAsAAAAAkJAWsaZt9caZOcfKSocWoSdAMdX2vjdGAAAAAK2JmbYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkpLXYHaqreODPnWFnp0CL0BGgp8o0RxhIAAACgpTLTFgAAAAAgIcnNtAWgafrs/X6BV+jWLP0AWob+ndY1ue6kJT2L1naxjeg8rqD6FavHN1NPgNbOeAGwfTPTFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABJSWszGqzfOzDlWVjq0CD0BWpt8Y4kxBwAAAGgJzLQFAAAAAEhIUWfaAgAAAK1Hxerxxe4C0EIZP7KZaQsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACTERmQArcSild2K3QVgO9G/07pid6FobJABAMC2YKYtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBCSrdVQ9UbZ+YcKysduq2aB8g75hibAAAAgNSYaQsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACSktNgdAAAAAFqHEZ3HFaXditXji9Iu0HyMH9nMtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABJSWuwOANA8+uz9fkH1F63s1kw9AVq7QsebSUt6FlT/uwXVLsyIzuMKql+xenwz9QRo7Yw3ANs3M20BAAAAABKyzWbalpUOzTlWvXFmg8oBNAdjDgAAANASmGkLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEKEtgAAAAAACRHaAgAAAAAkpLTYHQCgeUxa0rOg+v07rWumngDUrSWPNxWrxxe7CwAAbAfMtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASEhpsTsAAAAAtA4Vq8cXpd0RnccVpd1iPV+g9TPTFgAAAAAgIUWdaVtWOjTnWPXGmQ0qB1AXYwkAAADQUplpCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkpLXYHAEhDn73fb3LdRSu7NWNPgNRNWtKzoPr9O61rpp60PCM6j2ty3YrV45uxJ0DqvOcBtm9m2gIAAAAAJCS5mbZlpUNzjlVvnNmgcsD2yRgBAAAAtCZm2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJERoCwAAAACQEKEtAAAAAEBChLYAAAAAAAkR2gIAAAAAJKS02B1oiLLSoTnHqjfObFA5oPXI976P8N4HAAAAWhczbQEAAAAAEiK0BQAAAABIiNAWAAAAACAhLWJNWwDq17/TuoLqT1rSs2htA9ten73fL6B2t2brx/amYvX4YncB2IZGdB7X5LrGC4Dtm5m2AAAAAAAJEdoCAAAAACREaAsAAAAAkBBr2gIAAAA0wfTDf160tofNv75obdN6FbIWd6Gs5Z3NTFsAAAAAgIQIbQEAAAAAEtJil0coKx2ac6x648wGlwXSlu/97L0MAAAAbA/MtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABLSYjciy6e2TYpsaARp8x4FAAAA+DszbQEAAAAAEiK0BQAAAABIiNAWAAAAACAhrWpNWwCarn+ndU2u22fv9wtqe9HKbgXVh+1Roe+7SUt6NrluIeMFTTei87iC6lesHt9MPYHth/cdAMVipi0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQoS0AAAAAQEJKi92BbaGsdGjOseqNMxtUDmhe3nsAAAAAdTPTFgAAAAAgIUJbAAAAAICECG0BAAAAABIitAUAAAAASIjQFgAAAAAgIaXF7kCx5Nut3q720Ly8pwAAAAAaz0xbAAAAAICECG0BAAAAABIitAUAAAAASMh2u6YtAM1n0pKeBdW/8ryqgurPeHjPgupDMfTZ+/2C6v9kdveC6o/5yrqC6rPtVaweX1D96Yf/vKD6w+ZfX1B9KIYRnccVVL/Q9x0ANJXQdgs2J4Om814BAAAAaB6WRwAAAAAASIiZtgAAAECLVqylLCrmF6XZiCh8+Y+mevCD4txRWaw7OavH/6go7RZL2TjLwqTCTFsAAAAAgIQIbQEAAAAAEmJ5hHo0dHOy2spCa+PnHwAAAGDrMtMWAAAAACAhQlsAAAAAgIQIbQEAAAAAEiK0BQAAAABIiI3ImqC2DZdq26CpofWhmPz8AgAAAKTBTFsAAAAAgIQIbQEAAAAAEiK0BQAAAABIiNAWAAAAACAhNiJrRg3doCnfhk82d2Jb8jNIc+vfaV1B9X8zubyo7ffZ+/2C6i9a2a2g+rRMx/2uY0H19z1hYUH1J37Vzx2NM2z+9UVtf0TncQXVr1g9vpl6QktSPf5HBdUvG+fnBoCWyUxbAAAAAICECG0BAAAAABIitAUAAAAASIjQFgAAAAAgITYiK4J8Gz7l2xiqMfXBzxAAAABA62CmLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQkqL3QG+VFY6tMFlqzfObPZrki7fbwAAAIDti5m2AAAAAAAJEdoCAAAAACREaAsAAAAAkBBr2gJQdP07rStq+4fPeqeg+vMHN71u1wfPKqjtGd9+sqD6xXbc7zoWre19T3i6oPoTf9avsPpfLaw+tDQVq8cXVH9E53FNrvvgB4Wt/d/S9w6oHv+jorVdNu4PRa0PAC2V0LYFauiHRhtYpc33BwAAAIB8hLYAAAAALUyhdzA0VVlpcdotFjP+KRZr2gIAAAAAJERoCwAAAACQEKEtAAAAAEBCrGnbijX3hmVbo+2WwmsEAAAAwLZipi0AAAAAQEKEtgAAAAAACRHaAgAAAAAkRGgLAAAAAJAQG5GxVTbE2hobdxWTTcMAAAAA2FbMtAUAAAAASIjQFgAAAAAgIUJbAAAAAICECG0BAAAAABJSkslkMg0qWGLPMoCGyGQ2Nqne49/6P83cE7YHw+ZfX+wuFOTXvX9ZUP3+ndY1U09oSb4771+bXNdnWoCGaepnWuMsQMPUN86aaQsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJKclkMplidwIAAAAAgC+ZaQsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkBChLQAAAABAQoS2AAAAAAAJEdoCAAAAACREaAsAAAAAkJD/D2eWg9rWnlN/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1400x600 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_extraction(\n",
    "    image, kernel,\n",
    "\n",
    "    # Window parameters\n",
    "    conv_stride=3,\n",
    "    pool_size=2,\n",
    "    pool_stride=2,\n",
    "\n",
    "    subplot_shape=(1, 4),\n",
    "    figsize=(14, 6),    \n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d431ffd-2063-4f2c-b0d2-1f3e6f0e78ce",
   "metadata": {},
   "source": [
    "This seems to reduce the quality of the feature extracted. Our input circle is rather \"finely detailed,\" being only 1 pixel wide. A convolution with strides of 3 is too coarse to produce a good feature map from it.\n",
    "\n",
    "Sometimes, a model will use a convolution with a larger stride in it's initial layer. This will usually be coupled with a larger kernel as well. The ResNet50 model, for instance, uses $7 \\times 7$ kernels with strides of 2 in its first layer. This seems to accelerate the production of large-scale features without the sacrifice of too much information from the input."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f98fac1e-6fe3-420b-b197-06efe277aa70",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "In this lesson, we looked at a characteristic computation common to both convolution and pooling: the sliding window and the parameters affecting its behavior in these layers. This style of windowed computation contributes much of what is characteristic of convolutional networks and is an essential part of their functioning."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}