numbers(edit 4).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f4db1cce",
   "metadata": {},
   "source": [
    "(Fourth Edit) Beginning the data collecting by loading in MNIST original, a dataset with 70,000 small images of digits. The images are 28x28 pixels, and are grayscale. The images are already split into a training set (60,000 images) and a test set (10,000 images). The labels are the digits these images represent: integers from 0 to 9."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c74dda04",
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.io import loadmat\n",
    "\n",
    "mnist = loadmat(\"mnist-original.mat\")\n",
    "mnist_data = mnist[\"data\"].T\n",
    "mnist_label = mnist[\"label\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3698ad22",
   "metadata": {},
   "source": [
    "Once the data has been loaded in, preprocessing can begin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "42424757",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "import numpy as np\n",
    "#preprocessing the data.  We must first reshape the data to a 4D array(samples,width,height,number of colors), where each image is a different 2D matrix\n",
    "#Normalizing the values of the data, could experiment with minmax scaler too.  To do this, we must flatten the data to a 2D array, then turn it back into a 4D array\n",
    "#I do not need to look for null values because this data does not have any\n",
    "mnist_data = mnist_data.reshape(-1, 28, 28, 1)\n",
    "mnist_data_2d = mnist_data.reshape(-1, 28*28)\n",
    "scaler = StandardScaler()\n",
    "mnist_data_normalized = scaler.fit_transform(mnist_data_2d)\n",
    "#After the 2D data has been normalized, return it to a 4D array\n",
    "mnist_data_normalized = mnist_data_normalized.reshape(-1, 28, 28, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3e70cb3",
   "metadata": {},
   "source": [
    "The data must be split into a testing and a training set. The training set will be used to train the model, and the testing set will be used to test the model. The testing set will be 20% of the data, and the training set will be 80% of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "de82273a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(56000, 28, 28, 1) (56000,)\n",
      "(14000, 28, 28, 1) (14000,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "#Splitting the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(mnist_data_normalized, mnist_label, test_size=0.2, random_state=42)\n",
    "print(X_train.shape, y_train.shape)\n",
    "print(X_test.shape, y_test.shape)\n",
    "#print(X_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05782353",
   "metadata": {},
   "source": [
    "We have 2D representations of the matrixes, they are 28 by 28 pixels. We can visualize the data by reshaping the data into a 28 by 28 matrix, and then plotting it using matplotlib. The data of these images are normalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3ce2f77c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAdtElEQVR4nO3dfWyV9f3/8ddpbQ9taU9XS+9GYQVvWES7jEHXqExDA3SJEWWJd0tgMRJdMcPO6VhUdFvSDRNn9MvwP5mJqDMRiP5BJlVK3AobKCFks6G1GxhoUTLO6Q0cSvv5/dFw9jtSkOvinPO+eng+kiuh51zvXu9z9WpfXD3X9W7IOecEAECG5Vg3AAC4MhFAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMHGVdQNfNTY2pqNHj6q4uFihUMi6HQCAR845DQwMqKamRjk5Fz7PCVwAHT16VLW1tdZtAAAu05EjRzR9+vQLPh+4ACouLpYkFRYWcgaEjMu2yVR8D8GCc07Dw8OJn+cXkrYA2rBhg55//nn19fWpvr5eL7/8shYsWPC1dee+YUKhEN88wGXiewiWvu74S8tFCG+99ZZaW1u1bt06ffzxx6qvr9eSJUt0/PjxdGwOADAJhdIxDbuhoUHz58/X//3f/0kav7CgtrZWjz76qH75y19etDYWiykSiaioqIj/vSHj+BUccPmccxoaGlI0GlVJSckF10v5GdCZM2e0b98+NTU1/W8jOTlqampSZ2fneevH43HFYrGkBQCQ/VIeQF9++aVGR0dVWVmZ9HhlZaX6+vrOW7+trU2RSCSxcAUcAFwZzG9EXbt2raLRaGI5cuSIdUsAgAxI+VVw5eXlys3NVX9/f9Lj/f39qqqqOm/9cDiscDic6jYAAAGX8jOg/Px8zZs3T+3t7YnHxsbG1N7ersbGxlRvDgAwSaXlPqDW1latWLFC3/ve97RgwQK9+OKLGhoa0k9+8pN0bA4AMAmlJYDuueceffHFF3rmmWfU19en73znO9q+fft5FyYAAK5cabkP6HJk631AAdvNAAIkm37WSYb3AQEAcCkIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYSMs07MmCAaFIBY4j/7JtCKdfmTyGgrTPOQMCAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJi4oqdhI/OycXJ0Nr4mjAvS5OhU8XO8pms/cAYEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADARNYMI2UgpH+Z3HeZ2hav6fJkaginn+1k44DQoL8mr8fepa7PGRAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATWTOMFOOCPoQzyDV+Bfk1ZXLIZaYGiwZ9gKmfbQX9a5sunAEBAEwQQAAAEykPoGeffVahUChpmTNnTqo3AwCY5NLyHtANN9ygHTt2/G8jV/FWEwAgWVqS4aqrrlJVVVU6PjUAIEuk5T2gQ4cOqaamRrNmzdIDDzygw4cPX3DdeDyuWCyWtAAAsl/KA6ihoUGbNm3S9u3btXHjRvX29urWW2/VwMDAhOu3tbUpEokkltra2lS3BAAIoJBL8w0TJ0+e1MyZM/XCCy/owQcfPO/5eDyueDye+DgWi6m2tlZFRUWernPP5H0fQcb9L/5r/Aryawr6/S9Brsn0toK6HT+ccxoeHlY0GlVJSckF10v71QGlpaW67rrr1N3dPeHz4XBY4XA43W0AAAIm7fcBDQ4OqqenR9XV1eneFABgEkl5AD3++OPq6OjQv//9b/3tb3/TXXfdpdzcXN13332p3hQAYBJL+a/gPv/8c9133306ceKEpk2bpltuuUW7d+/WtGnTUr0pAMAklvIAevPNN1P9Ka9YmXoj3e92gvzmu9/XNDY2lpFtBf2imUy9+Z6TwzQwKdgXFKQTX30AgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAm0v4H6fwK+rBGr4I8WDTow0gzNSDUb102DiP105+fwaJ+hnAWFxd7rpk6darnGkkqKiryXHPVVd5/rH722Weea/x8X0jBGnzKGRAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwERgp2EjOycmZ6omLy/Pc40kFRQUeK4ZGhryXDMyMuK5xg8/E6olqbCw0HONn32en5/vucbPtGm/E6D9TJz2UxOkCdWZxBkQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwwjzTKZGvaZSVOnTvVcM2XKFF/b8jMU0s9wzHg87rnGz6BUv/zshyAPp82kwcFBzzWjo6Np6CT4OAMCAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABggmGk8M3PUMhIJOK5Ji8vz3ON34GVOTne/0+Wm5vruSYcDnuu8fOa/A65HB4e9lxz5swZzzUjIyOea0pLSz3X+NnfknT69GnPNf39/Z5r/Ax/9VMTNJwBAQBMEEAAABOeA2jXrl264447VFNTo1AopK1btyY975zTM888o+rqahUUFKipqUmHDh1KVb8AgCzhOYCGhoZUX1+vDRs2TPj8+vXr9dJLL+mVV17Rnj17VFRUpCVLlvj6XSoAIHt5vgihublZzc3NEz7nnNOLL76op556Snfeeack6bXXXlNlZaW2bt2qe++99/K6BQBkjZS+B9Tb26u+vj41NTUlHotEImpoaFBnZ+eENfF4XLFYLGkBAGS/lAZQX1+fJKmysjLp8crKysRzX9XW1qZIJJJYamtrU9kSACCgzK+CW7t2raLRaGI5cuSIdUsAgAxIaQBVVVVJOv9GrP7+/sRzXxUOh1VSUpK0AACyX0oDqK6uTlVVVWpvb088FovFtGfPHjU2NqZyUwCASc7zVXCDg4Pq7u5OfNzb26v9+/errKxMM2bM0Jo1a/Tb3/5W1157rerq6vT000+rpqZGy5YtS2XfAIBJznMA7d27V7fffnvi49bWVknSihUrtGnTJj3xxBMaGhrSqlWrdPLkSd1yyy3avn27pkyZkrquAQCTXsj5ndqYJrFYTJFIRIWFhVkxbO8cP7s5UzVFRUWeayT/Ax69yuQh6mcYqZ/+BgcHPdf4Gfbpdxip3zqviouLM1Lj91g9ceKE55p4PO65JtuGkTrnNDw8rGg0etH39c2vggMAXJkIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACY8/zkGBFthYaHnmkgk4mtbIyMjnmv8TFk+e/as5xo/k6MlaWxszHPNqVOnfG3Lq0xNR/fLz7HnpyY3N9dzjZ/p45K/yda4dJwBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMMEw0iyTk+P9/xR+BnBK/gZ++hncOTQ05LnGr1Ao5LkmU0NCMzmMdOrUqRmp8XO8+jkeBgYGPNdI/o4HPzVXKs6AAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmGAYaZYZHBzMSI0kjY6Oeq7xOxwzU/z052eYa6b2QyQS8VUXDodT3MnEhoeHPdf4GUbKgNBg4gwIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACYaRwtcwzWwV5GGpfgaLZmqoqCQNDAx4ronH455rcnL4f3O24CsJADBBAAEATHgOoF27dumOO+5QTU2NQqGQtm7dmvT8ypUrFQqFkpalS5emql8AQJbwHEBDQ0Oqr6/Xhg0bLrjO0qVLdezYscTyxhtvXFaTAIDs4/kihObmZjU3N190nXA4rKqqKt9NAQCyX1reA9q5c6cqKip0/fXX65FHHtGJEycuuG48HlcsFktaAADZL+UBtHTpUr322mtqb2/X73//e3V0dKi5uVmjo6MTrt/W1qZIJJJYamtrU90SACCAQu4ybnwIhULasmWLli1bdsF1PvvsM82ePVs7duzQokWLzns+Ho8n3QsQi8VUW1urwsJChUIhv60Fjp/dnKkav/cBBfk1+RXk1xT0+4CGhoY812TqPiC/P0v81AW5JlOccxoeHlY0GlVJSckF10v7ZdizZs1SeXm5uru7J3w+HA6rpKQkaQEAZL+0B9Dnn3+uEydOqLq6Ot2bAgBMIp6vghscHEw6m+nt7dX+/ftVVlamsrIyPffcc1q+fLmqqqrU09OjJ554Qtdcc42WLFmS0sYBAJOb5wDau3evbr/99sTHra2tkqQVK1Zo48aNOnDggP70pz/p5MmTqqmp0eLFi/Wb3/wmo7+LBgAE32VdhJAOsVhMkUiEixB81vi5oMDvIRDkCwqCfuGCn/c68/PzPdf4/R46efKk55qzZ896rvHTHxch+K/JlMBchAAAwEQIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACY8/zmGTAmFQp6mvQZsqDcuIMgTfP2aMmVKRmr8OHXqlK+6kZERzzXZ+LVFenEGBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwERgh5EyXDT4+8DP8Mmgv6a8vDzPNSUlJZ5r/OwHPwNCh4eHPdcAmcIZEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOBHUaabYI+hNOPTL0mP9spKCjwta2ioiJfdV75GRJ6+vRpzzVnz571XCP5GzQb5O0gmDgDAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYCKww0hDoZCnQYWZHPbpZ1t+hi4ywHRcOBz2XFNYWOi5xi8/Q0IHBwc91wT9eAjyAFO/vWVyW1cizoAAACYIIACACU8B1NbWpvnz56u4uFgVFRVatmyZurq6ktY5ffq0WlpadPXVV2vq1Klavny5+vv7U9o0AGDy8xRAHR0damlp0e7du/X+++9rZGREixcv1tDQUGKdxx57TO+++67efvttdXR06OjRo7r77rtT3jgAYHILuct4Z/OLL75QRUWFOjo6tHDhQkWjUU2bNk2bN2/Wj370I0nSp59+qm9/+9vq7OzU97///a/9nLFYTJFIREVFRVl1EYIfY2Njnmv89Ob39WRqW34uQpg6darnGsnfG8jxeNxzTTQa9VyTyWPcz37IyfH+G/0gb8dvXZAvxsgU55yGh4cVjUZVUlJywfUu6z2gc99EZWVlkqR9+/ZpZGRETU1NiXXmzJmjGTNmqLOzc8LPEY/HFYvFkhYAQPbzHUBjY2Nas2aNbr75Zs2dO1eS1NfXp/z8fJWWliatW1lZqb6+vgk/T1tbmyKRSGKpra312xIAYBLxHUAtLS06ePCg3nzzzctqYO3atYpGo4nlyJEjl/X5AACTg68bUVevXq333ntPu3bt0vTp0xOPV1VV6cyZMzp58mTSWVB/f7+qqqom/FzhcNjX7/gBAJObpzMg55xWr16tLVu26IMPPlBdXV3S8/PmzVNeXp7a29sTj3V1denw4cNqbGxMTccAgKzg6QyopaVFmzdv1rZt21RcXJx4XycSiaigoECRSEQPPvigWltbVVZWppKSEj366KNqbGy8pCvgAABXDk8BtHHjRknSbbfdlvT4q6++qpUrV0qS/vCHPygnJ0fLly9XPB7XkiVL9Mc//jElzQIAssdl3QeUDufuAyosLAz0de5eZeqemUzdO+S3Ljc313NNcXFxRrYjSaOjo55rvvzyS881mRpom8n7X7LxPiA/gnzvUKZk5D4gAAD8IoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCY8PUXURFcmZyqW1BQ4LmmqKjIc42fydFnzpzxXCNJAwMDvuq8ytTEZD+To/1ui8nW8IozIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYCO4w0FAp5GgToZ2BlJgV5qGFxcXHG6kZHRz3XnD171nON32GkfvrzO/DTq2wcRpqp7wsGmF6edL0mzoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYCOwwUq/8DMsL+gDT3NxczzUzZszwXFNTU+O5RpL++9//eq4ZHh72XHP8+HHPNfF43HON5G+fZ0qQh3363VY2DiPFpeMMCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgImsGUaajW644QbPNS0tLZ5r/vGPf3iukaTPPvvMc83evXs91wwNDXmuCfJQ0ckgyMM7g9wbvOEMCABgggACAJjwFEBtbW2aP3++iouLVVFRoWXLlqmrqytpndtuu02hUChpefjhh1PaNABg8vMUQB0dHWppadHu3bv1/vvva2RkRIsXLz7vd/QPPfSQjh07lljWr1+f0qYBAJOfp4sQtm/fnvTxpk2bVFFRoX379mnhwoWJxwsLC1VVVZWaDgEAWemy3gOKRqOSpLKysqTHX3/9dZWXl2vu3Llau3btRf8MczweVywWS1oAANnP92XYY2NjWrNmjW6++WbNnTs38fj999+vmTNnqqamRgcOHNCTTz6prq4uvfPOOxN+nra2Nj333HN+2wAATFK+A6ilpUUHDx7URx99lPT4qlWrEv++8cYbVV1drUWLFqmnp0ezZ88+7/OsXbtWra2tiY9jsZhqa2v9tgUAmCR8BdDq1av13nvvadeuXZo+ffpF121oaJAkdXd3TxhA4XBY4XDYTxsAgEnMUwA55/Too49qy5Yt2rlzp+rq6r62Zv/+/ZKk6upqXw0CALKTpwBqaWnR5s2btW3bNhUXF6uvr0+SFIlEVFBQoJ6eHm3evFk//OEPdfXVV+vAgQN67LHHtHDhQt10001peQEAgMnJUwBt3LhR0vjNpv+/V199VStXrlR+fr527NihF198UUNDQ6qtrdXy5cv11FNPpaxhAEB28PwruIupra1VR0fHZTUEALgyXNHTsP1O1f26IE6Va6+91nPNj3/8Y881536V6tVf/vIXzzV+7vNi+vHkwNdpcgjS14lhpAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEyEXKYma16iWCymSCSioqKiQA3Nu1wB280AAiSbftZJ4z/vhoaGFI1GVVJScsH1OAMCAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgImrrBv4qnMz07Jtdlq2vR4AuJBL/TkeuAAaGBiQJA0PDxt3AgC4HAMDA4pEIhd8PnDTsMfGxnT06FEVFxefNyE2FouptrZWR44cueiE1WzHfhjHfhjHfhjHfhgXhP3gnNPAwIBqamqUk3Phd3oCdwaUk5Oj6dOnX3SdkpKSK/oAO4f9MI79MI79MI79MM56P1zszOccLkIAAJgggAAAJiZVAIXDYa1bt07hcNi6FVPsh3Hsh3Hsh3Hsh3GTaT8E7iIEAMCVYVKdAQEAsgcBBAAwQQABAEwQQAAAE5MmgDZs2KBvfetbmjJlihoaGvT3v//duqWMe/bZZxUKhZKWOXPmWLeVdrt27dIdd9yhmpoahUIhbd26Nel555yeeeYZVVdXq6CgQE1NTTp06JBNs2n0dfth5cqV5x0fS5cutWk2Tdra2jR//nwVFxeroqJCy5YtU1dXV9I6p0+fVktLi66++mpNnTpVy5cvV39/v1HH6XEp++G2224773h4+OGHjTqe2KQIoLfeekutra1at26dPv74Y9XX12vJkiU6fvy4dWsZd8MNN+jYsWOJ5aOPPrJuKe2GhoZUX1+vDRs2TPj8+vXr9dJLL+mVV17Rnj17VFRUpCVLluj06dMZ7jS9vm4/SNLSpUuTjo833ngjgx2mX0dHh1paWrR79269//77GhkZ0eLFizU0NJRY57HHHtO7776rt99+Wx0dHTp69Kjuvvtuw65T71L2gyQ99NBDScfD+vXrjTq+ADcJLFiwwLW0tCQ+Hh0ddTU1Na6trc2wq8xbt26dq6+vt27DlCS3ZcuWxMdjY2OuqqrKPf/884nHTp486cLhsHvjjTcMOsyMr+4H55xbsWKFu/POO036sXL8+HEnyXV0dDjnxr/2eXl57u23306s869//ctJcp2dnVZtpt1X94Nzzv3gBz9wP/vZz+yaugSBPwM6c+aM9u3bp6ampsRjOTk5ampqUmdnp2FnNg4dOqSamhrNmjVLDzzwgA4fPmzdkqne3l719fUlHR+RSEQNDQ1X5PGxc+dOVVRU6Prrr9cjjzyiEydOWLeUVtFoVJJUVlYmSdq3b59GRkaSjoc5c+ZoxowZWX08fHU/nPP666+rvLxcc+fO1dq1awP3VwYCN4z0q7788kuNjo6qsrIy6fHKykp9+umnRl3ZaGho0KZNm3T99dfr2LFjeu6553Trrbfq4MGDKi4utm7PRF9fnyRNeHyce+5KsXTpUt19992qq6tTT0+PfvWrX6m5uVmdnZ3Kzc21bi/lxsbGtGbNGt18882aO3eupPHjIT8/X6WlpUnrZvPxMNF+kKT7779fM2fOVE1NjQ4cOKAnn3xSXV1deueddwy7TRb4AML/NDc3J/590003qaGhQTNnztSf//xnPfjgg4adIQjuvffexL9vvPFG3XTTTZo9e7Z27typRYsWGXaWHi0tLTp48OAV8T7oxVxoP6xatSrx7xtvvFHV1dVatGiRenp6NHv27Ey3OaHA/wquvLxcubm5513F0t/fr6qqKqOugqG0tFTXXXeduru7rVsxc+4Y4Pg436xZs1ReXp6Vx8fq1av13nvv6cMPP0z68y1VVVU6c+aMTp48mbR+th4PF9oPE2loaJCkQB0PgQ+g/Px8zZs3T+3t7YnHxsbG1N7ersbGRsPO7A0ODqqnp0fV1dXWrZipq6tTVVVV0vERi8W0Z8+eK/74+Pzzz3XixImsOj6cc1q9erW2bNmiDz74QHV1dUnPz5s3T3l5eUnHQ1dXlw4fPpxVx8PX7YeJ7N+/X5KCdTxYXwVxKd58800XDofdpk2b3D//+U+3atUqV1pa6vr6+qxby6if//znbufOna63t9f99a9/dU1NTa68vNwdP37curW0GhgYcJ988on75JNPnCT3wgsvuE8++cT95z//cc4597vf/c6Vlpa6bdu2uQMHDrg777zT1dXVuVOnThl3nloX2w8DAwPu8ccfd52dna63t9ft2LHDffe733XXXnutO336tHXrKfPII4+4SCTidu7c6Y4dO5ZYhoeHE+s8/PDDbsaMGe6DDz5we/fudY2Nja6xsdGw69T7uv3Q3d3tfv3rX7u9e/e63t5et23bNjdr1iy3cOFC486TTYoAcs65l19+2c2YMcPl5+e7BQsWuN27d1u3lHH33HOPq66udvn5+e6b3/ymu+eee1x3d7d1W2n34YcfOknnLStWrHDOjV+K/fTTT7vKykoXDofdokWLXFdXl23TaXCx/TA8POwWL17spk2b5vLy8tzMmTPdQw89lHX/SZvo9Utyr776amKdU6dOuZ/+9KfuG9/4hissLHR33XWXO3bsmF3TafB1++Hw4cNu4cKFrqyszIXDYXfNNde4X/ziFy4ajdo2/hX8OQYAgInAvwcEAMhOBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATPw/LvuaG/ziBhAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "random_nr = np.random.randint(0, 56000) #Selects random image from traning set. There are 56,000 images in traning set.\n",
    "\n",
    "#Printing out the 2D array of the image\n",
    "plt.imshow(X_train[random_nr].reshape(28,28), cmap=\"gray\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b29df949",
   "metadata": {},
   "source": [
    "Now that we are certain the dataset is valid and ready to be used, we can now begin image augmentation. Image augmentation artificially increases the size of a dataset by modifying images by rotating and zooming them. This is done to prevent overfitting, and to make the model more robust. The images are rotated by 10 degrees, and zoomed by 10%. The images are then reshaped back into a 28 by 28 matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff672ecb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(56000, 28, 28, 1)\n",
      "(56000, 28, 28, 1)\n",
      "(9, 28, 28, 1)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzQAAANECAYAAABmUGq2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAACWRklEQVR4nOzde3RU9b3//1fAZBIgFwIkIRDuCih4OQoURUVlcWntEi+1te0qntOftho8KvXY4qqinp7mWKuybKme1Qtoq7XVI9pqS4/lpraAgiCigoAg4ZJwMwkESCDZvz/8MhIun/ckn5nM7OT5WGvWgnnvvecze/Z+Z7/n8nmnBUEQCAAAAABCqEOyBwAAAAAALUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQNJ0ubNm5WWlqaf/vSncdvmokWLlJaWpkWLFsVtmwBSEzkEgC/yCFqKgibE5syZo7S0NC1fvjzZQ0moP/zhDxo9erQ6d+6svLw8XXjhhVqwYEFM6/7zn//UmDFj1KlTJxUVFenf//3ftX///gSPGAgHcoiNHAK4tYc8sm3bNl1//fXKy8tTTk6OrrrqKn388ccxr08eSbzTkj0AwOX+++/Xgw8+qOuuu0433nijDh8+rDVr1mjbtm3muqtWrdIVV1yhoUOH6tFHH9XWrVv105/+VOvXr9df//rXVhg9gGQjhwDwsX//fl122WWqrq7WPffco/T0dD322GO69NJLtWrVKnXr1s25PnmkdVDQIGUtXbpUDz74oB555BHdeeedzV7/nnvuUdeuXbVo0SLl5ORIkvr166ebbrpJ//d//6fx48fHe8gAUgg5BICvX/ziF1q/fr3eeustjRgxQpI0adIkDRs2TI888oh+/OMfO9cnj7QOvnLWxtXX1+u+++7T+eefr9zcXHXu3FkXX3yxFi5ceMp1HnvsMfXt21dZWVm69NJLtWbNmhOWWbt2ra677jrl5+crMzNTF1xwgf70pz+Z4zlw4IDWrl2r3bt3m8vOnDlTRUVFuv322xUEQbM+nq2pqdFrr72mb37zm9EEIknf+ta31KVLF/3xj3+MeVtAe0YOIYcAvsKcR1544QWNGDEiWsxI0pAhQ3TFFVeYeYA80nooaNq4mpoa/epXv9LYsWP10EMP6f7779euXbs0YcIErVq16oTln376aT3++OMqLS3V9OnTtWbNGl1++eWqrKyMLvP+++/rC1/4gj788EP94Ac/0COPPKLOnTtr8uTJmjt3rnM8b731loYOHaqf//zn5tjnz5+vESNG6PHHH1ePHj2UnZ2tnj17xrTue++9pyNHjuiCCy5ocn9GRobOPfdcrVy50twGAHIIOQTwF9Y80tjYqNWrV5+QByRp5MiR2rhxo/bt23fK9ckjrYevnLVxXbt21ebNm5WRkRG976abbtKQIUP0s5/9TL/+9a+bLL9hwwatX79evXr1kiRNnDhRo0aN0kMPPaRHH31UknT77berT58+evvttxWJRCRJt956q8aMGaPvf//7uvrqq73H/emnn2r37t36xz/+oQULFmjGjBnq06ePZs+erdtuu03p6en6zne+c8r1d+zYIUnq2bPnCbGePXvqjTfe8B4j0B6QQ8ghgK+w5pG9e/eqrq7ulHlAkrZv367BgwefdH3ySOvhE5o2rmPHjtEE0tjYqL1790bfLXjnnXdOWH7y5MnRBCJ99g7EqFGj9Je//EXSZyf3ggULdP3112vfvn3avXu3du/erT179mjChAlav36988e2Y8eOVRAEuv/++53jPvrVkD179uhXv/qV7rrrLl1//fV69dVXdeaZZ+pHP/qRc/2DBw9KUjTJHSszMzMaB+BGDiGHAL7CmkesPHDsMi1ZnzwSPxQ07cBTTz2ls88+W5mZmerWrZt69OihV199VdXV1Scse/rpp59w3xlnnKHNmzdL+uxdkyAIdO+996pHjx5NbjNmzJAk7dy503vMWVlZkqT09HRdd9110fs7dOigr371q9q6dau2bNlirl9XV3dC7NChQ9E4ABs5pClyCNB8Yc4jp8oDxy7TkvXJI/HDV87auN/97ne68cYbNXnyZP3Hf/yHCgoK1LFjR5WVlWnjxo3N3l5jY6Mk6a677tKECRNOusygQYO8xiwp+gO/vLw8dezYsUmsoKBA0mdfKenTp89J1z/68e7Rj3uPtWPHDhUXF3uPEWgPyCHkEMBXmPNIJBI5ZR6Q5MwF5JHWQ0HTxr3wwgsaMGCAXnzxRaWlpUXvP/oOxvHWr19/wn0fffSR+vXrJ0kaMGCApM/e9Rw3blz8B/z/dOjQQeeee67efvtt1dfXN/ne7fbt2yVJPXr0OOX6w4YN02mnnably5fr+uuvj95fX1+vVatWNbkPwKmRQ8ghgK8w55Hhw4eftGnosmXLNGDAAGVnZ59yffJI6+ErZ23c0XcmgyCI3rds2TItWbLkpMu/9NJLTb53+tZbb2nZsmWaNGmSpM/e2Rw7dqz+53/+56TvOOzatcs5nuZMlfjVr35VDQ0Neuqpp6L3HTp0SM8884zOPPPMJu9srF27tsnXR3JzczVu3Dj97ne/azIDyW9/+1vt379fX/nKV8zHB0AOIYcA/sKcR6677jq9/fbbTYqadevWacGCBSfkAfJI8vAJTRvwm9/8RvPmzTvh/ttvv11XXnmlXnzxRV199dX60pe+pE2bNunJJ5/UmWeeedKeDIMGDdKYMWN0yy23qK6uTjNnzlS3bt109913R5eZNWuWxowZo+HDh+umm27SgAEDVFlZqSVLlmjr1q169913TznWt956S5dddplmzJhh/hjvO9/5jn71q1+ptLRUH330kfr06aPf/va3+uSTT/TnP/+5ybJDhw7VpZdeqkWLFkXv+6//+i9deOGFuvTSS3XzzTdr69ateuSRRzR+/HhNnDjR+dhAe0IOIYcAvtpqHrn11lv1y1/+Ul/60pd01113KT09XY8++qgKCwv1ve99r8my5JEkChBas2fPDiSd8lZeXh40NjYGP/7xj4O+ffsGkUgkOO+884JXXnklmDJlStC3b9/otjZt2hRICh5++OHgkUceCUpKSoJIJBJcfPHFwbvvvnvCY2/cuDH41re+FRQVFQXp6elBr169giuvvDJ44YUXosssXLgwkBQsXLjwhPtmzJgR03OsrKwMpkyZEuTn5weRSCQYNWpUMG/evBOWkxRceumlJ9z/xhtvBBdeeGGQmZkZ9OjRIygtLQ1qampiemygrSOHfI4cArRMe8gj5eXlwXXXXRfk5OQEXbp0Ca688spg/fr1JyxHHkmetCA45vM/AAAAAAgRfkMDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaKVcY83GxkZt375d2dnZSktLS/ZwgHYrCALt27dPxcXF6tAhXO99kEeA5COHAPAVcx5JVIObn//859EGSiNHjgyWLVsW03rl5eXOBk3cuHFr3Vt5eXmi0oRTS3NIEJBHuHFLpVuyckgQcC3CjVtbuVl5JCGf0PzhD3/QtGnT9OSTT2rUqFGaOXOmJkyYoHXr1qmgoMC5bnZ2tiTp2muvVXp6eiKGlxICz36mvuuHge+7Yryr5ufw4cP63//93+g52Zp8coj0eR4588wz1bFjx0QPF8BJNDQ06IMPPkhKDpHicy1y3nnnkUNCjGul8GtoaNDKlSvNPJIWJODVHjVqlEaMGKGf//znkj776LakpES33XabfvCDHzjXrampUW5urr72ta8pIyMj3kNLGale0LRGErBOQgqa5Kqvr9dzzz2n6upq5eTktOpj++QQ6fM8Mnz4cC5GgCRpaGjQe++9l5QcIsXnWuSCCy7Qaael3LfzEaNYrmWSXfRwreN25MgRLV++3Mwjcf9Sa319vVasWKFx48Z9/iAdOmjcuHFasmTJCcvX1dWppqamyQ1A+9XcHCKRRwA0xbUI0L7EvaDZvXu3GhoaVFhY2OT+wsJCVVRUnLB8WVmZcnNzo7eSkpJ4DwlAiDQ3h0jkEQBNcS0CtC9Jn3Zk+vTpqq6ujt7Ky8uTPSQAIUMeAeCDHAKEW9y/GNq9e3d17NhRlZWVTe6vrKxUUVHRCctHIhFFIpF4DwNASDU3h0jkEQBNcS0CtC9x/4QmIyND559/vubPnx+9r7GxUfPnz9fo0aPj/XAA2hhyCABf5BGgfUnI1B3Tpk3TlClTdMEFF2jkyJGaOXOmamtr9a//+q+JeLiU5DtrRmNjo1fcV7Jn/ZDsmTusRm2+s6i19ZlDUhk5BIAv8kjiWdcKVtz3Wsb38VuD77VGopvStpVrnYQUNF/96le1a9cu3XfffaqoqNC5556refPmnfDjPAA4GXIIAF/kEaD9SEgfGh9tpQ8Nn9D44xOa5EpmHxpf9KEBki/ZfWh80IcmNnxCY0v0JzRtvY9N0vrQAAAAAEBroaABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILabuaCHfmTV8Z/5oaGjwWj/RM4/EwndudWt937g1OxazpCHVpcIMPy7W+FrjHOI8RXuW6GuVRG8/2TPCxiLR1yq+11KxzAQahjzJJzQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCiD81JxNK7wVrG6hPju/6RI0e81vftc+M7t7zk38fltNPch6/v3O3p6enOuDV3u28fm1iXQfuV6B4PyZaXl2cu06lTJ2d8+/btXmOw8gT9qJDKEp0jrGsRa/u+1zLW+rHkOGuZRPe0s+LW9q1rIStu7WMpPtczicYnNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0KIPzUnEow9NovvMHD58OKnr+44/Fta850OGDHHGe/To4Yy/++67zrj1HK3xZWRkOOPW3PKSPX98Ksz9jpZLdh8Za33f7Vvjt87RSZMmmY9x0003eT3GXXfd5Yy/9tprzriV66xzlD438JHqPfF8r0Xq6+sT+vixsM4xq2dd586dnfH8/Hxn/LzzznPG+/Xr54x/8MEHzvj777/vjEv234JUyEN8QgMAAAAgtChoAAAAAIQWBQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACK24FzT333+/0tLSmtysfiEAcBQ5BIAv8gjQviSkseZZZ52lv//9758/iNEcsLXF0jjT4tuQLtHNphId923MKdmNmHJzc53xL37xi874BRdc4Iz36dPHGZ8+fbozHkszKherGVcsrOaeYW3Kl+o5JBbxaNBr5ZFkN9Wzxmc1ly0uLnbGBw4c6IxL0uLFi51xq3Hl/v37nXHrOVq50DrHrHM4lga8Ft88EVZhzyOx5BDfczTRTb59rzXq6uq8Hj8ejTWt4yY7O9sZt641Bg8e7Iz/y7/8izN+1llnOeNjx451xv/0pz8545L06quvOuPxyFO+EnJ2n3baaSoqKkrEpgG0A+QQAL7II0D7kZCSav369SouLtaAAQP0jW98Q1u2bEnEwwBoo8ghAHyRR4D2I+6f0IwaNUpz5szR4MGDtWPHDj3wwAO6+OKLtWbNmpN+LFdXV9fkI8Wampp4DwlAiDQ3h0jkEQBNcS0CtC9xL2gmTZoU/ffZZ5+tUaNGqW/fvvrjH/+ob3/72ycsX1ZWpgceeCDewwAQUs3NIRJ5BEBTXIsA7UvCf8WTl5enM844Qxs2bDhpfPr06aquro7eysvLEz0kACFi5RCJPALAjWsRoG1LeEGzf/9+bdy4UT179jxpPBKJKCcnp8kNAI6ycohEHgHgxrUI0LbFvaC56667tHjxYm3evFn//Oc/dfXVV6tjx4664YYb4v1QANogcggAX+QRoH2J+29otm7dqhtuuEF79uxRjx49NGbMGC1dulQ9evSI90MlTGv0j/CdG96ae/3QoUMJjVtzx1v9JSRp+PDhzviFF17ojHfv3t0Zr6iocMarq6ud8czMTK/1O3Xq5IzHwrc/hNV/wiUe/ZpaIiw5xNo/sey/RPeZSXQPiqysLGfc9amaJOXn5zvj69atc8YlOxdZuWzHjh3O+M6dO51xq0eF1W/Kilvbj0f/B2sbYexnFZY84tIa1yLJ7jNjnZ++fWisHBeLgoICZ3z06NHOuHWtYvWxsXKUlSO++c1vOuPWtY5k96FJBXEvaJ577rl4bxJAO0IOAeCLPAK0L8lv7QkAAAAALURBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCK+7TNuMzqT43/MGDB53xAwcOOOODBg1yxi+//HJnXJLOPPNMZzwjI8MZ37JlizO+evVqZ7yystIZ37VrlzNeU1PjjFvi0bvB2ke+/SXQctY5Hotk95mx8syIESOc8V69ejnj1vFnnYOStGfPHmd8/fr1zvjWrVud8f379zvjVg8I6xyNRCLOuNWnxtq+ZI/R0tJeN8nqZRUWvtcJkv85nugc4dunxnp869i0el1J0sCBA53xM844wxnPyclxxq0+Mla/Les4GDx4sDNuXY8tWbLEGQ8LPqEBAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWfWhaKNF9Znznfq+rq/PafkFBgTN+8803O+Ndu3Z1xiV7H1p9Zt544w1n/KWXXnLGrf4OVn+I3NxcZ9zqD9GxY0dnXLLn2E9knxl6SCSe7z5OdB+a4uJiZ7yoqMjr8a08ZfWAkaR33nnHGd+wYYMzbo3ROoesHi/Wa2QdA/E4D60xWM/Bip8qD5FD3Frjtfe9VrHOD98cY42vU6dOzrjVZ8bqISNJ/fr1c8arqqqc8Q8//NAZ/+STT5xxax906dLFGR82bJgzvnHjRmd83rx5zrjU8l5UrSn1RwgAAAAAp0BBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGjRh+YkrHnZpcTP7e7bZ8aK9+zZ0xm//PLLnfHevXs741YPF0l67733nPG//vWvzvjrr7/ujFv70Lf/hLX9+vp6Z9zqUyP5z/Hv06cmlvMAieXbp8J6Da1eSL169XLGrePTGt+mTZuc8Y8//tgZl6SKigpzGRfrPLeeg+85aL0GvvFYxuB7HMV7vbYiUfu1OY/hOwbfaxnr8a2/gwMGDHDGTz/9dGe8sLDQGZfsPjPvv/++M759+3Zn3NpH1vVS9+7dnfGhQ4c641YvLt8cmir4hAYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFrN7kPz+uuv6+GHH9aKFSu0Y8cOzZ07V5MnT47GgyDQjBkz9Mtf/lJVVVW66KKL9MQTT5hzhbcma170WFhzs1u9CXx7mFhxa/vnnXeeM271oenWrZszHksfmrPOOssZX7dunTO+evVqZ9ya+916Da3+FNb6vv0BYnkM3zEkQ1vIIfFiHaO+8fT0dGe8X79+znjfvn2dcSvPWMffjh07nHGrP4Rk91jJzMx0xn33se855nuOx9LLxLeXiJULrdcgEdpDHkmF/O07hi5dujjjw4cPd8aHDBnijHfu3NkZr6ysdMYlae3atc64laes67HTTnNfalvnz8CBA51xq9fOBx984PX4YdHsZ1FbW6tzzjlHs2bNOmn8Jz/5iR5//HE9+eSTWrZsmTp37qwJEybo0KFD3oMFEH7kEAC+yCMAjtXsT2gmTZqkSZMmnTQWBIFmzpypH/7wh7rqqqskSU8//bQKCwv10ksv6Wtf+5rfaAGEHjkEgC/yCIBjxfVzpk2bNqmiokLjxo2L3pebm6tRo0ZpyZIlJ12nrq5ONTU1TW4A2qeW5BCJPALgc1yLAO1PXAuaiooKSSd+n6+wsDAaO15ZWZlyc3Ojt5KSkngOCUCItCSHSOQRAJ/jWgRof5L+S6Dp06eruro6eisvL0/2kACEDHkEgA9yCBBucS1oioqKJJ04q0RlZWU0drxIJKKcnJwmNwDtU0tyiEQeAfA5rkWA9ieuBU3//v1VVFSk+fPnR++rqanRsmXLNHr06Hg+FIA2iBwCwBd5BGh/mj3L2f79+7Vhw4bo/zdt2qRVq1YpPz9fffr00R133KEf/ehHOv3009W/f3/de++9Ki4ubjI/fKL59v+IZV5/3z4zie5D06NHD2fc6j9h9Znp1KmTM271fpDsuc+/+c1vOuMTJ050xp955hln/M9//rMzbo3Pmlve6t1gxWMRj220tjDkkFjEo0eEby6yztPevXs74927d3fGrfFZU+Bu377dGf/000+9Hl+ye+349puyWGO08ogVD+M53hraSh5xCcNrb51/gwYNcsatPjRWT7sPP/zQGd+4caMzLknbtm1zxg8ePGhuw8XKEVYe7t+/vzO+e/duZ/zY8+RkrOvJsGh2QbN8+XJddtll0f9PmzZNkjRlyhTNmTNHd999t2pra3XzzTerqqpKY8aM0bx582K6wAXQ9pFDAPgijwA4VrMLmrFjxzqrzbS0ND344IN68MEHvQYGoG0ihwDwRR4BcKykz3IGAAAAAC1FQQMAAAAgtChoAAAAAIQWBQ0AAACA0KKgAQAAABBazZ7lLAx8ezs0NDSYj+HbZ8aKW31urOdozf3et29fZ7xjx47O+IEDB5zxWOY1r6urc8at1ykrK8sZP/30053xvLw8Z9zqsWH1ofGNS/brQA+LxPHNI7H0s7JyjbWNgoICZ3zAgAHOuPUca2trnfF169Y54+Xl5c64lUdi0Rqvk4t1DlrncKLj8diG9Rxxclb+tfZrLMdmovudWT3nevbs6YwPHTrUGbeudZYvX+6Mf/LJJ854VVWVMy7Z1yvWGK197HstYG3/o48+csatXj3WtY5kjzEVckTyRwAAAAAALURBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGiFsg+N1XfAty+BNee4lPg+M1a8W7duzvjll1/ujJeUlDjj1rzs77//vjP+8ccfO+OS/TqMGTPGGbd6Jxw8eNAZz8zMdMat8VnrZ2RkOOPx6EPj2+fAFU+FeeWTyTfPxNLPyjrGunTp4oxbvZSsY7CmpsYZ37x5szNu5QHfHjCx9OGwlrHOEet18u0zk+h+VbGcp769SlqaR6xzpL1LhT5h1t8x61rB6nVl9ThZs2aNM27lIKtXlnUtJvn3xPM9h63ruR49ejjj27Ztc8at1yAe56m1jdY41tv3FQsAAACAUKOgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILTaZB8a3z4zscxbbi1jjcHqfdC5c2dnfPTo0V7xAwcOOOMrV650xp9++mln/KOPPnLGJSk7O9sZv+CCC5xxqzeCtX2rB4fVAyQSiTjj8ehDYy2Tnp7ujFv7KBX6ICSLbx6xzuF49KGxeilZPVCs52htf9euXc64lUutfRBLzy+LlSu7du3qjFtjtNa3+mBUV1c749Y5br3GVjwe22hpHmnP+SVVWK/tWWed5YwPGjTIGa+qqnLG161b54xbfWasaxXrWszqqRfLNnz/jlp/p/v16+eMW31y1q5d64xbfWhi6WUVhp5SfEIDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAitZhc0r7/+ur785S+ruLhYaWlpeumll5rEb7zxRqWlpTW5TZw4MV7jBRBy5BAAvsgjAI7V7MaatbW1Ouecc/Rv//Zvuuaaa066zMSJEzV79uzo/60GhCcTBMEpG/mkQmPNeDTVc7Ge48CBA722bzXUe+WVV5zxN9980xmPZR+ee+65zvjHH3/sjFv7YN++fc64dVxacatZltUwLx6NNRPVEM+K+WitHCL55RHfPGPFY1nGyiPWeRxLwzSXvXv3eq1vPT9rfMXFxeZjWHmgoKDAGbca4FpjtHLd0qVLnXGraZ51HsZynvpuo6XxRDbWbM08kijW/vE9fyX7+B86dKgz3qlTJ2f8nXfeccY/+eQTZ9zKYb55NpY8bPFtrJmfn++Md+/e3Rm39uGGDRuccevviJUDY2H9vWyNJrvNLmgmTZqkSZMmOZeJRCIqKipq8aAAtF3kEAC+yCMAjpWQ39AsWrRIBQUFGjx4sG655Rbt2bMnEQ8DoI0ihwDwRR4B2o9mf0JjmThxoq655hr1799fGzdu1D333KNJkyZpyZIlJ/16TF1dXZOP3GtqauI9JAAh0twcIpFHADTFtQjQvsS9oPna174W/ffw4cN19tlna+DAgVq0aJGuuOKKE5YvKyvTAw88EO9hAAip5uYQiTwCoCmuRYD2JeHTNg8YMEDdu3c/5Y+Wpk+frurq6uitvLw80UMCECJWDpHIIwDcuBYB2ra4f0JzvK1bt2rPnj3q2bPnSeORSCTlZh4BkDqsHCKRRwC4cS0CtG3NLmj279/f5B2OTZs2adWqVcrPz1d+fr4eeOABXXvttSoqKtLGjRt19913a9CgQZowYUJcBw4gnMghAHyRRwAcq9kFzfLly3XZZZdF/z9t2jRJ0pQpU/TEE09o9erVeuqpp1RVVaXi4mKNHz9e//mf/xnXdz58ezfEo4eM1cvGivvOjZ6Tk+OMZ2VlOeNW74Tly5c7477zsktyvuMu2XO3W8fUmjVrvNb37SNj9Yix4pK9n33jyZAKOUTyzyOt0YfGOo98+8RYvZSs48c6hq3XzOozM3jwYGdcsvtkWOeptQ+s7Vs9HP7lX/7FGbf6ePj28ollmVTME5ZUySPJZh3f1jnWt29fZ3zVqlXO+KZNm5xxq8+SL98cFY9tWNdjQ4YMccatHLJt2zZn3NrHsfS8s1h9ZlJBs5/l2LFjnU/sb3/7m9eAALRt5BAAvsgjAI4VvrdlAAAAAOD/oaABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKHlPzl1ggRB0OJ5r6314tGHJtG9bvr37++Md+7c2Rm3+sxYPVoOHjzojFtz+Xfv3t0Zl+weE9b8+dbrfMYZZzjjGzdudMatHiBWPB69H6z572Pp99Oe+eSRWLbtEo8+NFY/K6t/gdWfwDq++vTp44xbevXq5Yxb52gsPSQs1nlm7UOrT02XLl2c8aFDhzrjW7ZsccZ37drljMeSA3zzBHkmdVnH31lnneWMW32WampqvOJh6HFk5WHreuf00093xq08+MknnzjjVo7AZ1L/SAMAAACAU6CgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILRStg9NMsXSt8LqI2P1j7D6Q/Tu3dsZt/q8VFVVOeP//Oc/nXFLVlaWMz5y5EhzG1Z/htNOcx+e8+fPd8Zff/11Z9yaH9/qgWHF49GHxuLbKydRPVoQG6v/gRV/++23nXHrGOvXr58zfs455zjj1jls5TnrHI+F73lmjcGK+/aKqqio8Hr8ePShoc9MeFk53DpHrZ51hw4dcsZ9+7GlAuscPvvss53x4cOHO+P79u1zxlevXu2M79271xm3coT1GrRGL6vWkPpHGgAAAACcAgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAodUm+9BY82X79g2IhdWHJiMjwxkvLCx0xjt16uSMV1dXO+PvvfeeM15fX++MFxcXO+NTpkxxxiUpPz/fGbfmTn/zzTedcWsfZGZmOuO+fWZ81491GR+uYz0M885b0tLSEvY84tHbw7dPkJVnrD411jlSUlLijHfp0sUZt56fNf5Yjn/fPhjW+lYvIKuPzPbt253xVOhnhfCyzmGrZ5x1/A8aNMgZX7FihTN+8OBBZzw9Pd0Zt/rkRCIRZ7xHjx7OuGRfi/Tq1csZt56j1WfGyhHWc/TNcW3hb73EJzQAAAAAQoyCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNBqVh+asrIyvfjii1q7dq2ysrJ04YUX6qGHHtLgwYOjyxw6dEjf+9739Nxzz6murk4TJkzQL37xC7OvyvFc/SN85+W35v0/7TR7t1jLWGOw+tBY84Lv37/fGd+zZ48zfvrppzvj1vObOHGiM56bm+uMS3Z/h02bNjnjW7ZsccatXj3Wc7SOk0THpbYzP/yxWjOPuFjnaCx5wMU6vqX49Cpyqa2tdcY/+OADZ3znzp3OuJVHCgoKnHHf/gmSvZ/37t1rbsPl/fff93r8Q4cOOeOJ7qMTyzbCJlVySBj84x//cMYHDhzojHfv3t0ZHzVqlDNuHXtWLyrr73xOTo4zbo0/Flu3bnXG16xZ44zX1NQ441ZPvNbIEZYwXIs0K8stXrxYpaWlWrp0qV577TUdPnxY48ePb/JH884779Sf//xnPf/881q8eLG2b9+ua665Ju4DBxBO5BEAPsghAI7XrLcg582b1+T/c+bMUUFBgVasWKFLLrlE1dXV+vWvf61nn31Wl19+uSRp9uzZGjp0qJYuXaovfOEL8Rs5gFAijwDwQQ4BcDyvz6Grq6slSfn5+ZKkFStW6PDhwxo3blx0mSFDhqhPnz5asmTJSbdRV1enmpqaJjcA7Qd5BIAPcgiAFhc0jY2NuuOOO3TRRRdp2LBhkqSKigplZGQoLy+vybKFhYWqqKg46XbKysqUm5sbvZWUlLR0SABChjwCwAc5BIDkUdCUlpZqzZo1eu6557wGMH36dFVXV0dv5eXlXtsDEB7kEQA+yCEApGb+huaoqVOn6pVXXtHrr7+u3r17R+8vKipSfX29qqqqmrwzUllZqaKiopNuKxKJKBKJtGQYAEKMPALABzkEwFHN+oQmCAJNnTpVc+fO1YIFC9S/f/8m8fPPP1/p6emaP39+9L5169Zpy5YtGj16dHxGDCDUyCMAfJBDAByvWZ/QlJaW6tlnn9XLL7+s7Ozs6HdRc3NzlZWVpdzcXH3729/WtGnTlJ+fr5ycHN12220aPXp0XGcVsebD9u0zE8u7NIcPH3bGrT4zVm8Cq3fCgQMHnHGrD8zNN9/sjJ/qe8ZHdevWzRmPpe/B+vXrnfE33njDGbf2sW88PT3dGfftRdQa87oHQZCUdV1aM4+4+ln5svKMdfzEg/UaWXErD33yySfOuHUOHP8bhuN16dLFGY+ll8/27dudcauf1dEflJ+Kb78q3/XjkUesZcLQY+JYqXItEgYLFy50xj/99FNnvE+fPs74oEGDnPGGhgZn3OpDY/WRsXJYVVWVMy7Z1yJWjrGux3yvJZLdMy8WqZBDmlXQPPHEE5KksWPHNrl/9uzZuvHGGyVJjz32mDp06KBrr722STMrAJDIIwD8kEMAHK9ZBU0s79hmZmZq1qxZmjVrVosHBaDtIo8A8EEOAXA8rz40AAAAAJBMFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAACh1axZzlqTq3+Eb38Ma07vWPpHZGVlOePW3OuWVatWOePHT1d5vE6dOjnj1tzvAwcOdMatect37NjhjEvSK6+84oyXl5c745mZmc64bx8a5nZv36z+H1aPlFhef9/+INZjWMe41U/LyhPWeW7to169ejnj1vhiGYNvzzDfPjG+ecLafiw9v9B2Wa+/1WdmxYoVzrjVg8V6fOtaqa6uzhnft2+fM15TU+OMWz39JGnLli3OuHU953st4tunxjfHxHKdkQp99SxkQgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaKVsHxoXa75ra75sq49NLH1ofHvhWGO05l5/9dVXnfHLL7/cGR86dKgzbvV22LRpkzP++uuvO+OStHnzZmc8Eol4xX3nfrfWt+aG9+1PgcTy7QHTGmOwWMeQdYxbfWasPjJW/MCBA874+vXrnfFYeqxYy1j7wLfXj+/fI98+MrEcQ6nQIwKJ4ZvHdu/e7YxbfVys7Vs9XPbv3++MV1dXe20/lms162+51WfG91rFtw+Nb46JJQeFIYfwCQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEVigba1qsJkHxaBAUS/NNnzFYTR0/+OADZ/zdd991xs8880xn3GoEtWHDBme8vr7eGZekLl26OOOJbnzpG2+N4wypy7chYzwew2oaZzWds45xq3GmxXf9WPg2vkz29n3zCHmm7YpH01QrbuUAq/nu4cOHveJWjrKuRawcE8s+tPaBda3h28Tb91rE929NW2nOyyc0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhFazCpqysjKNGDFC2dnZKigo0OTJk7Vu3bomy4wdO1ZpaWlNbt/97nfjOmgA4UUeAeCDHALgeM3qQ7N48WKVlpZqxIgROnLkiO655x6NHz9eH3zwgTp37hxd7qabbtKDDz4Y/X+nTp3iN+JWYM35Ldnzfltxq/eANW+5Fbf6U5SXl3utb7Hmjpf892Gi49Zr5Lv9VJj73fd1bon2kkfiwTqGrNfPWt+3T4z1+K1xfPmeI4nuI+P7+GHo/9Da2ksOieW1t46/WK5nEsnKQda1jNWnJh58+7z4ru97reHbE62t5JhmHenz5s1r8v85c+aooKBAK1as0CWXXBK9v1OnTioqKorPCAG0KeQRAD7IIQCO5/XWUnV1tSQpPz+/yf3PPPOMunfvrmHDhmn69Ok6cOCAz8MAaMPIIwB8kEMAtPizyMbGRt1xxx266KKLNGzYsOj9X//619W3b18VFxdr9erV+v73v69169bpxRdfPOl26urqVFdXF/1/TU1NS4cEIGTIIwB8kEMASB4FTWlpqdasWaM333yzyf0333xz9N/Dhw9Xz549dcUVV2jjxo0aOHDgCdspKyvTAw880NJhAAgx8ggAH+QQAFILv3I2depUvfLKK1q4cKF69+7tXHbUqFGSpA0bNpw0Pn36dFVXV0dv1o/VAbQN5BEAPsghAI5q1ic0QRDotttu09y5c7Vo0SL179/fXGfVqlWSpJ49e540HolEYpoRC0DbQB4B4IMcAuB4zSpoSktL9eyzz+rll19Wdna2KioqJEm5ubnKysrSxo0b9eyzz+qLX/yiunXrptWrV+vOO+/UJZdcorPPPjshTwBAuJBHAPgghwA4XrMKmieeeELSZw2rjjV79mzdeOONysjI0N///nfNnDlTtbW1Kikp0bXXXqsf/vCHcRtwWFjzklvzjlv9IXznbvftH2HF4zF/vu/c6r7bt9b33X4qzP3uGkOixheWPJKMHj2tPQbfHiqJ5tsnR0p8nxhfqZAHwiYsOaQ1+B7fvtcq1vpHjhxxxhPdyyoeOc73WsA3bmkvfWYszf7KmUtJSYkWL17sNSAAbRt5BIAPcgiA46X223MAAAAA4EBBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCq1nTNremIAhaPP94KvSP8GXNS56RkeGMW/vA6u/QGn1oLMnu8+I7t3t7mfsdOJVEn4OxPIbv3wPOY4SZ799R3+379txrDWG/1sBn+IQGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0Eq5aZuPTrF5+PBh722kskRPVci0zf5jaGhoSOrjJ9vRczAM59Pxjo7Zeg1j2UYypcIYfLTGOcC0zanr6PkXxuM4HjkkDHyvFSzW+qkwbbPv9YxvjmiN66kwizWPpFxBs2/fPknSiy++mOSRAJA+Oydzc3OTPYxmOZpHPvjggySPBECYc8jKlSuTPBIAkp1H0oIUe+uksbFR27dvV3Z2ttLS0lRTU6OSkhKVl5crJycn2cMLJfahv/a4D4Mg0L59+1RcXGx+WpZqyCPxxz701972ITkEx2If+muP+zDWPJJyn9B06NBBvXv3PuH+nJycdvPiJQr70F9724dhe1f1KPJI4rAP/bWnfUgOwfHYh/7a2z6MJY+E6y0TAAAAADgGBQ0AAACA0Er5giYSiWjGjBmKRCLJHkposQ/9sQ/DjdfPH/vQH/swvHjt/LEP/bEPTy3lJgUAAAAAgFil/Cc0AAAAAHAqFDQAAAAAQouCBgAAAEBoUdAAAAAACK2UL2hmzZqlfv36KTMzU6NGjdJbb72V7CGlrNdff11f/vKXVVxcrLS0NL300ktN4kEQ6L777lPPnj2VlZWlcePGaf369ckZbAoqKyvTiBEjlJ2drYKCAk2ePFnr1q1rssyhQ4dUWlqqbt26qUuXLrr22mtVWVmZpBEjFuSQ5iGP+CGPtE3kkdiRQ/yQQ1ompQuaP/zhD5o2bZpmzJihd955R+ecc44mTJignTt3JntoKam2tlbnnHOOZs2addL4T37yEz3++ON68skntWzZMnXu3FkTJkzQoUOHWnmkqWnx4sUqLS3V0qVL9dprr+nw4cMaP368amtro8vceeed+vOf/6znn39eixcv1vbt23XNNdckcdRwIYc0H3nED3mk7SGPNA85xA85pIWCFDZy5MigtLQ0+v+GhoaguLg4KCsrS+KowkFSMHfu3Oj/Gxsbg6KiouDhhx+O3ldVVRVEIpHg97//fRJGmPp27twZSAoWL14cBMFn+ys9PT14/vnno8t8+OGHgaRgyZIlyRomHMghfsgj/sgj4UceaTlyiD9ySGxS9hOa+vp6rVixQuPGjYve16FDB40bN05LlixJ4sjCadOmTaqoqGiyP3NzczVq1Cj25ylUV1dLkvLz8yVJK1as0OHDh5vswyFDhqhPnz7swxREDok/8kjzkUfCjTwSX+SQ5iOHxCZlC5rdu3eroaFBhYWFTe4vLCxURUVFkkYVXkf3GfszNo2Njbrjjjt00UUXadiwYZI+24cZGRnKy8trsiz7MDWRQ+KPPNI85JHwI4/EFzmkecghsTst2QMAUlFpaanWrFmjN998M9lDARBS5BEAPsghsUvZT2i6d++ujh07njBrQ2VlpYqKipI0qvA6us/Yn7apU6fqlVde0cKFC9W7d+/o/UVFRaqvr1dVVVWT5dmHqYkcEn/kkdiRR9oG8kh8kUNiRw5pnpQtaDIyMnT++edr/vz50fsaGxs1f/58jR49OokjC6f+/furqKioyf6sqanRsmXL2J//TxAEmjp1qubOnasFCxaof//+TeLnn3++0tPTm+zDdevWacuWLezDFEQOiT/yiI080raQR+KLHGIjh7RQkiclcHruueeCSCQSzJkzJ/jggw+Cm2++OcjLywsqKiqSPbSUtG/fvmDlypXBypUrA0nBo48+GqxcuTL45JNPgiAIgv/+7/8O8vLygpdffjlYvXp1cNVVVwX9+/cPDh48mOSRp4ZbbrklyM3NDRYtWhTs2LEjejtw4EB0me9+97tBnz59ggULFgTLly8PRo8eHYwePTqJo4YLOaT5yCN+yCNtD3mkecghfsghLZPSBU0QBMHPfvazoE+fPkFGRkYwcuTIYOnSpckeUspauHBhIOmE25QpU4Ig+Gy6xHvvvTcoLCwMIpFIcMUVVwTr1q1L7qBTyMn2naRg9uzZ0WUOHjwY3HrrrUHXrl2DTp06BVdffXWwY8eO5A0aJnJI85BH/JBH2ibySOzIIX7IIS2TFgRBkNjPgAAAAAAgMVL2NzQAAAAAYKGggSRp8+bNSktL009/+tO4bXPRokVKS0vTokWL4rZNAKmJHALAF3kELUVBE2Jz5sxRWlqali9fnuyhJMy2bdt0/fXXKy8vTzk5Obrqqqv08ccfx7z+P//5T40ZM0adOnVSUVGR/v3f/1379+9P4IiB8GjrOaRfv35KS0s76e3000+PaRvkEMCtreeRo/7whz9o9OjR6ty5s/Ly8nThhRdqwYIFMa1LHkk8GmsiZe3fv1+XXXaZqqurdc899yg9PV2PPfaYLr30Uq1atUrdunVzrr9q1SpdccUVGjp0qB599FFt3bpVP/3pT7V+/Xr99a9/baVnASBZZs6cecJFwyeffKIf/vCHGj9+vLk+OQSAJN1///168MEHdd111+nGG2/U4cOHtWbNGm3bts1clzzSOihokLJ+8YtfaP369Xrrrbc0YsQISdKkSZM0bNgwPfLII/rxj3/sXP+ee+5R165dtWjRIuXk5Ej67B3bm266Sf/3f/8X0wUNgPCaPHnyCff96Ec/kiR94xvfMNcnhwBYunSpHnzwQT3yyCO68847m70+eaR18JWzNq6+vl733Xefzj//fOXm5qpz5866+OKLtXDhwlOu89hjj6lv377KysrSpZdeqjVr1pywzNq1a3XdddcpPz9fmZmZuuCCC/SnP/3JHM+BAwe0du1a7d6921z2hRde0IgRI6LFjCQNGTJEV1xxhf74xz86162pqdFrr72mb37zm9EEIknf+ta31KVLF3N9AJ8Jcw45mWeffVb9+/fXhRde6FyOHALET5jzyMyZM1VUVKTbb79dQRA066ti5JHWQ0HTxtXU1OhXv/qVxo4dq4ceekj333+/du3apQkTJmjVqlUnLP/000/r8ccfV2lpqaZPn641a9bo8ssvV2VlZXSZ999/X1/4whf04Ycf6gc/+IEeeeQRde7cWZMnT9bcuXOd43nrrbc0dOhQ/fznP3cu19jYqNWrV+uCCy44ITZy5Eht3LhR+/btO+X67733no4cOXLC+hkZGTr33HO1cuVK5+MD+ExYc8jJrFy5Uh9++KG+/vWvm8uSQ4D4CXMemT9/vkaMGKHHH39cPXr0UHZ2tnr27BnTuuSR1sNXztq4rl27avPmzcrIyIjed9NNN2nIkCH62c9+pl//+tdNlt+wYYPWr1+vXr16SZImTpyoUaNG6aGHHtKjjz4qSbr99tvVp08fvf3224pEIpKkW2+9VWPGjNH3v/99XX311d7j3rt3r+rq6tSzZ88TYkfv2759uwYPHnzS9Xfs2NFk2ePXf+ONN7zHCLQHYc0hJ/PMM89Iiu3rZuQQIH7Cmkc+/fRT7d69W//4xz+0YMECzZgxQ3369NHs2bN12223KT09Xd/5zndOuT55pPXwCU0b17Fjx2gCaWxs1N69e6PvFrzzzjsnLD958uRoApE++zRk1KhR+stf/iLps0JjwYIFuv7667Vv3z7t3r1bu3fv1p49ezRhwgStX7/e+SO5sWPHKggC3X///c5xHzx4UJKiSepYmZmZTZZpyfqudQF8Lqw55HiNjY167rnndN5552no0KHm8uQQIH7CmkeOfr1sz549+tWvfqW77rpL119/vV599VWdeeaZ0d/knQp5pPVQ0LQDTz31lM4++2xlZmaqW7du6tGjh1599VVVV1efsOzJpjI944wztHnzZkmfvWsSBIHuvfde9ejRo8ltxowZkqSdO3d6jzkrK0uSVFdXd0Ls0KFDTZZpyfqudQE0FcYccrzFixdr27ZtMX06I5FDgHgLYx45ep6np6fruuuui97foUMHffWrX9XWrVu1ZcsWc33ySOLxlbM27ne/+51uvPFGTZ48Wf/xH/+hgoICdezYUWVlZdq4cWOzt9fY2ChJuuuuuzRhwoSTLjNo0CCvMUtSfn6+IpFI9OPaYx29r7i4+JTrH/1491Tru9YF8Lmw5pDjPfPMM+rQoYNuuOGGmJYnhwDxE9Y8cnSygby8PHXs2LFJrKCgQNJnX0vr06fPSdcnj7QeCpo27oUXXtCAAQP04osvKi0tLXr/0Xcwjrd+/foT7vvoo4/Ur18/SdKAAQMkffZuxbhx4+I/4P+nQ4cOGj58+EkbdS1btkwDBgxQdnb2KdcfNmyYTjvtNC1fvlzXX3999P76+nqtWrWqyX0ATi2sOeRYdXV1+t///V+NHTs25gsIcggQP2HNIx06dNC5556rt99+W/X19U1+A7R9+3ZJUo8ePU65Pnmk9fCVszbu6DsKQRBE71u2bJmWLFly0uVfeumlJt87feutt7Rs2TJNmjRJ0mfvSIwdO1b/8z//c9J3HHbt2uUcT3OmSrzuuuv09ttvNylq1q1bpwULFugrX/lKk2XXrl3b5GPf3NxcjRs3Tr/73e+azIb229/+Vvv37z9hfQAnF+YcctRf/vIXVVVVOb9uRg4BEifMeeSrX/2qGhoa9NRTT0XvO3TokJ555hmdeeaZTd4kIY8kD5/QtAG/+c1vNG/evBPuv/3223XllVfqxRdf1NVXX60vfelL2rRpk5588kmdeeaZJ51LfdCgQRozZoxuueUW1dXVaebMmerWrZvuvvvu6DKzZs3SmDFjNHz4cN10000aMGCAKisrtWTJEm3dulXvvvvuKcf61ltv6bLLLtOMGTPMH+Pdeuut+uUvf6kvfelLuuuuu5Senq5HH31UhYWF+t73vtdk2aFDh+rSSy/VokWLovf913/9ly688EJdeumluvnmm7V161Y98sgjGj9+vCZOnOh8bKA9aas55KhnnnlGkUhE11577SmXIYcAftpqHvnOd76jX/3qVyotLdVHH32kPn366Le//a0++eQT/fnPf26yLHkkiQKE1uzZswNJp7yVl5cHjY2NwY9//OOgb9++QSQSCc4777zglVdeCaZMmRL07ds3uq1NmzYFkoKHH344eOSRR4KSkpIgEokEF198cfDuu++e8NgbN24MvvWtbwVFRUVBenp60KtXr+DKK68MXnjhhegyCxcuDCQFCxcuPOG+GTNmxPQcy8vLg+uuuy7IyckJunTpElx55ZXB+vXrT1hOUnDppZeecP8bb7wRXHjhhUFmZmbQo0ePoLS0NKipqYnpsYG2rj3kkOrq6iAzMzO45pprnMuRQ4CWaQ95pLKyMpgyZUqQn58fRCKRYNSoUcG8efNOWI48kjxpQXDM538AAAAAECL8hgYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQSrnGmo2Njdq+fbuys7OVlpaW7OEA7VYQBNq3b5+Ki4vVoUO43vsgjwDJRw4B4CvmPJKoBjc///nPow2URo4cGSxbtiym9crLy50Nmrhx49a6t/Ly8kSlCaeW5pAgII9w45ZKt2TlkCDgWoQbt7Zys/JIQj6h+cMf/qBp06bpySef1KhRozRz5kxNmDBB69atU0FBgXPd7OxsSdKQIUPUsWPHRAwPQAwaGhq0du3a6DnZmnxyiPR5HgnjO8PxFNA32RvvzrfcsZ9yJEM8rkW6devWrnNIsvnmsHjkwIEDBzrjr732mjNeVVXljH/lK19xxj/++GNnvK3nqMbGRu3Zs8fMI2lBAv7ijRo1SiNGjNDPf/7z6GBKSkp022236Qc/+IFz3ZqaGuXm5uqss86ioAGSqKGhQe+//76qq6uVk5PTqo/tk0Okz/NI79692/XFCAWNv7Z+sZBIjY2N2rp1a1JyiBSfa5EePXq06xySbKlQ0Jx++unO+D/+8Q9n3CpovvSlLznjGzZscMbbeo5qbGzUrl27zDwS97O0vr5eK1as0Lhx4z5/kA4dNG7cOC1ZsuSE5evq6lRTU9PkBqD9am4OkcgjAJriWgRoX+Je0OzevVsNDQ0qLCxscn9hYaEqKipOWL6srEy5ubnRW0lJSbyHBCBEmptDJPIIgKa4FgHal6R/jjp9+nRVV1dHb+Xl5ckeEoCQIY8A8EEOAcIt7pMCdO/eXR07dlRlZWWT+ysrK1VUVHTC8pFIRJFIJN7DABBSzc0hEnkEQFNciwDtS9w/ocnIyND555+v+fPnR+9rbGzU/PnzNXr06Hg/HIA2hhwCwBd5BGhfEjJt87Rp0zRlyhRdcMEFGjlypGbOnKna2lr967/+ayIeDimosbEx2UNIOmbGaTlySGySPQNQGGZRS/YMQMl+/PaMPJJ4ic4hvttPT083l+nWrZszftFFFznjL730kjO+efNmZ/zAgQPOeKLzbFvJUQkpaL761a9q165duu+++1RRUaFzzz1X8+bNO+HHeQBwMuQQAL7II0D7kZCCRpKmTp2qqVOnJmrzANo4cggAX+QRoH3gOzEAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQSNssZ/KR6H5d4zIue7B4W1tzrvnOzx+M1pJdN+2adI9b0sxUVFV7bT3SPCEss20/0eZzo9RM9fsAl2X1kfOPW+ZGfn++MS1KPHj2c8V27djnjf//7353xqqoqr7h1LdEaOSQMeYirJQAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaNGHJkES3UfGd+523/Eluz9FLBI9N7vVIyYej2+9TvSpCTffPjOlpaXOeFZWljP+4osvOuP//Oc/nfHDhw87420hTyT6PE/046N9S3SfGetvlO/6DQ0NzninTp2c8a5duzrjkpSenu6MW/28TjvNfSldW1vrjB86dMgZT/R1QFvJIVwNAQAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCiz40J5HoHjJS4vvI+MaPHDnijCe6z00sfPszWOsnOy7Zz4E+Nakrlv4PkUjEGR8+fLgzfsUVVzjja9eudcbHjBnjjPfo0cMZX7VqlTO+adMmZ9y3/0I8JDpPdOzY0evxLbGsn+ieXEge32uJRF+LWH1krHhmZqYzbuWo+vp6Z1yy+2n59pGx1rfiVg6xXiPr/I5HP7BUyCFc7QAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0KKgAQAAABBaFDQAAAAAQivuBc3999+vtLS0JrchQ4bE+2EAtFHkEAC+yCNA+5KQxppnnXWW/v73v3/+IKelVv/O1mjWlujGl77NqqzGmb6NNbt27eqMS9LZZ5/tjB88eNAZf++995xxq9mVdVxazays9a3XMJbzwrd5aFgbb6Z6DmktXbp0ccatxpmffPKJM75jxw5n3MoDffr0ccbz8/OdcWt827Ztc8Yl+zz3bTpnxX3zhLW+pTXO4VRomtcSYc8j8Wh46Ns407qWSPS1iu+xF0tjTWuZXbt2OeMHDhxwxhOdg3wbZ4b1/D5eQs7u0047TUVFRYnYNIB2gBwCwBd5BGg/EvLWzvr161VcXKwBAwboG9/4hrZs2ZKIhwHQRpFDAPgijwDtR9w/oRk1apTmzJmjwYMHa8eOHXrggQd08cUXa82aNcrOzj5h+bq6OtXV1UX/X1NTE+8hAQiR5uYQiTwCoCmuRYD2Je4FzaRJk6L/PvvsszVq1Cj17dtXf/zjH/Xtb3/7hOXLysr0wAMPxHsYAEKquTlEIo8AaIprEaB9SfivCfPy8nTGGWdow4YNJ41Pnz5d1dXV0Vt5eXmihwQgRKwcIpFHALhxLQK0bQkvaPbv36+NGzeqZ8+eJ41HIhHl5OQ0uQHAUVYOkcgjANy4FgHatrgXNHfddZcWL16szZs365///KeuvvpqdezYUTfccEO8HwpAG0QOAeCLPAK0L3H/Dc3WrVt1ww03aM+ePerRo4fGjBmjpUuXqkePHvF+qFPy7TMTy9zv1tzpvnO7W/0frPUPHz7sFbfGX1JS4owXFxc745KUnp7ujEciEWe8d+/ezvgHH3zgjFvHiTU+i2+fmlikah8ZH6mQQ1qL1UPl9ddfd8atYygzM9Pr8a0fRtfW1jrjVp6x+lVZeVDy7xFh9aCwzjHrNfDtAxIPic4TqdjHor3kEd/jK9k98SydO3d2xq1j28pBkrR7925nvKqqyhm39pFvr6rWyBFtQdwLmueeey7emwTQjpBDAPgijwDtS9t7excAAABAu0FBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCK+7TNrcFscz5neg+M1bct8+MNS+6qyu7JPXp08cZz8rKcsYl+zlaPS42bdrkjPu+Rtb89la8NeaOtx4jFftDIHZ79+51xpctW+aMW/0NMjIynHErT1jrWz1erOM3Ht3a9+3b54x/+umnznhdXZ0zHo9+Ui6tcQ5buYw8khjx6FGU7D40vn1qrD4zubm5zrh17Fq9tiSpurraGbeup3yvBXyvFehT8xk+oQEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhFYo+9D4zvvvOy+7ZM+t7js3u28fmkgk4owXFxc741afmczMTGc8FlVVVc74unXrnHFrH/bq1csZt+a/z8vLc8atHhzWa2jNfR/rMmi7rGO8vLzcGbfykHUep6enO+PWOWBt3zoHrT46kpSfn++MW71srOewc+dOZ/zAgQPOuG+/K2v9WHrEWMtY8Zb2uaA/RuL59jhJdB8b69jq1q2bM26dH1avrMrKSmdcsv9WW2NAauBVAgAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEVij70PjynXc9Ftbc7L59aqy53a0+MiUlJc54VlaWM75v3z5n3OqPIUlbtmxxxrt27eqMDx8+3Bm35re3nqPVh6Znz57O+DnnnOOM79mzxxmXpL/+9a/O+MqVK81tIHUlu0dETU2NM27lIatPjdWHplOnTs54ly5dnPFYlrF6chUWFjrjVh6y+tTs3bvXGbdeo3j8vUr0cRZLLxw0XzyuRXy3kejrJSvHWL2oDh065IxbPfukxPVhQuviExoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGg1uw/N66+/rocfflgrVqzQjh07NHfuXE2ePDkaD4JAM2bM0C9/+UtVVVXpoosu0hNPPKHTTz89nuN28u3NEI/HSPTc7d27d3fGCwoKnHGrd4PVX+LTTz91xvfv3++MS1Lfvn2d8f79+zvjVg8La/56qw9N586dveLWPrzhhhuccUm68sornfELL7zQ3EaqCUMOiUUY+oP49sOy4gcPHnTGrR4RvutLUm1trTPuex5bvXSsXNyhg/t9w+rqamc8Hn+vEu1Ux2Ei+3e0lTySbInusWKdP9b517FjR2f8wIEDzR5Tqkl0H6f20ken2Z/Q1NbW6pxzztGsWbNOGv/JT36ixx9/XE8++aSWLVumzp07a8KECTH9YQLQ9pFDAPgijwA4VrM/oZk0aZImTZp00lgQBJo5c6Z++MMf6qqrrpIkPf300yosLNRLL72kr33ta36jBRB65BAAvsgjAI4V19/QbNq0SRUVFRo3blz0vtzcXI0aNUpLliw56Tp1dXWqqalpcgPQPrUkh0jkEQCf41oEaH/iWtBUVFRIkgoLC5vcX1hYGI0dr6ysTLm5udFbSUlJPIcEIERakkMk8giAz3EtArQ/SZ/lbPr06aquro7eysvLkz0kACFDHgHggxwChFtcC5qioiJJUmVlZZP7Kysro7HjRSIR5eTkNLkBaJ9akkMk8giAz3EtArQ/cS1o+vfvr6KiIs2fPz96X01NjZYtW6bRo0fH86EAtEHkEAC+yCNA+9PsWc7279+vDRs2RP+/adMmrVq1Svn5+erTp4/uuOMO/ehHP9Lpp5+u/v37695771VxcXGT+eFj0djYmPC5uVOZ1WOld+/ezni3bt2c8YyMDGc8Eok44/369XPGrR4zknTkyBFnvK6uzhm3et1Y/Sms+e179OjhjPfq1csZt8b/ySefOOOStHr1anOZsGmtHJJo1uu/bds2cxvJ7iPj2+fGytFW/PDhw8641aNFss9zK0/k5eU541ZPLytXW3ErD+3du9cZb6/CkEfaS/8PF+tawuoHZ+UQ6zqioaHBGZfsXlGJ7gWV7OMklscPw/V4swua5cuX67LLLov+f9q0aZKkKVOmaM6cObr77rtVW1urm2++WVVVVRozZozmzZtnNlcC0D6QQwD4Io8AOFazC5qxY8c6q7m0tDQ9+OCDevDBB70GBqBtIocA8EUeAXCspM9yBgAAAAAtRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWs2e5ay1dOjQ4ZRzg1tzglvzZVtzjscyb7nvnNynnebe9a6u6JLUvXt3r+1b+8D3+dXX15vL7Nu3zxnfs2ePM75lyxav7Vv7qGfPns74v/zLvzjjBw8edMZ37NjhjEvSPffcYy6DxEhPT3fGb731Vmf8vvvuMx/D6sPi26fGt89MssUyPivX+PazsvpcFBYWOuNWz6+uXbs641VVVc54LLk60T0kTrX9MPSuSCTr+fv2eYrHNhL9Gll9aKy/w9bzs85vq8+NZPeCsnpdWXnc2r4V983jYf87ECs+oQEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhFbK9qFJpGTPyy5JOTk5zvjZZ5/ttX3fecetedWt3gw1NTXOuCStWLHCGbf60FiPYfV/yMvLc8aHDBnijFv9L6z+Ee+//74zLkmZmZnOuNXrpr33gfBxxhlnOONdunRxxq3+CJLdRybsfWYS3UcnlmWsuJXLdu7c6Yxbudw6TvLz851xK89ZeUby/5tHHkldyX5trJ521t8w6/y0erRY/eIOHTrkjMcyhuzsbHMbPtv37ZXVGnk+Hj2TEo1PaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAodUu+9C0BmvudKv3gNVDxeoT49uboaGhwRm3+p+8+eabzrgkbdu2zVzG5bTT3IdvJBJxxs866yxnvFu3bs641YdmwYIFzviaNWucccnez9ZxZs0N7+oDYvUICTvrHHjvvfec8WnTpnltP5Zlkt1Hxuf4kfzzTDz2ocV6jlYfCytu5Smrx4XVZ8bqUSGFo4cEThTL65LsHGH1WerUqZPX9q0cYf0NtK4DJHsfWtuwevFYOcDKIdZ1QCr8HUmFHMMnNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0Gp2QfP666/ry1/+soqLi5WWlqaXXnqpSfzGG29UWlpak9vEiRPjNV4AIUcOAeCLPALgWM1urFlbW6tzzjlH//Zv/6ZrrrnmpMtMnDhRs2fPjv4/lsZGzWE1MbKavbVGg5/i4mJn/KKLLnLGrUZLVrMpXwcOHHDGraaRlZWV3mOwXmer8eXpp5/ujJeUlDjj1j7+8MMPnfG1a9c647E0xPM91q31kyEVckg81NXVOeOxNCZNdMMzK9dZcd/xWfvANx4L6zla50hWVpYzbjVBzszMdMYtvjkglm34Hieniifyb21byCOJPv9ieQxfic5hvk0jYxmf9bfeaoxpnV9W88/OnTs741YOqq2tdcbbi2YXNJMmTdKkSZOcy0QiERUVFbV4UADaLnIIAF/kEQDHSsjbt4sWLVJBQYEGDx6sW265RXv27DnlsnV1daqpqWlyA9C+NSeHSOQRACfiWgRoP+Je0EycOFFPP/205s+fr4ceekiLFy/WpEmTTvmRXllZmXJzc6M362tAANq25uYQiTwCoCmuRYD2pdlfObN87Wtfi/57+PDhOvvsszVw4EAtWrRIV1xxxQnLT58+XdOmTYv+v6amhkQCtGPNzSESeQRAU1yLAO1Lwn8xPGDAAHXv3l0bNmw4aTwSiSgnJ6fJDQCOsnKIRB4B4Ma1CNC2Jbyg2bp1q/bs2aOePXsm+qEAtEHkEAC+yCNA29bsr5zt37+/yTscmzZt0qpVq5Sfn6/8/Hw98MADuvbaa1VUVKSNGzfq7rvv1qBBgzRhwoS4DhxAOJFDAPgijwA4VrMLmuXLl+uyyy6L/v/od06nTJmiJ554QqtXr9ZTTz2lqqoqFRcXa/z48frP//zPVp3/PR7z9vs688wznXGrd8GRI0ec8fT0dK/1P/30U2f8vffec8atGWCsedclex90797dGR84cKAz3rt3b2fc2ocbN250xq0+NPX19c64Nbe9lJp9ZHylSg5JdH+DWPoftMZjuCSq/0hrxWNhnUPWeWj1iMjNzXXGrTxj9fyy+lXFso9S4XWIt1TJI8mW6NfGOn+sv+NWDxXfXjzW+lZPP8nu45Kdne2Md+rUyXwMFytHxON6yyUef6tSIUc0u6AZO3as84n97W9/8xoQgLaNHALAF3kEwLHa3tu/AAAAANoNChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFrNnrYZnykoKHDGe/Xq5Yxbc7NbfWQOHz7sjG/fvt0ZX7t2rTNeWVnpjFtz01u9GyR7Hw4YMMAZz8/Pd8at/hIVFRXOuNVnxpq73ppbPhXmbW/rgiBoca+WttCHJux9amLpw+S7jby8PGfcykNWHxqrR8S+ffuc8dboQ4P2y7cPjBW3/g43NDQ441afJuta6ODBg864JO3Zs8cZt55jZmam1/pW3Or1Y+1DS1vJD3xCAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAIrTbZh6axsTHhj3HJJZc444MHD3bGrT4xVh+arVu3OuPr1q1zxnft2uWMW/OiWz1gioqKnHHJ7kOTk5PjjB86dMgZLy8vd8a3bNnijCe6z0w8+kfE0qcDqcs6z6xcluhcl+g+NvE4fq0+L1afGKtnWHZ2ttfjW31k9u7d64xbeS4evXraSh8KxJ91bFg5yFrf6lNj9Yix/g7X1dU545LUrVs3Z9zqVWWdg9Y+ss5x6zn69qGJRRhyBFdDAAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQapN9aFrD7t27nXFr3nCL1bvgo48+csat3gadOnVyxrt06eKM9+vXzxm35nWX7LnXKyoqnHGrz4z1Gllzw2dkZDjjVv+JeMzbHoa539ur1nhtrD4wvn1sfPk+vrV+JBIxx9C9e3dn3OqZZfW7ss5z6zlu27bNGf/000+dcXrIIJl8j6/Dhw8749Y5Xlxc7IxbPVhqamqcccm+HrJ65Vjq6+ud8aqqKmfc2ofkiM/wCQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAACh1ayCpqysTCNGjFB2drYKCgo0efJkrVu3rskyhw4dUmlpqbp166YuXbro2muvVWVlZVwHDSC8yCMAfJBDAByvWZNrL168WKWlpRoxYoSOHDmie+65R+PHj9cHH3ygzp07S5LuvPNOvfrqq3r++eeVm5urqVOn6pprrtE//vGPhDyBZLESY4cOfh9+ZWZmOuOFhYVe8YKCAmfc6sFy5MgRZ/zAgQPOuCR9/PHHzrjVv8Hq1WO9BtZztOaet/pT+K4fizDOL9+aeSQtLa3F+8h3bv9U6FPj2wfGOkat9a08Zp2jVg8ZSerZs6czbp3nVs8wqweE1e9q+/btzrjVR8PaR2HMAb7ay7VILK+tdQ4mmm+PFitHWX1qrPMnlhxiXc9YrOdg9cKxelFZfHNEW8khzToS582b1+T/c+bMUUFBgVasWKFLLrlE1dXV+vWvf61nn31Wl19+uSRp9uzZGjp0qJYuXaovfOEL8Rs5gFAijwDwQQ4BcDyvjxGqq6slfV4Br1ixQocPH9a4ceOiywwZMkR9+vTRkiVLfB4KQBtFHgHggxwCoMWfFTY2NuqOO+7QRRddpGHDhkmSKioqlJGRoby8vCbLFhYWqqKi4qTbqaurU11dXfT/1kdzANoO8ggAH+QQAJLHJzSlpaVas2aNnnvuOa8BlJWVKTc3N3orKSnx2h6A8CCPAPBBDgEgtbCgmTp1ql555RUtXLhQvXv3jt5fVFSk+vp6VVVVNVm+srJSRUVFJ93W9OnTVV1dHb2Vl5e3ZEgAQoY8AsAHOQTAUc0qaIIg0NSpUzV37lwtWLBA/fv3bxI///zzlZ6ervnz50fvW7dunbZs2aLRo0efdJuRSEQ5OTlNbgDaLvIIAB/kEADHa9ZvaEpLS/Xss8/q5ZdfVnZ2dvS7qLm5ucrKylJubq6+/e1va9q0acrPz1dOTo5uu+02jR49mllFAEgijwDwQw4BcLxmFTRPPPGEJGns2LFN7p89e7ZuvPFGSdJjjz2mDh066Nprr1VdXZ0mTJigX/ziF3EZbKysObmtOcNj6SGza9cuZzwrK8sZt/o7HJ1L/1QGDRrkjNfX1zvj1j44dOiQM75nzx6vuCStX7/eGbeeg9U/wjfu20cmHv0jrMfwnT/eNcZE9TcISx7xnbs/ljzie4xYx4fVX8HKM506dXLGrR4RVh+aLl26OOOxHN++fTCsH38f/7Wl41m50noN6DPTfGHJIWHge3xZfyes49/KYVafGSseS48Z6zlY+2jv3r3OuJVDrOdg5bjWyCFhyEPN+ksQywVOZmamZs2apVmzZrV4UADaLvIIAB/kEADH82tnDwAAAABJREEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAELLbwL/duzTTz91xleuXOmMb9261Rnv1q2bM271j7CmtbR6wBxtVHYqO3fudMb379/vjEv2vOYZGRkJjSe7D00sfUrCMPd7W+Xbh8Y6/iT7GOvatau5DRfrGLX6xFj9tKx+Vr79E2Jh9ZmwemIdOHDAGbf6zPgeJ4mOx7oM2qZEv/a1tbXOuHX+FRUVOeNWjxbfnnqS/RysbVjrJzpPtkYOsaRCjuETGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaNGH5iRi6Y1gzSs+Z84cZzwSiTjjeXl5zrjV52Xv3r1e61vPz+pzY/WvkOz9bM3Nnui4bx+ZePSh8dUajxFW1rz51jFurT906FBzDL169fIag9VDpaamxhk/fPiwM271eLHGZ+WR+vp6Z7yurs4Zl+weEFbPMItvD4hE95AAfPgef1afGOv83LVrlzNu5bB9+/Y547H0ofE9h32vNax+YanQyyoMuNoBAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0GqXjTWtJklWMzjJboRkNZyzGspt377da/uWTp06JXT7sTRq8m32lOjGmBbfZljxQOPM5LFe/7/97W/mNtasWeOML1++3BnfuHGjM75t2zZnvKqqyhm3muL5Nu7cs2ePM241zZMS37gy0dtvK03tkByJPv6s49+6VrByiHWOW9djVtzK05L9HK1tWPFEX6u0Rg4JQ57iaggAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQalZBU1ZWphEjRig7O1sFBQWaPHmy1q1b12SZsWPHKi0trcntu9/9blwHDSC8yCMAfJBDAByvWX1oFi9erNLSUo0YMUJHjhzRPffco/Hjx+uDDz5Q586do8vddNNNevDBB6P/t3qehFGq9x6w5j235o6PpRePr0T3d/B9/ET3eGmvPWTCkkes48s6h66//nrzMa655hpnfOTIkc74zp07nfFDhw4541YfGqtPzO7du51xax9a/bjikSd9+0WFPd4WhSWHpALfPjOJlug+OPHoied7rZDoPjPkkM80q6CZN29ek//PmTNHBQUFWrFihS655JLo/Z06dVJRUVF8RgigTSGPAPBBDgFwPK/SvLq6WpKUn5/f5P5nnnlG3bt317BhwzR9+nQdOHDglNuoq6tTTU1NkxuA9oM8AsAHOQRAsz6hOVZjY6PuuOMOXXTRRRo2bFj0/q9//evq27eviouLtXr1an3/+9/XunXr9OKLL550O2VlZXrggQdaOgwAIUYeAeCDHAJAktIC6wuIp3DLLbfor3/9q95880317t37lMstWLBAV1xxhTZs2KCBAweeEK+rq1NdXV30/zU1NSopKdFZZ51lfvc5UVrj9yPWbm9oaEjo9vkNDb+hsTQ0NOj9999XdXW1cnJyEvIYic4jvXv3Ttp+HjVqlLmM9Ruas88+2xl/5ZVXnPFVq1Y54xUVFc54sn9DE8ufJ9/vvyf7++lt+fvvjY2N2rp1a6hzSI8ePVI6V/v+rfe9FrDi1rWMtb7v+PkNTbhziPTZMbJr1y4zj7ToE5qpU6fqlVde0euvv+5MINLnf9RPlUQikYgikUhLhgEgxMgjAHyQQwAc1ayCJggC3XbbbZo7d64WLVqk/v37m+scfYewZ8+eLRoggLaFPALABzkEwPGaVdCUlpbq2Wef1csvv6zs7Ozo1xVyc3OVlZWljRs36tlnn9UXv/hFdevWTatXr9add96pSy65xPzqBID2gTwCwAc5BMDxmvUbmlN9z2727Nm68cYbVV5erm9+85tas2aNamtrVVJSoquvvlo//OEPY/7+bE1NjXJzc5P6G5p48P0NSgt/2hS39a3xx+M7xb7f20z1732m8veuY5Go39C0Zh5J5m9ofM9ByX43OTs72xnfsGGDM26dQ1YfG4vvPojHOZ7s75cne/vJlKjf0LRmDkn139BYkv0bm0Q/fjwk+zcqyY6nuoT8hsY6sEpKSrR48eLmbBJAO0MeAeCDHALgeOF92wEAAABAu0dBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCq1nTNiN21rz11tztye5NEOZ591NFa/TyQdu2Y8cOr7jFOkatXmCt0SPCV7JzabLXR/uW6OPH9+9YW+hDk+rbby+4ogIAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0Um7a5qNT9DU0NCR5JIllTZeKti/Vp7w9eg6m+jhP5uiYk3mehWG/+e6fMDzHZE+Jmuz1k+no8RWG4+R4qZBDWoPvtMm+r20qHBvWGJi2OblizSMpV9Ds27dPkrR27dokjwSA9Nk5mZubm+xhNMvRPLJ9+/YkjwRAmHPInj17kjwSAJKdR9KCVCiPj9HY2Kjt27crOztbaWlpqqmpUUlJicrLy5WTk5Ps4YUS+9Bfe9yHQRBo3759Ki4uDl0TUPJI/LEP/bW3fUgOwbHYh/7a4z6MNY+k3Cc0HTp0UO/evU+4Pycnp928eInCPvTX3vZh2N5VPYo8kjjsQ3/taR+SQ3A89qG/9rYPY8kj4XrLBAAAAACOQUEDAAAAILRSvqCJRCKaMWOGIpFIsocSWuxDf+zDcOP188c+9Mc+DC9eO3/sQ3/sw1NLuUkBAAAAACBWKf8JDQAAAACcCgUNAAAAgNCioAEAAAAQWilf0MyaNUv9+vVTZmamRo0apbfeeivZQ0pZr7/+ur785S+ruLhYaWlpeumll5rEgyDQfffdp549eyorK0vjxo3T+vXrkzPYFFRWVqYRI0YoOztbBQUFmjx5statW9dkmUOHDqm0tFTdunVTly5ddO2116qysjJJI0YsyCHNQx7xQx5pm8gjsSOH+CGHtExKFzR/+MMfNG3aNM2YMUPvvPOOzjnnHE2YMEE7d+5M9tBSUm1trc455xzNmjXrpPGf/OQnevzxx/Xkk09q2bJl6ty5syZMmKBDhw618khT0+LFi1VaWqqlS5fqtdde0+HDhzV+/HjV1tZGl7nzzjv15z//Wc8//7wWL16s7du365prrkniqOFCDmk+8ogf8kjbQx5pHnKIH3JICwUpbOTIkUFpaWn0/w0NDUFxcXFQVlaWxFGFg6Rg7ty50f83NjYGRUVFwcMPPxy9r6qqKohEIsHvf//7JIww9e3cuTOQFCxevDgIgs/2V3p6evD8889Hl/nwww8DScGSJUuSNUw4kEP8kEf8kUfCjzzScuQQf+SQ2KTsJzT19fVasWKFxo0bF72vQ4cOGjdunJYsWZLEkYXTpk2bVFFR0WR/5ubmatSoUezPU6iurpYk5efnS5JWrFihw4cPN9mHQ4YMUZ8+fdiHKYgcEn/kkeYjj4QbeSS+yCHNRw6JTcoWNLt371ZDQ4MKCwub3F9YWKiKiookjSq8ju4z9mdsGhsbdccdd+iiiy7SsGHDJH22DzMyMpSXl9dkWfZhaiKHxB95pHnII+FHHokvckjzkENid1qyBwCkotLSUq1Zs0ZvvvlmsocCIKTIIwB8kENil7Kf0HTv3l0dO3Y8YdaGyspKFRUVJWlU4XV0n7E/bVOnTtUrr7yihQsXqnfv3tH7i4qKVF9fr6qqqibLsw9TEzkk/sgjsSOPtA3kkfgih8SOHNI8KVvQZGRk6Pzzz9f8+fOj9zU2Nmr+/PkaPXp0EkcWTv3791dRUVGT/VlTU6Nly5axP/+fIAg0depUzZ07VwsWLFD//v2bxM8//3ylp6c32Yfr1q3Tli1b2IcpiBwSf+QRG3mkbSGPxBc5xEYOaaEkT0rg9NxzzwWRSCSYM2dO8MEHHwQ333xzkJeXF1RUVCR7aClp3759wcqVK4OVK1cGkoJHH300WLlyZfDJJ58EQRAE//3f/x3k5eUFL7/8crB69ergqquuCvr37x8cPHgwySNPDbfcckuQm5sbLFq0KNixY0f0duDAgegy3/3ud4M+ffoECxYsCJYvXx6MHj06GD16dBJHDRdySPORR/yQR9oe8kjzkEP8kENaJqULmiAIgp/97GdBnz59goyMjGDkyJHB0qVLkz2klLVw4cJA0gm3KVOmBEHw2XSJ9957b1BYWBhEIpHgiiuuCNatW5fcQaeQk+07ScHs2bOjyxw8eDC49dZbg65duwadOnUKrr766mDHjh3JGzRM5JDmIY/4IY+0TeSR2JFD/JBDWiYtCIIgsZ8BAQAAAEBipOxvaAAAAADAQkEDAAAAILQoaCBJ2rx5s9LS0vTTn/40bttctGiR0tLStGjRorhtE0BqIocA8EUeQUtR0ITYnDlzlJaWpuXLlyd7KAlx//33Ky0t7YRbZmZmzNv45z//qTFjxqhTp04qKirSv//7v2v//v0JHDUQHm09h0jStm3bdP311ysvL085OTm66qqr9PHHH8e8PjkEcGvreaRfv34nvRZJS0vT6aefHtM2yCOJd1qyBwBYnnjiCXXp0iX6/44dO8a03qpVq3TFFVdo6NChevTRR7V161b99Kc/1fr16/XXv/41UcMFkCL279+vyy67TNXV1brnnnuUnp6uxx57TJdeeqlWrVqlbt26OdcnhwCYOXPmCcXHJ598oh/+8IcaP368uT55pHVQ0CDlXXfdderevXuz17vnnnvUtWtXLVq0SDk5OZI+e6flpptu0v/93//FlIgAhNcvfvELrV+/Xm+99ZZGjBghSZo0aZKGDRumRx55RD/+8Y+d65NDAEyePPmE+370ox9Jkr7xjW+Y65NHWgdfOWvj6uvrdd999+n8889Xbm6uOnfurIsvvlgLFy485TqPPfaY+vbtq6ysLF166aVas2bNCcusXbtW1113nfLz85WZmakLLrhAf/rTn8zxHDhwQGvXrtXu3btjfg5BEKimpkbNmWG8pqZGr732mr75zW9GE4gkfetb31KXLl30xz/+MeZtAe1ZmHPICy+8oBEjRkSLGUkaMmSIrrjiCjMHkEOA+AlzHjmZZ599Vv3799eFF17oXI480nooaNq4mpoa/epXv9LYsWP10EMP6f7779euXbs0YcIErVq16oTln376aT3++OMqLS3V9OnTtWbNGl1++eWqrKyMLvP+++/rC1/4gj788EP94Ac/0COPPKLOnTtr8uTJmjt3rnM8b731loYOHaqf//znMT+HAQMGKDc3V9nZ2frmN7/ZZCyn8t577+nIkSO64IILmtyfkZGhc889VytXroz58YH2LKw5pLGxUatXrz4hB0jSyJEjtXHjRu3bt++U65NDgPgJax45mZUrV+rDDz/U17/+dXNZ8kjr4StnbVzXrl21efNmZWRkRO+76aabNGTIEP3sZz/Tr3/96ybLb9iwQevXr1evXr0kSRMnTtSoUaP00EMP6dFHH5Uk3X777erTp4/efvttRSIRSdKtt96qMWPG6Pvf/76uvvrquI196tSpGj16tCKRiN544w3NmjVLb731lpYvX97k3Y7j7dixQ5LUs2fPE2I9e/bUG2+8EZcxAm1dWHPI3r17VVdXd8ocIEnbt2/X4MGDT7o+OQSIn7DmkZN55plnJMX2dTPySOvhE5o2rmPHjtEE0tjYqL1790bfLXjnnXdOWH7y5MnRBCJ99k7mqFGj9Je//EXSZxcJCxYs0PXXX699+/Zp9+7d2r17t/bs2aMJEyZo/fr12rZt2ynHM3bsWAVBoPvvv98c++23366f/exn+vrXv65rr71WM2fO1FNPPaX169frF7/4hXPdgwcPSlI0yR0rMzMzGgfgFtYcYuWAY5dpyfrkECB2Yc0jx2tsbNRzzz2n8847T0OHDjWXJ4+0HgqaduCpp57S2WefrczMTHXr1k09evTQq6++qurq6hOWPdkUhGeccYY2b94s6bN3TYIg0L333qsePXo0uc2YMUOStHPnzoQ9l69//esqKirS3//+d+dyWVlZkqS6uroTYocOHYrGAdjCmEOsHHDsMi1ZnxwCNE8Y88jxFi9erG3btsX06YxEHmlNfOWsjfvd736nG2+8UZMnT9Z//Md/qKCgQB07dlRZWZk2btzY7O01NjZKku666y5NmDDhpMsMGjTIa8yWkpIS7d2717nM0Y93j37ce6wdO3aouLg4IWMD2pqw5pD8/HxFIpFT5gBJzjxADgHiJ6x55HjPPPOMOnTooBtuuCGm5ckjrYeCpo174YUXNGDAAL344otKS0uL3n/0HYzjrV+//oT7PvroI/Xr10/SZz/Ql6T09HSNGzcu/gM2BEGgzZs367zzznMuN2zYMJ122mlavny5rr/++uj99fX1WrVqVZP7AJxaWHNIhw4dNHz48JM2+1u2bJkGDBig7OzsU65PDgHiJ6x55Fh1dXX63//9X40dOzbmQoQ80nr4ylkbd7QJ5bFTHi9btkxLliw56fIvvfRSk++dvvXWW1q2bJkmTZokSSooKNDYsWP1P//zPyd9x2HXrl3O8TRnqsSTbeuJJ57Qrl27NHHixCb3r127Vlu2bIn+Pzc3V+PGjdPvfve7JjMZ/fa3v9X+/fv1la98xXx8AOHOIdddd53efvvtJkXNunXrtGDBghNyADkESJww55Gj/vKXv6iqqsr5dTPySPLwCU0b8Jvf/Ebz5s074f7bb79dV155pV588UVdffXV+tKXvqRNmzbpySef1JlnnnlC51vps49ox4wZo1tuuUV1dXWaOXOmunXrprvvvju6zKxZszRmzBgNHz5cN910kwYMGKDKykotWbJEW7du1bvvvnvKsb711lu67LLLNGPGDPPHeH379tVXv/pVDR8+XJmZmXrzzTf13HPP6dxzz9V3vvOdJssOHTpUl156qRYtWhS977/+67904YUX6tJLL9XNN9+srVu36pFHHtH48eNPKIiA9qyt5pBbb71Vv/zlL/WlL31Jd911l9LT0/Xoo4+qsLBQ3/ve95osSw4B/LTVPHLUM888o0gkomuvvfaUy5BHkihAaM2ePTuQdMpbeXl50NjYGPz4xz8O+vbtG0QikeC8884LXnnllWDKlClB3759o9vatGlTICl4+OGHg0ceeSQoKSkJIpFIcPHFFwfvvvvuCY+9cePG4Fvf+lZQVFQUpKenB7169QquvPLK4IUXXogus3DhwkBSsHDhwhPumzFjhvn8/r//7/8LzjzzzCA7OztIT08PBg0aFHz/+98PampqTlhWUnDppZeecP8bb7wRXHjhhUFmZmbQo0ePoLS09KTrA+1RW88hQRAE5eXlwXXXXRfk5OQEXbp0Ca688spg/fr1JyxHDgFapj3kkerq6iAzMzO45pprnMuRR5InLQia0X4dAAAAAFIIv6EBAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtBLWWHPWrFl6+OGHVVFRoXPOOUc/+9nPNHLkSHO9xsZGbd++XdnZ2UpLS0vU8AAYgiDQvn37VFxcrA4dWv+9j5bmEIk8AqSCZOcQiWsRIOxiziOJaG7z3HPPBRkZGcFvfvOb4P333w9uuummIC8vL6isrDTXLS8vdzZo4saNW+veysvLE5EmnHxySBCQR7hxS6VbMnJIEHAtwo1bW7pZeSQhjTVHjRqlESNG6Oc//7mkz97pKCkp0W233aYf/OAHznWrq6uVl5enzMxM3hUBkigIAh06dEhVVVXKzc1t1cf2ySHS53nkzDPPVMeOHRM9XAAn0dDQoA8++CApOUSKz7VIv379kvbpEoDPztvNmzebeSTuXzmrr6/XihUrNH369Oh9HTp00Lhx47RkyRJz/aNFTFpaGgUNkAJa+zz0zSHS52Pu2LEjBQ2QZMn4Wx6va5EOHTpQ0AApwMojcS9odu/erYaGBhUWFja5v7CwUGvXrj1h+bq6OtXV1UX/X1NTE+8hAQiR5uYQiTwCoCmuRYD2JelvO5SVlSk3Nzd6KykpSfaQAIQMeQSAD3IIEG5xL2i6d++ujh07qrKyssn9lZWVKioqOmH56dOnq7q6OnorLy+P95AAhEhzc4hEHgHQFNciQPsS94ImIyND559/vubPnx+9r7GxUfPnz9fo0aNPWD4SiSgnJ6fJDUD71dwcIpFHADTFtQjQviSkD820adM0ZcoUXXDBBRo5cqRmzpyp2tpa/eu//msiHg4nkYDJ63AcJq1IHHIIAF/kEaD9SEhB89WvflW7du3Sfffdp4qKCp177rmaN2/eCT/OA4CTIYcA8EUeAdqPhPSh8VFTU6Pc3FxlZWXxDriHFHtZ26S2fnwGQaCDBw+quro6dF+/OJpHhg8fzrTNQJI0NDTovffeC3UOGTBgANM2A0nU2Niojz/+2MwjnKUAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQSMm0z/CV7lrJkP34qsGYx891HbX2WNISf7zHuu35jY6PX+vHgO8OVdZ6TB9CWkUPIIa2FT2gAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFH5oESXQfl2TP7d4W+tQkeu72RPexieUx0L5Zx5gV9+3hkOzHjwerh4Rvj4hEbx/wkexz2Pfx4/EYFt+/9b7nMDnkM3xCAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILfrQtFCy+7gkem72tjD3u+/6qTB3e7Lnt0dyJboHhLV+Q0OD1/YTHZcSn0cikYgzfuaZZzrj5557rjM+ePBgZ3z79u3O+EsvveSMS1JNTY25DNqmZOcQ3+1bcStHxZIffP+O+vaa8o37jt/afqzLJFvqjxAAAAAAToGCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWnFvrHn//ffrgQceaHLf4MGDtXbt2ng/VMKkQtNI32ZVqR5vDYluhuUrlqaXvg2zwth4sy3kkFj5NrZMdGNMa/0jR44kNG49viR17NjRGe/evbszPnDgQGe8V69ezviAAQOc8SFDhjjj559/vjPes2dPZ3zZsmXOuCRVVVU542Fomtdc7SWPJLuxpW8O8c0xvjlOSnxjSitHnXaa+1LcWt83Hss+SoVG45a4FzSSdNZZZ+nvf//75w9ivFgAcCxyCABf5BGg/UjI2X3aaaepqKgoEZsG0A6QQwD4Io8A7UdCPmdev369iouLNWDAAH3jG9/Qli1bTrlsXV2dampqmtwAtG/NySESeQTAibgWAdqPuBc0o0aN0pw5czRv3jw98cQT2rRpky6++GLt27fvpMuXlZUpNzc3eispKYn3kACESHNziEQeAdAU1yJA+5IWJPjX21VVVerbt68effRRffvb3z4hXldXp7q6uuj/a2pqVFJSoqysrKT9yIhJAZgUQLJ/6Oe7/Xgc377bcK0fBIEOHjyo6upq5eTkeD2ODyuHSKfOI8OHDzd/EJlMTArApAC+kwJ87Wtfc8Yl6cMPP3TGEzUpQENDg957772k5xCp5dciAwYMSOlJE5gUgEkBfOOxXEfEYxst1djYqI8//tjMIwn/hVxeXp7OOOMMbdiw4aTxSCSiSCSS6GEACCkrh0jkEQBuXIsAbVvC33bYv3+/Nm7caL7LBAAnQw4B4Is8ArRtcf+E5q677tKXv/xl9e3bV9u3b9eMGTPUsWNH3XDDDfF+qKTiK2Wp/5UzX6nwHFJhbvfW1lZySCzHT6K/Upbor3wdPnzYGa+vr/d6/MzMTGdckvr37++MW18Zy8vLc8YzMjKccddvuySZE1pYXzfZuXOnM27tY8n+2k0Yekw0V1vII7HkkER/pSzRXxmz4laOicfXVq196PuVsvT0dHMMPnzP31i+lh2GnnZxL2i2bt2qG264QXv27FGPHj00ZswYLV26VD169Ij3QwFog8ghAHyRR4D2Je4FzXPPPRfvTQJoR8ghAHyRR4D2JXWn7gAAAAAAAwUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaMV9lrMwaI3+Isnu85LseCx85073nffcWt+ae953bnpr/ViWsXoIWOu75sdvbGzUwYMHnevj1OJxjiS6x4NvD4i6ujqv7RcUFDjj55xzjjMuyezubvV5sfbxoUOHnHGfc0ySsrOznXGrEaTVJwfhZR0bklRUVOSMW+eHlad8e01Zf2crKiqccSuHWOOvra11xiWppqbGGbeeozUG6zn4Xov4XgekQs+9eOATGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaLXLPjSw5z235iW35jWPpTeC1afFt8+MxbfPjdVfwtoHsewjq4eANf98ly5dnPGzzjrrlLHDhw/rj3/8o3P99sw6R6zXJpZt+MatMfj2ocnPz3fGBw8e7Ix36tTJGbeO31hYfWY2b97sjG/cuNEZHzRokDM+cuRIZ9zq9fTOO+8441YPDaQu62/gxRdfbG5j4sSJzvju3bud8W3btjnjO3fudMY//fRTZ9zq42T1iYolj7rEch1hLXPgwAFn3NrHlZWVzrjV58baB22lj4wvPqEBAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWfWhCypq/PpYeJy7WvOvW41v9JaTEzz/vOze71WvH2gdWnxqrx4wk5eXlOeN9+vRxxkePHu2MT5s27ZSxmpoa+tAkWLL70Fhxqz+D1YOlqKjIGbdyQCx27NjhjH/00UfO+N69e51xq4+G9fjV1dXOuNUnZ9++fc641cMCqSsrK8sZv/TSS81tfPGLX3TGX3vtNWfc6jPjm2Os49P6O2mx/k5b8VhYf6tzc3Odces5Wr2ArNfAt2dfonv+tRY+oQEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhFazmwC8/vrrevjhh7VixQrt2LFDc+fO1eTJk6PxIAg0Y8YM/fKXv1RVVZUuuugiPfHEEzr99NPjOe42z5o73Zq/3pr33JrX3Oq9EI+53Y8cOeKMHz582Bm35r+3+jv49uDwnf/e6mMjSXv27HHGKysrnfFNmzY54y+//PIpY9br01LkkPixjmHrPO/Ro4czXlxc7Iz36tXLGc/MzHTGd+3a5RWXpPXr1zvjtbW1zriVJ6zztEuXLs54dna2M2716fjHP/7hjFt9cCS7308Y+1C0hTxinZ9WDyPJ/jtp5XGr51xVVZUzbvWxsc7hnJwcZ9w6dq1rnVh64ll/q33/Fvbt29cZr6urc8at64DWEIYc0eyr0traWp1zzjmaNWvWSeM/+clP9Pjjj+vJJ5/UsmXL1LlzZ02YMMG8QAbQPpBDAPgijwA4VrM/oZk0aZImTZp00lgQBJo5c6Z++MMf6qqrrpIkPf300yosLNRLL72kr33ta36jBRB65BAAvsgjAI4V19/QbNq0SRUVFRo3blz0vtzcXI0aNUpLliyJ50MBaIPIIQB8kUeA9qfZn9C4VFRUSJIKCwub3F9YWBiNHa+urq7J9wdramriOSQAIdKSHCKRRwB8jmsRoP1J+ixnZWVlys3Njd5KSkqSPSQAIUMeAeCDHAKEW1wLmqKiIkknzrxUWVkZjR1v+vTpqq6ujt7Ky8vjOSQAIdKSHCKRRwB8jmsRoP2Ja0HTv39/FRUVaf78+dH7ampqtGzZMo0ePfqk60QiEeXk5DS5AWifWpJDJPIIgM9xLQK0P83+Dc3+/fu1YcOG6P83bdqkVatWKT8/X3369NEdd9yhH/3oRzr99NPVv39/3XvvvSouLm4yPzzsec8jkYgznpGR4Yxb89tbvQ8sVu8Gq/dDLNvw7bFh8e0z49vHxnr+kj3/vTV/vdXHwPUcfPfvqZBDWo/Vo6FPnz7OuOtTMcnuM2P1cLGO7w8//NAZj2UbvnnEt2eXNT7rNbL6cLRXbSGPWMfOP//5T3MbVi+p9957zxlfsWKFM/7xxx8741afGutawPo7a/0dtc5P61pKsvttWXnQt+ec1Svn008/dcbD0COmNTQ7Uy5fvlyXXXZZ9P/Tpk2TJE2ZMkVz5szR3XffrdraWt18882qqqrSmDFjNG/ePPMPH4D2gRwCwBd5BMCxml3QjB071lkRp6Wl6cEHH9SDDz7oNTAAbRM5BIAv8giAYyV9ljMAAAAAaCkKGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNBigvsEseZGt3oLWHOnW/OO+/YvsbZv9XawHl/y7/8QdrE8P98eGtb6PtuGm9WbQPLvZdS5c2dnvEuXLs641cPCykPW+N99911n/Ng+Iidz+PBhZ1yyc00sucglKyvLGe/Zs6cznp+f74xbfTwOHDjgjCO8rGNz+fLl5jbKy8udcasXWWVlpTNeU1PjjFs97ax+a1aOs3q4WKxeP5K0b98+r23069fPGbeeg9WLij4zseETGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaNGH5iRi6b+R6B4dVn8Hq8eF1ecmJyfHGbfmlvftcyPZ89P7Yu52JJJ1fMUjR1iPcd555znjVp8Zq8+LdY5aPSw+/vhjZ9zKE7H0kLFylZVLrVxp9YjIzc11xq0eFOvXr3fGt27d6oz79JpCclnn9/79+81tWH1mrHPIygG+ecx6jr551MpRsVxnWI9x8OBBr8ewcoiVgxJ9LdNWes7xCQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILToQ5Mk1tzvVm+BSCTiFbfmNc/MzHTGfccn2f0jamtrnfFEz48PpDqrv0Hnzp2d8ZqaGmfc6nFh9aGxznErbuVJye6zYfXssvrEWD27unfv7oxbPSwOHDjgjMejX5dvXzPgVHz/jlp/x61j1zr/Y+llZT2H7Oxscxs+Et3TLB59ZqxtpML1FFkMAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIQWBQ0AAACA0KKgAQAAABBaFDQAAAAAQqvZjTVff/11Pfzww1qxYoV27NihuXPnavLkydH4jTfeqKeeeqrJOhMmTNC8efO8B9uWWE2KrIZyVrMoq2Gd1UzOamZljT8jI8MZj2UZq9lbXV2dMx5LQy20PnLI56yGbcOHD3fG+/fv74xbTR2t5rXvvPOOM75jxw5n3MpDVp6x4rGwmvwOGjTIGT/77LOdcSuPffjhh874mjVrnHErj512Wvvsj00eiU2yG1D7No30bbwZj6aSXbp0ccat6ylrDNY5TmPc2DR7L9TW1uqcc87RrFmzTrnMxIkTtWPHjujt97//vdcgAbQd5BAAvsgjAI7V7Ld2Jk2apEmTJjmXiUQiKioqavGgALRd5BAAvsgjAI6VkM+pFi1apIKCAg0ePFi33HKL9uzZc8pl6+rqVFNT0+QGoH1rTg6RyCMATsS1CNB+xL2gmThxop5++mnNnz9fDz30kBYvXqxJkyapoaHhpMuXlZUpNzc3eispKYn3kACESHNziEQeAdAU1yJA+xL3XxN+7Wtfi/57+PDhOvvsszVw4EAtWrRIV1xxxQnLT58+XdOmTYv+v6amhkQCtGPNzSESeQRAU1yLAO1LwqdGGDBggLp3764NGzacNB6JRJSTk9PkBgBHWTlEIo8AcONaBGjbEl7QbN26VXv27FHPnj0T/VAA2iByCABf5BGgbWv2V87279/f5B2OTZs2adWqVcrPz1d+fr4eeOABXXvttSoqKtLGjRt19913a9CgQZowYUJcB97eWfOSW/0fLL5zt1t9dCQpKyvLGU9PT3fGrbnfrR4b8ehxgeYjh3zOOsZ79OjhjFvvIvv2s7J6PVl5wuqPYJ3DsfRXsLZRUFDgjI8cOdIZt14j68fj5eXlzvihQ4ec8dboM2O9jr69SBKBPJIafHugWH+HrWPPevxYrmWsczw3N9drDK7ff0rSvn37vNZPxfMzGZqdKZcvX67LLrss+v+j3zmdMmWKnnjiCa1evVpPPfWUqqqqVFxcrPHjx+s///M/zeZmANoHcggAX+QRAMdqdkEzduxYZ8X7t7/9zWtAANo2cggAX+QRAMdK+G9oAAAAACBRKGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCK/ET3CMhrHnHffvIWOtb8Vh6vPj2X7DmjrfiVq8e332I9i0vL89cZuzYsc748OHDnfGdO3c647t373bG161b54wfOHDAGfftI5ORkeGMDxw40BmXpC5dujjj/fr1c8at83zr1q3OuPUaVFdXO+OAj0T/nbKuNXz7xPg+fjzWt3pVWc/BegyrH5jVM8/i+xq1FXxCAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILfrQJIjvvN++c8v79qlpjbnfrT4zqT53OnO/wyU7O9tc5vTTT3fGMzMznXGr35PVA8XqU2Nt3+oBU1hY6IwPGjTIGe/UqZMzLklHjhxxxq3nYPXaWblypTNu9bOyeu1YedDqgeHb5wOpKx49ZhJ9LeH7d846fq3zNx7js/ppWWPwPQetnnxcS8SGTAgAAAAgtChoAAAAAIQWBQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACK022YfGd971WNa3lrH6M1jzmtfV1TnjDQ0Nzrgv394HsczLHolEvLZx+PBhZ9zqD5HoXjwW5pYPN+v4icc5YJ3nVh7p16+fM271udm/f78zbvVQ8c2DVo8ZSaqtrXXGq6qqnHGrF4/VI8K3j4zVA6M1+l2Ri9ovK48l+nrKygG+44tHLx8rD1vnsLW+db3nm2PaS888PqEBAAAAEFoUNAAAAABCi4IGAAAAQGhR0AAAAAAILQoaAAAAAKFFQQMAAAAgtChoAAAAAIRWs/rQlJWV6cUXX9TatWuVlZWlCy+8UA899JAGDx4cXebQoUP63ve+p+eee051dXWaMGGCfvGLX6iwsDDug2+peMxLbrHmBU9PT3fGfec1932O1rzk1vhi4duD4sCBA17bD8Pc62EYY3O1lTxi6dWrl7nMdddd54y/+uqrzrhvL6WcnBxn3OpTY52jvuew1WNGktauXeuMb9682RnPyMjwilu5MNE9ImLJEbH0RAqT9pJD4iHRf0Osv7NW3LfXVjyuhax+WdY+tM4vK4/Rqyo+mpXlFi9erNLSUi1dulSvvfaaDh8+rPHjxzd5se688079+c9/1vPPP6/Fixdr+/btuuaaa+I+cADhRB4B4IMcAuB4zfqEZt68eU3+P2fOHBUUFGjFihW65JJLVF1drV//+td69tlndfnll0uSZs+eraFDh2rp0qX6whe+EL+RAwgl8ggAH+QQAMfz+hy6urpakpSfny9JWrFihQ4fPqxx48ZFlxkyZIj69OmjJUuW+DwUgDaKPALABzkEQLM+oTlWY2Oj7rjjDl100UUaNmyYJKmiokIZGRnKy8trsmxhYaEqKipOup26ujrV1dVF/19TU9PSIQEIGfIIAB/kEACSxyc0paWlWrNmjZ577jmvAZSVlSk3Nzd6Kykp8doegPAgjwDwQQ4BILWwoJk6dapeeeUVLVy4UL17947eX1RUpPr6elVVVTVZvrKyUkVFRSfd1vTp01VdXR29lZeXt2RIAEKGPALABzkEwFHNKmiCINDUqVM1d+5cLViwQP37928SP//885Wenq758+dH71u3bp22bNmi0aNHn3SbkUhEOTk5TW4A2i7yCAAf5BAAx2vWb2hKS0v17LPP6uWXX1Z2dnb0u6i5ubnKyspSbm6uvv3tb2vatGnKz89XTk6ObrvtNo0ePTqlZhVpjfm0Dx065Ixb/R18exv49qHx7RFjzQ0vSYcPH/Z6DN8eHJbW6A/RHrWVPGL59NNPzWU+/vhjZ7xTp07OeCQSccatY9A6Tw8ePOgVt87xdevWOeMbN250xmN5DKvnl2/8tNPcf0Z94/Ho+dXWclF7ySGtIdl9aqwc5Bu3rrUkux+XlQOsPGxdD1o5wLdPjbV+W+lT1ayC5oknnpAkjR07tsn9s2fP1o033ihJeuyxx9ShQwdde+21TZpZAYBEHgHghxwC4HjNKmhiedc/MzNTs2bN0qxZs1o8KABtF3kEgA9yCIDjtY3PmQAAAAC0SxQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAodWsWc4Qu/r6emfcmpvdd15w3z401tzuVu+HWPrQWKz58a19lOp9YuLx+Ml+Du2Zte/Xrl1rbuP73/++Mz5lyhRn3OpvUFNT44zv2rXLGV+/fr0zfrT/x6lYvaSsPjZWnpSkjIyMpMZ9+9T45rFY/laQJ8IpltfN9/hIdDzR/dysx+/SpYszLtl9ZKx+YFYfG+t6zOoj49tnxhKP4ywV8AkNAAAAgNCioAEAAAAQWhQ0AAAAAEKLggYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC02mQfGmu+bN8eLfFg9XGxxpjsuCUe85q39Tiwbt06Z/wHP/iBM271ebHiVh6y4tb2fftRZWVlmctYfV58+8T4xq0eEonut4W2zTo+fHugJJp1/Frjs56f1SNGkvLz853xHj16eMU//vhjZ3zPnj3OuO85nuzXuLXwCQ0AAACA0KKgAQAAABBaFDQAAAAAQouCBgAAAEBoUdAAAAAACC0KGgAAAAChRUEDAAAAILQoaAAAAACEVptsrGmhEVnim4+GobGmJRUaZ3Kshlcsr51vwzOraaTVdM+KW9u3GmemQh6xGl/67qNkN84kR7Rd8XhtfY9P3+PXenwrx/jmkK5du5rLDBw40BkfNGiQM96rVy9nfOPGjc745s2bnXGrgbHv35G2kkP4hAYAAABAaFHQAAAAAAgtChoAAAAAoUVBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFrN6kNTVlamF198UWvXrlVWVpYuvPBCPfTQQxo8eHB0mbFjx2rx4sVN1vvOd76jJ598Mj4jbgWp0F/Emnvdd2523/XjIdX7xCT6OGgrc783V3vJI7GwjgGrv4B1Hvv2kGhsbPR6/Nbgex759ulI9vbbI3LI53yPHysH+K7vm8N8c0yXLl3MZc477zxnfNKkSc54Tk6OM/6Xv/zFGffdh5b2kmOadSQvXrxYpaWlWrp0qV577TUdPnxY48ePV21tbZPlbrrpJu3YsSN6+8lPfhLXQQMIL/IIAB/kEADHa9YnNPPmzWvy/zlz5qigoEArVqzQJZdcEr2/U6dOKioqis8IAbQp5BEAPsghAI7n9VljdXW1JCk/P7/J/c8884y6d++uYcOGafr06Tpw4MApt1FXV6eampomNwDtB3kEgA9yCIBmfUJzrMbGRt1xxx266KKLNGzYsOj9X//619W3b18VFxdr9erV+v73v69169bpxRdfPOl2ysrK9MADD7R0GABCjDwCwAc5BIAkpQUt/MXVLbfcor/+9a9688031bt371Mut2DBAl1xxRXasGGDBg4ceEK8rq5OdXV10f/X1NSopKREWVlZKf1DpmT/aJ9JAZgUINGCINDBgwdVXV1t/uixpRKdR4YPH+79g8pk8s0D1o/6feNtIY8wKUDiNDQ06L333gt1DhkwYID3D+eTKdE5xHf7ib7WGTRokLnMd77zHWfcd1KA2267zRl/7bXXnPGGhgZn3BL2HNPY2KiPP/7YzCMt+oRm6tSpeuWVV/T66687E4gkjRo1SpJOmUQikYgikUhLhgEgxMgjAHyQQwAc1ayCJggC3XbbbZo7d64WLVqk/v37m+usWrVKktSzZ88WDRBA20IeAeCDHALgeM0qaEpLS/Xss8/q5ZdfVnZ2tioqKiRJubm5ysrK0saNG/Xss8/qi1/8orp166bVq1frzjvv1CWXXKKzzz47IU8gWZL9EZ7v46fCV0UsfGWsbSKPxC7RPSZ8+9C0BdY+SPU81B6RQ2KX7D41yc4hR44cMZdxTRYhSTt37nTGjxbLp/Lxxx8749ZXysghsWlWQfPEE09I+qxh1bFmz56tG2+8URkZGfr73/+umTNnqra2ViUlJbr22mv1wx/+MG4DBhBu5BEAPsghAI7X7K+cuZSUlJzQmRcAjkUeAeCDHALgeOGdugMAAABAu0dBAwAAACC0KGgAAAAAhBYFDQAAAIDQoqABAAAAEFrNmuUMsbPmDbdmaWkP846n+nNM9fEB1jGa7B4U7aHfFRBmqZ5DLFaO2bFjh7mNH//4x15joFdVauATGgAAAAChRUEDAAAAILQoaAAAAACEFgUNAAAAgNCioAEAAAAQWhQ0AAAAAEIr5aZtPjoFXxim+/SR7OeX7MdH6gvzuXh0zA0NDUkeSfsWhmOHKVMT5+j5F4bj4HhHx9zY2JjkkcAlFY4tckhiHT0Hrdc65Qqaffv2SZIOHTqU5JEAkD47J3Nzc5M9jGY5mkc++OCDJI8EQJhzyObNm5M7EACS7DySFqRCeXuMxsZGbd++XdnZ2UpLS1NNTY1KSkpUXl6unJycZA8vlNiH/trjPgyCQPv27VNxcXHCm6PFG3kk/tiH/trbPiSH4FjsQ3/tcR/GmkdS7hOaDh06qHfv3ifcn5OT025evERhH/prb/swbO+qHkUeSRz2ob/2tA/JITge+9Bfe9uHseSRcL1lAgAAAADHoKABAAAAEFopX9BEIhHNmDFDkUgk2UMJLfahP/ZhuPH6+WMf+mMfhhevnT/2oT/24aml3KQAAAAAABCrlP+EBgAAAABOhYIGAAAAQGhR0AAAAAAILQoaAAAAAKGV8gXNrFmz1K9fP2VmZmrUqFF66623kj2klPX666/ry1/+soqLi5WWlqaXXnqpSTwIAt13333q2bOnsrKyNG7cOK1fvz45g01BZWVlGjFihLKzs1VQUKDJkydr3bp1TZY5dOiQSktL1a1bN3Xp0kXXXnutKisrkzRixIIc0jzkET/kkbaJPBI7cogfckjLpHRB84c//EHTpk3TjBkz9M477+icc87RhAkTtHPnzmQPLSXV1tbqnHPO0axZs04a/8lPfqLHH39cTz75pJYtW6bOnTtrwoQJOnToUCuPNDUtXrxYpaWlWrp0qV577TUdPnxY48ePV21tbXSZO++8U3/+85/1/PPPa/Hixdq+fbuuueaaJI4aLuSQ5iOP+CGPtD3kkeYhh/ghh7RQkMJGjhwZlJaWRv/f0NAQFBcXB2VlZUkcVThICubOnRv9f2NjY1BUVBQ8/PDD0fuqqqqCSCQS/P73v0/CCFPfzp07A0nB4sWLgyD4bH+lp6cHzz//fHSZDz/8MJAULFmyJFnDhAM5xA95xB95JPzIIy1HDvFHDolNyn5CU19frxUrVmjcuHHR+zp06KBx48ZpyZIlSRxZOG3atEkVFRVN9mdubq5GjRrF/jyF6upqSVJ+fr4kacWKFTp8+HCTfThkyBD16dOHfZiCyCHxRx5pPvJIuJFH4osc0nzkkNikbEGze/duNTQ0qLCwsMn9hYWFqqioSNKowuvoPmN/xqaxsVF33HGHLrroIg0bNkzSZ/swIyNDeXl5TZZlH6Ymckj8kUeahzwSfuSR+CKHNA85JHanJXsAQCoqLS3VmjVr9OabbyZ7KABCijwCwAc5JHYp+wlN9+7d1bFjxxNmbaisrFRRUVGSRhVeR/cZ+9M2depUvfLKK1q4cKF69+4dvb+oqEj19fWqqqpqsjz7MDWRQ+KPPBI78kjbQB6JL3JI7MghzZOyBU1GRobOP/98zZ8/P3pfY2Oj5s+fr9GjRydxZOHUv39/FRUVNdmfNTU1WrZsGfvz/wmCQFOnTtXcuXO1YMEC9e/fv0n8/PPPV3p6epN9uG7dOm3ZsoV9mILIIfFHHrGRR9oW8kh8kUNs5JAWSvKkBE7PPfdcEIlEgjlz5gQffPBBcPPNNwd5eXlBRUVFsoeWkvbt2xesXLkyWLlyZSApePTRR4OVK1cGn3zySRAEQfDf//3fQV5eXvDyyy8Hq1evDq666qqgf//+wcGDB5M88tRwyy23BLm5ucGiRYuCHTt2RG8HDhyILvPd73436NOnT7BgwYJg+fLlwejRo4PRo0cncdRwIYc0H3nED3mk7SGPNA85xA85pGVSuqAJgiD42c9+FvTp0yfIyMgIRo4cGSxdujTZQ0pZCxcuDCSdcJsyZUoQBJ9Nl3jvvfcGhYWFQSQSCa644opg3bp1yR10CjnZvpMUzJ49O7rMwYMHg1tvvTXo2rVr0KlTp+Dqq68OduzYkbxBw0QOaR7yiB/ySNtEHokdOcQPOaRl0oIgCBL7GRAAAAAAJEbK/obm/2+/jmkAAAAYBvl3PQm7m4ALAAAAHqEBAACyhAYAAMgSGgAAIEtoAACALKEBAACyhAYAAMgSGgAAIEtoAACALKEBAACyhAYAAMgSGgAAIGuEDWi0DVyUrQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1000x1000 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Attempting to use Keras's image augmentation\n",
    "import keras.preprocessing.image\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "print(format(X_train.shape))\n",
    "datagen = ImageDataGenerator(\n",
    "        rotation_range=25,  \n",
    "        zoom_range = 0.10,  \n",
    "        )\n",
    "datagen.fit(X_train)\n",
    "print(X_train.shape)\n",
    "augmented_images, augmented_labels = next(datagen.flow(X_train, y_train, batch_size=9)) #Aumgent 9 images in random from training set\n",
    "print(augmented_images.shape)\n",
    "fig, axs = plt.subplots(3, 3, figsize=(10, 10))\n",
    "for i, ax in enumerate(axs.flatten()):\n",
    "    ax.imshow(augmented_images[i].reshape(28, 28), cmap=\"gray\")\n",
    "    ax.set_title(f\"Label: {augmented_labels[i]}\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec352ea9",
   "metadata": {},
   "source": [
    "We can now apply CNN model to train and evaluate our dataset. We use multiple layers shown below, and use accuracy and MAE to evaluate the model. Overall, CNN performs well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f560a990",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " conv2d (Conv2D)             (None, 26, 26, 32)        320       \n",
      "                                                                 \n",
      " max_pooling2d (MaxPooling2D  (None, 13, 13, 32)       0         \n",
      " )                                                               \n",
      "                                                                 \n",
      " conv2d_1 (Conv2D)           (None, 11, 11, 64)        18496     \n",
      "                                                                 \n",
      " max_pooling2d_1 (MaxPooling  (None, 5, 5, 64)         0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " conv2d_2 (Conv2D)           (None, 3, 3, 64)          36928     \n",
      "                                                                 \n",
      " flatten (Flatten)           (None, 576)               0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 64)                36928     \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 10)                650       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 93,322\n",
      "Trainable params: 93,322\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "\n",
      "\n",
      "TRAINING PHASE:\n",
      "Epoch 1/4\n",
      "875/875 [==============================] - 15s 17ms/step - loss: 0.1640 - accuracy: 0.9506 - mae: 0.0162 - val_loss: 0.0608 - val_accuracy: 0.9812 - val_mae: 0.0056\n",
      "Epoch 2/4\n",
      "875/875 [==============================] - 15s 17ms/step - loss: 0.0457 - accuracy: 0.9856 - mae: 0.0046 - val_loss: 0.0600 - val_accuracy: 0.9830 - val_mae: 0.0048\n",
      "Epoch 3/4\n",
      "875/875 [==============================] - 15s 17ms/step - loss: 0.0327 - accuracy: 0.9889 - mae: 0.0033 - val_loss: 0.0461 - val_accuracy: 0.9871 - val_mae: 0.0039\n",
      "Epoch 4/4\n",
      "875/875 [==============================] - 15s 17ms/step - loss: 0.0262 - accuracy: 0.9915 - mae: 0.0026 - val_loss: 0.0450 - val_accuracy: 0.9866 - val_mae: 0.0036\n",
      "\n",
      "TESTING PHASE:\n",
      "438/438 [==============================] - 1s 2ms/step\n",
      "\n",
      "Evaluation metrics from testing phase:\n",
      "Accuracy: 0.9866\n",
      "Macro-Precision: 0.9866\n",
      "Macro-Recall: 0.9865\n",
      "Macro-F1-Score: 0.9865\n",
      "Micro-Precision: 0.9866\n",
      "Micro-Recall: 0.9866\n",
      "Micro-F1-Score: 0.9866\n",
      "Mean Absolute Error: 0.0476\n"
     ]
    }
   ],
   "source": [
    "from keras.utils import to_categorical\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense\n",
    "from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, mean_absolute_error\n",
    "\n",
    "# One-hot encode the labels\n",
    "y_train_onehot = to_categorical(y_train)\n",
    "y_test_onehot = to_categorical(y_test)\n",
    "\n",
    "# Define the model architecture\n",
    "model = Sequential()\n",
    "\n",
    "#Adds a 2D convolutional layer with 32 filters of size 3x3 and add ReLU activation function to the model. \n",
    "#The input_shape argument specifies the shape of the input data.\n",
    "model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))\n",
    "\n",
    "#2D max pooling layer to the model, which reduces the spatial dimensions of the output from the previous \n",
    "#layer by a factor of 2\n",
    "model.add(MaxPooling2D((2, 2)))\n",
    "\n",
    "#Another 2D convolutional layer with 64 filters of size 3x3 and a ReLU activation function to the model.\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "\n",
    "#Add another max pooling layer\n",
    "model.add(MaxPooling2D((2, 2)))\n",
    "\n",
    "#Adds another Convolution 2D layer.\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "\n",
    "#A flatten layer to the model, which reshapes the output of the previous layer into a one-dimensional array.\n",
    "model.add(Flatten())\n",
    "\n",
    "#A fully connected (dense) layer with 64 units and a ReLU activation function to the model.\n",
    "model.add(Dense(64, activation='relu'))\n",
    "\n",
    "#Another fully connected layer with 10 units and a softmax activation function to the model.\n",
    "model.add(Dense(10, activation='softmax'))\n",
    "\n",
    "# Compile the model\n",
    "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=[\"accuracy\", \"mae\"])\n",
    "\n",
    "#Obtain the number of models parameters in total and for each layer.\n",
    "print(model.summary())\n",
    "\n",
    "print(\"\\n\\nTRAINING PHASE:\")\n",
    "# Train the model\n",
    "history = model.fit(X_train, y_train_onehot, epochs=4, batch_size=64, validation_data=(X_test, y_test_onehot))\n",
    "\n",
    "print(\"\\nTESTING PHASE:\")\n",
    "#Get the predictions from the testing phase.\n",
    "y_pred = model.predict(X_test)\n",
    "\n",
    "# Convert predicted probabilities to predicted labels. This is b/c y_pred contians a list of probabilities for each sample.\n",
    "y_pred_labels = np.argmax(y_pred, axis=1)\n",
    "\n",
    "#Evaluate the model using various metrics\n",
    "accuracy = accuracy_score(y_test, y_pred_labels)\n",
    "\n",
    "macro_precision = precision_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_precision = precision_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "macro_recall = recall_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_recall = recall_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "macro_f1 = f1_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_f1 = f1_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "mae = mean_absolute_error(y_test, y_pred_labels)\n",
    "\n",
    "print(\"\\nEvaluation metrics from testing phase:\")\n",
    "print(\"Accuracy: {:.4f}\".format(accuracy))\n",
    "print(\"Macro-Precision: {:.4f}\".format(macro_precision))\n",
    "print(\"Macro-Recall: {:.4f}\".format(macro_recall))\n",
    "print(\"Macro-F1-Score: {:.4f}\".format(macro_f1))\n",
    "print(\"Micro-Precision: {:.4f}\".format(micro_precision))\n",
    "print(\"Micro-Recall: {:.4f}\".format(micro_recall))\n",
    "print(\"Micro-F1-Score: {:.4f}\".format(micro_f1))\n",
    "print(\"Mean Absolute Error: {:.4f}\".format(mae))\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787627af",
   "metadata": {},
   "source": [
    "We can now apply MLP model to train and evaluate our dataset. We use multiple layers shown below, and use accuracy and MAE to evaluate the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f41975a9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_11\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_32 (Dense)            (None, 256)               200960    \n",
      "                                                                 \n",
      " dropout_20 (Dropout)        (None, 256)               0         \n",
      "                                                                 \n",
      " dense_33 (Dense)            (None, 128)               32896     \n",
      "                                                                 \n",
      " dropout_21 (Dropout)        (None, 128)               0         \n",
      "                                                                 \n",
      " dense_34 (Dense)            (None, 10)                1290      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 235,146\n",
      "Trainable params: 235,146\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "\n",
      "\n",
      "TRAINING PHASE:\n",
      "Epoch 1/4\n",
      "875/875 [==============================] - 2s 2ms/step - loss: 0.5707 - accuracy: 0.8369 - mae: 0.0464 - val_loss: 0.2064 - val_accuracy: 0.9376 - val_mae: 0.0200\n",
      "Epoch 2/4\n",
      "875/875 [==============================] - 2s 2ms/step - loss: 0.3211 - accuracy: 0.9092 - mae: 0.0275 - val_loss: 0.1594 - val_accuracy: 0.9517 - val_mae: 0.0159\n",
      "Epoch 3/4\n",
      "875/875 [==============================] - 2s 2ms/step - loss: 0.2644 - accuracy: 0.9226 - mae: 0.0232 - val_loss: 0.1456 - val_accuracy: 0.9561 - val_mae: 0.0143\n",
      "Epoch 4/4\n",
      "875/875 [==============================] - 2s 2ms/step - loss: 0.2425 - accuracy: 0.9304 - mae: 0.0209 - val_loss: 0.1328 - val_accuracy: 0.9596 - val_mae: 0.0133\n",
      "\n",
      "TESTING PHASE:\n",
      "438/438 [==============================] - 0s 745us/step\n",
      "\n",
      "Evaluation metrics from testing phase:\n",
      "Accuracy: 0.9596\n",
      "Macro-Precision: 0.9595\n",
      "Macro-Recall: 0.9591\n",
      "Macro-F1-Score: 0.9592\n",
      "Micro-Precision: 0.9596\n",
      "Micro-Recall: 0.9596\n",
      "Micro-F1-Score: 0.9596\n",
      "Mean Absolute Error: 0.1511\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout\n",
    "\n",
    "# Reshape the data to have the flattened shape\n",
    "X_train_flat = X_train.reshape(-1, 784)\n",
    "X_test_flat = X_test.reshape(-1, 784)\n",
    "\n",
    "# One-hot encode the labels\n",
    "y_train_onehot = to_categorical(y_train)\n",
    "y_test_onehot = to_categorical(y_test)\n",
    "\n",
    "# Define the model architecture\n",
    "model = Sequential()\n",
    "\n",
    "# Adds a fully connected (dense) layer with 256 units and ReLU activation function to the model.\n",
    "model.add(Dense(256, activation='relu', input_shape=(784,)))\n",
    "\n",
    "# Adds a dropout layer to the model with a rate of 0.5, which randomly drops 50% of the connections during training to prevent overfitting.\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "# Adds another fully connected (dense) layer with 128 units and ReLU activation function to the model.\n",
    "model.add(Dense(128, activation='relu'))\n",
    "\n",
    "# Adds another dropout layer to the model with a rate of 0.5.\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "# Adds another fully connected (dense) layer with 10 units and softmax activation function to the model.\n",
    "model.add(Dense(10, activation='softmax'))\n",
    "\n",
    "#Obtain the number of models parameters in total and for each layer.\n",
    "print(model.summary())\n",
    "\n",
    "# Compile the model\n",
    "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=[\"accuracy\", \"mae\"])\n",
    "\n",
    "print(\"\\n\\nTRAINING PHASE:\")\n",
    "# Train the model and obtain the training history\n",
    "history = model.fit(X_train_flat, y_train_onehot, epochs=4, batch_size=64, validation_data=(X_test_flat, y_test_onehot))\n",
    "\n",
    "print(\"\\nTESTING PHASE:\")\n",
    "#Get the predictions from the testing phase.\n",
    "y_pred = model.predict(X_test_flat)\n",
    "\n",
    "# Convert predicted probabilities to predicted labels. This is b/c y_pred contians a list of probabilities for each sample.\n",
    "y_pred_labels = np.argmax(y_pred, axis=1)\n",
    "\n",
    "#Evaluate the model using various metrics\n",
    "accuracy = accuracy_score(y_test, y_pred_labels)\n",
    "\n",
    "macro_precision = precision_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_precision = precision_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "macro_recall = recall_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_recall = recall_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "macro_f1 = f1_score(y_test, y_pred_labels, average=\"macro\")\n",
    "micro_f1 = f1_score(y_test, y_pred_labels, average=\"micro\")\n",
    "\n",
    "mae = mean_absolute_error(y_test, y_pred_labels)\n",
    "\n",
    "print(\"\\nEvaluation metrics from testing phase:\")\n",
    "print(\"Accuracy: {:.4f}\".format(accuracy))\n",
    "print(\"Macro-Precision: {:.4f}\".format(macro_precision))\n",
    "print(\"Macro-Recall: {:.4f}\".format(macro_recall))\n",
    "print(\"Macro-F1-Score: {:.4f}\".format(macro_f1))\n",
    "print(\"Micro-Precision: {:.4f}\".format(micro_precision))\n",
    "print(\"Micro-Recall: {:.4f}\".format(micro_recall))\n",
    "print(\"Micro-F1-Score: {:.4f}\".format(micro_f1))\n",
    "print(\"Mean Absolute Error: {:.4f}\".format(mae))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9dec963a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}