gabarito.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Workshop Redes Neurais\n",
    "## Turing USP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Pytorch logo](https://upload.wikimedia.org/wikipedia/commons/9/96/Pytorch_logo.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: torch in /home/nelson/miniconda3/envs/py38/lib/python3.8/site-packages (1.10.0)\n",
      "Requirement already satisfied: torchvision in /home/nelson/miniconda3/envs/py38/lib/python3.8/site-packages (0.11.1)\n",
      "Requirement already satisfied: typing-extensions in /home/nelson/miniconda3/envs/py38/lib/python3.8/site-packages (from torch) (3.10.0.2)\n",
      "Requirement already satisfied: numpy in /home/nelson/miniconda3/envs/py38/lib/python3.8/site-packages (from torchvision) (1.21.3)\n",
      "Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /home/nelson/miniconda3/envs/py38/lib/python3.8/site-packages (from torchvision) (8.4.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install torch torchvision # Se você não os tiver no seu computador, pode levar um tempo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🔥 Básicos de Pytorch 🔥\n",
    "\n",
    "### Tensores\n",
    "*Os blocos de construção das redes neurais*\n",
    "\n",
    "Primeiro vamos ver alguns análogos entre **numpy** e **Pytorch**\n",
    "\n",
    "#### Matrizes\n",
    " - Em Pytorch, matrizes (*arrays*) são chamados de tensores.\n",
    " - Uma matriz $3\\times3$, por exemplo é um tensor $3\\times3$\n",
    " - Podemos criar um array numpy com o método `np.array()`\n",
    " - Podemos pegar o tipo do array com `type()`\n",
    " - Podemos pegar o formato do *array* com `np.shape()`. Linha $\\times$ Coluna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Array do tipo: <class 'numpy.ndarray'>\n",
      "Array de formato: (2, 3)\n",
      "[[1 2 3]\n",
      " [4 5 6]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "\n",
    "array = [[1,2,3],[4,5,6]]\n",
    "primeiro_array = np.array(array) # array 2x3\n",
    "print(f\"Array do tipo: {type(primeiro_array)}\")\n",
    "print(f\"Array de formato: {np.shape(primeiro_array)}\")\n",
    "print(primeiro_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Criamos um tensor com o método `torch.Tensor()`\n",
    "- `tensor.type`: tipo do objeto, nesse caso um tensor\n",
    "- `tensor.shape`: formato do tensor. Linha $\\times$ Coluna \n",
    "- `tensor.device` : por onde este tensor está sendo processado"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tipo: <built-in method type of Tensor object at 0x7fc60adbd810>\n",
      "Tensor de formato: torch.Size([2, 3])\n",
      "Tensor sendo armazenado em cpu\n",
      "tensor([[1., 2., 3.],\n",
      "        [4., 5., 6.]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "tensor = torch.Tensor(array)\n",
    "print(f\"Tipo: {tensor.type}\")\n",
    "print(f\"Tensor de formato: {tensor.shape}\")\n",
    "print(f\"Tensor sendo armazenado em {tensor.device}\")\n",
    "print(tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos fazer a alocação de *arrays* de maneira análoga nas duas linguagens:\n",
    " - `np.ones()` = `torch.ones()`\n",
    " - `np.random.rand()` = `torch.rand()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Numpy:\n",
      " [[1. 1. 1.]\n",
      " [1. 1. 1.]]\n",
      "\n",
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "print(f\"Numpy:\\n {np.ones((2,3))}\\n\")\n",
    "\n",
    "print(torch.ones((2,3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Numpy:\n",
      " [[0.74776459 0.70446929 0.07084426]\n",
      " [0.07801848 0.22158566 0.77696397]]\n",
      "\n",
      "tensor([[0.1803, 0.2471, 0.8223],\n",
      "        [0.3292, 0.9842, 0.9164]])\n"
     ]
    }
   ],
   "source": [
    "print(f\"Numpy:\\n {np.random.rand(2,3)}\\n\")\n",
    "\n",
    "print(torch.rand(2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Convertendo de numpy para torch e vice-versa\n",
    "\n",
    "Em muitos pontos **numpy** e **pytorch** são bem parecidos em suas estruturas, e muitas das vezes podemos utilizar os dois em conjunto. Assim normalmente convertemos resultados de redes neurais - que são tensores - para **arrays** de **numpy**.\n",
    "\n",
    "Os métodos para fazer a conversão entre tensores e arrays numpy:\n",
    " - `torch.from_numpy()`: de um array numpy para um tensor\n",
    " - `tensor.numpy()`: de um tensor para um array numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'numpy.ndarray'> \n",
      " [[0.48222387 0.4438185 ]\n",
      " [0.47211609 0.14998733]] \n",
      "\n",
      "tensor([[0.4822, 0.4438],\n",
      "        [0.4721, 0.1500]], dtype=torch.float64) \n",
      "\n",
      "<class 'numpy.ndarray'> \n",
      " [[0.48222387 0.4438185 ]\n",
      " [0.47211609 0.14998733]]\n"
     ]
    }
   ],
   "source": [
    "array = np.random.rand(2,2)\n",
    "print(f\"{type(array)} \\n {array} \\n\")\n",
    "\n",
    "de_numpy_para_tensor = torch.from_numpy(array)\n",
    "print(f\"{de_numpy_para_tensor} \\n\")\n",
    "\n",
    "tensor = de_numpy_para_tensor\n",
    "de_tensor_para_numpy = tensor.numpy()\n",
    "print(f\"{type(de_tensor_para_numpy)} \\n {de_tensor_para_numpy}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quando fazemos estas conversões também podemos fazer um *typecast* (mudança do tipo) das variáveis, isso pode ser útil já que o Pytorch faz uma série de computações de baixo nível, o qual o tipo primitivo das variáveis precisa ser bem especificado e definido, para isso podemos usar o método `tensor.type(torch.TipoDeTensor)`, alguns tipo de tensores nativos do Pytorch são:\n",
    "  - `torch.FloatTensor` - pontos flutuantes de 32-bits\n",
    "  - `torch.DoubleTensor` - pontos flutuantes de 64-bits\n",
    "  - `torch.IntTensor` - números inteiros de 32-bits\n",
    "  - `torch.LongTensor` - números inteiros de 64-bits\n",
    "É muito comum encontrarmos *bugs* causados pela utilização errada de algum tipo primitivo, você pode ler sobre todos eles na [documentação do Pytorch](https://pytorch.org/docs/stable/tensors.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'torch.Tensor'> \n",
      " tensor([[ 1., 10.],\n",
      "        [ 2., 20.]])\n",
      "\n",
      "<class 'torch.Tensor'> \n",
      " tensor([[ 1, 10],\n",
      "        [ 2, 20]])\n"
     ]
    }
   ],
   "source": [
    "array = np.array([[1,10],[2,20]])\n",
    "\n",
    "# Transformar em um tensor de Floats:\n",
    "tensor_float = torch.from_numpy(array).type(torch.FloatTensor)\n",
    "print(f\"{type(tensor_float)} \\n {tensor_float}\\n\")\n",
    "\n",
    "# Transformar em um tensor de Longs:\n",
    "tensor_long = torch.from_numpy(array).type(torch.LongTensor)\n",
    "print(f\"{type(tensor_long)} \\n {tensor_long}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Operações com tensores\n",
    "Existem mais de 100 operações implementadas para tensores, incluindo aritmética, álgebra linear, manipulação de matrizes etc. É interessante que você as cheque [aqui](https://pytorch.org/docs/stable/torch.html).\n",
    "\n",
    "O mais interessante, inclusive algo que possibilitou a utilização em massa de redes neurais, é o processamento dessas operações em GPU's (que geralmente possuem uma maior velocidade do que CPU's).\n",
    "\n",
    "Por padrão tensores são criados na CPU. Nós podemos explicitamente mover para GPU's utilizando o método `.to` (isso, claro, se você pode usar uma GPU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Para esse notebook podemos usar a GPU? False\n"
     ]
    }
   ],
   "source": [
    "tensor = torch.ones(3,3)\n",
    "print(f\"Para esse notebook podemos usar a GPU? {torch.cuda.is_available()}\")\n",
    "\n",
    "# Move nosso tensor para uma GPU se possível\n",
    "if torch.cuda.is_available():\n",
    "    tensor = tensor.to('cuda')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operações de splicing padrões:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Primeira linha:  tensor([1., 1., 1.])\n",
      "Primeira coluna:  tensor([1., 1., 1.])\n",
      "Última coluna: tensor([1., 1., 1.])\n",
      "tensor([[1., 0., 1.],\n",
      "        [1., 0., 1.],\n",
      "        [1., 0., 1.]])\n",
      "torch.Size([9]): tensor([1., 0., 1., 1., 0., 1., 1., 0., 1.]) \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('Primeira linha: ', tensor[0])\n",
    "\n",
    "print('Primeira coluna: ', tensor[:, 0])\n",
    "\n",
    "print('Última coluna:', tensor[..., -1])\n",
    "\n",
    "tensor[:,1] = 0 # Colocar a segunda coluna como 0's\n",
    "print(tensor)\n",
    "\n",
    "print(f\"{tensor.view(9).shape}: {tensor.view(9)} \\n\") # Podemos mudar o formato do tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Operações aritméticas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adição: \n",
      "tensor([[2., 2., 2.],\n",
      "        [2., 2., 2.],\n",
      "        [2., 2., 2.]]) \n",
      "\n",
      "Subtração: \n",
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.]]) \n",
      "\n",
      "Multiplicação elemento-a-elemento: \n",
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]]) \n",
      "\n",
      "Divisão elemento-a-elemento: \n",
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]]) \n",
      "\n",
      "Multiplicação de matriz:\n",
      "tensor([[3., 3., 3.],\n",
      "        [3., 3., 3.],\n",
      "        [3., 3., 3.]])\n",
      " \n"
     ]
    }
   ],
   "source": [
    "tensor = torch.ones(3,3)\n",
    "\n",
    "print(f\"Adição: \\n{torch.add(tensor, tensor)} \\n\")\n",
    "\n",
    "print(f\"Subtração: \\n{torch.sub(tensor, tensor)} \\n\")\n",
    "\n",
    "print(f\"Multiplicação elemento-a-elemento: \\n{torch.mul(tensor, tensor)} \\n\")\n",
    "\n",
    "print(f\"Divisão elemento-a-elemento: \\n{torch.div(tensor, tensor)} \\n\")\n",
    "\n",
    "print(f\"Multiplicação de matriz:\\n{torch.matmul(tensor, tensor.T)}\\n \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operações com só um tensor "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Soma: 15.0\n",
      "\n",
      "Média: 3.0 \n",
      "\n",
      "Desvio padrão: 1.5811388492584229 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "tensor = torch.Tensor([1,2,3,4,5])\n",
    "\n",
    "print(f\"Soma: {tensor.sum()}\\n\")\n",
    "\n",
    "print(f\"Média: {tensor.mean()} \\n\")\n",
    "\n",
    "print(f\"Desvio padrão: {tensor.std()} \\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Autograd e back propagation\n",
    "*Diferenciação automática e back prop com `torch.autograd`*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quando treinamos redes neurais, o algoritmo mais usado é a back propagation. Nesse algoritmo, parâmetros (os *weights* do modelo) são ajustados de acordo com o gradiente da função de perda em respeito com o parâmetro dado.\n",
    "\n",
    "Para computar esses gradientes, o PyTorch tem uma implementação de diferenciação (o cálculo de derivadas) chamado `torch.autograd`. Ele faz computações automáticas de gradientes para qualquer *computational graph*.\n",
    "\n",
    "Considere a rede neural mais simples de uma camada, com entrada `x`, parâmetros `w` e `b` e alguma função de perda. Ela pode ser definida da seguinte maneira:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Diagrama do computational graph](https://i.imgur.com/x6DBPFQ.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.ones(5)  # [1 1 1 1 1] vetor de entrada\n",
    "\n",
    "y = torch.ones(3)*2  # [2 2 2] valor esperado\n",
    "\n",
    "w = torch.full((5, 3), 3.0, requires_grad=True) # [ 3 3 3 ; 3 3 3 ; 3 3 3 ; 3 3 3 ; 3 3 3] matriz de pesos\n",
    "\n",
    "b = torch.ones(3, requires_grad=True) # [1 1 1] matriz de bias\n",
    "\n",
    "z = torch.matmul(x, w) + b # [16 16 16] \n",
    "\n",
    "loss = torch.sum(torch.pow(z,y)) # [768] função de perda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Agora podemos computar os gradientes seguindo esse diagrama:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![diagrama para calcular os gradiantes dos parâmetros](https://i.imgur.com/fSoQQBC.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para otimizar os pesos (weights) dos parâmetros da rede neural, precisamos computar as derivadas da nossa \"função de perda\" em respeito aos parâmetros. Precisamente $\\frac{\\partial \\, \\mathrm{loss}}{\\partial w}$ e $\\frac{\\partial \\, \\mathrm{loss}}{\\partial b}$ para valores fixos de `x` e `y`. Para computar as derivadas, utilizamos `loss.backward()` os valores ficam armazenados em `w.grad` e `b.grad`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[32., 32., 32.],\n",
      "        [32., 32., 32.],\n",
      "        [32., 32., 32.],\n",
      "        [32., 32., 32.],\n",
      "        [32., 32., 32.]])\n",
      "tensor([32., 32., 32.])\n"
     ]
    }
   ],
   "source": [
    "loss.backward()\n",
    "print(w.grad)\n",
    "print(b.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Por padrão, todos os tensores com parâmetro `requires_grad=True` estão monitorando seu histórico de funções computadas para calcular seu gradiente. Porém em alguns casos isso pode não ser necessário, isso pode acontecer em casos como:\n",
    " - Para marcar alguns parâmetros como **frozen parameters**. Algo comum quando você quer aperfeiçoar uma rede pré-treinada\n",
    " - Para **acelearar** as computações quando você está apenas passando pelo passo de **forward**, no qual computações com tensores que não monitoram gradientes são mais úteis.\n",
    "\n",
    "Podemos para de monitorar os gradientes colocando nosso código em um bloco com  `with torch.no_grad()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "# Monitora os gradientes\n",
    "z = torch.matmul(x, w) + b\n",
    "print(z.requires_grad)\n",
    "\n",
    "# Não monitora os gradientes\n",
    "with torch.no_grad():\n",
    "    z = torch.matmul(x, w) + b\n",
    "print(z.requires_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O mesmo resultado com o método `detach()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "z = torch.matmul(x, w) + b\n",
    "z_det = z.detach()\n",
    "print(z_det.requires_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datasets e Dataloader\n",
    "*Um meio de padronizar e otimizar dados para as redes neurais no PyTorch*\n",
    "\n",
    "O PyTorch fornece dois \"tipos primitivos\" (é como se fossem ints, floats, bools) para otimizar e padronizar datasets e depois dizer para a rede neural como ela deve ler esse dataset, eles são o `torch.utils.data.Dataset` e o `torch.utils.data.DataLoader`.\n",
    "\n",
    "O PyTorch também nos fornece alguns datasets já prontos, o [Fashion-MINIST](https://research.zalando.com/project/fashion_mnist/fashion_mnist/) é um deles, ele é um dataset com imagens de roupas em 28x28 com 60.000 imagens de treino e 10.000 de teste. Vamos usar ele tanto nesse exemplo quanto no exemplo de redes neurais.\n",
    "\n",
    "Mesmo que o dataset já esteja montado, vamos passar quais seriam os passos para criar um dataset do zero. Vale dizer que nem sempre precisamos realizar esses passos, muitas vezes podemos passar nossos dados de maneira \"cru\" como tensores, mas é interessante sabê-los."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import Dataset # estrutura de dataset de tensores \n",
    "from torchvision import datasets # datasets já existentes no pytorch\n",
    "from torchvision.transforms import ToTensor # para transformar as imagens em tensores\n",
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data = datasets.FashionMNIST(\n",
    "    root = 'data', # Onde vai armazenar o dataset\n",
    "    train = True, # Especifica que é o dataset de treino\n",
    "    download= True, # baixa o dataset da internet\n",
    "    transform= ToTensor() # transforma em tensor\n",
    ")\n",
    "\n",
    "test_data = datasets.FashionMNIST(\n",
    "    root = 'data', \n",
    "    train = False, \n",
    "    download= True, \n",
    "    transform= ToTensor() \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos dar uma olhada em como são as imagens do dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHRCAYAAAABukKHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABJ5UlEQVR4nO3debxdVX3///dHGTLP8wyEJIQpxjJTCaMEmSygRWTQKtpfrUOtv/Zr/QlqrS0/KwpWq/z4Wh9KQYtFg0CtTDKEGUOYA2Yg8zwTAoT1++OcfJv9WZ99z+bmDjn3vp6PBw9d666zzz73rLtX9v581lqWUhIAAMi9o7NPAACAPRWDJAAAJRgkAQAowSAJAEAJBkkAAEowSAIAUIJBEgC6CTO7zMweaOHnd5jZpR15Tnu6LjFImtmWXf57y8y27VK+qLPPDzCzhbv0y/VmdpuZje3s80LXZGbHm9lsM9toZuvM7EEzO6LR61JKM1NKP27huC0Osl1RlxgkU0p9dv4n6RVJZ+1Sd8POdma2V+ed5Z5zDug0Z9X76EhJKyVd28nngy7IzPpJ+rVq/WuQpNGSviJp+24et1teu7rEIFnGzGaY2RIz+xszWyHpR2a2r5l928yW1f/7tpntW2+f/SvJzJKZTaz//zPM7Dkz22xmS83sr3dpd6aZzTGzDfV/wR22y88W1s9hrqSt3bWzoSal9JqkmyVNlSQze5+Z/d7MNpnZYjO7ctf2ZnaJmS0ys7Vm9v/U+9MpnXDqaA6TJCmldGNKaUdKaVtK6b9TSnN3NjCzb9afaCwws5m71N9rZh+r///L6negV5vZWkk/k/Svko6pPxHZ0LEfq3N06UGyboRq/5oaL+lySX8n6WhJ0yQdLulISV+qeKzrJX0ipdRX0iGS7pYkM3uXpP8t6ROSBkv6gaRZOwffugslvU/SgJTSm7v3kdDMzKyXpA9KerhetVXSJZIGqNZH/tzMzq23nSrpe5IuUu0OtL9qdwZAmXmSdpjZj81sppkNdD8/StKLkoZIukrS9WZmJcc6StJ8ScMlfVjSJyU9VH9KN6Bdzn4P0x0GybckXZFS2p5S2qbaxearKaVVKaXVqj2GuLjisd6QNNXM+qWU1qeUnqzXXy7pBymlR+r/cvuxao82jt7ltdeklBbXzwHd0y/r//reKOlUSf+vJKWU7k0pPZ1Seqv+r/0bJZ1Qf835km5NKT2QUnpd0pclseAySqWUNkk6XrV+cp2k1WY2y8yG15ssSildl1LaIenHqv3ja3h8NC1LKV2bUnqzu167usMgubr+eGunUZIW7VJeVK+r4jxJZ0haZGa/M7Nj6vXjJX2+/qh1Q/1CONYdd3Grzh5dybn1f333kPQpSb8zsxFmdpSZ3WNmq81so2r/Wh9Sf80o7dJ3UkqvSlrbweeNJpNSej6ldFlKaYxqT71GSfp2/ccrdmn3av3/9ik5VLe/bnWHQdL/q3uZaoPaTuPqdVLtsVevnT8wsxGFA6X0WErpHEnDJP1S0s/rP1os6esppQG7/NcrpXRjC+eBbqr+tOE/Je1Q7V/8/y5plqSxKaX+qsV9dj7+Wi5pzM7XmllP1R7pA5WklF6Q9G+qDZZv++UNyl1edxgkvRslfcnMhprZENUeX/20/rOnJB1sZtPMrIekK3e+yMz2MbOLzKx/SukNSZtUe5Qr1R5pfLJ+R2Bm1ruejNG3wz4Vmka9j5wjaaCk5yX1lbQupfSamR0p6UO7NL9Z0llmdqyZ7aNanyyLHwEysylm9nkzG1Mvj1UtJ+Lhll9ZyUpJY+p9sVvojoPk30t6XNJcSU9LerJep5TSPElflXSnpJck+flAF0taaGabVHskdlH9dY9L+rik70paL+llSZe18+dA87nVzLao9g+sr0u6NKX0rKT/S9JXzWyzav9o2/mEQvWf/6Wkm1S7q9wiaZV2M50fXdpm1RJuHjGzraoNjs9I+nwbHPtuSc9KWmFma9rgeHs8Y9NloHmYWR9JGyQdmFJa0MmnA3R53fFOEmgqZnaWmfUys96SvqnaE5CFnXtWQPfAIAns+c5RLblsmaQDJf1p4hEQ0CF43AoAQAnuJAEAKMEgCQBAiRYX2jazPf5Z7LHHHpvVbd9ezI7funVr1qZXr16F8t5775212bFjR1b31ltvFcrveEf+74z+/fsXykuWLMnavPjii1ndnial1Cnz8Zqh37XWXnsV/+TefDNfxvfkk0/O6oYOHVoo33TTTW17YnuQzuh3XaXPjR8/Pqs76aSTCuWnn346a/PGG29kdTNmzCiUv/Od77TqnMqXhf0fnR32a6nPcScJAEAJBkkAAEowSAIAUIJBEgCAEi0m7uyJxowZUyjfcsstWZtXXnmlUH7ttdeyNvvsU1yfd/369VmbDRs2ZHU+cadHjx5ZG58oFCXu/K//9b+yOnQt73znO7M6n6gzZcqUrM2kSZOyulWrVhXK06ZNy9rMmTOnUI4SJjo7QaI78t9D9L3460pVRx99dKEcJXR96lOfKpSjZMPonM4///xCedCgQVmbK664ouE5VulzUeJklEzUGbiTBACgBIMkAAAlGCQBACjRdDHJcePGFcrPP/981mbRokWFcvRMfNiwYYXy2rVrszZLly7N6vzz/Oj5vq+Lnrf7GACxou7pvPPOy+q+/vWvN3zdn/3Zn2V1PiZJn+p4VeLAVb+X008/vVD+yEc+krXxC0+89NJLWZsLLrigUL7++uuzNl/84hezOn9tnT59etZm1qxZhfITTzyRtfnJT35SKM+fPz9rs6fEHyPcSQIAUIJBEgCAEgySAACUYJAEAKBE0yXuTJ48uVBet25d1mb16tWFct++fbM2fqGAaMcPv5uIJPXp06dQjnZx8HUDBgzI2vhdHfxkcTQfn7QR9amePXsWylFSVxXRxG6/sEW0iAYJY+2ryu8zSro666yzsrrRo0cXytF1bPny5YVy1C/69etXKEc7hUQ7gzz33HOF8sc//vGszebNmwvlU045JWvjdxNZsGBB1uaaa67J6nwiWmfhThIAgBIMkgAAlGCQBACgRNPFJP2C0FHcxcd9Xn/99ayNf5buFzyX8rihJG3ZsqVQjmIQfjGBaKLsqFGjCmViks3PL2gexat9TD1asKKKZcuWZXV+0fOHH344a1PlHFGdjylHf+uf+9znCuVPfOITWZtoMZNt27YVytG1zscbozyKQw89tFA+++yzszZR/O/SSy8tlKOFW/x10+eDROc4YcKErM2XvvSlhu/vN46Q8mttaxeKbwl3kgAAlGCQBACgBIMkAAAlGCQBACjRdIk7Bx10UKEcBYp9ok40md+LJn5HgfL+/fsXylEw2Sf3+EQiSdp3330bnhOay157Ff+coqQYn1zz5JNPtuq9Fi9enNX5nW3Q/qrsXnHMMccUyps2bcraRNeRIUOGFMrR9Wjjxo2FcpSk6PvFo48+mrVZv359VueTw/xiFVKeXOTLUp4AGS0A46+rknTGGWcUyv/xH/+RtWmPRB2PO0kAAEowSAIAUIJBEgCAEk0Xk/QxAD+ZVJJeffXVQjnaLdw/A/cLl0v5AsOS9MILLxTKUQzAi9pEk37R3KrEp3xf9P2pqnnz5mV1F110UcPXsXhAx/MLikc5ClVyG6LrmF8cIsp18Ase+IVUpHjxdB9fjN7f9/nBgwdnbfz1L7pmR9fII488slCOYpIdgTtJAABKMEgCAFCCQRIAgBIMkgAAlGi6xB0/eTbaqcPvqBHt8OEnz/qJu1K8g/eaNWsK5SjgvGTJkkI5WqggmpiL5ua/56jf9erVq1COEjaiBAm/28yKFSuyNn5nmSqqvBeqO/roo7M6/7ce9YsomcUn3ETfi28THccn10QT/v1OHVLeN6OkL//ZouvaokWLCuXoWhtdIwcOHJjVdQbuJAEAKMEgCQBACQZJAABKMEgCAFCi6RJ3/O4HRxxxRNbm6aefbngcnwDUu3fvrI3fcUSS7rrrrkI5WjnHr3rhV8WQ4hX90TyqJLwcd9xxWZsoUae9zun444/P2jzwwAOFctQ3WZWn9U499dSszn8vUXJNlDjjrxF+lxkpX10sWjmsSp+L+rNPnIl2L/H9J7qu+Z06omRLn9wj5b+TsWPHZm2i3XDaGneSAACUYJAEAKAEgyQAACWaLiY5Z86cQvnDH/5w1savaB9NnvUTWh9//PGszR/90R9ldVV2lo8mxnpLly5t2AZvj4/ZRLGfKjuZR5O2/XcaHdu3ee9735u1ifqLFx270XtJ+SIahx9+eNbGxyTbc+GA7rhQQRQ385856oNRLNHHG6vs8OEXO5HyBSyiCf9VrllRvLHKZ/PnHb1/tCjLgAEDCuXoekxMEgCATsQgCQBACQZJAABKMEgCAFCi6RJ35s+f37CNT+DYvHlz1sYHj++7776szTnnnJPV+WNVCYJHiUNRgB27pyMnwVdJdJg6dWpW981vfrNNjl3FAQcc0GHvFenqSTqRKVOmZHU+mSVawCFazMRfI6LriBcl14wcObJQ3rJlS9bG74ok5bvKRH9f/v2ixVV8ImVVPoFt//33b9Vxdhd3kgAAlGCQBACgBIMkAAAlmi4muXz58kI5iqn4nb+jSc0+BvDMM880fC9JGjx4cKH8hz/8IWvjJ++uXr06a9Md4zXtbebMmYWy35FdyuMq0eTnKGbk4yPR6/wC0FUmdo8ePTpr09qFJl544YVCeb/99svaXH755YVy//79szZ+8X9Jmjx5cqEcTeIeMWJEobxixYqszbe//e2srivx1wcpnyjvr09SfI2qEmP3x46+T/9+Ud+NrqM+tyN6nf+7iK5r/pyizxr9zfn3nzhxYtamI3AnCQBACQZJAABKMEgCAFCCQRIAgBJNl7izbt26QjlaKKBnz56FchQU9osALFy4MGsTJVAcffTRhXKUHOKD99FEXbS9c889t1D2OyRI+XcRfX/RxG7fX3w/lPLEnahPXXzxxYVytOOH36FGyj9LNGncJ8pEbXz/jfrmpEmTsjrvwAMPzOoOPfTQQjnaWaeri3bq8N9D1Cbik3KiSfm+z0WLGaxcubJQjhKHor7i+4a/rkr5TiVRUs6GDRsK5fHjx2dtqux8M2HChIZt2gN3kgAAlGCQBACgBIMkAAAlmi4m6SfY+ufdUh4/imKSUbzKixbr9RNcowm/vk30vB9tb+7cuYXy9OnTszZ+gr+P6UhxfMTHY9auXdvwfKIJ2r4v+MWnpTyGJEmvvPJKoVxl8fKHH344q/OLAkR9M4q3Tps2rVCOJq3739GsWbManmOz8/Hj6Dv3dVH/+v73v5/VnXrqqYVytMC5v9YNHDgwa+MXShk0aFDWJool+oXQo+thv379CuUo3nrnnXcWysOHD8/aDBs2LKvz11a/EEdH4U4SAIASDJIAAJRgkAQAoASDJAAAJZoucceLkhyGDh1aKEfB9ChQ7UUJDH6ieTQZ3YsC3mh7t956a6F80kknZW0GDBhQKEcJKNECFb4vRMlgJ5xwQqHsk10k6fbbby+UowSyKOHGJ8VEffqll14qlK+++uqsjd8p5LHHHsvaRH8bPsFn6tSpWRv/txB9jq5m//33L5SjhC6fyBd9d1/+8pezOr/whP/upHznlejYW7dubfE1UpwA6ftmlKTod/3wiTzROd18881Zm09+8pNZnd+FKbrWV9l5ZndxJwkAQAkGSQAASjBIAgBQgkESAIASTZ+4s3HjxqzOJ2dUWRUn4le4l/LVMqLVM956661COQqmo+35VWn8bixS3l9Wr16dtfH9JxKt/jF//vxCOUpc8QkwUR+LEmd8glGUVOZX4bnjjjuyNl6040ifPn2yOn+eURKJ/711h5Wm/M4UVVZrilbOifhdP6LrmE8kjBJwfJKZT7aR4u/cX8eiJEWflBglwo0ZM6ZQ/tu//duszTe+8Y2szifuRO/vd6whcQcAgA7EIAkAQAkGSQAASjR9TDJ6Tu+f5UfP4KvECZcsWZLV+Qm20TN4L5qcjvZ32223ZXXvec97CuUoJunjPFI+STraPaTRa6Q8Fh7t9h7xE8KjnRQOO+ywQjmKrUY7QHjRAgc+HjR27NiszcKFCxseu6sZP358oRzFzfzvs2qOgv/O99lnn6yN7z9+QYmojb8+SvFEfb9DTRTv9Ne2aBeS6O/JW79+fVbnr9vRAh7RriNtjTtJAABKMEgCAFCCQRIAgBIMkgAAlGj6xJ0ogcIHuKPAcZVdQPyODZL0z//8z4VylSSHaMEDtL+f/vSnWd2MGTMK5Sjw7xMmpDxp4PXXX8/a+CSyqG/4xJ2o/0YLDPgJ2RMnTsza+CSyKjtCRIkm0fv73RYOPPDArM0jjzyS1XV1PjkqSsTyiypUTdzZsmVLi+8l5ckt0WR6n2QVJTtGdb5vRItz+AU8ouNECWxV2viFCqK+WiURbXdxJwkAQAkGSQAASjBIAgBQouljktFkcB/3GT16dNamygR/v8CulC9gHE3M9e8fxabQ/qLvz0+ajiY/L126NKvz8aBoEQkf5476mO8vPtYYtZHyhcijmLqP2USL+Pu6KM4zdOjQrK5Hjx6F8mOPPZa1ieq6Oh8382UpXyi86gLn/ncexft8/DhaYN33pyjmHMUE/SL6w4YNy9r4/hTF6qtc/6K8Ef+79L8PKd5ooK1xJwkAQAkGSQAASjBIAgBQgkESAIASTZ+4s2DBgqzOB6+jYHYVVSb9RjuM+GAyu4DsOfxuAwcccEDDNlLeh6IkCp/EEO323ug1UrzAQZUEDb/gQZSU45NIoj4e7bZQJfniF7/4RcM2XU2V34tPQKmayOcXRYkSV/xiFFF/ir5PL0rm8QlcflEEqVriTJQc51W5jkdJQR2BO0kAAEowSAIAUIJBEgCAEk0fk4wWiPbxIh+HkeIJ01X4GE4UJ/DPzqMdxdH2/AT7KF7sJ7xPmjSp4XGiuigm6UWxRX+cqE2VOGHUp3wMMjq2f120cEE0sdsvbr127dqsTbQIQ1fnry1RrNi3iX53kZEjRxbK8+fPz9r4BTOiRch9f6qyCLuUf5YqsdQoRlkltyO6jvvFC6JcgSrx1t3FnSQAACUYJAEAKMEgCQBACQZJAABKNH3iTjQJ1idVRMHd1gZ8q+y04I9dJckDHePuu+8ulD/wgQ9kbaIEhWhivucThaKEsSr9Lkqs8EkU0bF90kSUVOY/W9R/o8/vjz179uysTXe0cOHCQjlKqPLJWt/5zncqHdv3lWiHEX/9Gzt2bNbGJ+VEiVlRkpdPCpoyZUrWZtmyZS2ej5TvYBNZs2ZNVjdo0KBCOeqXGzdubHjs3cWdJAAAJRgkAQAowSAJAEAJBkkAAEo0feJOFMz2SQ7RCiqtTabZunVroRytMOFX3ImC4mh70Qo73qpVqxq2GTBgQFa3bdu2QjlaWcUnykRJOv4co9VIot0O/LGj9/dJI1G/9+cUHSd6nU8UmjVrVtamO6qyM4W/RixZsqRV77Vly5aszifhRNc1/7rRo0dnbaK+6pNwovf3fysbNmzI2kQ77Xi33XZbVnfyyScXylGyWnRObY07SQAASjBIAgBQgkESAIASTR+TjCZ5++fyUZtownYVVXb48O/HYgJ7rueffz6rmzZtWlbnY9FVJtxHMRQfg4zaRPzrorih75vROfrFA6L+G8VJfezy2WefLT/ZbmTRokWFcvS789/xmDFjsjbRLhxelH/hdzNat25d1sbvDBLFUaPYtO8rmzdvztr4fIsotlll95Dos/nzjuKPq1evbnjs3cWdJAAAJRgkAQAowSAJAEAJBkkAAEo0feJOtAq8D55HQemorgofhI8SH1auXFkoV9lBAp3jkUceyeoOPfTQhq+LkhF88kG0uEGVyfxRne930fv7uiq7iUR9M1ogw+/28MILL2RtuqMHHnigUF6xYkXWxk/ef/HFF7M248ePz+p8UkqUFOOvdVFSzsCBAwvl6Dtv7U5JPuEo6vNVFneJEo58ok50rX/88ccbnuPu4k4SAIASDJIAAJRgkAQAoETTxySjBXX9c/lo9/XWxiT9QtdVdp+PdutG+6sSw7nvvvuyNh/96EezOh/v69u3b9bGx1CiuKE/p6qxIF8X9WkfH6+y4Hs0+T2KST799NMNjwVp6dKlWd273/3uQrl///5Zm2gBC78ISfR9+kUuIv56GC1cEMUp/bUtWkzA9/EoJjp//vxCefr06VmbqM/169evUPbX3o7CnSQAACUYJAEAKMEgCQBACQZJAABKNH3iTpQU4wPD0YRTP8G2Kp/wM2zYsKzNmjVrCuUqCRRoe1FSihclfkUTm0eMGFEoR9/p8OHDC+WqO3x4VXbhqJLcEyUO+Z3so2SIIUOGZHU333xzfLItvH+V339Xc/XVV2d1v/rVrwrlZ555Jmvjd/OQ8iSYqI1PFhs3blzD40Tfb7TwhO+/UZKQv/5Gx/ELJUQ7d/zoRz/K6mbPnl0oV9lNpD1wJwkAQAkGSQAASjBIAgBQwlqKl5lZUwbTvva1rxXKEyZMyNrceuuthfLPf/7zSseePHlyofytb30ra/Of//mfhfL1119f6dh7mpRSvhJxB+jsfnf00Udndf/wD/9QKEeTpnv06FEoRwtW+An/UdwyihP6yd7RhHAfA4x2e4/O2xs0aFBW94EPfKDh+7eVzuh3nd3nIn4y/cknn5y18XHwKNfC161fvz5rE32fvj9HbZYsWVIo+3wMSbrnnnuyuj1NS32OO0kAAEowSAIAUIJBEgCAEgySAACUaDFxBwCA7ow7SQAASjBIAgBQgkESAIASDJIAAJRgkAQAoASDJAAAJRgkAQAowSAJAEAJBkkAAEowSEoys8vM7IEWfn6HmV3akeeE7snM7jWzj5X8bJyZbTGzd3b0eWHPZ2bJzCa+3Z+hZd1qkDSz481stpltNLN1ZvagmR3R6HUppZkppR+3cNwWB1l0bfWBa+d/b5nZtl3KFwXtv2hmC+o/X2JmP6vyPimlV1JKfVJKO8ratDTIojnUv8P1ZrbvHnAul5nZjl3683wz+/M2Ova/mdnft8Wx2lO3GSTNrJ+kX0u6VtIgSaMlfUVSvjPt2zvuXrt/dmhm9YGrT0qpj6RXJJ21S90Nu7atP5G4WNIp9fZ/JOmu3T0Hq+k2f89dlZlNkPTHkpKkszv3bP6Ph3bp3+dJusrM3tXZJ9VRutMf1SRJSindmFLakVLallL675TS3J0NzOyb9X/BLTCzmbvU/59/ndf/ZfWgmV1tZmsl/UzSv0o6pv4vrQ0d+7HQZI6Q9JuU0h8kKaW0IqX0Q9dmfL2PbTaz/zazIVLtAlp/bLZXvXyvmX3dzB6U9Kqkn6h2gf1uvS9+t+M+FtrIJZIelvRvkgohnvqd17+Y2W31vvGImR0QHaT+1Gyxmc0IfrZv/Vr3ipmtNLN/NbOeVU4upfR7Sc9LOmiX451tZs+a2YZ6n9z1ZwfV6zbU25xdr79c0kWS/u96X721yvt3hu40SM6TtMPMfmxmM81soPv5UZJelDRE0lWSrjczKznWUZLmSxou6cOSPqn/+dfWgHY5e3QVD0u6xMy+YGZ/VBJf/JCkj0gaJmkfSX/dwvEulnS5pL6SLpN0v6RP1fvip9r0zNERLpF0Q/2/95rZcPfzP1XtCdhASS9L+ro/gJmdLulGSeellO4N3uMfVbtpmCZpompP1b5c5eTq4alJkh6vlyfV3+uzkoZKul3SrWa2j5ntLelWSf+tWl/+S0k3mNnk+j8Mb5B0Vb2vnlXl/TtDtxkkU0qbJB2v2mOM6yStNrNZu3TCRSml6+rxnh9LGqnaIBhZllK6NqX0ZkppW7ufPLqMlNJPVbtYvFfS7yStMrO/cc1+lFKaV+9bP1ftYlbm31JKz9b74hvtctLoEGZ2vKTxkn6eUnpC0h9U+wfTrm5JKT2aUnpTtUFmmvv5BZJ+IGlmSunR4D1MtX9UfS6ltC6ltFnSP6g2+JY5un4nuFnSo6o9sXip/rMPSrotpfTbev/7pqSeko6VdLSkPpL+MaX0ekrpbtVCXhdW+HXsMbrNIClJKaXnU0qXpZTGSDpE0ihJ367/eMUu7V6t/98+JYda3G4niS5jl2zULWa2ZWd9SumGlNIpkgao9hTia2b23l1eumKX//+qyvuhRF/sSi6V9N8ppTX18r/LPXJV477xWdUG2WdK3mOopF6SnqgPfBsk/Ve9vszDKaUBKaW+kkZIOli1gVWqXUMX7WyYUnpLtT45uv6zxfW6nRbVf9Y0utUguauU0guqPfc/pDUvb1AGds1G3Zn04H/+RkrpPyTNVev6oURf7BLqMcEPSDrBzFaY2QpJn5N0uJkd/jYOdYGkc83sMyU/XyNpm6SD6wPfgJRS/6h/RlJKKyX9QtLOx6PLVLv73fk5TNJYSUvrPxvrEsrG1X8mNUlf7TaDpJlNMbPPm9mYenmsarf9D7fB4VdKGmNm+7TBsdCF1RO/3mdmfc3sHfUEsYMlPdJGb7FS0v5tdCx0nHMl7ZA0VbVHqNNUS465X7U4ZVXLJJ0s6TPRVI36Xd11kq42s2GSZGaj3ZOMUmY2WNL7JT1br/q5pPeZ2cn1GOTnVZsxMFu1Pv2qask5e9eTiM6SdFP9tU3RV7vNIClps2oJN4+Y2VbVBsdnVPtSd9fdqnWaFWa2plFjdGubJH1RtakiG1RLEvvzlFJbzbP9jqTz61na17TRMdH+LlUtFv1KPeN5RUpphaTvSrro7Uw1Sym9otpA+bcWz5n9G9WSfh42s02S7pQ0uYVD7szc36JaZutq1eLqSim9qFry4rWq3aWepdoUqNdTSq/XyzPrP/uepEvqT/Ek6XpJU+uPfX9Z9fN1NEupKe54AQDocN3pThIAgLeFQRIAgBIMkgAAlGCQBACgRIsZU2a2x2X1nHjiiYXy2WfnawB/7nOf66jTqWTGjBlZ3Qc/+MFC+frrr8/aPP744+11SpWklMqW5WtXe2K/84YOzedeX3JJMVP//PPPz9r88IfFZVrXr19f6dh77VX8U73wwnzRklmzZhXK3/zmN7M2zaAz+l0z9Dm0n5b6HHeSAACUYJAEAKAEgyQAACUYJAEAKNHiijsdGcw+8sgjs7qbbropq1uyZEmhHG35uGPHjkL51lvz/Tznz59fKC9eXG0zhb59+xbK+++fLz04c+bMQnnq1KlZG/85+vTJ1xfu0aNHVnfaaacVymvW5KvgveMdxX/7vPXWW1mbKrpD4s5xxx2X1V177bVZ3ahRowrlgQP9dqT5d/rOd+ZbRfrv4qmnnsrarFq1quH7H3BAvtfugAEDCuWRI0dmbebOnVsof/e7+b7M1113XVbXkUjcQUcjcQcAgFZgkAQAoASDJAAAJTotJjlx4sRC+de//nXWZtmyZVmdj8H5+Jsk7bNPcVvHXr16ZW18DHDdunVZm9deey2r8zHIKCa4bdu2hsf2k8MHDx7csI0krVy5slC++OKLszatjUF6zRaTrBKL9TFdP7lfir93H+fevn17q95/3LhxhfKzzz6btXnhhReyurPOOqtQ3rp1a9bm9ddfb/F8JKlnz56F8r777pu1Wbp0aVb3hS98oVC+5557sjZthZgkOhoxSQAAWoFBEgCAEgySAACUYJAEAKBEpyXu3H333YVylKSyYcOGhsd58803G7aJdlrwr4uSe6Jz8kkdr776atbGJw5FiSAHH3xww/dasGBBVjd8+PBC+Sc/+UnW5oYbbsjqWqPZEneqePDBBwvlIUOGZG02bdqU1e29996FcpQU4xN1or8tX9evX7+sTe/evbO61atXF8q+j0n54gXR30aVv5doYQvfhw866KCGx2ktEnfQ0UjcAQCgFRgkAQAowSAJAECJPBDWQY4++uhCed68eVmbESNGZHV+geY33nij4Xv5RcmlfMJ/FGOK4jc+dugXlZbiOKXnY1yRLVu2ZHX9+/cvlNtzUnezu+SSS7I6v4jFihUrsjZRfNjHEluK5bfEf+/RdxwtFOD7Z9TvfZ+O4paNjivFMdlhw4YVyp/73OeyNldffXXD9wOaDXeSAACUYJAEAKAEgyQAACUYJAEAKNFpiwlMnz69UL7pppuyNhMmTMjq/CIEflcMKZ/UHSVi+B0TIlUmjEdt/K4f0S7yGzduLJT970OKk0P8rhHRLhZtlczT7IsJRMlgfheMKMkq+r37pBg/cV/KdwqJ+oaZtViW4t1DfMJPdN4+mSdalMDr0aNHVhclrPkkoCgpyO9w0losJtC+qlzXJOnTn/50oTx79uyszeOPP952J9YGTjzxxKyuyvWQxQQAAGgFBkkAAEowSAIAUKLTFhPwC4r7RQKk+Dn5oEGDCuXFixdnbfyE+2iB8yqxxSiW6euqLBwQLZ7ud3+PFjx4+eWXGx57zJgxDdt0F9OmTSuUo3ibn6gfLeoQxdt8vC+K20X91fP9rMproveL+qZfmDyKu/vXRf0+qvMx2ej9/e9/zpw5WRs0D78JQ5X4YxRj96ouxOGPFb3ukEMOKZR9zkrVc2oJd5IAAJRgkAQAoASDJAAAJRgkAQAo0WmJO4cffnihvO+++2ZtXnrppaxu8uTJhXK0Y4FPlKmyG0KUiFAlqSJKDvHvHyV5+N0f7r///qyNT0CS8uSMaBf57ur73/9+oRxNpt+8eXOhHAX1ozr/e/bHkfLvOUqAqfJeVdpFr3vttdcK5Sr9PjpOtFCCT9yJktGuvfbaQvmP//iPG74/Ol7VZLG77rqrUP74xz+etVmwYEGhvHz58tafmFMlwefyyy8vlM8///w2e/+duJMEAKAEgyQAACUYJAEAKMEgCQBAiU5L3BkyZEihHK18Eq2U40WJM16VBIqqr/Mr7IwcOTJr43cmiVbFGTVqVKH81FNPZW2ipBy/w8j++++ftemuTjnllEL5xhtvzNr4ZBL/+5TynUKkaivl+EQZv0pP9LrW9s0qCT9+V5Lo/aLkHr8qkSSNHj26UH7iiSeyNh/96EcbnhM6XpWVayLHHXdcobx9+/aszVVXXVUoR9exwYMHF8rRNTv6m/OJd9Hfk++X//RP/5S12V3cSQIAUIJBEgCAEgySAACU6LSY5IgRIwrlKjt1RKpMmK4imvDvJ2dLUr9+/Rq28QsFPPbYY1mbVatWFcpHHXVU1iaKZfrJu+PHj8/adFc+lnb22Wdnba688spC+YorrsjaPPTQQ1mdj6tEMRS/60bUf6tO5PZ8HCmKZfpzivq0323mmWeeydqcfvrpWd0111xTKH/mM58pP1l0CX7BlwcffLBhm+HDh2dtfF+NFo6J4pQ+JhktHPPcc88Vyv7vVMp3XHq7u4JwJwkAQAkGSQAASjBIAgBQgkESAIASnZa44wOs0UTVKAg7YcKEQvk3v/lN1ibaoaA1op1BfDDZLxwgSYMGDSqUo4CzX5QgSrLYsGFDVucTldgF5O3xiTu+XGbx4sWFcpSw5Sc7R0k6VXbzqCKaEO7/hqL395OvL7zwwqzNmWeemdVFCxN4rZ20js532GGHZXU+4Sa6Hvo+F+0C4q+Z0e480UIBPhEu+pvzO9ZE44j3dvsld5IAAJRgkAQAoASDJAAAJTotJuljadFiAj5+IuULoUfxPr94emtFccIBAwYUylFM0E9ejSZ+V4kl+rillE/EbavFFNAyH/uIYomticFVjUm2JnYZxZC8aKGLKP5YJd5IDHLPVOV7iRaQ8AuXRDkSfhGA6Frn+2HVa5bPLYkW3vf5H1G8c3dxJwkAQAkGSQAASjBIAgBQgkESAIASnZa444Oy0cTnKPFg27ZthXK0erxPsoiO7QPMUZJOdGw/wXXcuHFZm3vuuadQ9sFlKf9s0WeNzrvKhHV/rOiz4e2JJjJ7/rvw/VBqfQKMr4sSeaok9/j+W2Xyddk5oeuIEnfmzp1bKI8aNSprE10jPf93EP1dRMli/joWtfHjSLQr0pNPPtnwHFvCnSQAACUYJAEAKMEgCQBAiQ6JSUYTTP1z6ShuFsVYosUDvCpxO39O0bP16HXPPvtsoRzFdIYNG9bwHL1oomzE/56i9/cx0FWrVr3t8+mqWrsIt+8vUd+ociwfV6kaW6wSb/Tv7xfekKRly5Y1PE6kyufv6tpqAYndeb+2eP+vfe1rWd3ChQsbHtsvpCLl19oo3lhlUYvo+u/7WHSctWvXFsonnnhi1uaWW25p+P4t4U4SAIASDJIAAJRgkAQAoASDJAAAJTokcSdKZPFJBVEigN/xQsp3xoiSgnydn0At5SvRR0Hh+fPnZ3UjRowolKMdtf1nic7RiwLeEX/e0e/IT7DF7vPfaZWkiqhP+cSdKPGirRI2osSdNWvWNHxda9+vq+vo30GV9/PXg+had9555xXKBx98cNZmxYoVWV2/fv0aHtufYzThv8rniP5W/Oui46xcubJQPuSQQxq+19vFnSQAACUYJAEAKMEgCQBACQZJAABKdEjizpAhQ7K6KOHFq7IKThTw9UkwPrgt5UHooUOHZm1OPfXUrO7FF18slKOVIqrsuuE/2/r167M20TlVWekkWhkDu8cnwURJDL7fRQk4VRKAqiTzVEnuiRLG1q1b1/B1ke6YuFNldaYq30NrVfmd+344c+bMrM3HPvaxQvnf//3fszYHHHBAVueTcA466KCsjf/80d+FP06VBKCoLvpd+2P37Nkza3P88ccXyg888EDWpiXcSQIAUIJBEgCAEgySAACU6JCYZJ8+fbK6Ks/b/cIBUrWYnI/FRMfxcbsobup3/IiO1bt374bnUyVGGbWJ4q3+s1V9HXZP1cUedhX18Sox9SqieGOVhQqiyd5ovc6O1X7wgx8slE844YSszTnnnFMoRzHBCy+8MKsbM2ZMobxhw4aszebNmwvlKP+jyoID0d/XokWLCuXo2j9w4MBC+eWXX87a+M9GTBIAgDbCIAkAQAkGSQAASjBIAgBQokMyPKrsghGJAsw+cSYKFG/durVQHjt2bNZmypQphfI999yTtYmSKnzCz8aNG7M2/pxam4D02muvZXVVEkha+/tGOZ9sEE1s9gsOVNkFJNJWiwlEx4mS6BDryKScQYMGZXXnn39+oXzSSSdlbf7whz8UyldeeWXWxl8PjjvuuKzNu971rqzOX7c2bdqUtVm6dGmhHO08469ZUbJhdK3z18gqCxVEf1/jxo3L6t4OrqYAAJRgkAQAoASDJAAAJTokJtm/f/+szj9fjuI3UfzE76Adxfb8wuDR8/5bb721YZuIf07f2gn//ryjz1FlEYJIFKfF7qkS5/XxkCqvidpUiVdXiZdFx2nNogjd1cEHH1woH3nkkVkb/5337du34XFHjBiR1UULjD/99NOF8o9+9KOsje8/J554YtbGxxKPOuqorE20MLjfhKFfv34N3z/qX8OHDy+Uo80c9t1334bvv2TJkqxNlWtkr169GrZpCXeSAACUYJAEAKAEgyQAACUYJAEAKNEhiTt+Ar6UB3yrJCtIeeJMlNyzdu3aQvmll15qeE7R+0d1VZJifDA5Chz7NlEChw9cS/lni9pEE3qxe1555ZVC2ScjSHnSQpUJ/1V3RPDHihJ3/Pfelok7Vd6/mUUJeKecckqhPHjw4KzNo48+WijfddddWRufzBMl7qxevbrhOQ4bNiyr80mRfueO6P2jfvnGG29kdT5RJ7rW+N9JNOHfX9ui5KbodX6Bgeha76+j0d9TaxMgd+JOEgCAEgySAACUYJAEAKBEpy1wXjUG6W3btq1QjmIJ/hl0lUWdo+fWVSaDR5/Dvy6KY1ZZTCGKAfhjR8/yiUmWa21srcrr/HfTlgvN+/er8vcTxZ569+7dZufUlVxwwQVZnd/1PprwPnPmzEL58MMPz9r4TRCivhNN1Pd1URzcxyCja42Pd0afwy/AIlVbHMP3wyrXnujzV/l7inJbfEw2uh56hx12WMM2hfN4W60BAOhGGCQBACjBIAkAQAkGSQAASnRI4k6UgFIl8aDKqvNRoLg1k0ejoHRrEy/8Z4uScnyAOXqvKp8jCtRHv2/sHj8JP+q/PlEm+t6raKt+FyXu0DdijzzySFbnd6uIFhMYNWpUoRztJjR27NhCOUqeihZ58IuQRIkr/juOdvOYMmVKoewn6UtxMk9rEtGivwt/7OiaHV3rN2zYUCj7HaAkac2aNYXyqlWrsjb+Wjt58uSsTUu4kwQAoASDJAAAJRgkAQAowSAJAECJDkncCd/YJTVESSo+KC7lQWC/Ao+UB2qjYHKV94/4xAe/K4mUB7ijBA7fprUr/kSJO6yq0vb89x6tEOL7WZXVmKokAEV1VVYtidpEO9JU0dV3AZk/f35Wt2DBgkI5WvFmyJAhhXKUuOMTfqLdPKKdMXyCy8iRI7M2BxxwQKEcXUf8OUY7fixdujSr80kwfgciSdq+fXuh7FfAkaStW7cWysuWLcvaRHwfi66jvj9HSUmbNm0qlOfNm1fp/XfiThIAgBIMkgAAlGCQBACgRIfEJKPn5P55e7R6e/Tsety4cYXys88+m7XxcZ7o/X1dFFuM4n1VYkpVYpJV3j/iJwtHz+DbcveJrqa1sbQqk/D991xlZ4Oo/0S7q1f5TqvEDavGgxoduzvwv79oMntU5/nfXTSZPopJ+tyCqA/4a1R07Kg/eT5uKOWxvOga5XNEJk2alLXxfzs+jll2jv79ojHCn6MvR8d+u9cArqYAAJRgkAQAoASDJAAAJRgkAQAo0SGJO1Fygg8wRxNs/QrvkjR9+vRCOQrU+gB3laSLKAEmSvhZvXp1w2P5ycPR+/uV6KM2UeJQ9DvxSNwpVyW5JdrtwffXKPnAHzv6HqLdHqqocuwqWvv+aD3fx6IElGiiflS3p/ELEzz55JOddCbth6spAAAlGCQBACjBIAkAQIlOW0zAT2qO4kA33HBDVnfaaacVymPGjMna+DhllQXGI62N+/iYQxRb7NOnT6EcxRqvuOKKrG706NGFchQjjWLAqPExueh3FfVF/x1Gk6/9satMkI76WDRx38e1qsQWo+NEfwsAynEnCQBACQZJAABKMEgCAFCCQRIAgBIdEsWPkhP8Dt7RZP7HHnssq/NJOc8999xunt2e67777svq/uIv/qJQjpJDot3RUVNlN4tRo0ZldT5xJ9q1we+IMHTo0KxNa3ch8aLPsXLlyoav80llVXaoAboz7iQBACjBIAkAQAkGSQAASnRITPKhhx7K6k466aRCOVqofM6cOQ2PXSWm0gwTqKM40O9///us7umnny6Uo53m77nnnrY7sS6mSrzt3nvvzepmzJhRKPtd46U8luljlJI0cODAQjlaaCKKW/q4fhST3LBhQ6EcLZDtFyGoGn9sq1gq0Gy4kwQAoASDJAAAJRgkAQAowSAJAEAJIyAPAECMO0kAAEowSAIAUIJBEgCAEgySAACUYJAEAKAEgyQAACUYJAEAKMEgCQBACQZJAABKMEgCAFCiSw+SZrbQzLaZ2WYz22Bms83sk2bWpT83OoaZbdnlv7fqfW1n+aLOPj90b2b2ITN7vN4fl5vZHWZ2/G4e814z+1hbnWMz6A6DxVkppb6Sxkv6R0l/I+n6qKGZvTOqByIppT47/5P0imp9bWfdDTvbmVmn7/q9J5wDOo6Z/ZWkb0v6B0nDJY2T9D1J53TiaTWl7jBISpJSShtTSrMkfVDSpWZ2iJn9m5l938xuN7Otkk40s1Fm9gszW21mC8zs0zuPYWZH1v9ltsnMVprZt+r1Pczsp2a2tn7H+piZDe+kj4pOZmYzzGyJmf2Nma2Q9CMz29fMvm1my+r/fdvM9q23v8zMHnDHSGY2sf7/zzCz5+pPRJaa2V/v0u5MM5uzy5OSw3b52cL6OcyVtJWBsnsws/6SvirpL1JK/5lS2ppSeiOldGtK6QsN+uJAM/t1/fq3vv7/x9R/9nVJfyzpu/W70+923qfsON1mkNwppfSopCWqfdmS9CFJX5fUV9JsSbdKekrSaEknS/qsmb233vY7kr6TUuon6QBJP6/XXyqpv6SxkgZL+qSkbe3+YbAnGyFpkGpPMC6X9HeSjpY0TdLhko6U9KWKx7pe0ifqT0QOkXS3JJnZuyT9b0mfUK3f/UDSrJ0XvLoLJb1P0oCU0pu795HQJI6R1EPSLSU/b6kvvkPSj1Trt+NUu459V5JSSn8n6X5Jn6o/LflUO53/HqXbDZJ1y1S7gEnSr1JKD6aU3pJ0qKShKaWvppReTynNl3SdpD+tt31D0kQzG5JS2pJSeniX+sGSJqaUdqSUnkgpberAz4M9z1uSrkgpbU8pbZN0kaSvppRWpZRWS/qKpIsrHusNSVPNrF9KaX1K6cl6/eWSfpBSeqTe734sabtqF8CdrkkpLa6fA7qHwZLWtPCPotK+mFJam1L6RUrp1ZTSZtVuIE7okLPeQ3XXQXK0pHX1/794l/rxkkbVH11tMLMNkr6o2jN9SfozSZMkvVB/pHpmvf4nkn4j6ab644urzGzvdv8U2JOtTim9tkt5lKRFu5QX1euqOE/SGZIWmdnvzOyYev14SZ93/XWsO+5iobtZK2lIC4/XS/uimfUysx+Y2SIz2yTpPkkDunO+RrcbJM3sCNUGyZ0xoF13nV4saUFKacAu//VNKZ0hSSmll1JKF0oaJumfJN1sZr3rz/u/klKaKulYSWdKuqTDPhT2RH4382WqDWo7javXSdJWSb12/sDMRhQOlNJjKaVzVOt3v9T/POZfLOnrrr/2Sind2MJ5oOt7SLUnCueW/Lylvvh5SZMlHVUPK72nXm/1/+12/anbDJJm1q9+53eTpJ+mlJ4Omj0qaXM92aGnmb2znuBzRP0YHzazofVHsxvqr3nLzE40s0Pr/9rapNrjsbfa/1Ohidwo6UtmNtTMhkj6sqSf1n/2lKSDzWyamfWQdOXOF5nZPmZ2kZn1Tym9oVr/2tm3rpP0STM7ymp6m9n7zKxvh30q7HFSShtV61//Ymbn1u8O9zazmWZ2lVrui31Vi0NuMLNBkq5wh18paf+O+SR7hu4wSN5qZptV+1f330n6lqSPRA1TSjtUuwucJmmBpDWS/j/VknIk6XRJz5rZFtWSeP60HusZIelm1S5gz0v6nWqPYIGd/l7S45LmSnpa0pP1OqWU5qmWjXinpJf0P085drpY0sL6469PqhZTUkrpcUkfVy2xYr2klyVd1s6fA00gpfTPkv5KtYSc1apd/z6l2pOI0r6o2rSRnqpd+x6W9F/u0N+RdH498/Wadv0QewhLqdvdPQMAUEl3uJMEAKBVGCQBACjBIAkAQAkGSQAASjBIAgBQosUFj81sj099HTUqX7TkZz/7WaF88803Z2169eqV1XkLFy7M6k488cSGrzvmmGMK5enTp2dt3njjjYbH6WwpJWvcqu01Q7+LXHNNMSN+wYIFWRv/vW/evDlrc9BBB2V1v/71rwvlnj17Zm3mzZtXKPftm0+XfOaZZ7K6PU1n9LvO7nPveEd+v/LWW29/qvXMmTOzuo98pDjjbfv27VmbqD8tW7asUP70pz+dtfGiz+FnUOyJMypa6nPcSQIAUIJBEgCAEgySAACUYJAEAKBE0+9U/oUvfCGrmzBhQqH8y1/+MmtzxhlnNDz2q6++mtVt2LChUP6Xf/mXrM3Pf/7zQnnGjBlZm9/+9rcN3x97rsGDB2d1f/mXf1koRwlj69evL5S3bt2atTn//POzuoceeqhQPuWUU7I2v/nNbwrlKDlt6dKlLZ4P2p9ZniPS2mSWww8/vFCOrkdDhgwplJ9//vmszZtv5ltPDh06tFA+88wzszY+oaxKslFbJSl1FO4kAQAowSAJAEAJBkkAAEo0fUwyijd+9rOfLZR79+6dtYnqvCim4ydsX3DBBVmbffbZp1COJpWjuU2ePDmr83HDKO68du3aQjmaxD169OisbsmSJYXy/fffn7UZMGBAoezj51LeN9G2qsQbWxt/9Nc1SZo6dWqh/Ktf/Sprc9ZZZxXKP/zhD7M2F110UcP3P+2007K6iy++uFC+4gq/R7P0wgsvFMp7cvwxwp0kAAAlGCQBACjBIAkAQAkGSQAASjR94s7vfve7hm0uu+yyrM5PqvZJD5I0adKkrM4nQ4wfPz5r4yfvrl69uuE5orlEu8Fs2rSpUI6+d58wtmPHjqyNTwCSpEGDBjV8nd9RJEoi8RPEV65cmbVBx/PJNZJ06qmnFsoDBw7M2vhEsH333Tdr4/vKt771raxNv379srrvfe97hXK0c9Hrr79eKP/VX/1V1sYvWDF79uysTZRwtKfgThIAgBIMkgAAlGCQBACgRNPHJCMPPPBAobz33ntnbaKFpT3/vF2SnnrqqULZxx8lac6cOYXyxo0bG74XmsuwYcOyOr9wdLSQs48ZRZP7ffxRkvbaq/in+s53vjNr4xepfu211xoeB20r+s59TPC8887L2px99tlZnY9xR4vR+4X2999//6yNX8xkxYoVWZso3ukn/Y8ZMyZr4xcKiD6/z/eINnxYuHBhVuevtVGfj2LzbY07SQAASjBIAgBQgkESAIASDJIAAJToklF8nzizZcuWrI3f0XvZsmVZm+eeey6rO+GEEwrlPn36ZG2qLHCA5hYlSPh+F+3C4ZNrop1mRowYkdX16NGjUI4Sfvxk72i3BX8ctK0qiSQHHXRQVhclWfnrVvR9+gUjokVRfL988cUXszaf+cxnsro/+ZM/KZRvuummrI3/O1izZk3WxicgRT7wgQ9kdT5xpyOSdCLcSQIAUIJBEgCAEgySAACU6JIxyfvuu69Qfs973pO1mTBhQqEcxR9XrVqV1flJt9Hk7GgBX3QtUezHx5Wiyc89e/YslKMFK6JYpp+kvW3btobnuH379qzOx0TR8caNG5fV+Qn/Un5tOeyww7I2vv9EcUN/nNGjR2dtogXGfUw0WgT91VdfLZSjBfOnTp2a1XnLly9v2KazcCcJAEAJBkkAAEowSAIAUIJBEgCAEl0yceeII44olKMki5dffrlQ9gFoKd9FXpJ+//vfF8rvf//7szb9+/evcppoYtHEbl/nk3SkfML/u9/97qzNSSedlNU98sgjhXK0A/29995bKEeJQ9Hu8mhf/voTXWsmTpyY1a1bt65QjpJbfHLWqFGjsjZ+14/o/aNr5Nq1awvlaDejsWPHFsp+cQNJevLJJwvloUOHZm2iRTX8Dif+fDoKd5IAAJRgkAQAoASDJAAAJRgkAQAo0SUTd/xqOj7ZRspX2PGvkaQDDzwwq1u4cGGhHK2of/DBBxfKd9xxR8mZollVWbkmWvHGJx/cf//9WZvp06dndX73Gb9DgiTtvffehfKgQYOyNqtXr45PFu3GXw+iHT/GjBmT1flVll555ZWsjV9Nx1+fpHxVnv32269hGylPwonO0fe5aJeZ4cOHF8pRQllU53dLeeCBB7I2HYE7SQAASjBIAgBQgkESAIASTR+TPOecc7K6pUuXFsp+546oTTTBddq0aVmdf3a/bNmyrI2f0Dt+/PiszaJFi7I6NI8o3rjPPvsUyn4XBUnq27dvoRzFYk4//fSszscufSwoer9owYPovNG+/PVn2LBhWZtopwy/G0yUWzFy5MhCed68eVkbv7hJtJtHFG+cM2dOoRztQrJ169ZC+cEHH8zaXHXVVYXyf/3Xf2VtooUKDjnkkEKZmCQAAHsYBkkAAEowSAIAUIJBEgCAEk2fuHPuuedmdT54fNxxx2VtfF00CTdaYMAH03/72982PMczzjgjq/v+97/f8HXYc0WLCfiEiCqTpn1/KuMnoEc71PhEHT8ZXYqTidC+/I4t0U4ZUV+ZNWtWoRwtbuKTgKJdXlJKDY8T9SefZBYd2+9UcuaZZ2Zt/K4j0XtFiWhVFuzoCNxJAgBQgkESAIASDJIAAJRo+pjkkCFDsjq/MMCjjz7a8DjRc/oofuMXFI4WCvCTyqO4KTHJ5hYtIuF3d492W/d1Tz/9dKX3u+uuuwrlY445puFropjOnhLn6U78tWX06NFZm2jRc7+IfRS383FCv5i4lC+c4hfZl/K4oSQNHjy4UH744YezNuPGjSuUo2vm5ZdfXih/4xvfqPT+a9asyeo6A3eSAACUYJAEAKAEgyQAACUYJAEAKNH0iTtRIoJPqogC3n6H9mgV/jvvvDOr87uAREF4dn/v+qLveMSIEYVyNEHcJ1pUXUzg+eefL5SjxB2fxBYdm8UEOp7fFahnz55Zm6g/LViwoFCOFiXZvHlzoRztpuGvWT6xUIp3BvFJZRMnTsza+M8Svb/fqcQn+0jS3Llzs7rDDz88q+sM3EkCAFCCQRIAgBIMkgAAlGj6mGS0mIB/Bh9N6vaiNtEEVx9fiOKdfkdvvwABml8U7/O7u/t+KOX9ZdOmTZXez08I94uZS3msKYo9RXFSdKw+ffpkdVUWzI/i0L5fRDFBH89+4YUXsjb+uiZJkydPLpQPOuigrI2/tkW5Hffff3+hfN9992Vt/CLwUr5Af9R3d+zYkdW1Ne4kAQAowSAJAEAJBkkAAEowSAIAUKLpM0r8zguSdOyxxxbKzz33XNbGJ15EgfMomce/btWqVVkbHwT3u4ej+UU7FPjvPUrK6d+/f6EcJXVFO8D794sWBfATy6O+SRJZ++rbt29W53fdWLFiRdbGT7iX8sSvKOHHLzgQJRumlAplv+iFFE/w9+ft+5ck7bfffoVylWSxaMelKOHI9/EJEyZkbf7whz9kdW2NO0kAAEowSAIAUIJBEgCAEgySAACUaPoofpRAsWjRokK5d+/eWRufTBMlAA0dOjSr87t+zJ49O2szadKkQtnvSoLmF624M378+EI5SgbzddFKI1HizvLlywvlKPnC91e/i4Mkbdu2LatD24l20/Ar1UQrMUWryfg+9uijj2Zt/LVu4MCBWRv/nUfJNX7lHklavHhxoRxdI/05Rm18ApBPXpPinVH8Zxs8eHDWhsQdAAA6EYMkAAAlGCQBACjR9DHJaFK1f74ePYP3z8WjuOXWrVuzuoULFzZ8/+h16FruuuuurM7HmvxEaymPPUXxx8j27dsL5YcffjhrM2jQoEI5mnwd7QCBtuO/JynvF9Fk+vXr12d1fmGAKN7o8x+iXZGeeuqp+GR3Ee0qM3369EI5WijB79QRLTiwbt26QjnaOSm6/vrFMKK4ZUfgThIAgBIMkgAAlGCQBACgBIMkAAAlmj5xxyfSSHkQePXq1VmbGTNmFMpRMDmaMO6PfeSRRzY8x3vvvbdhGzSXKEHjxRdfLJSjRAs/sTo6TsTMCuUxY8ZkbXw/95PYpbwvVn1/VDNlypSszu+8Ei0uEiWl+ESsaOGU++67r1A+88wzszZ+h5FoAQu/SEr0fitXrsza+Otm9Dl8m2hXpkMPPTSr89ffaKeQjsCdJAAAJRgkAQAowSAJAECJpo9JRs/pTzjhhEL59ttvz9r42GKV3eClfAHfaBKsj/M8/vjjWRs0t2hB6iqi2HcVfnf54cOHZ238QtazZs3K2vj+21lxnu7ET7j3C+FL8QLjfoJ/FBM86qijCuXo+/QLT0QLDkTXOh8HjxbH93H3aHEVH6eMNo6IXuff74ADDsjazJkzJ6tra9xJAgBQgkESAIASDJIAAJRgkAQAoETTJ+7cdtttWd11111XKN95551ZG79TR5S4EwW4fbt58+ZlbS644IKG74/m9tprr2V1PrkmmrTtkxF69OhR6dhetNPMpk2bCuVoZ4dotwm0HZ+kI+ULSEQLoCxatCirO/bYYwvl4447Lmvz8ssvN3z/ww47rFD2u3JIcV/1fTNaeMK3iXYKed/73lco+909pDgB0h8r2s2pI3AnCQBACQZJAABKMEgCAFCi6WOS0a7b/pl3NIF79uzZhfIRRxyRtXnXu96V1T377LOF8sSJE7M2fifyxx57LGuDrsfHCaPFnn1cJWpTJSYZTez27x/FeXzfRNuKFrX3sbVoIYpXXnklq7v11lsL5Y9//ONZG99Xotji5MmTC+UoVt2/f/+szudf9OnTJ2uzfPnyQtnHXyVpxYoVhXLUL6Pfm/89+ffqKNxJAgBQgkESAIASDJIAAJRgkAQAoETTJ+5EiwD4nb9PO+20rM0tt9xSKEer8J977rlZnZ9Q++abb2Zt/I7ar776atYGXY/vG9HEar+zQrQjQmsn/Pt+FiWIRH8vaDuDBg3K6lavXl0oRzu4RBP1fRJMtJhAW4mScvy1LUr4aU0i2Mc+9rGsrkq/jJKSOgJ3kgAAlGCQBACgBIMkAAAlGCQBACjR9Ik7Eb8zx9SpU7M2J554YouvkaQBAwZkdT54HK3eH60oga7PJx/069cva+OTa6LkniqiHWp8okV0bL9LxI4dO1r1/ohFu7P4VZWiRL7Nmzc3PLZP+pLynWdaa8uWLW1ynCqiFaWipKDnnnuuUO6svsqdJAAAJRgkAQAowSAJAECJLhmTHDZsWKEc7ZjgY5DRKvhVntNPmjQpq4smBqPrq7ILyIEHHlgot7avrFy5MqvzE9mjnSVYTKB9+YVMpDyWttde+WV3zZo17XZOPpYZxTajOi+Kf/q66Dg+3ugXV5DiOKXfMSe6RncE7iQBACjBIAkAQAkGSQAASjBIAgBQoksm7owePbphG5/AMGHChKzN7bffntXtt99+hfLee++dtRk1alTD90fX8+53v7tQ9rvBSNKcOXMK5WgSdaRHjx6FcpS44xcKiHaoQfvyiVlS/j2sW7cua9Oek/l9ck1bLUAQ8X0wEv1dRH8HvXr1KpSjXW06AneSAACUYJAEAKAEgyQAACW6ZEzSGzlyZFbnFxOYMWNG1mbgwIFZnX+eHu0sH00oRtf3la98pVD+0Ic+lLUZPHhwoTx//vxKx/aTrZ966qmsjY+FR/0e7SuKt23atKlQ9nkNkrR8+fKGx44WIdjTFoeosijB4sWLKx3Lx+E767NyJwkAQAkGSQAASjBIAgBQgkESAIAS3SJx5/jjj8/q/O4d0QIAzzzzTFbndxW/9NJLszY+KQjdw9ixYwvlMWPGZG38TiHDhw+vdOwpU6YUyu9///uzNnfccUehHCWRoH0NGDAgq/OJOtGOF1FdM6qyUIG/hkr5346UJ55Fuzndcsstb+PsWoc7SQAASjBIAgBQgkESAIASXTImeeWVVxbKS5cuzdq8/vrrhfLs2bOzNnPnzs3qJk6cWCj/8Ic/zNrceOONVU4TXYyPj/Tu3Ttr4/vUggULKh37pZdeKpSffPLJrM3zzz/f4mvQ/p544omszl8zokUB/IIDkfZcmLytVDnH6LP6vitJa9asKZTvv//+1p/YbuBOEgCAEgySAACUYJAEAKAEgyQAACWsGYLBAAB0Bu4kAQAowSAJAEAJBkkAAEowSAIAUIJBEgCAEgySAACU+P8BNiRaDmeo6fMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 576x576 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels_map = {\n",
    "    0: \"T-Shirt\",\n",
    "    1: \"Trouser\",\n",
    "    2: \"Pullover\",\n",
    "    3: \"Dress\",\n",
    "    4: \"Coat\",\n",
    "    5: \"Sandal\",\n",
    "    6: \"Shirt\",\n",
    "    7: \"Sneaker\",\n",
    "    8: \"Bag\",\n",
    "    9: \"Ankle Boot\",\n",
    "}\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 3\n",
    "for i in range(1, cols * rows + 1):\n",
    "    sample_idx = torch.randint(len(training_data), size=(1,)).item()\n",
    "    img, label = training_data[sample_idx]\n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.title(labels_map[label])\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img.squeeze(), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preparando seu dataset para o treino com o Dataloader\n",
    "\n",
    "Quando você treinar seu modelo com seus dados do dataset, você vai querer passá-los como _minibatches_ (quantos dados serão alimentados antes de treinar o modelo), e também embaralhar esses minibathcs a cada _epoch_ (uma passada por todo o dataset) para que o modelo não veja informação na sequência dos dados. O **Dataloader** é a ferramenta do PyTorch que nós possibilita fazer isso de maneira facilitada."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "\n",
    "train_dataloader = DataLoader(training_data, batch_size=64, shuffle=True) # Aqui definimos o tamanho do batch como 64 e que embaralhe a amostra\n",
    "test_dataloader = DataLoader(test_data, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Formato do batch das features: torch.Size([64, 1, 28, 28])\n",
      "Formato do batch das labels: torch.Size([64])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAR+klEQVR4nO3dX4yV9ZkH8O9XQAREYIQZECb+qX+JZukGycaSjcZsQzVGuSH1onGN2emFTdqkiWvci3qzidls2+3Fpsl0NdBN16axNXJhlgKpMb0QQYMMiquIQ2AcHf4EHEER8NmLeTUjzvs843nfc94Tn+8nITNznnnn/Oad+XLOnOf9/X40M4jIN99FTQ9ARDpDYRdJQmEXSUJhF0lCYRdJYmYn74ykXvpvg0WLFpXWZs70f8SnTp1y65999lmlem9vb2nt5MmT7rHj4+NuXaZmZpzq9kphJ7kOwK8AzADwX2b2RJWv183IKc8fAKDp9uWdd95ZWlu6dKl77EsvveTWT58+Xan+8MMPl9a2bNniHrtt2za3XsVFF/lPaqOfadM/81a0/DSe5AwA/wngewBWArif5Mq6BiYi9aryN/saAPvN7ICZfQrg9wDurWdYIlK3KmFfDuDQpI8PF7d9CckBkrtI7qpwXyJSUdtfoDOzQQCDgF6gE2lSlUf2EQD9kz5eUdwmIl2oSth3AriO5NUkLwbwfQCb6xmWiNSNVVoIJO8C8B+YaL09ZWb/Gnx+1z6N91prQLVWi9cHB4B77rnHrd92221ufWhoqLS2cqXfIFm+/Csvs3zJnDlz3Pr58+fdutfi2rhxo3tsZP/+/W599+7dpbVz585Vuu9u1pY+u5k9D+D5Kl9DRDpDl8uKJKGwiyShsIskobCLJKGwiyShsIskUanP/rXvrME+e9U++rx580prGzZscI+Net3vvfeeW4/mnJ89e7a0Njw87B47f/58t3711Ve79bGxMbe+Z8+e0lp/f39pDQAWL17s1qOxe7306Ly88MILbt0750B7r9uIlPXZ9cgukoTCLpKEwi6ShMIukoTCLpKEwi6SRJrWWyRabfSRRx4prUVtmKNHj7r1M2fOuPWLL77Yrc+YMaO0Fo0tauudOHHCrc+ePdute+2xTz/91D02mj4b1b37jn7e0RTYZ555xq03Sa03keQUdpEkFHaRJBR2kSQUdpEkFHaRJBR2kSQ6umVzN3vwwQfd+rXXXlta85YsBuJ+sjd9Foh75Z6oFz1r1iy33tfX59ajLZu9sUfXeES98GjsXj06p0uWLHHrN954o1t/88033XoT9MgukoTCLpKEwi6ShMIukoTCLpKEwi6ShMIukoT67IU1a9a4da8nHC23HM1nj+arR73sdoqWRPbm0kei7yvqs8+dO9ete730q666quVjAeCGG25w693YZ68UdpLDAMYBnAdwzsxW1zEoEalfHY/sd5iZ/9AlIo3T3+wiSVQNuwH4M8lXSA5M9QkkB0juIrmr4n2JSAVVn8avNbMRkr0AtpJ808xenPwJZjYIYBDo7gUnRb7pKj2ym9lI8XYMwLMA/Je0RaQxLYed5DyS8z9/H8B3Aeyta2AiUq8qT+P7ADxb9GFnAvgfM/vfWkbVgGj74H379pXWli5d6h4bzQmP+smffPKJW/f61VEfPJrvHo0tUmUufiRaJ2DhwoWltZtuusk91ttqGoi3k+5GLYfdzA4A+JsaxyIibaTWm0gSCrtIEgq7SBIKu0gSCrtIEmmmuC5fvtytR9NMx8fHS2urVq1yj73lllvcutfWA+JtkU+fPl1ai7YejpZzjlpn0TLYVb52NAU2+t689tiiRYsq3fecOXPc+iWXXOLWo3ZqO+iRXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSSJNH3266+/3q3PnOmfih07dpTWbr31VvdYb6olAPT09Lj1qJ985syZ0lo0xTWqR/3i6PoEbxpq1Kv2vi8g7lV703OjqbvR9Nn58+e79RUrVrj1/fv3u/V20CO7SBIKu0gSCrtIEgq7SBIKu0gSCrtIEgq7SBJp+uzRcs5V5n1Hveiohx/Nrfbm0gPVlnuO5pTPmjWr0n17ffjonHvz9IF47N4aBtGWzUeOHHHr0bUTl112mVtvgh7ZRZJQ2EWSUNhFklDYRZJQ2EWSUNhFklDYRZJI02fv7e1160ePHnXr3rzv6GtH2yLv3etvax+tzR71oz3RnPGoPnfuXLfu9eGjtdmj7/vEiRNu3RtbtI9AtIV39DONfieaED6yk3yK5BjJvZNu6yG5leTbxVv/qhARadx0nsZvBLDugtseBbDdzK4DsL34WES6WBh2M3sRwPELbr4XwKbi/U0A7qt3WCJSt1b/Zu8zs9Hi/fcBlF54TnIAwECL9yMiNan8Ap2ZGcnSWSJmNghgEAC8zxOR9mq19fYByWUAULwdq29IItIOrYZ9M4AHivcfAPBcPcMRkXYJn8aTfBrA7QAWkzwM4GcAngDwB5IPATgIYEM7B1mHaJ3vaG60J1oj/OTJk2696l7d3jUAUS87Wvc98vHHH7f89auuaX/s2DG37vX4o7X6ozUIousLZs+e7dabEIbdzO4vKd1Z81hEpI10uaxIEgq7SBIKu0gSCrtIEgq7SBJpprhGrZCo/eUtHRxNd4ymz0Zji6aZRu21KqL2V9SyrLJls7d8NxC3U19++eXS2t133+0eW7Ut2I2tNz2yiyShsIskobCLJKGwiyShsIskobCLJKGwiySRps8eibbo9Xq6x49fuETflx04cMCtR9v/jo6OunVvOmbUgydZqR5tV+2JppFWufYBAN56663SWjQ9NtpyOdrKusp5aRc9soskobCLJKGwiyShsIskobCLJKGwiyShsIskkabPHvWLo+1/ve2DlyxZ4h4bbcl88803u/XDhw+7da+nG/Wqo3nX586dc+tRH99bzjn62tEy19GWziMjI6W16LqK6LycOnXKrUfz3ZugR3aRJBR2kSQUdpEkFHaRJBR2kSQUdpEkFHaRJNL02aP5x9Ha7FdccUVpLZrPPjQ05NbvuOMOt+71qgG/1x0dG4m2Jo565d55jcYWrccfrRvvbSfd19fnHhvNR4/69NHvWxPC3wSST5EcI7l30m2Pkxwhubv4d1d7hykiVU3nv/2NANZNcfsvzWxV8e/5eoclInULw25mLwLwn6eKSNer8gfdj0juKZ7mLyr7JJIDJHeR3FXhvkSkolbD/msA3wKwCsAogJ+XfaKZDZrZajNb3eJ9iUgNWgq7mX1gZufN7DMAvwGwpt5hiUjdWgo7yWWTPlwPwJ/DKSKNC/vsJJ8GcDuAxSQPA/gZgNtJrgJgAIYB/LB9Q6xHtA74O++849a9udXj4+PusYcOHXLr0Vx7r18MxP3oKqKvXWUf86rXAETHe3vDRz+z6PuO9qWP5uI3IQy7md0/xc1PtmEsItJGulxWJAmFXSQJhV0kCYVdJAmFXSSJNFNco1ZKtH2w10KKlhWuuvXwggUL3LrXmoumoFYVtQ296bfRtOJINMXV+95Pnz7tHhu11qLzGrVLm6BHdpEkFHaRJBR2kSQUdpEkFHaRJBR2kSQUdpEk0vTZvemOQLx0cJUteKsumRz1hL3jo22No2sEoq2Lo/PqTfWMzkvUh68yRTbaonvOnDluPbouoxunuOqRXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSSJNH32qCfbzm2Ro22Po+19o2sAvD581EeP7tv7voG4D+/N5Y960dG1DVV64dFc+OhrR/VoG+8m6JFdJAmFXSQJhV0kCYVdJAmFXSQJhV0kCYVdJIk0ffYq89EjUS866idHvepovrvXx4/mm1fZchkAzKzl46PzFq1JH23D7V3/MDY25h4bnbeoTx9tCd2E8JGdZD/Jv5B8g+TrJH9c3N5DcivJt4u3i9o/XBFp1XSexp8D8FMzWwng7wA8THIlgEcBbDez6wBsLz4WkS4Vht3MRs3s1eL9cQD7ACwHcC+ATcWnbQJwX5vGKCI1+Fp/s5O8CsC3AewA0Gdmo0XpfQB9JccMABioMEYRqcG0X40neSmAPwL4iZl9OLlmE6/STPlKjZkNmtlqM1tdaaQiUsm0wk5yFiaC/jsz+1Nx8wcklxX1ZQD8lzdFpFHh03hO9D+eBLDPzH4xqbQZwAMAnijePteWEdYkWo65ynTL0dHR0hoQt9Y++ugjtx4tqXzppZeW1qLpsVGLqeoy2F5bMNrW+MMPP3Tr0bbJPT09pbWqU3cj0VLVTZjO3+zfAfADAEMkdxe3PYaJkP+B5EMADgLY0JYRikgtwrCb2V8BlF3dcGe9wxGRdtHlsiJJKOwiSSjsIkko7CJJKOwiSaSZ4nry5Em3Hk1Z9KZyRr3qhQsXuvWonxzxlkyOevSXX355pfs+e/asW/f68FGPPtoW+cCBA249mgJbhdfDB+LfiSbokV0kCYVdJAmFXSQJhV0kCYVdJAmFXSQJhV0kiTR99mjp4CuvvNKte33TqOcazZWPrgHo7++v9PU90ZzyaEvnaLlnb154dA1AdH1CX9+UK6F9YefOnaU1bytpIL4GIKp343x2PbKLJKGwiyShsIskobCLJKGwiyShsIskobCLJJGmz378+HG3Hs0/9uZtRz1Vb+10IF6bPVpXvspc+0g0tirz2aved3Reve89mgsfzfOP+vTRPgVN0CO7SBIKu0gSCrtIEgq7SBIKu0gSCrtIEgq7SBLT2Z+9H8BvAfQBMACDZvYrko8D+CcAR4pPfczMnm/XQKvasmWLW1+3bp1b9+ZOb9++3T323XffdetRT/fgwYNu3es3ez14IL5GIJqv7u1bD/h99miPdG/feSCex3/s2LHSWnR9wDXXXOPWozUIoj5+E6ZzUc05AD81s1dJzgfwCsmtRe2XZvbv7RueiNRlOvuzjwIYLd4fJ7kPwPJ2D0xE6vW1/mYneRWAbwPYUdz0I5J7SD5FclHJMQMkd5HcVW2oIlLFtMNO8lIAfwTwEzP7EMCvAXwLwCpMPPL/fKrjzGzQzFab2erqwxWRVk0r7CRnYSLovzOzPwGAmX1gZufN7DMAvwGwpn3DFJGqwrBz4uXYJwHsM7NfTLp92aRPWw9gb/3DE5G6TOfV+O8A+AGAIZK7i9seA3A/yVWYaMcNA/hhG8ZXm2jKYdSC8to8a9eudY8dHBx069u2bXPrVUTTRKOlotspar1F7bEq1q9f79aPHDni1l977TW3Hn1vTZjOq/F/BTBVs7Vre+oi8lW6gk4kCYVdJAmFXSQJhV0kCYVdJAmFXSSJNEtJR6I++4IFC0prw8PD9Q6mRlG/N9o2+ZvK284ZAHp7e936qVOn6hxOR+iRXSQJhV0kCYVdJAmFXSQJhV0kCYVdJAmFXSQJRksN13pn5BEAk9dFXgzgaMcG8PV069i6dVyAxtaqOsd2pZktmarQ0bB/5c7JXd26Nl23jq1bxwVobK3q1Nj0NF4kCYVdJImmw+4vztasbh1bt44L0Nha1ZGxNfo3u4h0TtOP7CLSIQq7SBKNhJ3kOpL/R3I/yUebGEMZksMkh0jubnp/umIPvTGSeyfd1kNyK8m3i7dT7rHX0NgeJzlSnLvdJO9qaGz9JP9C8g2Sr5P8cXF7o+fOGVdHzlvH/2YnOQPAWwD+AcBhADsB3G9mb3R0ICVIDgNYbWaNX4BB8u8BfATgt2Z2c3HbvwE4bmZPFP9RLjKzf+6SsT0O4KOmt/EuditaNnmbcQD3AfhHNHjunHFtQAfOWxOP7GsA7DezA2b2KYDfA7i3gXF0PTN7EcDxC26+F8Cm4v1NmPhl6biSsXUFMxs1s1eL98cBfL7NeKPnzhlXRzQR9uUADk36+DC6a793A/Bnkq+QHGh6MFPoM7PR4v33AfQ1OZgphNt4d9IF24x3zblrZfvzqvQC3VetNbO/BfA9AA8XT1e7kk38DdZNvdNpbePdKVNsM/6FJs9dq9ufV9VE2EcA9E/6eEVxW1cws5Hi7RiAZ9F9W1F/8PkOusXbsYbH84Vu2sZ7qm3G0QXnrsntz5sI+04A15G8muTFAL4PYHMD4/gKkvOKF05Ach6A76L7tqLeDOCB4v0HADzX4Fi+pFu28S7bZhwNn7vGtz83s47/A3AXJl6RfwfAvzQxhpJxXQPgteLf602PDcDTmHhadxYTr208BOByANsBvA1gG4CeLhrbfwMYArAHE8Fa1tDY1mLiKfoeALuLf3c1fe6ccXXkvOlyWZEk9AKdSBIKu0gSCrtIEgq7SBIKu0gSCrtIEgq7SBL/D/XMRifJqr8GAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label: Pullover\n"
     ]
    }
   ],
   "source": [
    "# Podemos visualizar uma batch do nosso dataloader\n",
    "train_features, train_labels = next(iter(train_dataloader)) # O dataloader é um objeto iterável\n",
    "print(f\"Formato do batch das features: {train_features.size()}\") # repare como são 64 imagens de 28x28 pixels\n",
    "print(f\"Formato do batch das labels: {train_labels.size()}\") # repare como são 64 labels\n",
    "img = train_features[0].squeeze() # remove todas as dimensões com valores 1\n",
    "label = train_labels[0]\n",
    "plt.imshow(img, cmap=\"gray\")\n",
    "plt.show()\n",
    "print(f\"Label: {labels_map[label.item()]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Criando um dataset customizado\n",
    "Como tinha dito antes, normalmente não teremos um dataset bonitinho assim na natureza, e normalmente você terá que o fazer. Muitas vezes, pode-se usar apenas tensores, sem criar o dataset e o dataloader, porém essa geralmente não é a opção mais padronizada nem mais optimizada para imagens e textos. Segue um exemplo de como construir seu próprio dataset para imagens:\n",
    "\n",
    "Toda classe customizada de dataset deve conter 3 métodos: \n",
    "- `__init__` : Método que é executado quando você instância (cria/chama) o dataset. Normalmente você vai passar o endereço dos seus dados, de suas categorias e se precisar, alguma transformação neles.\n",
    "- `__len__` : Serve para falar quantas amostras existem no seu dataset\n",
    "- `__getitem__` : Serve quando você tem que pegar um elemento do seu dataset dado um índice `idx`. Vai identificar o endereço do elemento no disco e converter para um tensor e aplicar as transformações, caso você as tenha solicitado. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para dados de tabela:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomDataset(Dataset):\n",
    "    def __init__(self, X_data, y_data):\n",
    "        self.X = X_data\n",
    "        self.y = y_data\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.X)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return self.X[idx], self.y[idx]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Não vai rodar pq não temos o dataset\n",
    "# train_data = CustomDataset(torch.FloatTensor(X_train),torch.FloatTensor(y_train.values))\n",
    "# test_data = CustomDataset(torch.FloatTensor(X_test),torch.FloatTensor(y_test.values))\n",
    "\n",
    "# train_loader=DataLoader(train_data,batch_size=batch_size,shuffle=True)\n",
    "# test_loader=DataLoader(test_data,batch_size=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para dataframes simples, você também pode pular a parte de criar a classe e usar um `TensorDataset`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Não vai rodar pq não temos o dataset\n",
    "# Pegar as colunas do dataframe\n",
    "# target_train = df['Target'].values\n",
    "# features_train = df.drop('Target', axis=1).values\n",
    "\n",
    "# target_validation = df_validation['Target'].values\n",
    "# features_validation = df_validation.drop('Target', axis=1).values\n",
    "\n",
    "# Transformar em dataset de tensores\n",
    "# treino = torch.utils.data.TensorDataset(features_train, target_train)\n",
    "# validacao = torch.utils.data.TensorDataset(features_validation, target_validation)\n",
    "\n",
    "# Preparamos o dataset para ser iterado pela rede\n",
    "# treino_loader = DataLoader(treino, batch_size=tamanho_batch, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para imagem:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "from torchvision.io import read_image\n",
    "\n",
    "class CustomImageDataset(Dataset):\n",
    "    def __init__(self, annotations_file, img_dir, transform=None, target_transform=None):\n",
    "        \"\"\"\n",
    "        Parâmetros\n",
    "        ----------\n",
    "            annotations_file : str\n",
    "                endereço do CSV das labels das imagens\n",
    "            img_dir : str\n",
    "                endereço do diretório onde estão as imagens\n",
    "            transform : function\n",
    "                funções de transformação que podem ser aplicadas nas imagens\n",
    "            target_transform : function\n",
    "                funções de transformação que podem ser aplicadas nas labels\n",
    "\n",
    "        \"\"\"\n",
    "        self.img_labels = pd.read_csv(annotations_file)\n",
    "        self.img_dir = img_dir\n",
    "        self.transform = transform\n",
    "        self.target_transform = target_transform\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.img_labels)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        img_path = os.path.join(self.img_dir, self.img_labels.iloc[idx, 0]) # pega o endereço de uma única imagem\n",
    "        image = read_image(img_path) # transforma essa imagem em tensor\n",
    "        label = self.img_labels.iloc[idx, 1] # pega a label dessa imagem\n",
    "        # Se houverem transformações a serem apliacadas, aplicá-las\n",
    "        if self.transform:\n",
    "            image = self.transform(image)\n",
    "        if self.target_transform:\n",
    "            label = self.target_transform(label)\n",
    "        return image, label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🧠 Construíndo as redes neurais 🧠\n",
    "\n",
    "Redes neurais são basicamente compostas por diversas camadas, cada uma com um tipo de operação. O módulo `torch.nn` possui todos os blocos que precisamos para a construção dessas redes. Todas as redes neurias no PyTorch são filhos da classe `nn.Module`, por isso precisamos que nossa rede dependa dele."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Usando cpu\n"
     ]
    }
   ],
   "source": [
    "# Fazer o processamento em uma GPU caso seja possível\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "print(f'Usando {device}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layers mais usados\n",
    "\n",
    "Vamos fazer um exemplo com um minibatch de 3 \"imagens\" feitas de pontos aleatório e ver como seriam as etapas de cada camada de uma rede neural de maneira individual, para que depois possamos juntá-los em uma rede só.\n",
    "\n",
    "**Dados de entrada**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A dimensão da entrada é : torch.Size([3, 28, 28])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAACRCAYAAABOvIgbAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjy0lEQVR4nO2dedjPZfr+L9ompYVixtKmIhHJHrJWKKTsKk1EkZhUE5KZmkYjaVFqDkOpCRm7VPalCBMiRJKlGWQpNFmr3z/f+eM+z/N5eh/fI8/3+B3H+frvulzPp8/z/tzv993nuc77vPL99NNPYYwxxpjcyf9//QaMMcaY/x/whmmMMcZkwBumMcYYkwFvmMYYY0wGvGEaY4wxGfCGaYwxxmTg5Nz+cdmyZXTm5M9//nMS33333fRzu3fvptyKFSsod8YZZyTxli1bqObo0aOUa9asWRLfddddVPPQQw9R7qSTTqLcvHnzkvi6666jmooVK1Ju2bJlSayO5xQtWpRy33zzDeVKliyZxHPmzKGap556inJvvvlmEp977rlUc/rpp1NuwoQJlHvjjTeSePLkyVTTo0ePfJQ8AQwePJguJn4GPXv2pJ/r3r075U477TTK7dq1K4krV65MNVOnTqVc+/btk3jVqlVU06NHD8qNGTOGcng/nH322VSzc+dOyhUuXDiJTznllJ997YiIAgUKUK5SpUpJfOGFF1LNxo0bKbdu3bok/s1vfkM16l4rU6YM5ebPn5/EZ511FtU8//zzJ3zdNW7cmNbcs88+m8QtW7akn2vUqBHl1DME11OtWrWoBtdlRMSll16axEeOHKGaYsWKUW7t2rWUw3WBn39ExMkn85aAn4lac+q5dtlll1Fu0qRJSYxrKSLiiiuuoFydOnWSGJ9XERE1a9ak3L59+yj32WefJTHe1xERHTt2lGvO3zCNMcaYDHjDNMYYYzLgDdMYY4zJQL7crPG6detG/3j++ecnsfqbd7169Si3evVqys2dOzeJDx8+TDVVqlSh3DXXXJPEv/rVr6jmzDPPpNyoUaMoV65cuSRevnw51Tz//POUGzp0aBJPmTKFajp37ky5/fv3Uw57CStXrqSaGTNmUA57BKrP+cILL1Duu+++o9zMmTOTuEmTJuq18qSH2atXL1p3DRs2TOL169fTz23YsIFyt956K+VOPfXUJP773/9ONdhfj4i48sork1j1k7799lvKqd4U9nxU/3X8+PGUw/vhhhtuoJohQ4ZQTvVDv//++ySuWrUq1ahrivfDiBEjqEbpES666CLK4b2r+vfjxo074etuyJAhtOawNzhy5Ej6OdQfRERccMEFlMPnH/aiI3h9RfDn9umnn1KN6iGr5x++ltI3YM80ImLRokVJXL16dapRz+klS5ZQ7vPPP09i9ZxRvW7sO6rrrtYOPjciIt5+++0kXrBgAdWsXbvWPUxjjDHmf4s3TGOMMSYD3jCNMcaYDOR6DrNEiRKUa9CgQRKrnpz6u3Hfvn0pd+211ybxxx9/TDV45jIi4ssvv0zi9957j2pUX/Occ86hHJ6Zuuqqq6hGnfnBOvV3d3U2KX9+/n+UzZs3/+z7rFatGuWwT4HXJUKfQ7z33nspt2bNmiRW58vyCnVmEM8Dql6L6msMGjSIcvXr109i1XPasWMH5QoVKpTEqi+/Z88eyrVo0YJy2JNu3Lgx1Tz22GOU++KLL5J42LBhVNO7d2/K4fk3hbr/1BrG84kLFy7M9B4++ugjyuHPqvOJeYHqPeO1VdqMY8eOUe4///kP5cqWLZvE6qy66kViL1it+3z5uN2mXr906dJJrM68qs8If+7iiy+mGnX2F/eKiIgffvghidV9dujQIcph71Nd43//+9+U27ZtG+VwTT/wwANUkxP+hmmMMcZkwBumMcYYkwFvmMYYY0wGvGEaY4wxGchV9KOMz5cuXZrE6oDy2LFjKYdm5er169atSzXY7I2IGD58eBIrQREedI2I2L59O+UeffTRJO7VqxfVdOjQgXJonqAa1UpMot7r7Nmzk/jgwYNUow4Zly9fPokffvhhqvnjH/9IOSXUQlGREhDlFb/+9a8ph0Kgf/3rX1SjPnMlPEDxDgoRIvTnif9NZXavjDz+8Y9/UA7X/ksvvUQ16vfB++Hxxx+nmgcffJBy6iA7Co/U/dGuXTvKobG6Evio964ERKVKlUpiJdzKC5SQ5bbbbkviTz75hGpq1KhBOfUcmzhxYhKjCChCG4p07NgxiZWJijJDV88LFCYWKVKEapRxCwrN1D2lxELKEATNVpRxAYrKIthU4/3336caZTaydetWyqGBhho6kBP+hmmMMcZkwBumMcYYkwFvmMYYY0wGvGEaY4wxGchV9HP8+HHKYTNZuQEpR33V8K9YsWISK7f5ypUrUw6FDq+++irVKBcLNbkCHVaU84SaFIJCB+XOoybBjB49mnLY2FfvQQlMsrik9OzZk3LKleimm25K4r1791JNXrFq1SrK4VpRky+U24hq6KPTj3L66NKlC+U2bdqUxGpavJo4gaKuCBZszZ8/n2o6depEOXSwUoIbNU0CBSwR7IwyYMAAqlGOKrgWcepQBIs7IrQLEr4vJcDKC5RYC6+jEiQpdyQ1ZQaFbMWLF6eaP/zhD5RDtyf1bFWOUDiJJoId0SpUqEA1aqrN5ZdfnsRKiKlEcmpv+O1vf5vEatKNcorDCU5FixalGvVcK1iwIOVw4tbrr79ONWoKUIS/YRpjjDGZ8IZpjDHGZMAbpjHGGJMBb5jGGGNMBvIpYcp/OXz4MP0jOsdgAzVCN4C/+uoryuHoFdUIx/FbESzyue+++6hGjeRSQg50vVHii5tvvplyKIZQI5zQkSgi4sCBA5RD4UCbNm2oRjXaccSYGjelmuPTp0+nXNOmTZNYNd7Lli3Lc4ROAPXq1aN1h44g5513Hv2cukYoFohgwY0aq/T2229TDt+DEhmgMCiCxyNFsPOLGtuknLbQsadw4cJUo5y2sjgJqVFICxYsoNzVV1+dxGqUXvv27SmnXGoWL16cxGeffTbVzJgx44SvuzZt2tCaQ3ep1157jX7uqaeeotypp55KORwDpoR9yrEH144SoyghoRLOoTuZctR66623KNe2bdskVoJAtY+gWCiCXYN27dpFNUrYhGvsoYceohrlqNWtWzfK/f73v09idBH6nxq55vwN0xhjjMmAN0xjjDEmA94wjTHGmAzkalzw8ssvUy5//nSPVT2Hv/3tb5RTPTH8G7f627869I29u3379lFNnz59KKeMEbAfo97DO++8QznsraqD4aq/oQ554yF8NSlETT7ZuHFjEquemppIoHp2OG1ATQNRExZOBNhrieB1p6YqqF6h6gth3+zOO++kGjWFAvvdaiLEhg0bKKfuETRLUPeMMsPAz1wduFf9UNXfwQk1NWvWpBo1QeO0005LYjX9Rj07VO8Lnwuqb5sX1K5dm3KTJ09O4kWLFlHN4MGDKYcmGxFspKLWpTKJwAkyEyZMoJp3332XcqoXiboO1XtW+hOcTqNMEdRzU02swftYPdeUeQWasqh+spo4oybMYH9dvc+c8DdMY4wxJgPeMI0xxpgMeMM0xhhjMuAN0xhjjMlArqKf3bt3U65Ro0ZJrA5qDxo0iHLKER4nO6xdu5ZqzjjjDMrhYXwltFBiCHWIH9351VQVJdrAw7U9evSgmq5du1Lu/vvvpxxODVDvYejQoZRDkU+DBg2oZsmSJZQ7fPgw5VCApYQE+NmfKNTUFVw/SthSqlQpyq1fv55y99xzTxKPHDmSan788UfKoagIpz9EaMGHErw9+eSTSdyqVSuqwWkiEWwkcP3111ONEiPhwfkIFnape7Rv376UQ3MPNbFDGY4oERMaL6j3mRcocSEKgdTnrdZqoUKFKIefpRLezZw5k3IoxkORYoSeaqOuIz7/1H9P/T5oOoPCswhtXKDeKxoqqOe7Wr8olkThZ0TE119/TTmcjhIR8dxzzyWxmmyVE/6GaYwxxmTAG6YxxhiTAW+YxhhjTAa8YRpjjDEZyFX0gw4PEeyKoBwllEu9ml6ADjqqSTxr1izKoROMcshQDiXKHQYFN2oyiRIQvfjii0msnC5Uw/nDDz+kHDqnKOGIcvHB975161aqGTVqFOU6d+5MuW+//TaJlXtSXqEma6D7xwcffEA1anIOTveIYOGB+nyvuOIKyuGEFPzcIiKKFStGuXbt2lEO1/qKFSuoRgnqcJKNqlFTKJQ7C4o3lJPT7bffTrm6desm8SWXXEI1Srim1ieuayUgyguaN29OOXQGU+9twIABlFPPIxT5nHvuuVSjxDs4bUcJVNT6VROJSpQokcR16tShmmnTplEOxU9qUohy3lGvdfz48STu3bs31cybN49yKEBV7lIq9/TTT1MOXa+UWDAn/A3TGGOMyYA3TGOMMSYD3jCNMcaYDOTaw8QDqxF80FhNq87ai8SD9uimHxHx17/+lXJ46PvgwYNUow62q0Oy2M9Svc9KlSpRDg8nq75F8eLFKYdT3NXPKrOBW2+9lXJ4iF0ZLIwfP55yakI79rPUweC8QvWT9uzZk8TKfEGZFKiD/XgAXU2aUYe6P//88yRWvUmccJHT6+NkkHXr1lEN3msRvNaVwQT2OdV/L4LXBvaXIrQBAfaTVF9N9dzV8wRNEFQ/Li9YuHAh5fB3UAfq8f1HRFx77bWUw363ema9//77lENDC/V5q2ePWpvY78Z7KkJPMEFNhXqmqGedMqpo2rRpEqtnUfXq1Sk3bNiwJH7iiSeoRk15Qp1HBGsRlH4gJ/wN0xhjjMmAN0xjjDEmA94wjTHGmAx4wzTGGGMykE+5zP+XN998k/4Rp3uow9vYVI2IKF++POVUwxzBqSAR7Lq/d+9eqqlcuTLl0HQhIuKLL75I4oIFC1KNmj6Ar68c/Hv27Em5LOICdQhYTTfAa6NETeo6KOEAHrhXkyt27tyZj5IngFq1atG6O3LkSBLXr1+ffk4d2FdrEU0QVq9eTTVK6IXTPbZs2UI1OIEnQq8pFCgpUZwy0cDroAwWlPBICVGymEGoCThoBqHuP3VNixQpQjkUfeAkiYiIjh07nvB1V7lyZVpzf/rTn5JYTW9S16x169aUGzx4cBJXqVKFaqZMmUI5NHxRE2yUAcHSpUsph88L9exWRh8ooETxW4Q2DcH1FRGRL1/6UdarV49qlOAOjVTUM1mZx+B+FcEGEep5+Pzzz8s152+YxhhjTAa8YRpjjDEZ8IZpjDHGZMAbpjHGGJOBXG01lIv7RRddlMTo/B6hXT7UlA50/1cO/kpMg41pJSbA6Rs5vQd04kfnnwg9sQFdOe655x6qUaIf1aBHIRBOg4iImDhxIuXQYUW5Whw7doxyjRs3phy6a+CUhLxECW5wQo0SOCnBiHJIQsGFEq2oKTzogqIcVpTwSIkffvzxxyS+4IILqObo0aOUQ7ecqVOnUk3p0qUpd8MNN1Cue/fuSayEIkqcgsIjFHJE6M/ngQceoBwK8dR6zQtatGhBORSDKBcc5XL0wgsvUA5FMeqavfrqq5R75ZVXklh9tmvXrqWcuodw7ag1ceDAAcrhMx6d1iL0VCTlaoZTc2bPnk016jmGLksXX3wx1YwYMYJyymVrwoQJSXzvvfdSTU74G6YxxhiTAW+YxhhjTAa8YRpjjDEZ8IZpjDHGZCBX0c/WrVsp98033ySxarSecsoplEOBTwQ7Yqjmda1atSiHrjfKqUG5cqhRT+haoZwn1NgxHAOmBA1q3Na2bdsoh2ILdLWI0GOP0MVCjXVSoik1zmbgwIFJrJx08go13mf//v1JrEYMKUcddDCKYAGVqlGiCRzBpUQ56jNXwriWLVsm8a5du6hGCUpwnN727dupRgln1BrGta6EZWPGjKEcrlflxnXnnXdSrkuXLpRr0qRJEu/YsYNq8oKSJUtSDu8vdHqK0OMN1ec2d+7cJFbXQrkGlStXLomVw820adMopxxu8L327duXavr160c5/Iz69+9PNZdddhnl1H2M4ja1x6CoLIKfm+jQFqFdr5TDHDoJqeun7v8If8M0xhhjMuEN0xhjjMmAN0xjjDEmA7lOKxk3bhz9I/Yrjh8/Tj+Hh8wjdC9t2bJlSawOBjdr1oxy+B7UYfGvv/460/vCv18rwwM1IaBVq1ZJrP6mrg7Nq7/PY49AHaTFg+4R3M9QvWPVY1F9EOwdHTp0iGryYmpERETx4sVp3eG0BzSO+J+fo5w6BI25l19+mWoOHz5MOezJqP5hxYoVKaf6mtinVhMarrvuOsqtWbPmZ/97WfvWON1BHWTPn5//nxqnXKhniDIJKVu2LOU2bNiQxAcPHqSakSNHnvB1N2TIEPol8NnQsGFD+jmlN1DPsSVLliSxmh6j+oDnnHNOEuN9EKGfa+p5sWnTpiRGM4AIPW0HJ9bgFJeIiLvvvptyqj+J9yjuARHawAY1HKilidDPLPUe8PdRz422bdt6Wokxxhjzv8UbpjHGGJMBb5jGGGNMBrxhGmOMMRnI1bgAD/VH8BSCEiVKUI06qH3fffdRDpvQF154IdWoSRIozFEHz3EKQkTE4sWLKVetWrUkViIKNcEED63XqFGDapSjvppksH79+iRWEyLUxAuccqKa12rqCAotIlgkoF6rY8eOlDsR4ASZCBYlnXTSSVSjDuyr3NixY5NYCQPU54kHy1H8EqEnmKDgI4KvN06ziIgYMGAA5fCw+dChQ6nmqquuopwShOGhePX7qHWHwjU0lYjQh9bVZ9G+ffskVp9rXqBMU+64444kVoI9fIZFaHEQPi/UdBolZClatGgSqzWhnmso6IqIePHFF5NYTWhRz01cF+o5rUR4ShCFRjQ4QSWCp1Gp16pcuTLVKBGeWr+4Z6nfOSf8DdMYY4zJgDdMY4wxJgPeMI0xxpgMeMM0xhhjMpCr6Ec17idPnpzEymViy5YtlFMNcxR3KHeVdu3aUW7+/PlJrJrljRo1opyaqpAvX2rogK8doQVLOH1A/X7K7UQJlHDKhrp+Dz74IOXQJQOnuERoByIlCkERgpo2oVxGTgRFihShHAqVlDCgTJkylMMJIxHs3KREDEokgy40t9xyC9XMmDGDcnXr1qXcRRddlMTKiUeJn3BihrpnlHBG3SMoYlKiHCViQuGfcghSa79t27aUQ4cjJdxQAo9fGuUwhFNH8FkRoZ9PavIMOvaoGvUZoWuQclU677zzKKcEm0899VQSq89NXX+cFHL55ZdTzZw5cyin3H9QJKVEOereRvcf9YzEaxyhXZc6deqUxMpZLSf8DdMYY4zJgDdMY4wxJgPeMI0xxpgM5NrDVIdfsfcyevRoqqlZsybl8JB9BB+IVX+Dxp5pRMT06dOT+Oabb6Ya1Q9Qh/HxwDL+fhERixYtotyePXuSuECBAlSjJmWoPghOVlH9DdVbaNOmTRKryTGqRzBp0iTKDRw4MInVlJi8QvUwV69encQdOnSgGuwFR+ieNPZu1KFr1fvs1atXEk+cOJFqli9fTjns+UVEjB8/PolVz1jdD2iCUK9ePapRJghqog++1iWXXEI1qq+Jn4/ql6kJLWoqDD5j6tevTzV5gbrvR4wYkcQ45SJCT9tQU2a+//77JL7pppuoRplQYD9PaSU++eQTymH/NYL7tMqkRfVy8T2o58zOnTspp+5HNAm47bbbqGbevHmUw/WLfdUIfU3VhBzst3755ZdUkxP+hmmMMcZkwBumMcYYkwFvmMYYY0wGvGEaY4wxGchV9IPimghuOqP7fERE1apVKTdu3DjKYdNZHcBVwgc8cL1gwQKqUYdfL7vsMsrhwe9Dhw5RjTrgi4fd1RQB1fRWDWYUmNSpU4dqZs2aRTmc2lKyZEmqUQfbr7/+esphsx+nYuQlSoCB60AdzlfCKCX0wsPyOBEiIqJUqVKUw89ATW9Rhgdqog9OK1GTQtTUHzRwUEKaN954g3LK1AGn6SxdupRqlHHB1VdfncRqIk758uUpp0xOUAyjDByUiOaX5tJLL6UcivbU1CL1TEFziQieiKOEOmrN/fOf/0xiJTxSRiRqPaEABicARehn97vvvks5RN1n6nrhGlMCoizCy+bNm1ONmhxTuHBhyj366KNJrIRUOeFvmMYYY0wGvGEaY4wxGfCGaYwxxmTAG6YxxhiTgVxFP6qRiw3SZ599lmq6detGOSUOQrcGJXK49tprKXfjjTcmsRI0PPPMM5RDoUUEC42UaGP9+vWUQ+f/o0ePUo0SPijnFLwO+/fvpxrlwoKCAzVFQDkx7du3j3J33HFHEi9evJhqUOxxovjoo48ohy4ew4cPpxolBFLTClCMpT4TnI4QwZ/x2LFjqUaJ1LLk1CQd5WCFjkAoComI6N69O+XUNd28eXMS9+/fn2qeeOIJyuFUHHQMitBiIZwSE8GuREp8kxegKCeCRYlKzKNQTlX4eyoHLrVOcNKNqlETiRo2bEi51157LYmVC456NmRxY1LCNiXeQWeqvXv3Uo16BuNEkWnTplGNWjsPP/ww5XDiixJ15oS/YRpjjDEZ8IZpjDHGZMAbpjHGGJMBb5jGGGNMBnIV/aiGbP786R7bp08fqlHN8eeee45yF1xwQRJ36dKFapTzRJUqVZL49ttvpxrV9D75ZP51UcSk3CnUz6EgSjXxlXPHm2++SbmuXbsmsRqXpFx8cMQYfjYRejzaSy+9RDkcXaUcj9q1a0e5E4EaRYVjulSNGmmkBCl4vdExKUKP5ML/prreauTXd999RzkUarRs2ZJqlEimX79+SawEJmoNV69enXLolqKcX7IIiNRnMXfuXMqhO1YEC4i2bNlCNUr89EuDjkMRPIrv9NNPpxrlJKOco9CF5quvvqIaJapDh5saNWpkeg8obIlgRyAlglTublncmJToZ9SoUZRDMZ1a48ohDR3L1Pg4dR+ra9qoUaMkVsLVnPA3TGOMMSYD3jCNMcaYDHjDNMYYYzKQaw/z66+/phxOulB/19+4cSPlVC8ED6gOGzaManbv3v2zr6XMDZTDvprcgX2bd955h2ruv/9+yuHUEdVbmDJlCuVatGhBuZEjRyaxMjzAyRIR3NdT10q9FvYrIyKaNm2axE8++STV5BWqB459ZHUwXk2OUL8/HlJXh8GVCQL2piZMmEA12PeK0P0k7G+r6RXqfdWsWTOJy5UrRzWqp6UmOaCpgzLfUP10fO8FChSgGlxPERGVKlWiHN5vHTp0oJr/K/B5hNqJCP1MUT1dpYNAlLkETuCZMWMG1Vx55ZWUU31zXPfqWqtpSrhOvvnmG6pBQ4II/TujKYvaY1q3bk05nEilTAqU2YiagIV9UxsXGGOMMb8w3jCNMcaYDHjDNMYYYzLgDdMYY4zJQD4lSPgvTZo0oX/Ew9q1a9emn1MHulVzHBvYamJAz549KYcN39WrV1MNHuqP0Aen33///SRWzWv1WsiBAwcop4QWSsiBh+TVa+E0kQg+QKwOzaNIK0KLL5CFCxdSbvTo0axoOQG8/vrrtO5QnKWEZU2aNKGcOkDfsWPHJFYHlwcNGkS5hx56KImVoECtMSVsQGHIsmXLqKZz586Uw4k+Slwza9YsyqnJEXhwvVOnTlQzYsQIyuHvrYR/aoLGwYMHKYdCGiVqGTt27AlfdytWrKA1hyYjSvTz2GOPUU6ZmOB6Ugf2S5cuTTmcRqMO4r/99tuUU0IgFOuoe1wZvqCZhBLlTJo0iXIKfB4VK1aMatSaq1ChQhKrCVJKvKOuKT4T1HN62LBhcs35G6YxxhiTAW+YxhhjTAa8YRpjjDEZ8IZpjDHGZCBX+4lq1apRDpvVypWlQYMGlFPTAJo1a5bEakKEErugM45ytUBHlAh2xolgAZFqqivhCLoLrVy5kmoUOKElgqdslC1blmqUg//MmTOTWDl3KAd/5czy4osvJrGajpJXqKkjKJxRTj/btm3L9PpLly5NYpyYERGxdu1ayqGTkhK8qfe1fPlyyrVv3z6J1cQdNRUC3VOUo5US8qkJGig0UoIb9XMo+vnwww+pRt1rOCUigp2E1H2bFzz99NOUQ+eYZ555hmoeeeQRyuF9GcFr7vzzz6ca5aq0Zs2aJK5YsSLVqNdCF7WIiMaNGydx/fr1qWb8+PGUO+2005IYXXci2A0ogsV1Ebym1XQa5RB0ySWXJLFa42oCjHJu27dvXxLfdtttVJMT/oZpjDHGZMAbpjHGGJMBb5jGGGNMBnLtYR47doxy2P/Cvy1HRAwdOpRy/fv3pxz+zV793V2ZEuBUCjXpQU1eUMYId999dxKrA8WfffYZ5dCIQf0tfvPmzZT74IMPKIeH0d977z2qUX0inKCupiSovogyYrjrrruSWB1qzivmzJlDOTzwrPrKqnekDmKPHj06idX0FvU5YX8STS8itCmEOlCNh/jVulNmBtjjU/1enHARoSdMYD8MD8lHaHMGNEZQPSc1OUbd3x9//HESK91EXqB6tTi5Q5lEHDlyhHKVK1emHPbEVQ9OmQ3069cvidWhfvXe1TNxw4YNSayMGFQ/FPuoSqOijCqUoQJqOFTvU00iwb6jmt40e/Zsyim9Bt7bL7zwAtUo7UyEv2EaY4wxmfCGaYwxxmTAG6YxxhiTAW+YxhhjTAZyFf3ceuutlMOG6VtvvUU1Xbp0odzixYsp98Ybb/zsz6kJHGguoCYxqOkoyrkeRUvKYCFLI1xNbFAHndXvg9ewRo0aVKNMHdatW5fEl19+OdUo1CF5FNqoQ8B5hRJS9O3bN4mVQKJQoUKUU2IpNHhQazPLQWwlMihTpgzllLkHrtnBgwdTzS233EI5/FzU9BslMlLrDgU96p5Rax/FMIULF6YaJfpRr1+qVKkkViYSeYESh+DnvWvXLqrBezBCCyFvvPHGJP7000+pRon28L4sX7481ai1qgxY0EADRYMR/NlG8ASnJUuWUI0SEqpnKa65K6644mffZwSbJ4waNYpq+vTpQzkl6GnZsmUSq+dGTvgbpjHGGJMBb5jGGGNMBrxhGmOMMRnwhmmMMcZkIFfRj3IaQRHA7373O6pRzjjKCQedJpQbSf78vKej0GHHjh0/+9oRETt37vzZ15o3bx7VKBHO9OnTk3jRokVUg8KgCC3MQfGI+p2VCAedWpQoARvcEVqUgE5MOEkmL1GTUlBUpZyclPjk+PHjlBs3blwSK3cZ5SSC4h21Lk466STKqfsBnUTUZBIlRFm/fn0So2NQhBagqfWD665cuXJUo9y+Tj311CRW11gJt4YPH065xx9/PImVg9M999xDuV+aM888k3JTpkxJYvVs+Mtf/kI5df3xWh84cIBq9u7dSzkUKqoadd8rVyUUFSlBnJr8hKIiJShSLlvq/sDpSRs3bqSaYsWK/ezP4bSoCP1MUNNvWrduncRTp06lmpzwN0xjjDEmA94wjTHGmAx4wzTGGGMy4A3TGGOMyUCuoh/V3EVxC4oQIrSjBwoFIiJ69+6dxKtWraKaV155hXLXXXddEiuxAo6DidDuIyjkOPfcc6lGiZEeeeSRJMbxPRF6tNYPP/xAuapVqyaxcm9R1w9zqoGuXJDatm1LORS5qHFTeYUaM4VCpRIlSlCNum5KEPH0008nMTpORWjxQ/Xq1ZNYjbFr3rw55fLly0e5zp07J7ESJygxEv7eSrCgnFjUyCkco3TNNddQjbon8bXUGDslAlHXFMf3qXFieYFaJ/Xr109idCWK0Ne/du3alEMBlxJhqTWA70s9B9T6QmFLRMRrr72WxEqUg8/WCHbCUqO8lEjw5ZdfphwKj5QzVsGCBSmH9/aYMWOoZtu2bZRDYWQEi7LUMzIn/A3TGGOMyYA3TGOMMSYD3jCNMcaYDOTawxwwYADlGjdunMRnnXUW1ai/G6vD6OjYr6Y6KHd+nGig+m2qv6R6sjfddFMSHzp0iGpGjBhBOZx4oXqfLVq0oJzqH951111JvHXrVqrp3r075fDgseoZqM9ww4YNlMNeUoUKFagmr1BTZfCgsjp0/eOPP1JOHezHnmXXrl2pBg+tR7C5A06giNA9JrXusM81cOBAqlHTHtDUQk2e2bRpE+VWrlxJOZwU8dNPP1FN0aJFKYf3m+rtqQP26rA5vq/t27dTTV6g1smsWbOSuG7dulSjprmo6SFomqIMTJTZA07uUKYjqnc3aNAgyuF0EtW7w2lUERFnn312Ejdo0IBq1D2Ez8gI7uWiIUGE1mJg37FVq1ZUo3qy7733HuV69uyZxKp3nBP+hmmMMcZkwBumMcYYkwFvmMYYY0wGvGEaY4wxGcinGv3GGGOMSfE3TGOMMSYD3jCNMcaYDHjDNMYYYzLgDdMYY4zJgDdMY4wxJgPeMI0xxpgM/D+/M77iCA27LAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x576 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "input_image = torch.randn((3,28,28))\n",
    "print(f\"A dimensão da entrada é : {input_image.size()}\")\n",
    "\n",
    "# Para mostrar as \"imagens\"\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 1\n",
    "for i in range(1,4):\n",
    "    img = input_image[i-1]\n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img.squeeze(), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Camada de achatamento**\n",
    "\n",
    "A camada de ``nn.Flatten`` converte imagens 2D em um único vetor. No caso, uma imagem de 28x28 se torna um vetor de 784 elementos, em que cada elmento é um pixel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dimensão das imagens achtadas torch.Size([3, 784])\n"
     ]
    }
   ],
   "source": [
    "flatten = nn.Flatten()\n",
    "flat_image = flatten(input_image)\n",
    "print(f\"Dimensão das imagens achtadas {flat_image.size()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Camada Linear**\n",
    "\n",
    "A camada linear é onde aplicamos as operações entre os pesos (weights), bias e dados. Possui esse nome porque essa operação é uma \"Transformação linear\": $W \\cdot X + b$, repare como esse formato lembra da \"equação linear\" que aprendemos na escola. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dimensão da imagem depois de ter passado pela primeira camada: torch.Size([3, 20])\n"
     ]
    }
   ],
   "source": [
    "layer1 = nn.Linear( in_features=28*28,out_features=20) # Aqui definimos qual a dimensão da entrada e qual será a dimensão da saída\n",
    "hidden1 = layer1(flat_image) # Passamos nossa imagem achatada para a camada\n",
    "print(f\"Dimensão da imagem depois de ter passado pela primeira camada: {hidden1.size()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Camada de Ativação**\n",
    "\n",
    "**A função de ativação retificadora linear** (*Rectified Linear Activation Function*) - ReL \n",
    "\n",
    "Para conseguirmos passar um sinal para a próxima camada, necessitamos de funções de ativação. Duas funções comuns são as [sigmoid](https://en.wikipedia.org/wiki/Sigmoid_function) e [tangente hiperbólica](https://mathworld.wolfram.com/HyperbolicTangent.html), ambas funções não lineares, uma propriedade que ajuda nosso modelo a compreender funções mais complexas. Porém, como elas são funções com limites bem estabelecidos, elas acabam \"saturando\" suas saídas, sendo sensíveis apenas para seus valores intermediários. A solução é utilizar a função de ativação retificadora linear (ReL) nos *hidden layers*. Dizemos que um nó (ou neurônio) com essa função de ativação é uma unidade de ativação retificadora linear (ReLU)\n",
    "\n",
    "$$\n",
    "f(x) = \\begin{cases}\n",
    "    x & \\text{se } x > 0, \\\\\n",
    "    0 & \\text{caso contrário}.\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "![Funções de Ativação](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fcdn-images-1.medium.com%2Fmax%2F1200%2F1*ZafDv3VUm60Eh10OeJu1vw.png&f=1&nofb=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Antes do ReLU: tensor([[-0.9703,  1.4260, -0.8053,  0.5087, -0.0765, -0.6215,  0.6229, -0.7089,\n",
      "         -0.8182, -0.2579, -0.1015, -0.1266, -0.0315, -0.9359,  1.0535, -0.3224,\n",
      "         -0.6802, -0.4018,  0.0539, -0.1499],\n",
      "        [ 0.4533,  0.3297,  0.2099, -0.2643,  0.4186,  0.0341, -0.4178,  0.3023,\n",
      "         -0.4167,  0.3338, -0.8953, -1.0487, -0.3839, -0.2220,  0.3507, -0.2475,\n",
      "         -0.6685,  0.6706,  0.0205,  1.2893],\n",
      "        [ 0.3237, -0.3199,  0.4203, -0.3860, -0.4498, -0.7379, -0.8022,  0.5070,\n",
      "         -0.5468,  0.5099, -0.2052,  0.8070, -0.1265, -0.2001, -0.3509, -0.5909,\n",
      "          0.4863, -0.4447, -0.1298,  0.6129]], grad_fn=<AddmmBackward0>)\n",
      "\n",
      "\n",
      "Depois ReLU: tensor([[0.0000, 1.4260, 0.0000, 0.5087, 0.0000, 0.0000, 0.6229, 0.0000, 0.0000,\n",
      "         0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 1.0535, 0.0000, 0.0000, 0.0000,\n",
      "         0.0539, 0.0000],\n",
      "        [0.4533, 0.3297, 0.2099, 0.0000, 0.4186, 0.0341, 0.0000, 0.3023, 0.0000,\n",
      "         0.3338, 0.0000, 0.0000, 0.0000, 0.0000, 0.3507, 0.0000, 0.0000, 0.6706,\n",
      "         0.0205, 1.2893],\n",
      "        [0.3237, 0.0000, 0.4203, 0.0000, 0.0000, 0.0000, 0.0000, 0.5070, 0.0000,\n",
      "         0.5099, 0.0000, 0.8070, 0.0000, 0.0000, 0.0000, 0.0000, 0.4863, 0.0000,\n",
      "         0.0000, 0.6129]], grad_fn=<ReluBackward0>)\n"
     ]
    }
   ],
   "source": [
    "print(f\"Antes do ReLU: {hidden1}\\n\\n\")\n",
    "activation1 = nn.ReLU()\n",
    "hidden1 = activation1(hidden1)\n",
    "print(f\"Depois ReLU: {hidden1}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Juntando todas as camadas sequencialmente**\n",
    "\n",
    "o ``nn.Sequential`` é uma espécied de container de módulos. Os dados são passados para ele na mesma ordem que definimos, ele server para simplificar a criação das redes neurais. Vamos criar um exemplo com as camadas que fizemos até agora:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.1849, -0.2601,  0.0499, -0.0943, -0.0679, -0.1442,  0.1640, -0.1429,\n",
       "          0.1742,  0.2893],\n",
       "        [-0.1016, -0.1994, -0.0074,  0.0562, -0.1001, -0.1043,  0.1753, -0.1512,\n",
       "          0.2368,  0.1603],\n",
       "        [-0.2525, -0.0955, -0.0016, -0.0239, -0.0212, -0.0595,  0.2503, -0.1955,\n",
       "          0.1452,  0.2539]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seq_modules = nn.Sequential(\n",
    "    flatten,\n",
    "    layer1,\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(20, 10)\n",
    ")\n",
    "input_image = torch.rand(3,28,28) # batch de 3 imagens de entrada\n",
    "logits = seq_modules(input_image) # Valores de saída são \"logits\", valores qeu relacionam probabilidades com numeros reais\n",
    "logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Função ``nn.Softamx``**\n",
    "\n",
    "Como a última camada retorna \"logits\", podemos passá-los por uma função chamada **Softmax** que transforma esses números reais em valores de probabilidades (valores entre 0 ou 1). Mostrando qual a categoria mais provável para aquela imagem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0837, 0.0776, 0.1058, 0.0916, 0.0941, 0.0871, 0.1186, 0.0873, 0.1198,\n",
       "         0.1344],\n",
       "        [0.0897, 0.0813, 0.0986, 0.1050, 0.0898, 0.0895, 0.1183, 0.0854, 0.1258,\n",
       "         0.1166],\n",
       "        [0.0767, 0.0897, 0.0985, 0.0963, 0.0966, 0.0930, 0.1267, 0.0812, 0.1141,\n",
       "         0.1272]], grad_fn=<SoftmaxBackward0>)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "softmax = nn.Softmax(dim=1)\n",
    "pred_probabilities = softmax(logits)\n",
    "pred_probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A rede neural\n",
    "\n",
    "Agora que você conhece todos os blocos da rede, podemos juntá-los para criar a rede neural! Para isso ciramos uma classe que é inicializada no método ``__init__`` com os elementos da rede. De maneira similar às classes do ``Dataloader``, qualquer filho do módulo ``nn.Module`` (ou seja, qualquer rede neural que criarmos no PyTorch), precisa de um método ``forward``, o método que passa os dados pela rede neural."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NeuralNetwork(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork, self).__init__() # Herda os atributos do nn.Module do pytorch\n",
    "        self.flatten = nn.Flatten() # Define o método de achatar imagens\n",
    "        self.nn = nn.Sequential(   # Inicia o container de camadas\n",
    "            nn.Linear(28*28, 512), # camada linear com 784 entradas e 512 saídas\n",
    "            nn.ReLU(),             # unidade de ativação\n",
    "            nn.Linear(512, 512),   # camada linear de 512 entradas e 512 saídas\n",
    "            nn.ReLU(),             # unidade de ativação\n",
    "            nn.Linear(512, 10)     # última camada linear com 512 entradas e 10 saídas \n",
    "        )\n",
    "\n",
    "    def forward(self, x):    # método que passa os dados para a rede neural\n",
    "        x = self.flatten(x)  # achata as imagens para uma dimensão\n",
    "        logits = self.nn(x)  # pasas os dados pela rede, retornando 10 logits\n",
    "        return logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Treinando a rede neural\n",
    "\n",
    "Vamos agora para a parte mais importante das redes neurais, treiná-las! Para isso vamos usar os dados de treino e teste que já preparamos antes ``train_dataloader`` e ``test_dataloader``. Vamos também inicializar a nossa classe de rede neural no objeto ``model``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = NeuralNetwork()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hiperparâmetros\n",
    "\n",
    "São parâmetros que são ajustáveis e permitem que possámos controlar o modelo. Tipos diferentes de hiperparâmetros podem ter grandes efeitos no modelo. No nosso caso, vamos ter 3:\n",
    " - Número de epochs: número de vezes que nosso modelo vai **passar por todo o dataset**\n",
    " - Tamanho do batch: número de **amostras do nosso dataset** passadas pro modelo antes de atualizar seus parâmetros\n",
    " - Learning Rate: O quanto nosso modelo vai **atualizar seus parâmetros**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 1e-3\n",
    "batch_size = 64\n",
    "epochs = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Função de perda (loss)\n",
    "\n",
    "Quando nosso modelo está aprendendo ele não sabe a resposta certa. A **função de perda (loss)** é um meio de medir o quão certo ou errado nosso modelo estava da predição desejada, e é esse valor que queremos minimizar durante o treinamento (ou seja, que ele tenha o menor erro possível). Para computar esse erro, passamos o valor predito por nosso modelo e o valor alvo e calculamos essa diferença de algum modo.\n",
    "\n",
    "Existem diferentes tipos de calcular o erro do modelo. Algumas funções comuns são [`nn.MSELoss`](https://pytorch.org/docs/stable/generated/torch.nn.MSELoss.html#torch.nn.MSELoss) (Mean Squared Error) para regressões, [`nn.NNLoss`](https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html#torch.nn.NLLLoss) (Negative Log Likelihood) para classificações e [`nn.CrossEntropyLoss`](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss) que combina a função `nn.LogSoftmax` com o `nn.NNLoss`.\n",
    "\n",
    "Para nosso modelo, iremos usar a ``nn.CrossEntropyLoss` já que já normaliza nossos logits em valores de probabilidade e calcula uma loss comum em tarefas de classificação."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Otimizador\n",
    "\n",
    "Com nossa função de perda definida, temos que definir agora qual técnica utilizaremos para chegar reduzir o erro em cada passo de treino. Existem [diferentes tipos de algoritmos de otimização](https://pytorch.org/docs/stable/optim.html) para redes neurais, sendo o **SGD** (Stocastic Gradient Descent) o mais simples e o que vamos utilizar para nosso modelo. Alguns menções de algoritmos que usamos bastante também é o ADAM e o RMSProp, que podem funcionar melhor dependendo do seu tipo de modelo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate) # Passamos os parâmetros do nosso modelo e o leraning rate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quando formos fazer nosso loop de treinamento, geralmente o passo de otimização é realizado vom 3 passos:\n",
    " - Chamar ``optimizer.zero_grad()`` para resetar os gradientes dos parâmetros do modelo. Gradientes por padrão se somam, para evitar que sejam contados duas vezes em treinamentos diferentes, nos explicitamente zeremos eles a cada iteração.\n",
    " - Faz a backpropagation da loss com ``loss.backwards()``. O autograd do PyTorch automaticamente depoisita os gradientes em relação a cada parâmetro. \n",
    " - Com os gradientes calculados, podemos usar ``optimizer.setp()`` para ajustar os parâmetros dos gradientes coletados em cada passo anterior.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loop de otimização\n",
    "Com nossos hiperparâmetros e otimizador prontos, podemos nos preparar par definir o loop de treino. Cada iteração desse loop de otimiação é chamado de **epoch**. O epoch consiste de duas partes principais\n",
    "- **O loop de treino**: itera sobre o dataset de treino, tentando convergir os parâmetros para o melhor possível.\n",
    "- **O loop de teste**: itera sobre o dataset de teste para checar como o modelo está se saindo. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_loop(dataloader, model, loss_fn, optimizer):\n",
    "    \"\"\"\n",
    "    Loop de treino do modelo\n",
    "\n",
    "    Parâmetros\n",
    "    ----------\n",
    "        dataloader: dataloader do nosso dataset de treino definido anteriormente\n",
    "        model: objeto com o modelo da nossa rede neural\n",
    "        loss_fn: nossa função de perda\n",
    "        optimizer: o otimizador definido\n",
    "    \"\"\"\n",
    "    size = len(dataloader.dataset) # pega o tamanho do dataset\n",
    "    for batch, (X, y) in enumerate(dataloader):\n",
    "        # Computa a predição do modelo e a loss\n",
    "        pred = model(X)\n",
    "        loss = loss_fn(pred, y)\n",
    "\n",
    "        # Backpropagation\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # A cada 100 treinos printamos as métricas\n",
    "        if batch % 100 == 0:\n",
    "            loss, current = loss.item(), batch * len(X)\n",
    "            print(f\"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]\")\n",
    "\n",
    "\n",
    "def test_loop(dataloader, model, loss_fn):\n",
    "    \"\"\"\n",
    "    Loop de teste do modelo\n",
    "\n",
    "    Parâmetros\n",
    "    ----------\n",
    "        dataloader: dataloader do nosso dataset de teste definido anteriormente\n",
    "        model: objeto com o modelo da nossa rede neural\n",
    "        loss_fn: nossa função de perda\n",
    "    \"\"\"\n",
    "    size = len(dataloader.dataset)\n",
    "    num_batches = len(dataloader)\n",
    "    test_loss, correct = 0, 0\n",
    "\n",
    "    # Como não vamos otimizar nada, não vamos acompanhar os gradientes\n",
    "    with torch.no_grad():\n",
    "        for X, y in dataloader:\n",
    "            pred = model(X)  # computa a predição do modelo\n",
    "            test_loss += loss_fn(pred, y).item() # loss dessa predição\n",
    "            correct += (pred.argmax(1) == y).type(torch.float).sum().item() # quantas vezes nosso modelo acertou a predição\n",
    "\n",
    "    test_loss /= num_batches\n",
    "    correct /= size\n",
    "    print(f\"Erros do teste: \\n Acurácia: {(100*correct):>0.1f}%, loss média: {test_loss:>8f} \\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos finalmente treinar nosso modelo!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1\n",
      "-------------------------------\n",
      "loss: 2.291909  [    0/60000]\n",
      "loss: 2.284972  [ 6400/60000]\n",
      "loss: 2.267519  [12800/60000]\n",
      "loss: 2.273911  [19200/60000]\n",
      "loss: 2.242418  [25600/60000]\n",
      "loss: 2.208034  [32000/60000]\n",
      "loss: 2.220039  [38400/60000]\n",
      "loss: 2.176813  [44800/60000]\n",
      "loss: 2.181044  [51200/60000]\n",
      "loss: 2.144097  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 46.0%, loss média: 2.140186 \n",
      "\n",
      "Epoch 2\n",
      "-------------------------------\n",
      "loss: 2.149046  [    0/60000]\n",
      "loss: 2.141654  [ 6400/60000]\n",
      "loss: 2.080613  [12800/60000]\n",
      "loss: 2.104068  [19200/60000]\n",
      "loss: 2.053595  [25600/60000]\n",
      "loss: 1.980902  [32000/60000]\n",
      "loss: 2.013377  [38400/60000]\n",
      "loss: 1.925538  [44800/60000]\n",
      "loss: 1.934078  [51200/60000]\n",
      "loss: 1.859145  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 60.0%, loss média: 1.860133 \n",
      "\n",
      "Epoch 3\n",
      "-------------------------------\n",
      "loss: 1.891657  [    0/60000]\n",
      "loss: 1.865061  [ 6400/60000]\n",
      "loss: 1.741046  [12800/60000]\n",
      "loss: 1.788998  [19200/60000]\n",
      "loss: 1.689588  [25600/60000]\n",
      "loss: 1.626987  [32000/60000]\n",
      "loss: 1.650402  [38400/60000]\n",
      "loss: 1.545289  [44800/60000]\n",
      "loss: 1.576880  [51200/60000]\n",
      "loss: 1.472128  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 62.0%, loss média: 1.488313 \n",
      "\n",
      "Epoch 4\n",
      "-------------------------------\n",
      "loss: 1.552855  [    0/60000]\n",
      "loss: 1.522441  [ 6400/60000]\n",
      "loss: 1.368197  [12800/60000]\n",
      "loss: 1.451789  [19200/60000]\n",
      "loss: 1.338904  [25600/60000]\n",
      "loss: 1.324719  [32000/60000]\n",
      "loss: 1.346894  [38400/60000]\n",
      "loss: 1.262178  [44800/60000]\n",
      "loss: 1.303636  [51200/60000]\n",
      "loss: 1.212903  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 63.7%, loss média: 1.229432 \n",
      "\n",
      "Epoch 5\n",
      "-------------------------------\n",
      "loss: 1.300912  [    0/60000]\n",
      "loss: 1.287441  [ 6400/60000]\n",
      "loss: 1.119360  [12800/60000]\n",
      "loss: 1.238376  [19200/60000]\n",
      "loss: 1.115256  [25600/60000]\n",
      "loss: 1.133718  [32000/60000]\n",
      "loss: 1.163434  [38400/60000]\n",
      "loss: 1.089454  [44800/60000]\n",
      "loss: 1.133903  [51200/60000]\n",
      "loss: 1.060545  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 65.0%, loss média: 1.070005 \n",
      "\n",
      "Epoch 6\n",
      "-------------------------------\n",
      "loss: 1.133990  [    0/60000]\n",
      "loss: 1.141040  [ 6400/60000]\n",
      "loss: 0.956652  [12800/60000]\n",
      "loss: 1.103764  [19200/60000]\n",
      "loss: 0.978199  [25600/60000]\n",
      "loss: 1.005490  [32000/60000]\n",
      "loss: 1.048359  [38400/60000]\n",
      "loss: 0.980530  [44800/60000]\n",
      "loss: 1.023831  [51200/60000]\n",
      "loss: 0.963030  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 65.9%, loss média: 0.966934 \n",
      "\n",
      "Epoch 7\n",
      "-------------------------------\n",
      "loss: 1.018738  [    0/60000]\n",
      "loss: 1.046483  [ 6400/60000]\n",
      "loss: 0.845994  [12800/60000]\n",
      "loss: 1.013077  [19200/60000]\n",
      "loss: 0.892478  [25600/60000]\n",
      "loss: 0.915242  [32000/60000]\n",
      "loss: 0.972019  [38400/60000]\n",
      "loss: 0.910659  [44800/60000]\n",
      "loss: 0.948834  [51200/60000]\n",
      "loss: 0.897511  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 67.4%, loss média: 0.897173 \n",
      "\n",
      "Epoch 8\n",
      "-------------------------------\n",
      "loss: 0.934972  [    0/60000]\n",
      "loss: 0.981305  [ 6400/60000]\n",
      "loss: 0.767885  [12800/60000]\n",
      "loss: 0.949114  [19200/60000]\n",
      "loss: 0.836360  [25600/60000]\n",
      "loss: 0.849808  [32000/60000]\n",
      "loss: 0.918000  [38400/60000]\n",
      "loss: 0.864687  [44800/60000]\n",
      "loss: 0.895699  [51200/60000]\n",
      "loss: 0.850767  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 68.8%, loss média: 0.847487 \n",
      "\n",
      "Epoch 9\n",
      "-------------------------------\n",
      "loss: 0.871020  [    0/60000]\n",
      "loss: 0.932871  [ 6400/60000]\n",
      "loss: 0.709982  [12800/60000]\n",
      "loss: 0.902003  [19200/60000]\n",
      "loss: 0.797000  [25600/60000]\n",
      "loss: 0.800839  [32000/60000]\n",
      "loss: 0.876948  [38400/60000]\n",
      "loss: 0.833152  [44800/60000]\n",
      "loss: 0.856553  [51200/60000]\n",
      "loss: 0.815373  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 70.0%, loss média: 0.810063 \n",
      "\n",
      "Epoch 10\n",
      "-------------------------------\n",
      "loss: 0.820013  [    0/60000]\n",
      "loss: 0.894206  [ 6400/60000]\n",
      "loss: 0.665338  [12800/60000]\n",
      "loss: 0.865947  [19200/60000]\n",
      "loss: 0.767482  [25600/60000]\n",
      "loss: 0.763446  [32000/60000]\n",
      "loss: 0.843834  [38400/60000]\n",
      "loss: 0.809854  [44800/60000]\n",
      "loss: 0.826107  [51200/60000]\n",
      "loss: 0.787091  [57600/60000]\n",
      "Erros do teste: \n",
      " Acurácia: 71.2%, loss média: 0.780460 \n",
      "\n",
      "Acabou!\n"
     ]
    }
   ],
   "source": [
    "for t in range(epochs):\n",
    "    print(f\"Epoch {t+1}\\n-------------------------------\")\n",
    "    train_loop(train_dataloader, model, loss, optimizer)\n",
    "    test_loop(test_dataloader, model, loss)\n",
    "print(\"Acabou!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Brincando com o modelo\n",
    "Eu fiz uns desenhos toscos e vou ver como o modelo classifica eles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHRCAYAAAABukKHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABBqklEQVR4nO3debjdVXn3/88NEshERpKQmRkjk8yTqBSkTBZxpMpQ24dHLdRev9SxT5/q42VLbf09DtRLpf4ELSBSFUGLggQEglCQKWEKATKThMxzgLB+f5yder73utc53+wzn/N+XReXrnXuvfd37732Xvnu+/6uZSklAQCA3G49fQAAAPRWTJIAABQwSQIAUMAkCQBAAZMkAAAFTJIAABQwSUoys8vM7P42/n67mV3anceE/sePMzNLZnZgTx4TIElmdo+Z/UXhb1PNbJOZ7d7dx9UbDKhJ0sxONbMHzGy9ma0xs9lmdlx7t0spnZ1Suq6N+21zkkX/Y2YLzGxr48tjhZlda2bDevq4MHA0xt7O/95oNR43mdmHg/jPm9lLjb8vMbOb6jxOSmlRSmlYSmlHG8dSnGT7ugEzSZrZ3pJ+IembkkZLmiTpi5K2d/B+39Txo0MfdX5KaZikoyUdK+l/9fDxtImx2r80Jq5hjTG4SI3x2Pjv+taxjV/CLpZ0RiP+WEl3dfQYrEW/nkf69ZNzDpaklNKNKaUdKaWtKaU7UkpP7gwws38xs7WNf22d3ar/v/+V1DhrnG1m/9fMVku6SdK3JZ3U+Bfauu59WuhpKaWlkm6XdFjjJ9T/nozq/gvbzEaY2Q/M7BUzW2hm/8vMdjOzPc1snZkd1ip2n8ZZw7hG+zwze7wR94CZHdEqdoGZfcbMnpS0mYlywDpO0q9TSi9IUkppeUrpuy5mWuO7baOZ3WFmYyXJzKa3HteNMf1lM5staYukH0p6m6SrG9+BV3ff0+p6A2mSnCdph5ldZ2Znm9ko9/cTJD0naaykr0j6nplZ4b5OkPSipPGSPiLpY5J+1/gX3MguOXr0WmY2RdI5ktZ24G6+KWmEpP0lvV3SJZL+LKW0XdJPJV3UKvYDkn6bUlppZm+V9P9J+p+Sxkj6jqRbzWzPVvEXSTpX0siU0usdOEb0XQ9KusTMPmVmxxbyi38q6c8kjZM0SNLftHF/F0u6XNJwSZdJuk/SFY3vwCs69ch72ICZJFNKGySdKilJukbSK2Z2q5mNb4QsTCld0/jd/TpJ+6plEowsSyl9M6X0ekppa5cfPHqrWxq/HNwv6beS/qGZO2l8YX1I0udSShtTSgskfVUtX0SSdEPj7zv9aaNPavmi+k5K6aHGLyTXqSWFcGKr+G+klBYzVgeulNK/S7pS0llqGasrzewzLuz7KaV5jXHyY0lHtXGX16aUnmp8B77WJQfdSwyYSVKSUkrPpJQuSylNlnSYpImSvtb48/JWcVsa/7dUiLG4yw4SfckFKaWRKaVpKaVPSGp2EhoraQ9JC1v1LVRL3lyS7pY0xMxOMLPpavny+lnjb9MkzWz81LquMWlPUcvY3onxOoC0qkbdZGabdvanlK5PKZ0haaRafv36kpmd1eqmy1v9/y0qf/9JA2hMDahJsrWU0rOSrlXLZLnLN2+njYFpc+N/h7Tqm1DjdqskvaaWCW+nqZKWSlLj140fq+Vn04sk/SKltLERt1jSlxuT9c7/hqSUbmx1X4zPAaRVNerOoh7/99dSSjdLelLNff9JA+g7cMBMkmZ2qJnNNLPJjfYUtXzhPNgJd79C0mQzG9QJ94U+KqX0ilomto+Y2e5m9lFJB9S43c5J8MtmNtzMpkn6fyT9e6uwGyR9UNKH9YefWqWW1MHHGmeZZmZDzexcMxveSU8L/UCj4PDcxvjarVGY+BZJD3XSQ6xQSz693xkwk6SkjWopuHnIzDarZXKcK2lmJ9z3LElPSVpuZqs64f7Qd/0PSZ+StFotX0IP1LzdlWo5E31RLTnOG9RSkCNJSik91Pj7RLVU0u7sf6TxmFerpXBovloKKYDWNkj6vFouFVmnluLEj6eUOuv67q9Lel/j6oBvdNJ99grGpssAAMQG0pkkAAC7hEkSAIACJkkAAAqYJAEAKGCSBACgoM3Fjs2M0tcBLKVUWru2SzHuBraeGHeMuYGtrTHHmSQAAAVMkgAAFDBJAgBQwCQJAEABu5QDqNhzzz2zvu3bt/fAkQA9jzNJAAAKmCQBAChgkgQAoICcJPqtQYPyPbCPPvroSvuUU07JYqZNm5b1nX/++ZX2lClTspjdd999Vw+xx61duzbrmz17dta39957t3u773znO5X27bffnsUAfQ1nkgAAFDBJAgBQwCQJAEABkyQAAAUU7qDfOuuss7K+iy++uNJevnx5FvPKK69kfb/73e8q7ZEjR2YxUV9v99xzz2V9jzzySNb3+uuvV9oTJkzIYi666KJKm8KdFuPGjau0f/SjH2Uxr732WtbnX/OtW7dmMRs2bGj3fl5++eVKe86cOVlMNOaffvrpSnvVqlVZzEDAmSQAAAVMkgAAFDBJAgBQ0Ktzkin17GbhZt2+QTo60ec+97msz+cWozzLm96UfyzGjh1baQ8dOrSDR9c7HHDAAVnfiBEjsj6fu128eHEW41+TaHGFHTt27Ooh9nkrVqyotKPXYPXq1VnfvHnzKm2/oIMkjR8/vtKOXvOTTz650vYLY5SOyedEo5hhw4ZV2kcddVQW09e/RzmTBACggEkSAIACJkkAAAqYJAEAKOg1hTs9XaTTlbryufX1pHhnmjRpUqW9fv36LMbvXhEV6Rx88MFZ33HHHdfu7fqiMWPGZH1nnnlm1nfLLbdU2lu2bMli9tprr0o7eo36W+FOM5/turvFTJ48udLec889sxhfFLR9+/YsZvPmzZW2X4BAyhcckKSlS5dW2r4ASMoL2iJ9/fuPM0kAAAqYJAEAKGCSBACgwNr6vdjMuuzH5P6cg+xtmv3dPqXUIwnPZsfd6NGjK21/MbYkPfTQQ23eRpIOO+ywrM9fKN+fc8H+InIpfy2jRbKPOOKISvvII4/MYqIFuL2eGHd1xlxXfmc9+uijWd/8+fMr7SiX+OKLL1bab7zxRruPNXjw4KzPL8IuSYccckilHX2edtutep516qmnZjEzZsxo95g6S1d813EmCQBAAZMkAAAFTJIAABQwSQIAUNAthTvNJryjC4/9rg0bN27MYupchNufCy/qqPP8+1rhjn9OUQGKL2yILuwe6GMj4l83f4G6JA0aNKjS9osL1NVbCne6qlAnKoCZO3du1udf86goZsKECZ13YO14/vnns75f/vKXlfb++++fxbz97W+vtKNdZrpSR7/rOJMEAKCASRIAgAImSQAACpgkAQAo6NVbGaxZsybre+CBByrtdevWZTF+x4Y3v/nNWUzdlfjRd/hCi6jwor/s3tHd/Moqw4cP76Ej6RqdVaQTFRv6HVP8TjSStGnTpqzPF8GMHDmyYwfXQdHqVIcffnilHRV0bd26tdLu7sKdjuJMEgCAAiZJAAAKmCQBACjokgRNZ/2+H+UN/e/ZUW7EX2Dr8ynI36P+eAH9ggULsr4DDjig+w8E/Zbf1SS64H7lypWVdpR/HDJkSNbnx2q0KEp3ir5r995770o7ykn6xRPGjBmTxeyxxx4dPLquw+wBAEABkyQAAAVMkgAAFDBJAgBQ0KuvrI4unn3rW99aaU+dOjWLWb9+faXdG4tS/Ar/0aIIURLcP5foNRo6dGibtxkoXn755ayv2cId/35t3749i/GFFVHB2CmnnFJp+8Ux6uqqHSrqPl70/FesWFFpR4V30cX2fcW2bduyvkceeaTSjoprfCHh4sWLs5howZN99tmn0o5eO/+94QuJpHpjJRqrgwcPrrT9Li+SNG7cuErbjwEp/+wsWrQoi+nNBXWcSQIAUMAkCQBAAZMkAAAFvTonGf1OHi2yW+d2Xp0cS7Szvb/ot9mFCjZs2FBpP/HEE1nM8uXLsz6fX4wWC/Z5L3/B70DhL+LuiI0bN1bajz32WBbj8+XRe9NsDtKLck9Lly6ttF999dUsZr/99qu0617E7T8LzzzzTBbj83MTJ07MYqJ8XG+1evXqSvu5557LYnye7sADD8xiRo0aVWlPnjw5i9lrr72yPv8ez58/P4vx73k0LnwuNcqtRn0+vxqN59NOO63SPu+887KYJ598stL2xyzF31E+J9tTOJMEAKCASRIAgAImSQAACpgkAQAo6NWFO13JJ6rnzp2bxUQXxvqiGJ+Ur+tNb6q+9FFSPFpMwF88/MILL2QxvjDA7x4+UPjXuCP86/zggw9mMb4Y7Iwzzui0x/emT5+e9X32s5+ttLds2ZLFnHPOOZX2YYcdlsVEi0/4wrao+MLvbhFdfN5b3Xvvve3GRLtgHHXUUZV29Nr5vuji/qhw5u67766077///izGF1RFj+/fh2jBg+i98osXPPvss1mMXzAiKtw55JBDKu2HH344i4mKovz3XzTmuwNnkgAAFDBJAgBQwCQJAEBBl+Qk6/wG38z9dCa/CLq/EFqKd7b3C6o3m5P0i5AffPDBWUz0G7zPMy1btiyL6c2LBXclf0H2jBkzOu2+fe4nulD/hhtuqLSjHdhfeeWVTjmeaCd7n6/eunVrFuMvSI/GXZSz8vmhaEF+vzBB9Br1Vp/85CezPv9a+Zxrs6Lvw1WrVmV9PgcYLc7wjne8o9KeNGlSFlPnezRaOMV//0U5UT8OokXY/ecyWiQl+q71izlMmTIli4kW0e9snEkCAFDAJAkAQAGTJAAABUySAAAUdMtiAlHiuLt3Vvf8rtvRivM+cSzFRQ3N8K/JsGHD2o2JRKv+d0cyuzcaOXJkpR0VI9QRFR940UIFvoihKwvPoovPfeFMVLjjx3S0YEU0xv1r4nebl/IL0pt9/XvC448/3m2PFY2vqBDKFxeefPLJWYwv1Inu2xdQ+R2IJGnChAlZn99xKdrxyN93nWItX/woSXPmzMn6/GsS7dTkC9i64jPHmSQAAAVMkgAAFDBJAgBQwCQJAEBBj+0C0pVFDXX4Ff1PP/30LOatb31r1hclnXuSL9YYyHzBUlQMVUe0e4bf9SIav0uWLKm016xZ09Tj1xEVDvlj8iudSHlhRbSKjC/YiESFO1FhB3JR0VVUTOOLG6MVuPyY8zuHSHkBV1QAs99++2V9J554YqUdjTlfOBg9Ny8q7vE7hfQmjGoAAAqYJAEAKGCSBACgoMdykj3N50+iPEyd3Ax6D7+YwPjx45u6n7Vr12Z9fpeGaJd6n7fs7ovpfU4yyldHucQ6fO4pyiv5nHCdRRkGoui1i/KE/v2Mdn7xr7H/DEh5vjMa31G+8ZRTTqm0o0VK/Hhqdnz1ZpxJAgBQwCQJAEABkyQAAAVMkgAAFAzYwh30P35HhGYXWoh2xvAX3f/617/OYvyuDd2tTuGOX2AhKtiI+CKkqPjE7wLS0wuG9FbR6xK9nr64MHqv/AID0Q4uvnBn7733zmIuvPDCrM+/n3V2F+qPC0r0v2cEAEAnYZIEAKCASRIAgIJ+mZP0CwPX2a178ODBWQw5lb5l3Lhxu3wbP1akOCfpF26OdpL3C0l3N5+zihZTOOaYYyrtuq9Znc+Cfy37Y36qM0R5w2jM+RxgtAiAf4+jBQd8PjnKSQ4dOrTd4xwxYkQW4xeZ6G0bQHQGRjEAAAVMkgAAFDBJAgBQwCQJAEBBvyzc8SvRv/LKK1nMCy+8UGkfeOCBWUxU1NBMMUKdlfEpcui4CRMm7PJt6hR1RaJCC1/E0N3q7BrhCz3qLibgFybYa6+9shj/ut14441ZzLve9a42bzMQRK9dtKuML9xZtGhRuzF+0Qspf439ghJSXEzkj3Ps2LFZjC9oiz4XfR3fzAAAFDBJAgBQwCQJAEBBv8xJ+jzTsmXLspj58+dX2lE+Kfrt3vdFu8+vWbOm0l69enUW4/NFEydOzGKwa0499dRdvk30/kWLAvjcT0/nH7ubH/djxozJYl588cVK+7HHHsti9t1330p74cKFnXB0fUtUf3DEEUdkfbNmzaq0H3/88SzmiSeeqLSjcbl9+/ZKO6q1iHKS/jij/L3/HosW5+jrOJMEAKCASRIAgAImSQAACpgkAQAo6JeFO/7C52nTpmUx/qLb0aNHZzFRgt0vDLBy5cos5oEHHqi0ox3rR40aVWlHFxhj1xx66KG7fJuoSCd6v/xF91HBT0/zRRPRIha+sKNuAZIv0IgKTTZu3FhpRwVr7KwTi4ppLrvsskr70UcfzWL8+xcVG/rvmuOPPz6LWbp0adbnCx6jx7/ggguyvv6GM0kAAAqYJAEAKGCSBACggEkSAICCflm444sDoqKcY445ptKOinQGDx6c9fliiKjIwxeDREn5N7/5zZV2tMI+do3fSeHBBx/MYnyhgy82keLdFgYNGlRp+90Pups/Hik/piVLlmQx9913X7v3U0edgqeocMgXqFHI0yJ6HQ455JBKO9qpyK+mE/HvsS8slKSnn34665s8eXKl/fa3vz2L2W+//dp9/L6OM0kAAAqYJAEAKGCSBACgYEDkJOuou3q9z13uv//+WYzPZfqLeaU8fxblRFEW7VrgL3b+/e9/n8X4RQCinQ2iXFqU1+5JfsEMKc9PRflWv/hF9FmJPgvRa9IevwCDJE2fPr3SjnboQczvRCPlizzUcdppp9XqQwu+mQEAKGCSBACggEkSAIACJkkAAAr6ZeFOM7ioue/zOyBERQ3NFu74C6ufeOKJLKY7F4SIFjw47LDDKu1oMQH/3OqOe19YVud2Q4cOzfpuvvnmSjtalADoTTiTBACggEkSAIACJkkAAAqsrYvozazeFfbol1JKPZKobXbc+RxktLB8HdFF8KtWraq0o4Xt6y5I0VX88/cLVkjNL1rhn1udxQV8/lfKFzjwC843Hqvbx113ftfdddddWV+0ULnP6R555JFZzIgRIzrvwDpBX63taGvMcSYJAEABkyQAAAVMkgAAFDBJAgBQwGIC6De2bNlSaS9YsKBnDqSH+Ofv2+gd/KIXkvSb3/wm69u0aVOl/eSTT2YxF154YaU9ceLELObOO++stKNFJjZv3pz1+cUxxo8fn8W8/PLLlfagQYOymFdffTXr60s4kwQAoIBJEgCAAiZJAAAKWEwARX1tMQH0D/19MYG6fE7w3nvvzWJ8DjJaXGDZsmWVdrSAQ7TwxHPPPVdpr1u3Lovxj3faaadlMdEmAr0NiwkAANAEJkkAAAqYJAEAKGCSBACgoM3CHQAABjLOJAEAKGCSBACggEkSAIACJkkAAAqYJAEAKGCSBACggEkSAIACJkkAAAqYJAEAKGCSbJKZJTM7sEbc9Ebsm7rjuABgV5nZPWb2F4W/TTWzTWa2e3cfV2/Q7yZJMzvVzB4ws/VmtsbMZpvZcT19XBjYzGyBmW1tfNmsNbNfmtmUnj4u9F2NsbTzvzdaja9NZvbhIP7zZvZS4+9LzOymOo+TUlqUUhqWUipuDNnWJNvX9atJ0sz2lvQLSd+UNFrSJElflLS9J48LaDg/pTRM0r6SVqhlnAJNaUxcwxpjapEa46vx3/WtY83sUkkXSzqjEX+spLs6egzWol/NI15/e3IHS1JK6caU0o6U0taU0h0ppSfN7AAzm2Vmq81slZldb2Yjd96w8S/9vzGzJxtnoTeZ2V6t/v4pM3vZzJaZ2UdbP6iZnWtmj5nZBjNbbGZf6K4njL4npbRN0n9ImiG1P37M7BIzW9gYu3/XGKtn9MCho+86TtKvU0ovSFJKaXlK6bsuZlrjl7eNZnaHmY2V8pRR46zxy2Y2W9IWST+U9DZJVzfOUq/uvqfV9frbJDlP0g4zu87MzjazUa3+ZpL+UdJESW+WNEXSF9ztPyDpjyXtJ+kISZdJkpn9saS/kXSmpIMk+S+ozZIukTRS0rmSPm5mF3TSc0I/Y2ZDJH1Q0oONruL4MbMZkr4l6cNqOQMdoZZfSIBd8aCkSxr/2D+2kF/8U0l/JmmcpEFq+c4ruVjS5ZKGq+V78j5JVzTOYq/o1CPvYf1qkkwpbZB0qqQk6RpJr5jZrWY2PqU0P6V0Z0ppe0rpFUn/r6S3u7v4RkppWUppjaTbJB3V6P+ApO+nlOamlDbLTa4ppXtSSnNSSm+klJ6UdGNw38AtZrZO0nq1/IPrn6V2x8/7JN2WUro/pfSqpP+tlvEN1JZS+ndJV0o6S9JvJa00s8+4sO+nlOallLZK+rH+8P0XuTal9FRK6fWU0mtdctC9RL+aJCUppfRMSumylNJkSYep5czxa2Y23sx+ZGZLzWyDpH+XNNbdfHmr/79F0rDG/58oaXGrvy1sfSMzO8HM7jazV8xsvaSPBfcNXJBSGilpL0lXSPqtmU1oZ/xUxl5KaYuk1d183OhDWlWjbjKzTTv7U0rXp5TOUMsvFh+T9CUzO6vVTUvff5HFbfytX+l3k2RrKaVnJV2rlsnyH9TyL/DDU0p7S/qIWn6CreNltfw8u9NU9/cbJN0qaUpKaYSkb+/CfWOAaeTLfypph1p++Whr/LwsafLO25rZYEljuveI0Ze0qkbdWdTj//5aSulmSU+q5buxqYdpp91v9KtJ0swONbOZZja50Z4i6SK1/B4/XNImSevNbJKkT+3CXf9Y0mVmNqORT/p79/fhktaklLaZ2fFq+W0fCDUqAv9E0ihJz6jt8fMfks43s5PNbJBafurnH2DYJWZ2WaNAbLiZ7WZmZ0t6i6SHOukhVkjav5Puq1fpV5OkpI2STpD0kJltVsvkOFfSTLVcCnK0WvJBv5T007p3mlK6XdLXJM2SNL/xv619QtL/MbONaskZ/bhDzwL91W2Nn782SPqypEtTSk+pjfHT+PuVkn6klrPKTZJWisuasGs2SPq8Wi4VWSfpK5I+nlK6v5Pu/+uS3mct1wB/o5Pus1ewlPrtWTLQ75jZMLV8yR2UUnqphw8H6Pf625kk0O+Y2flmNsTMhkr6F0lzJC3o2aMCBgYmSaD3+xNJyxr/HSTpQ4mfgIBuwc+tAAAUcCYJAEBBm9s3mRmnmQNYSqlHLjXoq+Nut92q/+Y86aSTsphPfOITlfZrr+WLleyxxx5Z35veVP2ofv/7389i7r777kp7+/a+WQDbE+Ous8Zc9N4NHz68M+66lo0bN2Z90Rjzevq4I/651HkezWprzHEmCQBAAZMkAAAFTJIAABQwSQIAUNDmJSB9tYACnYPCnbIxY/I1xr/61a9W2mb5yzdv3rxK2xf7SNKUKVOyvnHjxlXaW7duzWLuvffeSvu6667LYrZs2ZL19TZ9qXDHF7yMHj06i5k5c2ZzB9UEPwal5gt3uvO4I/65rFmzJovprGIeCncAAGgCkyQAAAVMkgAAFJCTRBE5yT/Yc889K+1LLrkki/mjP/qjSvu+++7LYu66665K+/nnn89iDjrooKzvne98Z6X97ne/O4vZvHlzpX3HHXdkMd/73vcq7R07dmQxPa235iSjvJ3PQUZ5PJ93njZtWhYzYsSIrG/SpEmV9htvvJHFLFmypNJ+8cUXs5j169dnfd7rr7+e9e23336VdrQ4RZSb9/xxL168OIt59dVX272fq666KuuL8pTNICcJAEATmCQBAChgkgQAoIBJEgCAgjZ3AQEGomj3gyuuuKLSPuqoo7KYF154odL+5S9/mcUsWLCg3cd/9tlnsz6/C4gvqpCkI444otIeOnRoFvOud72r0r799tvbPR60iMaFL9SJFod4xzveUWmPHTs2i5k8eXLW5++rTnHLc889l/XVGXN+sQpJmjBhQqUdLZSw++67V9pRAdKoUaMq7Q0bNmQxt912W9bnC9F6CmeSAAAUMEkCAFDAJAkAQAE5SQxo0SLkH/jAB7K+ww8/vNKOFhifNWtWpV0nF1TX3LlzK+3//M//zGL84unvfe97sxh/0fiJJ56YxfzXf/1X1hddyI7cMccck/X5nNzUqVOzmGgc+oUeooUCfMwJJ5yQxZx00kmVts9vS/FC4VGc53OS++67bxbjFyHwiySUbjd//vxKO1rMwfd11oLnrXEmCQBAAZMkAAAFTJIAABQwSQIAUEDhDgYUv5vHhz70oSzGX/wtSQsXLqy0V61alcXce++9HTu4XXDPPfdkff5i92j3h3PPPbfSHjlyZBYT7ewQLYyA/DWPil2ihQK8qODEF35t2rQpi/GFM0OGDMlifLFWdIxR4ZBfzCAq3vIX/K9duzaL8QsVRDvP+N1MItEOK35nkM7aFaQ1ziQBAChgkgQAoIBJEgCAAnKS6Ld8/lHKc5DnnHNOFhNdTO9zRg888EAWU2cB6q50xx13VNrHHntsFnP00UdX2qeddloWE+Vbjz/++Eo7eo0GIr+IfJST83ngH/7wh1nMunXrsj6/YMWgQYOyGJ83TCkVj7V0v1Kcp/T5zohfGCDKZ/tjeuihh7KYW2+9td3H8ovzdxfOJAEAKGCSBACggEkSAIACJkkAAAoo3EG/4YsPPvWpT2UxRxxxRKX9ne98J4uJdsbwFyk/+eSTzRxil/K7LUQFIqtXr660fVGJJF1wwQVZ30EHHVRpP/vss1lMtON8f7d8+fJK+wc/+EG7t7n44ouzvosuuijrmzZtWqU9ePDgLGbYsGGVtl84IOqLiouiXTj8Rf/RLhz77LNPpR19Lp566qlK+9Of/nQWU2f3jkcffTTr27hxY7u36yjOJAEAKGCSBACggEkSAIACJkkAAAoo3EG/ceWVV1ba06dPz2L8rg3nnXdeFhPttnDjjTdW2j29uk4d8+fPz/r8Sjljx47NYnxxkyQ9+OCDlfaMGTPajelvoiIRvwuFX9GoLl9QJUlbtmzZ5fuOVsnxRTHRSlTRDh917vvxxx+vtOvshBMV6XTF7h2dhTNJAAAKmCQBAChgkgQAoMDaWjXezNpfUh79Vkop3668GzQ77r75zW9W2lFO8uGHH660ox0Rrrnmmqwv2qWhP7jllluyPr+zhZQ//7322iuL8Re21/HOd74z6+uJcded33XRRfmjR4/O+mbOnNnufb3jHe+otKPdPHy+cfPmzVnMyJEjsz4/N0R5+N/+9rdt3kaSvvrVr1baUf6xzmICXamtMceZJAAABUySAAAUMEkCAFDAJAkAQAGLCaBPOuaYY7K+MWPGtHs7v2vB5z73uSwmWkygq5x22mlZ37e//e12bxcVSPhdQKICiahQyYsuLN9vv/0q7ZUrV7Z7P4jVvZjeF7xEhTz+PY/4mKi4J2JWrWV5/vnns5i2Cj938s+3p4t0dhVnkgAAFDBJAgBQwCQJAEABOUn0SV/60peyvvHjx1fa0QIAhx12WKVdZ0HmrhTtCL/bbvm/XRcvXlxpr1ixIovxucTofqZMmVJpT5s2LYuJ8o3PPPNMpX3AAQdkMd/61rcqbX+he+RXv/pVuzEDQZSnq5O727ZtW6U9ePDgLMbnof3npK5ly5Y1dbu+jjNJAAAKmCQBAChgkgQAoIBJEgCAAgp3ukF0wa2/ULc33ndvcvfdd1faI0aMyGIWLVpUaUeFK764JdptvjstWbIk64vevyFDhlTa0S4cfif76H58MdPy5cuzmKhgZPbs2ZX2Qw89lMX4HT0WLFiQxfhCpe5cuKE/8q+fXyxDktavX19p1y3cef311yvtOgsH9EecSQIAUMAkCQBAAZMkAAAF5CR3kc/z+IusJWnWrFmV9iWXXJLF/Nu//VulfdJJJ2UxO3bsaPd45syZ025Mf+AXab7mmmvavY2/uF2Sbr311ko7unA+WgB69913b/fxvGihcN8XxUS5H//4UU7SX0g+duzYLGbo0KGV9ve///0sxl+gLuU710cLs/uccGT06NGVdp0xjjI/DqI8tB9j0WsejW8fF923H6t+UXap5/P+HcWZJAAABUySAAAUMEkCAFDAJAkAQAGFOw177rln1nfggQdmfeedd16l/bWvfS2L8RfvzpgxI4s54YQTKu0xY8ZkMVEBiS9g+cQnPpHFDB8+vNLu64lzKX9N/esn5Re4R6+N98UvfjHri3YP8Y/vd1aIDBo0KOsbN25cpT1s2LAsxhfJSNInP/nJdh+vDj82ooKxaDEB//lYuHBhFrN58+ZKOyoG8UUk/XHhi+7ki7Wioi+/EEVULOYXDojuKyoW85+DZncz6c04kwQAoIBJEgCAAiZJAAAKmCQBACgYEIU7UVHO5ZdfXmlHq4VEiWpfnHDKKadkMQcccEClffzxx2cxzazgIkmPPPJIpR09t76eKK/DF9JI0jnnnFNpRwUKDz/8cKX9T//0T7Uez68Us/fee2cxvlAnKrx6+umnaz2e5wuV9ttvvyzG7wJSp0DDj2cpXnFnzZo1lXa0w8oee+xRadcpyhmoO0t0Fl+IFfHFPVGRTvR+enUKd/ojziQBAChgkgQAoIBJEgCAgl6Tk4wuvD744IOzvjPPPLPSXrZsWRbjczGvvvpqFuNzkNFF3ZdeemnW53OQ0S4Sfrf3xYsXZzH+8aLf+6NjeuGFFyptnwcaKKKcnM+lbdiwIYs56qijKu0obxjlyXwONMrreHV3+GjGE0880W5MlBP04yXKRUXPzefQo9v5XLjPhUUGQv68K9XJ+/r3LnrN/YIDUj4Oou9oL/o+8n197T3nTBIAgAImSQAACpgkAQAoYJIEAKCgw4U7UeL4yCOPrLTf//73ZzH+ovioyCHii2B27NiRxfgdNaLdPPwx7b///rUe35s+fXrWN3bs2ErbF/JIecFPdHF6VFTy/PPPV9pRMn2g7qzgx1BUIBAVWtXh7ysar/69iBaM8OM1up+o+MEXykSFXr7QIho/dUSfKV/8VufxR40alcX4Y/rc5z6XxVx44YW1jrM/qVPwEvHjMrqNL8qJxkWdvjrHM3PmzKzvqquuqrT9whS9HWeSAAAUMEkCAFDAJAkAQEGHc5LR7u9LliyptOfOnZvF+Iuqo7zhX//1X2d9EyZMqLTr5F1WrVqV9S1durTd2zXLLwIQPbc65s2b126MX3hbynNj/XER4iiX5xf4rnPRcpRniW7nc4J18oZ18ux1F7r3n5cob+gfP1q4IFpYo84xDR06tNKOXqOpU6dW2iNGjMhitm/fXmlH3w0DUbRQuc/v1VlUPuLf82hThDp8rYWUf9f3R5xJAgBQwCQJAEABkyQAAAVMkgAAFHS4cKdO4vbEE0/M+v7qr/6qow9dW3Sh/sKFCyvtaDeRiRMndtkx1REtCuBfyzvvvDOLqbswQ1+2efPmrM8Xs0RFKr4AJSqGiBZo8BfKRztl+EKZ6H3oyh0Q6hQX1dmFJCrc8UU40WvkC6eigrE6hUOI+bEr5e9VNC79OIi+V6JCMH/fUSHWQMCZJAAABUySAAAUMEkCAFCwyzlJ/7t4nR2t/Y7xkvTMM89U2lGOadOmTVmfz4XUuajc7yov5Rc1P/XUU+3GSNLkyZMr7bPPPjuLqcMv1P6LX/wii1m9enXW5xdPiBYqqHOBcV8Xve8+1xLl1nxMnRxddF91FpGP8p3+Qu4otxd9puouOtAef9x1F8ivk5P1uf/oPYrqA1CPX6QkEr130Tisw382ml0wv6/jTBIAgAImSQAACpgkAQAoYJIEAKBglzOxvsAmKng55phjKu05c+ZkMb///e8r7SjhHBVVnHXWWZV2lJT2u6aPHDkyi/Er2kdJ6aioYd9996206xRwRHwBQ53dVCLRce+///6Vdl/bCbyOtWvXZn1jxoyptKPCqzoXs0e382No3LhxWYy/ILvOwgFRccvGjRvbve/owm5fTDN48OAsxhcF1SnSkfJxFn3u/Gsb7Wzxx3/8x1kf6mm26KnZoi//OfDfqwMFZ5IAABQwSQIAUMAkCQBAAZMkAAAF1taKI2ZWbzkS9EsppeaqkjooGndTp06ttP/5n/85u50vSomKS3yhTLT7QcR/TqJVpJpd2aQZdVZfiYqU/vZv/7bSfvrppzvtmDpLT4y7nv6uGz16dNb32c9+ttI++uijs5h99tmn0o7Gc52Vc6Lb+V1col1IZs+eXWlHK6d99atfrbSjQsKu3B2njrbGHGeSAAAUMEkCAFDAJAkAQMHAXNYdfc6iRYsq7Q9+8IPt3uZXv/pV1ud3lqmbR/S5zM985jNZjD9GIBLt0hP1+cUgosUZvGg8+3xjVIdSdzccb9SoUZV2lJOcOXNmpX3VVVdlMb15wRPOJAEAKGCSBACggEkSAIACJkkAAAoo3EG/xY4T6I2iAhxf3CLlO81EO8b4gptoAQlfzBPFRDvf+EUHosf3u+PU2bmobuFSTy8wsBNnkgAAFDBJAgBQwCQJAEABOUkA6EI+3xbl3yJjxoxp93Z+of2f/OQnWczZZ59dab/88stZzOrVq7O+E088sdK+//77s5gZM2Zkfe2J8q9+EXQpX2Cgp3KUnEkCAFDAJAkAQAGTJAAABUySAAAUULgDAF3ILx4QFa6YWdbnL9Rfu3ZtFjN37txKO9rNY/fdd6+0o0UB/E4hUr7AQPT4UV8zotfE7xbSUzuFcCYJAEABkyQAAAVMkgAAFJCTBIAe5hcTj/ruvPPOLGbIkCGV9ogRI7KY+fPnV9orVqzIYjZs2JD1+ZxktJhBlN/sbziTBACggEkSAIACJkkAAAqYJAEAKLDo4lMAAMCZJAAARUySAAAUMEkCAFDAJAkAQAGTJAAABUySAAAUMEkCAFDAJAkAQAGTJAAABUySQC9iZveY2V8U/jbVzDaZ2e7dfVzoH8zsMjO7v42/325ml3bnMfV2A3KSNLMFZra18YWz1sx+aWZTevq40Dc1xtHO/95oNbY2mdmHg/jPm9lLjb8vMbOb6jxOSmlRSmlYSmlHG8dSnGQxcJjZqWb2gJmtN7M1ZjbbzI5r73YppbNTSte1cb9tTrL90YCcJBvOTykNk7SvpBWSvtnDx4M+qjFxDWuMp0VqjK3Gf9e3jm38K/1iSWc04o+VdFdHj8FaDOTPMxrMbG9Jv1DLd9poSZMkfVHS9rZuV+N+852hB4AB/6FKKW2T9B+SZkiSmZ1rZo+Z2QYzW2xmX2gdb2aXmNlCM1ttZn/XOCs9owcOHX3TcZJ+nVJ6QZJSSstTSt91MdMa//LfaGZ3mNlYSTKz6WaWdn5ZNc4av2xmsyVtkfRDSW+TdHXjLPXq7nta6EUOlqSU0o0ppR0ppa0ppTtSSk/uDDCzf2n8ivaSmZ3dqv+/f4lonDXONrP/a2arJd0k6duSTmqMr3Xd+7R6xoCfJM1siKQPSnqw0bVZ0iWSRko6V9LHzeyCRuwMSd+S9GG1nIGOUMu/0oC6HpR0iZl9ysyOLeQX/1TSn0kaJ2mQpL9p4/4ulnS5pOGSLpN0n6QrGmexV3TqkaOvmCdph5ldZ2Znm9ko9/cTJD0naaykr0j6nplZ4b5OkPSipPGSPiLpY5J+1xhfI7vk6HuZgTxJ3tL4l9B6SWdK+mdJSindk1Kak1J6o/Evrxslvb1xm/dJui2ldH9K6VVJ/1sSe42htpTSv0u6UtJZkn4raaWZfcaFfT+lNC+ltFXSjyUd1cZdXptSeiql9HpK6bUuOWj0KSmlDZJOVct30zWSXjGzW81sfCNkYUrpmkZu+zq1/IN/fHxvWpZS+mZjfG3t8oPvhQbyJHlB419Ce0m6QtJvzWyCmZ1gZneb2Stmtl4t/3Ia27jNREmLd95BSmmLpNXdfNzoI1pVo24ys007+1NK16eUzlDLrxUfk/QlMzur1U2Xt/r/WyQNa+NhFrfxNwxQKaVnUkqXpZQmSzpMLd9dX2v8eXmruC2N/1saYwN+fA3kSVKS1PjN/qeSdqjlX183SLpV0pSU0gi1/Aa/86eIlyVN3nlbMxssaUz3HjH6ilbVqDuLevzfX0sp3SzpSbV8kTX1MO20McCllJ6VdK2aG2MDfnwN+EmyURX4J5JGSXpGLbmdNSmlbWZ2vFryQzv9h6TzzexkMxsk6Qv6wwQKtKtRDHGumQ03s90aRRNvkfRQJz3ECkn7d9J9oQ8ys0PNbKaZTW60p0i6SH+ou+iIFZImN77/BoSBPEne1vgJbIOkL0u6NKX0lKRPSPo/ZrZRLTnHH++8QePvV0r6kVrOKjdJWqkOllZjQNkg6fNquVRknVoKJz6eUuqsa8++Lul9jcrFb3TSfaJv2aiWgpuHzGyzWibHuZJmdsJ9z5L0lKTlZraqE+6v17OUBtzZc6cxs2Fq+aI7KKX0Ug8fDgCgkw3kM8mmmNn5ZjbEzIZK+hdJcyQt6NmjAgB0BSbJXfcnkpY1/jtI0ocSp+MA0C/xcysAAAWcSQIAUMAkCQBAQZurupsZv8UOYCmlHrkGtKfH3fDhw7O+cePGVdqLFi3KYl57jVXhOkNPjLueHnPoWW2NOc4kAQAoYJIEAKCASRIAgAImSQAACtos3AH6u2OPPTbru/XWW7O+V199tdJ+/vnns5iJEydW2jNmzMhiynvbdtyECRMq7V//+tdZzLp16yrtTZs2ZTFvvPFG1nfKKadU2qNHj27iCIG+hzNJAAAKmCQBAChgkgQAoICcJAa0f/zHf8z69tlnn6xvw4YNlXaUtxs8eHClHS0u4NdKbjZHGa25/Oyzz1baK1euzGJ833nnnZfF3H333Vnf5s2b271vv+AC6tt9992zvj333LPS9u+BJN1/f3Ub0gMPPLCpx58/f37Wd9FFF1Xaa9asaeq+m7V9e3Wb3h07dnTr4+/EmSQAAAVMkgAAFDBJAgBQwCQJAEABhTsY0IYOHZr1RUUUfmeQaKGAhx9+uNL++c9/nsVcfvnllfa2bdtqHae3ZcuWrO/xxx+vtFesWJHFnHXWWZX2M888k8VExR9+0YGo0AP1CnDq3s4Xi0VOPfXUdmMuuOCCdmMiixcvbup2nfX4s2bNqrSjMd8dxTycSQIAUMAkCQBAAZMkAAAFFl2U/N9/ZLfuXm/69OlZn1/E2rfr6okd4qXuHXf33Xdf1nfQQQdlfXfeeWelfdxxx2UxN998c6W9bNmyLMYvQr7XXntlMX5RAinPvfgLraX8fb700kuzmKeeeqrSPvroo7MYn9uUpJEjR1baW7duzWLOPffcrK8ZPTHu6oy5OvnGKOb0009v9pjajfHf37fccktTj1VHlFt805uqZS2jRo3KYvzmAFFuMVp4w/M5yui+ms1RtjXmOJMEAKCASRIAgAImSQAACpgkAQAooHCnj1m7dm2l/dBDD2Uxu+1W/bfP5z//+SzmkUceafexBkLhzoknnpj1PfDAA1mfLz6YM2dOFnPXXXdV2hs3bsxi/O4ZgwYNymKiHUa8qEBk2rRplfZ73vOeLObHP/5xpX3KKadkMb///e+zvpNOOqnSHjt2bBYTFTw1o7cW7gwZMiTrO/PMM9u97z322KPSjhawiBYcqLNjjP+sRzuF+EUJouKeqCjHP9+oyMz3RQU4r7/+eqUdjV3/+ZLyz080V3XWggMU7gAA0AQmSQAACpgkAQAoYIHzNkS5KW/SpElZn/8tP9rp3jvmmGOyPn8BtyT97ne/q7Sj39sPOOCASjvKn6HFgw8+mPX5PI+U50MOP/zwLMb3Re/NP/zDP1Taq1evzmLe8pa3ZH1Tp06ttKPFr88777xKO1qUwD/fD3/4w1lMlPu68sorK+06Oe2ByC+EH/VF4yLKQ/v3IcrJ+XzfsGHDspho4Yc6fL6xTq48em7ROPSiz5zP065atSqL8Qs1+EU/pDhPuSs4kwQAoIBJEgCAAiZJAAAKmCQBACigcKcN0a4GTz75ZKUdFV4sXbq00o52VfjWt75VaUcXcB966KFZn0/UX3/99VnMTTfdlPWhY+rsyHDIIYdU2o8++mgWM3r06Ep7zZo1WUxUlOMff968eVnMjTfeWGl/9KMfzWJuvfXWNtvYNf59qVOkEl1wH11g7+OihSd8TLOFM9ECA5dddlmlHRXXeNGCA5s2baq0o8UU/G4ikrRt27ZKO3oeHS3KqYMzSQAACpgkAQAoYJIEAKCASRIAgAIKd9rgd9yQpClTprR7O5+Yjgp3/vzP/7zSjnZsiBL8xx57bKX97ne/u93jQff4+te/Xmk3u/pHtHvI7NmzK22/Ao8UF/yga/mCm2hVHL9STVSkE93OFwVF3wc+xhe7SPkKYBdeeGEWc+2117Z7u0idnUp837p167KYUaNGZX3+dYqKgvxqQtFr6/vq7ArSGmeSAAAUMEkCAFDAJAkAQAE5yS7gdwePdmz3F4OvWLEii/E7mktxPgG9Q5Qz8tavX19p18lFSXkeJcqr+N3doxi/4MFzzz1XPli0yy/u4dtSvd08oh02fH4xysn5fFu0c5AfF9FOJdH3j99RxC8KINVb8MAvAuA/A1J83H6BgWjBAf+a+F1BpLw2YFcXIOBMEgCAAiZJAAAKmCQBAChgkgQAoIDCnW7gC3kk6Z577qm0o6T08ccfn/V95StfqbSjxQSii3XRMRMmTKi0f/CDH2Qxhx9+eKV9++23ZzG+iGLMmDFZTFTE4YsfouKPVatWVdo///nPs5jf/OY3lXadxTEGKl8UE12o7kVFV/69igqqtm/fnvX5opTodv57Iyoe87eLCgKjgiNfzBPt3uGPOyrcicaq9/LLL2d9e++9d7uP718jv7hAZ+BMEgCAAiZJAAAKmCQBACggJ9kN5s+fn/X5HMBLL72UxZxzzjlZ31NPPVVpf+QjH8lirr766l09RLTDLzoe5X1vuummSnvx4sVZjF80euzYsVnMypUrsz6fk4zy3HvuuWelvXTp0izmkUceqbSj3eajnOhA5F/P6EJ1L3o9m3ksqd7iED4nF713Pk8aLQrgH0vKn0t0EX60wIEX5Vvr8PnVKN9bZwGPjuJMEgCAAiZJAAAKmCQBAChgkgQAoKDPF+5ESWGfmI5WmPdJ6Sgp3FmihLtPgi9cuDCLeeWVV7I+n6i+4oorspi///u/r7T9RblSXCiAMl8oExVIbNiwodKO3vc64yzaEd5fyB5dfO2PKSq08Bd7R48VFXagnuhiel8UE42daGeMOvftx1O0KIl//GuvvTaLWbBgQdb3r//6r5V2VCS4zz77VNrRwgFRUVBnqbNTSUdxJgkAQAGTJAAABUySAAAU9Oqc5OrVq7M+f9H9fffdl8X43+6PPfbYLKbORbCd5cgjj8z63vzmN1fap5xyShbzox/9KOurc1G5z7e+973vzWJ8Lpcc5a45//zzs76rrrqq0o7yM/79ivI1Ps8iSWvXrq20o/fL57WiC619zipa7BrNixYKb3aBAZ9vHDFiRLu3iR7rG9/4RqXd7HffT3/606zv8ssvr7SjGpEoN+75RS4k6W1ve1u7t+vKWpKdOJMEAKCASRIAgAImSQAACpgkAQAo6DWFO9GuCj/72c+yvtGjR1faUVLYFzX85Cc/yWI++MEP7uIR1ueT12vWrMli/A7ad911VxYzadKkrM9fVB7tIuEfP9qh/n3ve1/WhxZRMYsvdoh2qd+2bVu79+Pfm2j8+iIdSRo+fHilHV007gt3mt19AZ3LF3BFiwJEhStRwWF7ou/Rv/zLv6y0/TiVpC9+8YtZ3xe+8IVKO1rcxBcSRsVqvhAtevzejDNJAAAKmCQBAChgkgQAoIBJEgCAgl5TuLNixYqsL9q9w4t2MfCiFSaiBHMzouIIv1LQddddl8UcdNBBlbZPgEv1dpGIHn/lypWVNrs67Jq3vOUtWd/Xvva1Sjt6b/bdd99KOyp08MVYy5cvz2Lq7AISPb4Xrbjj7zuKQddq9rsnWs3HF3ANHTo0i/HfGdHYiVZ+8o8XfY/6+4oKyvwuRHXHXPR8vc76Hm8LZ5IAABQwSQIAUMAkCQBAQY/lJP3v2353Dyle0b7Ob9D+N/go7xNd0OtF+T5/wfbChQuzmDlz5lTahx56aBbjf2+PLk6PjtHnAKLXyO8WHuU70TF+ZwVJWrp0aaUd7ebhF7qI8kPRWKiz24EfC9F9H3fccZU2+eoy//mfNWtWFnP66ad31+GE331vvPFGuzF1dnoZP3581ucXPInGk89B+uORmt+po07ustkdVnYFZ5IAABQwSQIAUMAkCQBAAZMkAAAFPVa44y9e9cUmkvTAAw9kfatWraq0Dz744CzGX8z/nve8p937mT9/fhYzd+7crM8X5UQX2PpEdZTM9rerm4D2cVFS3ifY61x4jj948cUXs76NGzdW2lHBiy/Kid53XwwSXXwdjSm/M0h00bh/n6OCic2bN1fahxxySBbz3HPPZX0DUW/7HEVjJRpjzYiKBP19R4/fla9JswU/nY0zSQAACpgkAQAoYJIEAKCgwznJ6OLVOr8l+wtVN2zYkMVMnTq13fuJfpNfs2ZNpf2zn/0si/E50Ggx3ei5+Qu9fR5Kqvf8m/293d8uuh9/3HUuJsYfRGPRixaa8PliP8ajmCgXHeUb/SIW0QIRo0aNqrSj8XvzzTdX2k888UQWE+VEUU8zi53UFS0y4b//otzili1bKu1oMfPoPfd5+GhxDJ+nrLMoeV11XsvOysm2hTNJAAAKmCQBAChgkgQAoIBJEgCAgg4X7kRJ4JUrV1baUZHDCy+8UGn/5Cc/yWL8jtZSvvt7xCd8o4UK2ruNFCfBfaI6SsL7vu7YPbs1nzzv7ouA+yM/FuoUP0S7GPhFCKLxExXujB49utKOPlMTJ06stCnK6X7R58qPg2Z35YnGnC/Kq/O5vv7667O+aOGUG264odL+yle+ksX4hTei4iKvTkxddQqFos/KruBMEgCAAiZJAAAKmCQBACjocE5y+fLlWZ9fmLzOBfdHHHFEU49f52LSOhfvDhkypKnHr7OzfFdeYBw9f39MUUx0oTtaHH/88VmfXwj8zjvvzGJ87mn//ffPYhYtWlRpR/mZaCz6xQSWLVuWxfhxNnz48CzGX1iOzlUnJ9hsjUJ03z5XHo2nq6++utL2Y1CK8+Depz/96azvsssua/N4pHzBjM6s0aiz8EdH6y84kwQAoIBJEgCAAiZJAAAKmCQBACjocOFOtMPEtm3bKu2oKKXORc11dhipczF/HdFtoovB69y3f/7RxfzR7g/eunXrsr6RI0dW2tHFtP4Yo4tpO3qBbX8RFRrMnDkz6/MXVvtCGkkaMWJEpe135ZCkJUuWVNrR2Gh215bNmze3ez/+M/WrX/2qqcdCff6z3mwhSfSdUaco5rOf/WylPX78+Czm7/7u77K+97///ZV2tAuI/x6PCmn892izO4XUWaihK3AmCQBAAZMkAAAFTJIAABQwSQIAUNDhwp0ocepXuY8KCOoUwHRWTJ3bRQnvKFHuk8fR7fyqF3VWvY9WxYlWwahTuORXOIp2U+nunUl6q7Fjx2Z9vjhKkhYvXlxpT506NYvxO3xE46fZHRD8+xXdt4+J3uM673u0SwUrNDXPv+e+sK+u6HvUf2/UKUB86aWXaj2e/xzUefzoO2v16tWV9uzZs7OYaIcTr853bVQU5GN2tXCKM0kAAAqYJAEAKGCSBACgoMM5ye9+97tZ35QpUyrt6Ldk/9t1lAdpNt9Y537879TR7gjRggc+FxTlEv3zjS7cjy5i96Lf133ey1/ALuW5A3+RudT8ricDVZ3ch39PozHl35top45o1xz/eNEiBD7XUieHFI3xMWPGZH1+EQTEou+DZnNi/rumTh46ynnXiYlqS/zt6nzXRd+1/nMxadKkLKZOnjT6zPl5Y+PGjVmM/zzt6k44nEkCAFDAJAkAQAGTJAAABUySAAAUdLhw53vf+17WN2fOnI7ebZerUwjRn0ULDAxEURFFVKDgi2Cii+vHjRtXaUcXjfuiiWjhgqioy4/XOjtCRM/N90WFQ9OnT8/6KNypJyocqSP6/vELXUQFkL4osM73WDR2ovHsx3ydQsros+OLBKdNm5bFRONr1qxZlfbpp5/e7uP//Oc/b/fxdxVnkgAAFDBJAgBQwCQJAEBBhxNxy5cvz/r22Wefjt4t0C2indSji5Z9PihakNmP+yjP4nOC0eP7BSOix4/ySj6/WWeX+Ei0Az3iRUGayZs99thjWd9b3/rWrM/nr6P3LsoBenVyiVGNwrp16yrtaOESf99RHtxfzB+N3ToLs/vXWmo+B7wrOJMEAKCASRIAgAImSQAACpgkAQAoGFhX0ANOVGgQ7UjjL6yOiih8YcO8efOyGH+76PGjxTh8sUNU6OALbqICJL/AwdSpU7OYqEAC8Xvld5S48847272fuhe3+2KtOrvK1LFy5cpacTfeeGOlfckll2QxfjenaFz6vmjBg5NPPjnru/feeyvtqHCqO3AmCQBAAZMkAAAFTJIAABSQk8SAFi0KcOqpp2Z9t912W6Ud5S19zijKCfp8lM/pSHGeyS8wED2+Fz03n7eMLiKPboeYz1PW2fXeX1xf4t+HaJGH6MJ8z+eh99prryzmggsuyPpuueWWNu9Hysdz9Pz9GI+ef50xF+Uk6+R3O5rL5EwSAIACJkkAAAqYJAEAKGCSBACgwKLdrgEAAGeSAAAUMUkCAFDAJAkAQAGTJAAABUySAAAUMEkCAFDw/wP+wtgJEhEXlAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 576x576 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from torchvision.io import ImageReadMode\n",
    "\n",
    "custom = []\n",
    "for i in range(1,10):\n",
    "    path = \"./desenhos/cam_\" + str(i) + \".png\"\n",
    "    img = read_image(path, ImageReadMode.GRAY)\n",
    "    custom.append(img)\n",
    "\n",
    "def predict(img):\n",
    "    return labels_map[torch.argmax(model(img/255)).item()]\n",
    "\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "cols, rows = 3, 3\n",
    "for i in range(1, 10):\n",
    "    img = custom[i-1]\n",
    "    label = predict(img)\n",
    "    figure.add_subplot(rows, cols, i)\n",
    "    plt.title(label)\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img.squeeze(), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Código Completo da Seção"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1\n",
      "-------------------------------\n",
      "loss: 2.312982  [    0/60000]\n",
      "loss: 2.292838  [ 6400/60000]\n",
      "loss: 2.272382  [12800/60000]\n",
      "loss: 2.259651  [19200/60000]\n",
      "loss: 2.248788  [25600/60000]\n",
      "loss: 2.217512  [32000/60000]\n",
      "loss: 2.226016  [38400/60000]\n",
      "loss: 2.192277  [44800/60000]\n",
      "loss: 2.180309  [51200/60000]\n",
      "loss: 2.152212  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 45.1%, Avg loss: 2.146545 \n",
      "\n",
      "Epoch 2\n",
      "-------------------------------\n",
      "loss: 2.156515  [    0/60000]\n",
      "loss: 2.145649  [ 6400/60000]\n",
      "loss: 2.083417  [12800/60000]\n",
      "loss: 2.101507  [19200/60000]\n",
      "loss: 2.057301  [25600/60000]\n",
      "loss: 1.984425  [32000/60000]\n",
      "loss: 2.021983  [38400/60000]\n",
      "loss: 1.932130  [44800/60000]\n",
      "loss: 1.931154  [51200/60000]\n",
      "loss: 1.879296  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 58.7%, Avg loss: 1.867551 \n",
      "\n",
      "Epoch 3\n",
      "-------------------------------\n",
      "loss: 1.890308  [    0/60000]\n",
      "loss: 1.867879  [ 6400/60000]\n",
      "loss: 1.743336  [12800/60000]\n",
      "loss: 1.796476  [19200/60000]\n",
      "loss: 1.694892  [25600/60000]\n",
      "loss: 1.631329  [32000/60000]\n",
      "loss: 1.666520  [38400/60000]\n",
      "loss: 1.554984  [44800/60000]\n",
      "loss: 1.582912  [51200/60000]\n",
      "loss: 1.500697  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 61.8%, Avg loss: 1.505270 \n",
      "\n",
      "Epoch 4\n",
      "-------------------------------\n",
      "loss: 1.557742  [    0/60000]\n",
      "loss: 1.535189  [ 6400/60000]\n",
      "loss: 1.381200  [12800/60000]\n",
      "loss: 1.465485  [19200/60000]\n",
      "loss: 1.355629  [25600/60000]\n",
      "loss: 1.331740  [32000/60000]\n",
      "loss: 1.357726  [38400/60000]\n",
      "loss: 1.273749  [44800/60000]\n",
      "loss: 1.314472  [51200/60000]\n",
      "loss: 1.230960  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 63.3%, Avg loss: 1.247899 \n",
      "\n",
      "Epoch 5\n",
      "-------------------------------\n",
      "loss: 1.311782  [    0/60000]\n",
      "loss: 1.303910  [ 6400/60000]\n",
      "loss: 1.137996  [12800/60000]\n",
      "loss: 1.248314  [19200/60000]\n",
      "loss: 1.132865  [25600/60000]\n",
      "loss: 1.138724  [32000/60000]\n",
      "loss: 1.167215  [38400/60000]\n",
      "loss: 1.098202  [44800/60000]\n",
      "loss: 1.143564  [51200/60000]\n",
      "loss: 1.073119  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 64.4%, Avg loss: 1.085267 \n",
      "\n",
      "Epoch 6\n",
      "-------------------------------\n",
      "loss: 1.145539  [    0/60000]\n",
      "loss: 1.155935  [ 6400/60000]\n",
      "loss: 0.973886  [12800/60000]\n",
      "loss: 1.108593  [19200/60000]\n",
      "loss: 0.990799  [25600/60000]\n",
      "loss: 1.007280  [32000/60000]\n",
      "loss: 1.048179  [38400/60000]\n",
      "loss: 0.984227  [44800/60000]\n",
      "loss: 1.029751  [51200/60000]\n",
      "loss: 0.974027  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 65.6%, Avg loss: 0.978867 \n",
      "\n",
      "Epoch 7\n",
      "-------------------------------\n",
      "loss: 1.028921  [    0/60000]\n",
      "loss: 1.058770  [ 6400/60000]\n",
      "loss: 0.860069  [12800/60000]\n",
      "loss: 1.014515  [19200/60000]\n",
      "loss: 0.900162  [25600/60000]\n",
      "loss: 0.914752  [32000/60000]\n",
      "loss: 0.970257  [38400/60000]\n",
      "loss: 0.909898  [44800/60000]\n",
      "loss: 0.950951  [51200/60000]\n",
      "loss: 0.908737  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 66.9%, Avg loss: 0.906658 \n",
      "\n",
      "Epoch 8\n",
      "-------------------------------\n",
      "loss: 0.943535  [    0/60000]\n",
      "loss: 0.991678  [ 6400/60000]\n",
      "loss: 0.778539  [12800/60000]\n",
      "loss: 0.948423  [19200/60000]\n",
      "loss: 0.839779  [25600/60000]\n",
      "loss: 0.847935  [32000/60000]\n",
      "loss: 0.915847  [38400/60000]\n",
      "loss: 0.860879  [44800/60000]\n",
      "loss: 0.894711  [51200/60000]\n",
      "loss: 0.862630  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 68.0%, Avg loss: 0.855251 \n",
      "\n",
      "Epoch 9\n",
      "-------------------------------\n",
      "loss: 0.878494  [    0/60000]\n",
      "loss: 0.941814  [ 6400/60000]\n",
      "loss: 0.718055  [12800/60000]\n",
      "loss: 0.899845  [19200/60000]\n",
      "loss: 0.796923  [25600/60000]\n",
      "loss: 0.798380  [32000/60000]\n",
      "loss: 0.875064  [38400/60000]\n",
      "loss: 0.827433  [44800/60000]\n",
      "loss: 0.852808  [51200/60000]\n",
      "loss: 0.827825  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 69.2%, Avg loss: 0.816707 \n",
      "\n",
      "Epoch 10\n",
      "-------------------------------\n",
      "loss: 0.826864  [    0/60000]\n",
      "loss: 0.901940  [ 6400/60000]\n",
      "loss: 0.671408  [12800/60000]\n",
      "loss: 0.862941  [19200/60000]\n",
      "loss: 0.764555  [25600/60000]\n",
      "loss: 0.760666  [32000/60000]\n",
      "loss: 0.842443  [38400/60000]\n",
      "loss: 0.803161  [44800/60000]\n",
      "loss: 0.820262  [51200/60000]\n",
      "loss: 0.800041  [57600/60000]\n",
      "Test Error: \n",
      " Accuracy: 70.6%, Avg loss: 0.786286 \n",
      "\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torch.utils.data import DataLoader\n",
    "from torchvision import datasets\n",
    "from torchvision.transforms import ToTensor\n",
    "\n",
    "training_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=True,\n",
    "    download=True,\n",
    "    transform=ToTensor()\n",
    ")\n",
    "\n",
    "test_data = datasets.FashionMNIST(\n",
    "    root=\"data\",\n",
    "    train=False,\n",
    "    download=True,\n",
    "    transform=ToTensor()\n",
    ")\n",
    "\n",
    "class NeuralNetwork(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Linear(28*28, 512),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(512, 512),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(512, 10),\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.flatten(x)\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        return logits\n",
    "\n",
    "def train_loop(dataloader, model, loss_fn, optimizer):\n",
    "    size = len(dataloader.dataset)\n",
    "    for batch, (X, y) in enumerate(dataloader):\n",
    "        # Compute prediction and loss\n",
    "        pred = model(X)\n",
    "        loss = loss_fn(pred, y)\n",
    "\n",
    "        # Backpropagation\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        if batch % 100 == 0:\n",
    "            loss, current = loss.item(), batch * len(X)\n",
    "            print(f\"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]\")\n",
    "\n",
    "\n",
    "def test_loop(dataloader, model, loss_fn):\n",
    "    size = len(dataloader.dataset)\n",
    "    num_batches = len(dataloader)\n",
    "    test_loss, correct = 0, 0\n",
    "\n",
    "    with torch.no_grad():\n",
    "        for X, y in dataloader:\n",
    "            pred = model(X)\n",
    "            test_loss += loss_fn(pred, y).item()\n",
    "            correct += (pred.argmax(1) == y).type(torch.float).sum().item()\n",
    "\n",
    "    test_loss /= num_batches\n",
    "    correct /= size\n",
    "    print(f\"Test Error: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")\n",
    "\n",
    "model = NeuralNetwork()\n",
    "\n",
    "train_dataloader = DataLoader(training_data, batch_size=64)\n",
    "test_dataloader = DataLoader(test_data, batch_size=64)\n",
    "\n",
    "learning_rate = 1e-3\n",
    "batch_size = 64\n",
    "epochs = 10\n",
    "\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n",
    "\n",
    "for t in range(epochs):\n",
    "    print(f\"Epoch {t+1}\\n-------------------------------\")\n",
    "    train_loop(train_dataloader, model, loss_fn, optimizer)\n",
    "    test_loop(test_dataloader, model, loss_fn)\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 📝Recomendação de exercícios\n",
    "- 🔢 Um data set com [números desenhados a mão](https://www.kaggle.com/c/digit-recognizer) (MNIST clássico), para treinar os básicos\n",
    "- 🦘 Um dataset que utiliza csv, [predição de chuvas na australia](https://www.kaggle.com/jsphyg/weather-dataset-rattle-package)\n",
    "- 🐟 Um dataset de [imagens de peixes](https://www.kaggle.com/crowww/a-large-scale-fish-dataset), para treinar criar datasets/dataloaders\n",
    "- 🦠 Um data set de [Tweets sobre o coronavirus ](https://www.kaggle.com/datatattle/covid-19-nlp-text-classification), para treinar classificação de texto "
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "fe7cbed4bc17059f1642dccf61018811625bb91516f85815cc0235fd6d517ea8"
  },
  "kernelspec": {
   "display_name": "Python 3.8.12 64-bit ('py38': conda)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}