Image Classifier Project.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Image-Classifier-Project.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "7MDFp5bA3Kth",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Developing an AI application\n",
        "\n",
        "## **NOTE:** \n",
        "\n",
        "This is slightly edited notebook to help you with downloaing and loading image datasets. I have left the exercise parts untouched and you have to fill them, ofcourse. If you have any issue with this notebook, feel free to contact me `avinash` on Slack :D\n",
        "\n",
        "Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications. \n",
        "\n",
        "In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application. We'll be using [this dataset](http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html) of 102 flower categories, you can see a few examples below. \n",
        "\n",
        "<img src='https://github.com/udacity/pytorch_challenge/blob/master/assets/Flowers.png?raw=1' width=500px>\n",
        "\n",
        "The project is broken down into multiple steps:\n",
        "\n",
        "* Load and preprocess the image dataset\n",
        "* Train the image classifier on your dataset\n",
        "* Use the trained classifier to predict image content\n",
        "\n",
        "We'll lead you through each part which you'll implement in Python.\n",
        "\n",
        "When you've completed this project, you'll have an application that can be trained on any set of labeled images. Here your network will be learning about flowers and end up as a command line application. But, what you do with your new skills depends on your imagination and effort in building a dataset. For example, imagine an app where you take a picture of a car, it tells you what the make and model is, then looks up information about it. Go build your own dataset and make something new.\n",
        "\n",
        "First up is importing the packages you'll need. It's good practice to keep all the imports at the beginning of your code. As you work through this notebook and find you need to import a package, make sure to add the import up here."
      ]
    },
    {
      "metadata": {
        "id": "bQ0VpMDB68bR",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Tips, Model Performance, Submission Troubleshooting, etc \n",
        "\n",
        "The amazing people at Slack have compiled an amazing list of tips for this project. Check it here - [link](https://docs.google.com/document/d/1-MCDPOejsn2hq9EoBzMpzGv9jEdtMWoIwjkAa1cVbSM/edit#heading=h.yspy8tt3f0xe)"
      ]
    },
    {
      "metadata": {
        "id": "4IX8JftM5ZLy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# google colab does not come with torch installed. And also, in course we are using torch 0.4. \n",
        "# so following snippet of code installs the relevant version\n",
        "\n",
        "from os.path import exists\n",
        "from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
        "platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
        "cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\\.\\([0-9]*\\)\\.\\([0-9]*\\)$/cu\\1\\2/'\n",
        "accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'\n",
        "\n",
        "!pip install -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gcQnqSIu5aQq",
        "colab_type": "code",
        "outputId": "e70e8bb0-f199-4c10-9df7-9d152584a56a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# we will verify that GPU is enabled for this notebook\n",
        "# following should print: CUDA is available!  Training on GPU ...\n",
        "# \n",
        "# if it prints otherwise, then you need to enable GPU: \n",
        "# from Menu > Runtime > Change Runtime Type > Hardware Accelerator > GPU\n",
        "\n",
        "import torch\n",
        "\n",
        "# check if CUDA is available\n",
        "train_on_gpu = torch.cuda.is_available()\n",
        "\n",
        "if not train_on_gpu:\n",
        "    print('CUDA is not available.  Training on CPU ...')\n",
        "else:\n",
        "    print('CUDA is available!  Training on GPU ...')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "CUDA is available!  Training on GPU ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "a6WqsETVV-_B",
        "colab_type": "code",
        "outputId": "7588f3f3-c96e-41d5-e9ce-bb363b0096bb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        }
      },
      "cell_type": "code",
      "source": [
        "# we need pillow version of 5.3.0\n",
        "# we will uninstall the older version first\n",
        "!pip uninstall -y Pillow\n",
        "# install the new one\n",
        "!pip install Pillow==5.3.0\n",
        "# import the new one\n",
        "import PIL\n",
        "print(PIL.PILLOW_VERSION)\n",
        "# this should print 5.3.0. If it doesn't, then restart your runtime:\n",
        "# Menu > Runtime > Restart Runtime"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Uninstalling Pillow-5.3.0:\n",
            "  Successfully uninstalled Pillow-5.3.0\n",
            "Collecting Pillow==5.3.0\n",
            "  Using cached https://files.pythonhosted.org/packages/62/94/5430ebaa83f91cc7a9f687ff5238e26164a779cca2ef9903232268b0a318/Pillow-5.3.0-cp36-cp36m-manylinux1_x86_64.whl\n",
            "Installing collected packages: Pillow\n",
            "Successfully installed Pillow-5.3.0\n",
            "5.3.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "JzKELNjh3Ktj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# IMPORTS\n",
        "\n",
        "# Torch Features\n",
        "from torch import nn\n",
        "from torch import optim\n",
        "import torch.nn.functional as F\n",
        "from torchvision import datasets, transforms, models\n",
        "from torch.utils.data.sampler import SubsetRandomSampler\n",
        "from torch.optim.lr_scheduler import StepLR, ReduceLROnPlateau\n",
        "\n",
        "# Calc Utilities\n",
        "import numpy as np\n",
        "from collections import OrderedDict\n",
        "\n",
        "# Plot Utilities\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "# Miscellaneous\n",
        "import time\n",
        "import json\n",
        "import warnings\n",
        "warnings.simplefilter(action='ignore', category=FutureWarning)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "nHGZkiGB3Ktn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Load the data\n",
        "\n",
        "Here you'll use `torchvision` to load the data ([documentation](http://pytorch.org/docs/0.3.0/torchvision/index.html)). You can [download the data here](https://s3.amazonaws.com/content.udacity-data.com/courses/nd188/flower_data.zip). The dataset is split into two parts, training and validation. For the training, you'll want to apply transformations such as random scaling, cropping, and flipping. This will help the network generalize leading to better performance. If you use a pre-trained network, you'll also need to make sure the input data is resized to 224x224 pixels as required by the networks.\n",
        "\n",
        "The validation set is used to measure the model's performance on data it hasn't seen yet. For this you don't want any scaling or rotation transformations, but you'll need to resize then crop the images to the appropriate size.\n",
        "\n",
        "The pre-trained networks available from `torchvision` were trained on the ImageNet dataset where each color channel was normalized separately. For both sets you'll need to normalize the means and standard deviations of the images to what the network expects. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`, calculated from the ImageNet images.  These values will shift each color channel to be centered at 0 and range from -1 to 1."
      ]
    },
    {
      "metadata": {
        "id": "vyw-aBuE4Az8",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# we will download the required data files\n",
        "!wget -cq https://github.com/udacity/pytorch_challenge/raw/master/cat_to_name.json\n",
        "!wget -cq https://s3.amazonaws.com/content.udacity-data.com/courses/nd188/flower_data.zip\n",
        "!rm -r flower_data || true\n",
        "!unzip -qq flower_data.zip"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "zJf-hrbx3Kto",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "data_dir = '/content/flower_data'\n",
        "train_dir = data_dir + '/train'\n",
        "test_dir = data_dir + '/valid'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "87IDM9w3EDLj",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# number of subprocesses to use for data loading\n",
        "num_workers = 0\n",
        "# how many samples per batch to load\n",
        "batch_size = 20\n",
        "# percentage of training set to use as validation\n",
        "valid_size = 0.2\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "2KvmdFb-3Kts",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# TODO: Define transforms for the training data and testing data\n",
        "train_transforms = transforms.Compose([transforms.RandomRotation(30),\n",
        "                                       transforms.RandomResizedCrop(224),\n",
        "                                       transforms.RandomHorizontalFlip(),\n",
        "                                       transforms.ToTensor(),\n",
        "                                       transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))])\n",
        "\n",
        "test_transforms = transforms.Compose([transforms.Resize(255),\n",
        "                                       transforms.CenterCrop(224),\n",
        "                                       transforms.ToTensor(),\n",
        "                                       transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))])\n",
        "\n",
        "\n",
        "# Pass transforms in here, then run the next cell to see how the transforms look\n",
        "train_data = datasets.ImageFolder(train_dir, transform=train_transforms)\n",
        "test_data = datasets.ImageFolder(test_dir, transform=test_transforms)\n",
        "\n",
        "# obtain training indices that will be used for validation\n",
        "num_train = len(train_data)\n",
        "indices = list(range(num_train))\n",
        "np.random.shuffle(indices)\n",
        "split = int(np.floor(valid_size * num_train))\n",
        "train_idx, valid_idx = indices[split:], indices[:split]\n",
        "\n",
        "# define samplers for obtaining training and validation batches\n",
        "train_sampler = SubsetRandomSampler(train_idx)\n",
        "valid_sampler = SubsetRandomSampler(valid_idx)\n",
        "\n",
        "# prepare data loaders (combine dataset and sampler)\n",
        "train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,\n",
        "    sampler=train_sampler, num_workers=num_workers)\n",
        "valid_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, \n",
        "    sampler=valid_sampler, num_workers=num_workers)\n",
        "test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, \n",
        "    num_workers=num_workers)\n",
        "\n",
        "# specify the image classes\n",
        "with open('cat_to_name.json', 'r') as f:\n",
        "    classes = json.load(f)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1vGuzGEl3Ktv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Label mapping\n",
        "\n",
        "You'll also need to load in a mapping from category label to category name. You can find this in the file `cat_to_name.json`. It's a JSON object which you can read in with the [`json` module](https://docs.python.org/2/library/json.html). This will give you a dictionary mapping the integer encoded categories to the actual names of the flowers."
      ]
    },
    {
      "metadata": {
        "id": "kfHsUYS93Ktz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Building and training the classifier\n",
        "\n",
        "Now that the data is ready, it's time to build and train the classifier. As usual, you should use one of the pretrained models from `torchvision.models` to get the image features. Build and train a new feed-forward classifier using those features.\n",
        "\n",
        "We're going to leave this part up to you. If you want to talk through it with someone, chat with your fellow students! You can also ask questions on the forums or join the instructors in office hours.\n",
        "\n",
        "Refer to [the rubric](https://review.udacity.com/#!/rubrics/1663/view) for guidance on successfully completing this section. Things you'll need to do:\n",
        "\n",
        "* Load a [pre-trained network](http://pytorch.org/docs/master/torchvision/models.html) (If you need a starting point, the VGG networks work great and are straightforward to use)\n",
        "* Define a new, untrained feed-forward network as a classifier, using ReLU activations and dropout\n",
        "* Train the classifier layers using backpropagation using the pre-trained network to get the features\n",
        "* Track the loss and accuracy on the validation set to determine the best hyperparameters\n",
        "\n",
        "We've left a cell open for you below, but use as many as you need. Our advice is to break the problem up into smaller parts you can run separately. Check that each part is doing what you expect, then move on to the next. You'll likely find that as you work through each part, you'll need to go back and modify your previous code. This is totally normal!\n",
        "\n",
        "When training make sure you're updating only the weights of the feed-forward network. You should be able to get the validation accuracy above 70% if you build everything right. Make sure to try different hyperparameters (learning rate, units in the classifier, epochs, etc) to find the best model. Save those hyperparameters to use as default values in the next part of the project."
      ]
    },
    {
      "metadata": {
        "id": "xW2PnSKK3Kt0",
        "colab_type": "code",
        "outputId": "25c60f9a-4d16-41fa-d852-d68a6a9c3bce",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1476
        }
      },
      "cell_type": "code",
      "source": [
        "model = models.resnet18(pretrained=True)\n",
        "model"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ResNet(\n",
              "  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
              "  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "  (relu): ReLU(inplace)\n",
              "  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
              "  (layer1): Sequential(\n",
              "    (0): BasicBlock(\n",
              "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    )\n",
              "    (1): BasicBlock(\n",
              "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    )\n",
              "  )\n",
              "  (layer2): Sequential(\n",
              "    (0): BasicBlock(\n",
              "      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (downsample): Sequential(\n",
              "        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      )\n",
              "    )\n",
              "    (1): BasicBlock(\n",
              "      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    )\n",
              "  )\n",
              "  (layer3): Sequential(\n",
              "    (0): BasicBlock(\n",
              "      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (downsample): Sequential(\n",
              "        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      )\n",
              "    )\n",
              "    (1): BasicBlock(\n",
              "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    )\n",
              "  )\n",
              "  (layer4): Sequential(\n",
              "    (0): BasicBlock(\n",
              "      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (downsample): Sequential(\n",
              "        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
              "        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      )\n",
              "    )\n",
              "    (1): BasicBlock(\n",
              "      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "      (relu): ReLU(inplace)\n",
              "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
              "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
              "    )\n",
              "  )\n",
              "  (avgpool): AvgPool2d(kernel_size=7, stride=1, padding=0)\n",
              "  (fc): Linear(in_features=512, out_features=1000, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "metadata": {
        "id": "T7u4dFGk3SFW",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Freeze parameters so we don't backprop through them\n",
        "for param in model.parameters():\n",
        "    param.requires_grad = True\n",
        "\n",
        "from collections import OrderedDict\n",
        "classifier = nn.Sequential(OrderedDict([\n",
        "                          ('fc1', nn.Linear(512, 128)),\n",
        "                          ('relu', nn.ReLU()),\n",
        "                          ('fc2', nn.Linear(128, 102)),\n",
        "                          ('output', nn.LogSoftmax(dim=1))\n",
        "                          ]))\n",
        "    \n",
        "model.fc = classifier"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "M3uiGpa1JCF5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def train_model(model, checkpoint_path = False):\n",
        "    \n",
        "    # Start epoch variable\n",
        "    start_epoch = 0\n",
        "    \n",
        "    # Define the Loss\n",
        "    criterion = nn.NLLLoss()\n",
        "\n",
        "    # Only train the classifier parameters, feature parameters are frozen\n",
        "    optimizer = optim.Adam(model.fc.parameters(), lr=0.1)\n",
        "    \n",
        "    # Set StepLR to reduce the learning rate\n",
        "        # Assuming optimizer uses lr = 0.05 for all groups\n",
        "        # lr = 0.05     if epoch < 30\n",
        "        # lr = 0.005    if 30 <= epoch < 60\n",
        "        # lr = 0.0005   if 60 <= epoch < 90\n",
        "    scheduler_step = StepLR(optimizer, step_size=5, gamma=0.1)\n",
        "    \n",
        "    #scheduler_plat = ReduceLROnPlateau(optimizer)\n",
        "    \n",
        "    #Check if there is an old run\n",
        "    if checkpoint_path:\n",
        "      cfile = torch.load(checkpoint_path)\n",
        "      start_epoch = cfile['epochs']\n",
        "      #optimizer.load_state_dict(cfile['optim'])\n",
        "\n",
        "\n",
        "    # number of epochs to train the model\n",
        "    n_epochs = 50 # you may increase this number to train a final model\n",
        "\n",
        "    valid_loss_min = np.Inf # track change in validation loss\n",
        "\n",
        "    for epoch in range(1, n_epochs + 1):\n",
        "        start_time = time.time()\n",
        "      \n",
        "        # Move model to GPU if CUDA is available\n",
        "        if train_on_gpu:\n",
        "            model.cuda()\n",
        "\n",
        "        # keep track of training and validation loss\n",
        "        train_loss = 0.0\n",
        "        valid_loss = 0.0\n",
        "        \n",
        "        # reduce the learning rate\n",
        "        scheduler_step.step()\n",
        "        \n",
        "\n",
        "        ###################\n",
        "        # train the model #\n",
        "        ###################\n",
        "        model.train()\n",
        "        for data, target in train_loader:\n",
        "            # move tensors to GPU if CUDA is available\n",
        "            if train_on_gpu:\n",
        "                data, target = data.cuda(), target.cuda()\n",
        "            # clear the gradients of all optimized variables\n",
        "            optimizer.zero_grad()\n",
        "            # forward pass: compute predicted outputs by passing inputs to the model\n",
        "            output = model(data)\n",
        "            # calculate the batch loss\n",
        "            loss = criterion(output, target)\n",
        "            # backward pass: compute gradient of the loss with respect to model parameters\n",
        "            loss.backward()\n",
        "            # perform a single optimization step (parameter update)\n",
        "            optimizer.step()\n",
        "            # update training loss\n",
        "            train_loss += loss.item()*data.size(0)\n",
        "\n",
        "        ######################    \n",
        "        # validate the model #\n",
        "        ######################\n",
        "        correct = 0\n",
        "        total = 0\n",
        "        model.eval()\n",
        "        for data, target in valid_loader:\n",
        "            # move tensors to GPU if CUDA is available\n",
        "            if train_on_gpu:\n",
        "                data, target = data.cuda(), target.cuda()\n",
        "            # forward pass: compute predicted outputs by passing inputs to the model\n",
        "            output = model(data)\n",
        "            # calculate the batch loss\n",
        "            loss = criterion(output, target)\n",
        "            # update average validation loss \n",
        "            valid_loss += loss.item()*data.size(0)\n",
        "            \n",
        "            #calculate accuracy\n",
        "            pred = torch.max(output, 1)[1] \n",
        "            total += len(target)\n",
        "            correct += (pred == target).sum()\n",
        "\n",
        "        # calculate average losses\n",
        "        train_loss = train_loss/len(train_loader.dataset)\n",
        "        valid_loss = valid_loss/len(valid_loader.dataset)\n",
        "        accuracy = 100 * correct/ float(total)\n",
        "        duration = (time.time() - start_time)/60\n",
        "        \n",
        "        # reduce the learning rate\n",
        "        #scheduler_plat.step(valid_loss)\n",
        "\n",
        "        # print training/validation statistics \n",
        "        print('Epoch: {} \\tTraining Loss: {:.6f} \\tValidation Loss: {:.6f} \\tValidation Acc. {:.2f} \\tDuration: {:.2f} min.'.format(\n",
        "            start_epoch + epoch, train_loss, valid_loss,accuracy,duration))\n",
        "\n",
        "        # save model if validation loss has decreased\n",
        "        if valid_loss <= valid_loss_min:\n",
        "            print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(\n",
        "                  valid_loss_min,\n",
        "                  valid_loss))\n",
        "            model.class_to_idx = train_data.class_to_idx\n",
        "            model.cpu()\n",
        "            torch.save({'arch': 'Resnet18',\n",
        "                        'epochs': epoch,\n",
        "                        'state_dict': model.state_dict(), \n",
        "                        'class_to_idx': model.class_to_idx,\n",
        "                        'optim': optimizer.state_dict()},'classifier-resnet18-trained.pth')\n",
        "\n",
        "\n",
        "            valid_loss_min = valid_loss\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IGTiSJZ6HMIM",
        "colab_type": "code",
        "outputId": "d9f3855b-6434-4082-920c-05c9105bfbc7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1279
        }
      },
      "cell_type": "code",
      "source": [
        "train_model(model)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 1 \tTraining Loss: 4.829283 \tValidation Loss: 0.914306 \tValidation Acc. 2.00 \tDuration: 2.88 min.\n",
            "Validation loss decreased (inf --> 0.914306).  Saving model ...\n",
            "Epoch: 2 \tTraining Loss: 3.649984 \tValidation Loss: 0.914609 \tValidation Acc. 2.00 \tDuration: 2.84 min.\n",
            "Epoch: 3 \tTraining Loss: 3.644846 \tValidation Loss: 0.912097 \tValidation Acc. 3.00 \tDuration: 2.85 min.\n",
            "Validation loss decreased (0.914306 --> 0.912097).  Saving model ...\n",
            "Epoch: 4 \tTraining Loss: 3.646843 \tValidation Loss: 0.910963 \tValidation Acc. 3.00 \tDuration: 2.85 min.\n",
            "Validation loss decreased (0.912097 --> 0.910963).  Saving model ...\n",
            "Epoch: 5 \tTraining Loss: 3.649821 \tValidation Loss: 0.914711 \tValidation Acc. 3.00 \tDuration: 2.85 min.\n",
            "Epoch: 6 \tTraining Loss: 3.619767 \tValidation Loss: 0.905946 \tValidation Acc. 3.00 \tDuration: 2.87 min.\n",
            "Validation loss decreased (0.910963 --> 0.905946).  Saving model ...\n",
            "Epoch: 7 \tTraining Loss: 3.606509 \tValidation Loss: 0.904795 \tValidation Acc. 3.00 \tDuration: 2.86 min.\n",
            "Validation loss decreased (0.905946 --> 0.904795).  Saving model ...\n",
            "Epoch: 8 \tTraining Loss: 3.604630 \tValidation Loss: 0.904541 \tValidation Acc. 3.00 \tDuration: 2.85 min.\n",
            "Validation loss decreased (0.904795 --> 0.904541).  Saving model ...\n",
            "Epoch: 9 \tTraining Loss: 3.604334 \tValidation Loss: 0.904723 \tValidation Acc. 3.00 \tDuration: 2.85 min.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-13-4c7aaa9735fa>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtrain_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m<ipython-input-12-5df222267675>\u001b[0m in \u001b[0;36mtrain_model\u001b[0;34m(model, checkpoint_path)\u001b[0m\n\u001b[1;32m     73\u001b[0m         \u001b[0mtotal\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     74\u001b[0m         \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meval\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 75\u001b[0;31m         \u001b[0;32mfor\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvalid_loader\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     76\u001b[0m             \u001b[0;31m# move tensors to GPU if CUDA is available\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     77\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mtrain_on_gpu\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m__next__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    312\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_workers\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# same-process loading\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    313\u001b[0m             \u001b[0mindices\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msample_iter\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# may raise StopIteration\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 314\u001b[0;31m             \u001b[0mbatch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcollate_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdataset\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mindices\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    315\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpin_memory\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    316\u001b[0m                 \u001b[0mbatch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpin_memory_batch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    312\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_workers\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# same-process loading\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    313\u001b[0m             \u001b[0mindices\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msample_iter\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# may raise StopIteration\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 314\u001b[0;31m             \u001b[0mbatch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcollate_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdataset\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mindices\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    315\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpin_memory\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    316\u001b[0m                 \u001b[0mbatch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpin_memory_batch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torchvision/datasets/folder.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, index)\u001b[0m\n\u001b[1;32m    101\u001b[0m         \u001b[0msample\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    102\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransform\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 103\u001b[0;31m             \u001b[0msample\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msample\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    104\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtarget_transform\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    105\u001b[0m             \u001b[0mtarget\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtarget_transform\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, img)\u001b[0m\n\u001b[1;32m     47\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mimg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     48\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransforms\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 49\u001b[0;31m             \u001b[0mimg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     50\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mimg\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     51\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, pic)\u001b[0m\n\u001b[1;32m     74\u001b[0m             \u001b[0mTensor\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mConverted\u001b[0m \u001b[0mimage\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     75\u001b[0m         \"\"\"\n\u001b[0;32m---> 76\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mF\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpic\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     77\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     78\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__repr__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/torchvision/transforms/functional.py\u001b[0m in \u001b[0;36mto_tensor\u001b[0;34m(pic)\u001b[0m\n\u001b[1;32m     81\u001b[0m     \u001b[0mimg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mimg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtranspose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtranspose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcontiguous\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     82\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mByteTensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 83\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mimg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdiv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m255\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     84\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     85\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mimg\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "metadata": {
        "id": "WyGjw9mD4Gpo",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def calc_accuracy_one(model, data, cuda=False):\n",
        "    model.eval()\n",
        "    if train_on_gpu:\n",
        "        model = model.cuda()    \n",
        "    \n",
        "    with torch.no_grad():\n",
        "        for idx, (inputs, labels) in enumerate(test_loader):\n",
        "\n",
        "            if cuda:\n",
        "                inputs, labels = inputs.cuda(), labels.cuda()\n",
        "\n",
        "            # obtain the outputs from the model\n",
        "            outputs = model.forward(inputs)\n",
        "\n",
        "            # max provides the (maximum probability, max value)\n",
        "            _, predicted = outputs.max(dim=1)\n",
        "\n",
        "            equals = predicted == labels.data\n",
        "            \n",
        "            if equals.sum() < 20:\n",
        "              print(f'Misclassified class:'.format(predict))\n",
        "              print(predicted) #the predicted class\n",
        "              print(labels) # the predicted probability\n",
        "              print(equals)\n",
        "\n",
        "#            print(equals.float().mean())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "g1JvuXcG_jJC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def calc_accuracy(model, loader, train_on_gpu=False):\n",
        "  model.eval()\n",
        "  \n",
        "  if train_on_gpu:\n",
        "      model = model.cuda()  \n",
        "  \n",
        "  correct = 0\n",
        "  total = 0\n",
        "\n",
        "  for data, target in loader:\n",
        "          # move tensors to GPU if CUDA is available\n",
        "          if train_on_gpu:\n",
        "              data, target = data.cuda(), target.cuda()\n",
        "          # forward pass: compute predicted outputs by passing inputs to the model\n",
        "          output = model(data)\n",
        "          pred = torch.max(output, 1)[1] \n",
        "          total += len(target)\n",
        "          correct += (pred == target).sum()\n",
        "          accuracy = 100 * correct/ float(total)    \n",
        "          # print training/validation statistics \n",
        "          print('Accuracy: {:.2f}%'.format(accuracy),'\\r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "jwKG_7mvCbaJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print(calc_accuracy_one(model, 'test', True))\n",
        "#print(calc_accuracy(model, test_loader, train_on_gpu))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3oE_YhrS3Kt3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Save the checkpoint\n",
        "\n",
        "Now that your network is trained, save the model so you can load it later for making predictions. You probably want to save other things such as the mapping of classes to indices which you get from one of the image datasets: `image_datasets['train'].class_to_idx`. You can attach this to the model as an attribute which makes inference easier later on.\n",
        "\n",
        "```model.class_to_idx = image_datasets['train'].class_to_idx```\n",
        "\n",
        "Remember that you'll want to completely rebuild the model later so you can use it for inference. Make sure to include any information you need in the checkpoint. If you want to load the model and keep training, you'll want to save the number of epochs as well as the optimizer state, `optimizer.state_dict`. You'll likely want to use this trained model in the next part of the project, so best to save it now."
      ]
    },
    {
      "metadata": {
        "id": "J4vw7s603Kt3",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#Since it was saved every epoch in which the validation loss decreases, this step only save it to drive.\n",
        "\n",
        "# It has two functions: Load and train the classifier and give the file name for saving.\n",
        "path_to_classifier = 'classifier-resnet152.pth'\n",
        "\n",
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')\n",
        "\n",
        "path = F\"/content/gdrive/My Drive/Colab Notebooks/PyTorch Udacity/{path_to_classifier}\" "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "2Qw2_jEE198S",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "torch.save({'arch': 'Resnet152',\n",
        "            'state_dict': model.state_dict(), \n",
        "            'class_to_idx': model.class_to_idx},path)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kTzSWddW3Kt6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Loading the checkpoint\n",
        "\n",
        "At this point it's good to write a function that can load a checkpoint and rebuild the model. That way you can come back to this project and keep working on it without having to retrain the network."
      ]
    },
    {
      "metadata": {
        "id": "HCZN5U523Kt8",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# TODO: Write a function that loads a checkpoint and rebuilds the model\n",
        "def load_model(checkpoint_path):\n",
        "    cfile = torch.load(checkpoint_path)\n",
        "    \n",
        "    model = models.resnet152(pretrained=True)\n",
        "    for param in model.parameters():\n",
        "        param.requires_grad = False\n",
        "\n",
        "    model.class_to_idx = cfile['class_to_idx']  \n",
        "\n",
        "    classifier = nn.Sequential(OrderedDict([\n",
        "                              ('fc1', nn.Linear(2048, 128)),\n",
        "                              ('relu', nn.ReLU()),\n",
        "                              ('fc2', nn.Linear(128, 102)),\n",
        "                              ('output', nn.LogSoftmax(dim=1))\n",
        "                              ]))\n",
        "\n",
        "    model.fc = classifier\n",
        "    \n",
        "    model.load_state_dict(cfile['state_dict'])\n",
        "       \n",
        "    return model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "lw2MJKs7BmiF",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = load_model(path)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "H4WOBWshdZb3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Keep training if needed\n",
        "\n",
        "If the number of epochs was not enough, you can continue to training after loading the model. Just uncomment the line below.\n"
      ]
    },
    {
      "metadata": {
        "id": "10Lz8MNaB9g9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "train_model(model,path_to_classifier)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "k1QqyrwG3Kt_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "# Inference for classification\n",
        "\n",
        "Now you'll write a function to use a trained network for inference. That is, you'll pass an image into the network and predict the class of the flower in the image. Write a function called `predict` that takes an image and a model, then returns the top $K$ most likely classes along with the probabilities. It should look like \n",
        "\n",
        "```python\n",
        "probs, classes = predict(image_path, model)\n",
        "print(probs)\n",
        "print(classes)\n",
        "> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]\n",
        "> ['70', '3', '45', '62', '55']\n",
        "```\n",
        "\n",
        "First you'll need to handle processing the input image such that it can be used in your network. \n",
        "\n",
        "## Image Preprocessing\n",
        "\n",
        "You'll want to use `PIL` to load the image ([documentation](https://pillow.readthedocs.io/en/latest/reference/Image.html)). It's best to write a function that preprocesses the image so it can be used as input for the model. This function should process the images in the same manner used for training. \n",
        "\n",
        "First, resize the images where the shortest side is 256 pixels, keeping the aspect ratio. This can be done with the [`thumbnail`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) or [`resize`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) methods. Then you'll need to crop out the center 224x224 portion of the image.\n",
        "\n",
        "Color channels of images are typically encoded as integers 0-255, but the model expected floats 0-1. You'll need to convert the values. It's easiest with a Numpy array, which you can get from a PIL image like so `np_image = np.array(pil_image)`.\n",
        "\n",
        "As before, the network expects the images to be normalized in a specific way. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`. You'll want to subtract the means from each color channel, then divide by the standard deviation. \n",
        "\n",
        "And finally, PyTorch expects the color channel to be the first dimension but it's the third dimension in the PIL image and Numpy array. You can reorder dimensions using [`ndarray.transpose`](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.ndarray.transpose.html). The color channel needs to be first and retain the order of the other two dimensions."
      ]
    },
    {
      "metadata": {
        "id": "8FuziAdu3KuA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def process_image(image_path):\n",
        "    ''' \n",
        "    Scales, crops, and normalizes a PIL image for a PyTorch       \n",
        "    model, returns an Numpy array\n",
        "    '''\n",
        "    # Open the image\n",
        "    from PIL import Image\n",
        "    img = Image.open(image_path)\n",
        "\n",
        "    # Resize\n",
        "    if img.size[0] > img.size[1]:\n",
        "        img.thumbnail((10000, 256))\n",
        "    else:\n",
        "        img.thumbnail((256, 10000))\n",
        "\n",
        "    # Crop \n",
        "    left_margin = (img.width-224)/2\n",
        "    bottom_margin = (img.height-224)/2\n",
        "    right_margin = left_margin + 224\n",
        "    top_margin = bottom_margin + 224\n",
        "\n",
        "    img = img.crop((left_margin, bottom_margin, right_margin,   \n",
        "                      top_margin))\n",
        "    # Normalize\n",
        "    img = np.array(img)/255\n",
        "    mean = np.array([0.485, 0.456, 0.406]) #provided mean\n",
        "    std = np.array([0.229, 0.224, 0.225]) #provided std\n",
        "    img = (img - mean)/std\n",
        "    \n",
        "    # Move color channels to first dimension as expected by PyTorch\n",
        "    img = img.transpose((2, 0, 1))\n",
        "    \n",
        "    return img"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Vh-4XSJZ3KuD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "To check your work, the function below converts a PyTorch tensor and displays it in the notebook. If your `process_image` function works, running the output through this function should return the original image (except for the cropped out portions)."
      ]
    },
    {
      "metadata": {
        "id": "Amoybpeo3KuD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def imshow(image, ax=None, title=None):\n",
        "    \"\"\"Imshow for Tensor.\"\"\"\n",
        "    if ax is None:\n",
        "        fig, ax = plt.subplots()\n",
        "    \n",
        "    # PyTorch tensors assume the color channel is the first dimension\n",
        "    # but matplotlib assumes is the third dimension\n",
        "    image = image.transpose((1, 2, 0))\n",
        "    \n",
        "    # Undo preprocessing\n",
        "    mean = np.array([0.485, 0.456, 0.406])\n",
        "    std = np.array([0.229, 0.224, 0.225])\n",
        "    image = std * image + mean\n",
        "    \n",
        "    # Image needs to be clipped between 0 and 1 or it looks like noise when displayed\n",
        "    image = np.clip(image, 0, 1)\n",
        "    \n",
        "    ax.imshow(image)\n",
        "    \n",
        "    return ax"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "-T17dp7sWn8Q",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_path = data_dir + '/valid/22/image_05352.jpg'\n",
        "img = process_image(image_path)\n",
        "imshow(img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "PNUU_TQu3KuG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Class Prediction\n",
        "\n",
        "Once you can get images in the correct format, it's time to write a function for making predictions with your model. A common practice is to predict the top 5 or so (usually called top-$K$) most probable classes. You'll want to calculate the class probabilities then find the $K$ largest values.\n",
        "\n",
        "To get the top $K$ largest values in a tensor use [`x.topk(k)`](http://pytorch.org/docs/master/torch.html#torch.topk). This method returns both the highest `k` probabilities and the indices of those probabilities corresponding to the classes. You need to convert from these indices to the actual class labels using `class_to_idx` which hopefully you added to the model or from an `ImageFolder` you used to load the data ([see here](#Save-the-checkpoint)). Make sure to invert the dictionary so you get a mapping from index to class as well.\n",
        "\n",
        "Again, this method should take a path to an image and a model checkpoint, then return the probabilities and classes.\n",
        "\n",
        "```python\n",
        "probs, classes = predict(image_path, model)\n",
        "print(probs)\n",
        "print(classes)\n",
        "> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]\n",
        "> ['70', '3', '45', '62', '55']\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "ZlgLO1653KuH",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def predict(image_path, model, top_num=5):\n",
        "  \n",
        "    # move tensors to GPU if CUDA is available\n",
        "    if train_on_gpu:\n",
        "        model = model.cuda()\n",
        "    \n",
        "    model.eval()\n",
        "  \n",
        "    # Process image\n",
        "    img = process_image(image_path)\n",
        "    \n",
        "    # Numpy -> Tensor\n",
        "    image_tensor = torch.from_numpy(img).type(torch.cuda.FloatTensor)\n",
        "\n",
        "    # Add batch of size 1 to image\n",
        "    model_input = image_tensor.unsqueeze(0)\n",
        "    \n",
        "    # Probs\n",
        "    probs = torch.exp(model.forward(model_input))\n",
        "    \n",
        "    # Top probs\n",
        "    top_probs, top_labs = probs.topk(top_num)\n",
        "    top_probs = top_probs.cpu().detach().numpy().tolist()[0] \n",
        "    top_labs = top_labs.cpu().detach().numpy().tolist()[0]\n",
        "    \n",
        "    # Convert indices to classes\n",
        "    idx_to_class = {val: key for key, val in    \n",
        "                                      model.class_to_idx.items()}\n",
        "\n",
        "    top_labels = [idx_to_class[lab] for lab in top_labs]\n",
        "    top_flowers = [classes[idx_to_class[lab]] for lab in top_labs]\n",
        "\n",
        "    return top_probs, top_labels, top_flowers"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "b5qmbXst3KuJ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Sanity Checking\n",
        "\n",
        "Now that you can use a trained model for predictions, check to make sure it makes sense. Even if the validation accuracy is high, it's always good to check that there aren't obvious bugs. Use `matplotlib` to plot the probabilities for the top 5 classes as a bar graph, along with the input image. It should look like this:\n",
        "\n",
        "<img src='https://github.com/udacity/pytorch_challenge/blob/master/assets/inference_example.png?raw=1' width=300px>\n",
        "\n",
        "You can convert from the class integer encoding to actual flower names with the `cat_to_name.json` file (should have been loaded earlier in the notebook). To show a PyTorch tensor as an image, use the `imshow` function defined above."
      ]
    },
    {
      "metadata": {
        "id": "QmjvEOpx3KuK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def plot_solution(image_path, model):\n",
        "    # Set up plot\n",
        "    plt.figure(figsize = (6,10))\n",
        "    ax = plt.subplot(2,1,1)\n",
        "\n",
        "    # Set up title\n",
        "    flower_num = image_path.split('/')[4]\n",
        "    title_ = classes[flower_num]\n",
        "\n",
        "    # Plot flower\n",
        "    img = process_image(image_path)\n",
        "    imshow(img, ax, title = title_);\n",
        "\n",
        "    # Make prediction\n",
        "    probs, labs, flowers = predict(image_path, model) \n",
        "\n",
        "    # Plot bar chart\n",
        "    plt.subplot(2,1,2)\n",
        "    sns.barplot(x=probs, y=flowers, color=sns.color_palette()[0]);\n",
        "    plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "a7QhJgVDvOaG",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_path = data_dir + '/valid/10/image_07107.jpg'\n",
        "plot_solution(image_path, model)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "n-wESbTb6xz-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}