Day2_overview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Day 2 overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](Day2/figs/deep-learning-reseaux-neurones.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Overview of various deep learning strategies for image and pixel classification\n",
    "    * Image classification = 'image recognition'\n",
    "    * Pixel classification = 'semantic segmentation'\n",
    "\n",
    "\n",
    "* We'll be developing classifiers to\n",
    "    * Classify landuse in satellite imagery\n",
    "    * Classify pixels in oblique images of shoreline environments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to DL for image classification\n",
    "* \n",
    "\n",
    "> Workbook: Day2/2.1.DL_forimageclass.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating label/ground-truth images\n",
    "* Creating label images using weakly supervised annotation\n",
    "* Overview of the conditional random field\n",
    "\n",
    "> Workbook: Day2/2.2.Labeling_images.ipynb\n",
    "\n",
    "> ```dl_tools:create_groundtruth:label_1image_crf.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# From here, we go 2 different routes!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Route 1\n",
    "\n",
    "* Using Mobilenets model to classify \n",
    "    1. whole images\n",
    "    2. regions (tiles) of images\n",
    "        * refining those tile classifications using a CRF to get pixel classifications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Route 2 \"Fully convolutional\"\n",
    "\n",
    "* Using VGG model to classify \n",
    "    1. pixels of images (directly)\n",
    "        * refining those tile classifications using a CRF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Route 1 (\"Hybrid\")\n",
    "\n",
    "![](Day2/figs/dl_tools.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transfer learning for whole image recognition\n",
    "* Retraining an existing deep convolutional neural network model to classify landuse in satellite imagery\n",
    "* Evaluating classifier performance\n",
    "\n",
    "> Workbook: Day2/2.3.retrain_imrecog_whole.ipynb\n",
    "\n",
    "> ```dl_tools:train_dcnn_tfhub:retrain.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating image tiles\n",
    "* Use label imagery to automatically generate tiles of specified size for DCNN model retraining\n",
    "\n",
    "> Workbook: Day2/2.4.tile_images.ipynb\n",
    "\n",
    "> ```dl_tools:create_library:retile.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transfer learning for image tile recognition\n",
    "* Retraining an existing DCNN using the tiles we generated in the previous exercise\n",
    "* Lake Ontario shoreline environments\n",
    "* California coastal environments\n",
    "\n",
    "> Workbook: Day2/2.5.retrain_imrecog_tiles.ipynb\n",
    "\n",
    "> ```dl_tools:train_dcnn_tfhub:retrain.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating accuracy of retrained DCNN models for image recognition\n",
    "* Applying retrained models to tile classification\n",
    "* Computing accuracy, F1-score, confusion matrix\n",
    "\n",
    "> Workbook: Day2/2.6.eval_tileclass.ipynb\n",
    "\n",
    "> ```dl_tools:eval_imrecog:test_class_tiles.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hybrid DCNN-CRF semantic segmentation\n",
    "* Preparing inputs\n",
    "* Running classification with local files\n",
    "* Running classification with a pretrained model file from Google drive\n",
    "* Exercise\n",
    "\n",
    "> Workbook: Day2/2.7.hybrid_semseg.ipynb\n",
    "\n",
    "> ```dl_tools:semseg_crf:semseg_cnn_crf.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating accuracy of hybrid DCNN-CRF semantic segmentation\n",
    "* Evaluating classification pixel-by-pixel\n",
    "\n",
    "> Workbook: Day2/2.8.eval_semseg.ipynb\n",
    "\n",
    "> ```dl_tools:eval_semseg:test_pixels.py```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Route 2 (\"Fully convolutional\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](Day2/figs/dl_tools_fullyconv.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fully convolutional semantic segmentation\n",
    "* Preparing the training and validation data\n",
    "* Retraining VGG16\n",
    "* Testing the model\n",
    "* Testing the model with CRF post-processing\n",
    "\n",
    "> Workbook: Day2/2.9.fullyconv_semseg.ipynb\n",
    "\n",
    "> ```dl_tools:semseg_fullyconv:train.py```\n",
    "\n",
    "> ```dl_tools:semseg_fullyconv:infer.py```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:conda]",
   "language": "python",
   "name": "conda-env-conda-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}