Deep_Learning_Classification_of_HDPE_and_PET_Spectra.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.7"
    },
    "colab": {
      "name": "Deep Learning Classification of HDPE and PET Spectra.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/wincowgerDEV/OpenSpecyAI/blob/main/Deep_Learning_Classification_of_HDPE_and_PET_Spectra.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p19PnVUvqRyQ"
      },
      "source": [
        "# Deep Learning Classification of HDPE and PET Spectra\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wtW4kRAJqRyR"
      },
      "source": [
        "### Aim:\n",
        "    This project aims to develop a Convolution Network to perform identification of HDPE and PET FTIR spectra.\n",
        "    \n",
        "    Tensorflow and Keras APIs were used for the development of a 1D Sequential CNN of 7 Layers.\n",
        "    \n",
        "    The code was adapted for spectral analysis using a model developed for accelerometer data:https://github.com/bharatm11/1D_CNN_Human_activity_recognition/blob/master/Deep_Learning_for_Human_Activity_Recognition.ipynb\n",
        "\n",
        "    The dataset comes from Chabuka et al. 2020 https://doi.org/10.1177%2F0003702820923993 and was augmented with additional noise and baseline shift. The test data can be downloaded from GitHub.\n",
        "    \n",
        "     "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8a7a3KNWw96T"
      },
      "source": [
        "Getting Started\n",
        "\n",
        "Download the training data from: https://osf.io/bes7h/\n",
        "\n",
        "Put the data into a known location on you Google drive. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VRd7USpIqRyS"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from scipy import stats\n",
        "from sklearn import metrics\n",
        "import tensorflow as tf\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout, Conv1D, MaxPooling1D,GlobalAveragePooling1D,GlobalAvgPool1D, Reshape, Activation\n",
        "from keras import optimizers\n",
        "import keras\n",
        "from keras.utils import np_utils\n",
        "#%matplotlib inline\n",
        "plt.style.use('ggplot')\n",
        "from keras.optimizers import SGD\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndo5cR4kqRyV"
      },
      "source": [
        "### Pipeline\n",
        "\n",
        "The training process starts by reading the data and normalizing it. This normalized data is then segmented into slices of window size 1300 which translates to individual spectra. These chunks are then randomly split into training and test sets. For the results shown in this report, 70% data as taken into the test set and the remaining was used in the test set for validation of the training algorithm. This training data was fed to a 1D CNN network which is described below.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MpfdC4y9qRyW",
        "outputId": "2a311aaf-d9f8-427b-b1c3-298e8e923860",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "'''***********************'''\n",
        "#DEFINE NETWORK PARAMETERS\n",
        "trainSplitRatio = 0.7 # split ratio for test and validation\n",
        "window_size = 1300 #Length of slice.\n",
        "numFilters1 = 100 # number of filters in first Conv1D layer\n",
        "kernalSize = 10 # kernal size of the Conv2D layer\n",
        "batchSize = 10\n",
        "numNueronsFCL2 = 160 # number of filters in fully connected output layer \n",
        "dropout = 0.5 #dropout rate. % of neurons converted to 0 weight before softmax\n",
        "epochs = 50\n",
        "'''***********************'''"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'***********************'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W1rM_4Dsqa0H",
        "outputId": "7ac03648-02ae-4607-d43a-08532f3e04ae",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p7pjcU_BqRyY",
        "outputId": "12a93a7c-932b-4ee8-8b28-41f429936825",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "#DEFINE HELPER FUNCTIONS\n",
        "def read_data(file_path):\n",
        "    print(\"reading data\")\n",
        "    column_names = ['Wavelength','Absorbance','Polymer', 'SpectrumID']\n",
        "    data = pd.read_csv(file_path,header = None, names = column_names)\n",
        "    print(\"finished reading data\")\n",
        "    return data\n",
        "\n",
        "def feature_normalize(dataset):\n",
        "    mu = np.mean(dataset,axis = 0)\n",
        "    sigma = np.std(dataset,axis = 0)\n",
        "    return (dataset - mu)/sigma\n",
        "\n",
        "\n",
        "def windows(data, size):\n",
        "    start = 0\n",
        "    while start < data.count():\n",
        "        yield int(start), int(start + size)\n",
        "        start += (size / 2)\n",
        "\n",
        "def segment_signal(data,window_size):\n",
        "    segments = np.empty((0,window_size,2))\n",
        "    labels = np.empty((0))\n",
        "    for (start, end) in windows(data[\"Wavelength\"], window_size):\n",
        "        #x = data[\"x-axis\"][start:end]\n",
        "        #y = data[\"y-axis\"][start:end]\n",
        "        #z = data[\"z-axis\"][start:end]\n",
        "        x = data[\"Wavelength\"][start:end]\n",
        "        y = data[\"Absorbance\"][start:end]\n",
        "        if(len(dataset[\"Wavelength\"][start:end]) == window_size):\n",
        "            segments = np.vstack([segments,np.dstack([x,y])])#,z])])\n",
        "            labels = np.append(labels,stats.mode(data[\"Polymer\"][start:end])[0][0]) #activity\n",
        "    return segments, labels\n",
        "#READ AND NORMALIZE DATA\n",
        "dataset = read_data('/content/gdrive/My Drive/GrayLab/Projects/Plastics/ActiveProjects/OpenSpecy/Data/Processed Data/DataAugmentation/ChabukaAugmentedDataProcessed.txt') #You will need to change this path to the location that you place the file in your own google drive that you link to.\n",
        "dataset.dropna(axis=0, how='any', inplace= True)\n",
        "print(\"normalizing x\")\n",
        "dataset['Wavelength'] = feature_normalize(dataset['Wavelength'])\n",
        "print(\"normalizing y\")\n",
        "dataset['Absorbance'] = feature_normalize(dataset['Absorbance'])\n",
        "#print(\"normalizing z\")\n",
        "#dataset['z-axis'] = feature_normalize(dataset['z-axis'])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "reading data\n",
            "finished reading data\n",
            "normalizing x\n",
            "normalizing y\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Oil6ELkpsARq",
        "outputId": "c028d81b-52e9-49d8-8057-c23b41fbd6be",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 402
        }
      },
      "source": [
        "dataset"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Wavelength</th>\n",
              "      <th>Absorbance</th>\n",
              "      <th>Polymer</th>\n",
              "      <th>SpectrumID</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-1.730573</td>\n",
              "      <td>2.677583</td>\n",
              "      <td>HDPE</td>\n",
              "      <td>HDPESample 1Beauty C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-1.727908</td>\n",
              "      <td>2.670636</td>\n",
              "      <td>HDPE</td>\n",
              "      <td>HDPESample 1Beauty C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-1.725244</td>\n",
              "      <td>2.673689</td>\n",
              "      <td>HDPE</td>\n",
              "      <td>HDPESample 1Beauty C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-1.722579</td>\n",
              "      <td>2.674474</td>\n",
              "      <td>HDPE</td>\n",
              "      <td>HDPESample 1Beauty C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-1.719914</td>\n",
              "      <td>2.672644</td>\n",
              "      <td>HDPE</td>\n",
              "      <td>HDPESample 1Beauty C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3889595</th>\n",
              "      <td>1.710435</td>\n",
              "      <td>0.403935</td>\n",
              "      <td>PET</td>\n",
              "      <td>PETSample 135Beauty C10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3889596</th>\n",
              "      <td>1.713099</td>\n",
              "      <td>0.339660</td>\n",
              "      <td>PET</td>\n",
              "      <td>PETSample 135Beauty C10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3889597</th>\n",
              "      <td>1.715764</td>\n",
              "      <td>0.201673</td>\n",
              "      <td>PET</td>\n",
              "      <td>PETSample 135Beauty C10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3889598</th>\n",
              "      <td>1.718428</td>\n",
              "      <td>0.559599</td>\n",
              "      <td>PET</td>\n",
              "      <td>PETSample 135Beauty C10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3889599</th>\n",
              "      <td>1.721094</td>\n",
              "      <td>0.115099</td>\n",
              "      <td>PET</td>\n",
              "      <td>PETSample 135Beauty C10</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3889600 rows × 4 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "         Wavelength  Absorbance Polymer               SpectrumID\n",
              "0         -1.730573    2.677583    HDPE     HDPESample 1Beauty C\n",
              "1         -1.727908    2.670636    HDPE     HDPESample 1Beauty C\n",
              "2         -1.725244    2.673689    HDPE     HDPESample 1Beauty C\n",
              "3         -1.722579    2.674474    HDPE     HDPESample 1Beauty C\n",
              "4         -1.719914    2.672644    HDPE     HDPESample 1Beauty C\n",
              "...             ...         ...     ...                      ...\n",
              "3889595    1.710435    0.403935     PET  PETSample 135Beauty C10\n",
              "3889596    1.713099    0.339660     PET  PETSample 135Beauty C10\n",
              "3889597    1.715764    0.201673     PET  PETSample 135Beauty C10\n",
              "3889598    1.718428    0.559599     PET  PETSample 135Beauty C10\n",
              "3889599    1.721094    0.115099     PET  PETSample 135Beauty C10\n",
              "\n",
              "[3889600 rows x 4 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bBKNHZG2qRyb"
      },
      "source": [
        "### This section plots one window size long plots for each class of the normalized data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "zwJt6dlEqRyd",
        "outputId": "659de7c7-ad1d-41fd-bcba-205e8ac756cb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "#SEGMENT DATA, LABELS INTO WINDOW_SIZE\n",
        "print(\"segmenting data into windows\")\n",
        "segments, labels = segment_signal(dataset,window_size)\n",
        "labels = np.asarray(pd.get_dummies(labels), dtype = np.int8)\n",
        "reshaped_segments = segments.reshape(len(segments), 1,window_size, 2)\n",
        "print(\"segmented data in windows\")\n",
        "#SPLIT DATA INTO TEST AND TRAINING SETS\n",
        "print(\"Splitting data into test and training sets\")\n",
        "train_test_split = np.random.rand(len(reshaped_segments)) < trainSplitRatio\n",
        "train_x = reshaped_segments[train_test_split]\n",
        "train_y = labels[train_test_split]\n",
        "test_x = reshaped_segments[~train_test_split]\n",
        "test_y = labels[~train_test_split]\n",
        "print(\"Ready for training\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "segmenting data into windows\n",
            "segmented data in windows\n",
            "Splitting data into test and training sets\n",
            "Ready for training\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "drUJYdIds9Tv",
        "outputId": "d1e1dc1c-9613-4b9f-ce21-72ca75d0e007",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        }
      },
      "source": [
        "segments[100]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.73057311,  3.1828699 ],\n",
              "       [-1.72790835,  3.17505994],\n",
              "       [-1.72524387,  3.17379161],\n",
              "       ...,\n",
              "       [ 1.72543644,  2.67631326],\n",
              "       [ 1.72810037,  2.67746493],\n",
              "       [ 1.73076568,  2.67732151]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6tK39TmDuaWN",
        "outputId": "d993e336-1328-49fe-8890-148439c00f65",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "labels[100]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 0], dtype=int8)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N_WNjTqvutzF",
        "outputId": "8d430ce5-73b0-4f09-8015-b11f5300cb08",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 941
        }
      },
      "source": [
        "train_x"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[[[-1.73057311e+00,  2.81590240e+00],\n",
              "         [-1.72790835e+00,  2.81215857e+00],\n",
              "         [-1.72524387e+00,  2.80310570e+00],\n",
              "         ...,\n",
              "         [ 1.72543644e+00,  2.53924801e+00],\n",
              "         [ 1.72810037e+00,  2.53982392e+00],\n",
              "         [ 1.73076568e+00,  2.53998005e+00]]],\n",
              "\n",
              "\n",
              "       [[[ 1.42797108e-03,  2.38835894e+00],\n",
              "         [ 4.09328196e-03,  2.38933465e+00],\n",
              "         [ 6.75721113e-03,  2.38772291e+00],\n",
              "         ...,\n",
              "         [-6.56519815e-03,  2.46968322e+00],\n",
              "         [-3.90126898e-03,  2.46792369e+00],\n",
              "         [-1.23595810e-03,  2.46758476e+00]]],\n",
              "\n",
              "\n",
              "       [[[-1.73057311e+00,  2.62004893e+00],\n",
              "         [-1.72790835e+00,  2.61570875e+00],\n",
              "         [-1.72524387e+00,  2.61926459e+00],\n",
              "         ...,\n",
              "         [ 1.72543644e+00,  2.46896444e+00],\n",
              "         [ 1.72810037e+00,  2.47012871e+00],\n",
              "         [ 1.73076568e+00,  2.47145679e+00]]],\n",
              "\n",
              "\n",
              "       ...,\n",
              "\n",
              "\n",
              "       [[[-1.73278384e+00, -1.48752967e-02],\n",
              "         [-1.73011908e+00,  1.09328066e-01],\n",
              "         [-1.72745460e+00, -2.04917581e-02],\n",
              "         ...,\n",
              "         [ 1.72322571e+00,  5.37315895e-02],\n",
              "         [ 1.72588964e+00, -1.64951892e-01],\n",
              "         [ 1.72855495e+00, -5.46679118e-02]]],\n",
              "\n",
              "\n",
              "       [[[ 2.11882408e-03, -3.42251829e-01],\n",
              "         [ 4.78413496e-03, -3.39300656e-01],\n",
              "         [ 7.44806413e-03, -6.24855675e-01],\n",
              "         ...,\n",
              "         [-1.62371402e-02, -4.65263966e-01],\n",
              "         [-1.35732110e-02, -3.33352584e-01],\n",
              "         [-1.09079001e-02, -4.06408626e-01]]],\n",
              "\n",
              "\n",
              "       [[[-1.74024505e+00, -4.03896047e-01],\n",
              "         [-1.73758030e+00, -2.72537806e-01],\n",
              "         [-1.73491581e+00, -3.37262985e-01],\n",
              "         ...,\n",
              "         [ 1.71576450e+00,  2.01672524e-01],\n",
              "         [ 1.71842842e+00,  5.59598995e-01],\n",
              "         [ 1.72109374e+00,  1.15098806e-01]]]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BjsakI4gu16a",
        "outputId": "a7d3b761-12a8-4149-adbe-e585edfe3892",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        }
      },
      "source": [
        "train_y"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0, 1, 0, 0, 0, 0],\n",
              "       [0, 1, 0, 0, 0, 0],\n",
              "       [0, 1, 0, 0, 0, 0],\n",
              "       ...,\n",
              "       [0, 0, 1, 0, 0, 0],\n",
              "       [0, 0, 1, 0, 0, 0],\n",
              "       [0, 0, 1, 0, 0, 0]], dtype=int8)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "POfjECSEqRyg",
        "outputId": "36764fae-d5fe-4c5f-8a73-58ee6f8a8a1a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "#EXTRACT DATASET PARAMETERS\n",
        "numOfRows = segments.shape[1]\n",
        "print(numOfRows)\n",
        "numOfColumns = segments.shape[2]\n",
        "print(numOfColumns)\n",
        "print(train_x.shape[2])\n",
        "print(train_y.shape[1])\n",
        "num_classes = labels.shape[1]\n",
        "num_data_parameters = train_x.shape[3]\n",
        "input_shape = window_size*num_data_parameters"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1300\n",
            "2\n",
            "1300\n",
            "2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IP5tV-tZusnA"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "41qqyuu4qRyi"
      },
      "source": [
        "### CNN Network\n",
        "\n",
        "A 1D CNN network was used considering the dimensions of the data. Each row of the data consists of the corresponding Wavelength and Absorbance from the spectrum and the height of the layer determines the number of instances of data equalling the window size which is 1300 in our case. Only the size of the input and output layers needs to be specified explicitly. The network estimates the size of the hidden layers on it's own.\n",
        "\n",
        "The network used here is of sequential type which means that it's basically a stack of layers. These layers include:\n",
        "* Input layer\n",
        "* First 1D CNN Layer\n",
        "* A max pooling layer\n",
        "* Second 1D CNN Layer \n",
        "* An average pooling layer\n",
        "* A dropout layer\n",
        "* A fully connected Softmax Activated layer\n",
        "\n",
        "**Input Layer:** The input data consists of 1300 spectral point slices long instances of the spectrum. Hence, the size of the input layer needs to be reshaped to 1300x2. The data passes through the input layer as a vector. The output for this layer is 1300x2.\n",
        "\n",
        "**First 1D CNN Layer:** This defines a filter of kernel size 10. 100 such filters are defined in this layer to enable it to learn 100 different features.\n",
        "\n",
        "**A max pooling layer:** This is used to reduce the complexity of the output and to prevent overfitting of the data. Using a pooling layer size of 3 reduces the size of the output matrix to 1/3rd of the input matrix.\n",
        "\n",
        "**Second 1D CNN Layer:** This layer enables the network to pick up higher level features which were missed in the First CNN layer. \n",
        "\n",
        "**Average pooling layer:** This averages the value of two weights in the network thereby further reducing overfitting. \n",
        "\n",
        "**Dropout layer:** This randomly assignms a weight of 0 to the neurons in the network. A value of 0.5 indicates that 50% of the neurons turn 0.\n",
        "\n",
        "**Fully connected Softmax Activated layer:** This reduces the output to the desired height of 6 which indicates the number of activity classes in the data. Softmax forces all six outputs of the neural network to sum up to one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1-nmroznqRyj",
        "outputId": "4da85dd0-7433-4e53-9cb1-b88bd0040af2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 403
        }
      },
      "source": [
        "#DEFINE CNN MODEL\n",
        "# 1D CNN neural network\n",
        "\n",
        "model_m = Sequential()\n",
        "model_m.add(Reshape((window_size, num_data_parameters), input_shape=(1,numOfRows,numOfColumns)))\n",
        "model_m.add(Conv1D(numFilters1, kernalSize, activation='relu', input_shape=(window_size, num_data_parameters)))\n",
        "model_m.add(MaxPooling1D(3))\n",
        "model_m.add(Conv1D(numNueronsFCL2, 10, activation='relu'))\n",
        "model_m.add(GlobalAveragePooling1D())\n",
        "\n",
        "model_m.add(Dropout(dropout))\n",
        "\n",
        "model_m.add(Dense(num_classes, activation='softmax'))\n",
        "print(model_m.summary())\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "reshape (Reshape)            (None, 1300, 2)           0         \n",
            "_________________________________________________________________\n",
            "conv1d (Conv1D)              (None, 1291, 100)         2100      \n",
            "_________________________________________________________________\n",
            "max_pooling1d (MaxPooling1D) (None, 430, 100)          0         \n",
            "_________________________________________________________________\n",
            "conv1d_1 (Conv1D)            (None, 421, 160)          160160    \n",
            "_________________________________________________________________\n",
            "global_average_pooling1d (Gl (None, 160)               0         \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 160)               0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 2)                 322       \n",
            "=================================================================\n",
            "Total params: 162,582\n",
            "Trainable params: 162,582\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "id": "gTSWO3LuqRym",
        "outputId": "c22aac7f-e871-4f8a-c333-e513cb3cb51e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "callbacks_list = [\n",
        "    keras.callbacks.ModelCheckpoint(\n",
        "        filepath='best_model.{epoch:02d}-{val_loss:.2f}.h5',\n",
        "        monitor='val_loss', save_best_only=True),\n",
        "    keras.callbacks.EarlyStopping(monitor='acc', patience=1)\n",
        "]\n",
        "\n",
        "model_m.compile(loss='categorical_crossentropy',\n",
        "                optimizer='adam', metrics=['accuracy'])\n",
        "\n",
        "BATCH_SIZE = 400\n",
        "EPOCHS = epochs\n",
        "\n",
        "history = model_m.fit(train_x,\n",
        "                      train_y,\n",
        "                      batch_size=BATCH_SIZE,\n",
        "                      epochs=EPOCHS,\n",
        "                      callbacks=callbacks_list,\n",
        "                      validation_split=0.2,\n",
        "                      verbose=1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.6931 - accuracy: 0.5397WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.6931 - accuracy: 0.5397 - val_loss: 0.6734 - val_accuracy: 0.6066\n",
            "Epoch 2/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.6458 - accuracy: 0.6486WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.6458 - accuracy: 0.6486 - val_loss: 0.6262 - val_accuracy: 0.7190\n",
            "Epoch 3/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.6105 - accuracy: 0.6963WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 34s 4s/step - loss: 0.6105 - accuracy: 0.6963 - val_loss: 0.6008 - val_accuracy: 0.7237\n",
            "Epoch 4/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5764 - accuracy: 0.7174WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5764 - accuracy: 0.7174 - val_loss: 0.5720 - val_accuracy: 0.7178\n",
            "Epoch 5/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5580 - accuracy: 0.7280WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5580 - accuracy: 0.7280 - val_loss: 0.5788 - val_accuracy: 0.7272\n",
            "Epoch 6/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5400 - accuracy: 0.7385WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5400 - accuracy: 0.7385 - val_loss: 0.5703 - val_accuracy: 0.7330\n",
            "Epoch 7/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5310 - accuracy: 0.7397WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5310 - accuracy: 0.7397 - val_loss: 0.5434 - val_accuracy: 0.7237\n",
            "Epoch 8/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5310 - accuracy: 0.7397WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5310 - accuracy: 0.7397 - val_loss: 0.5379 - val_accuracy: 0.7131\n",
            "Epoch 9/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5282 - accuracy: 0.7461WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5282 - accuracy: 0.7461 - val_loss: 0.5321 - val_accuracy: 0.7190\n",
            "Epoch 10/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5285 - accuracy: 0.7403WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5285 - accuracy: 0.7403 - val_loss: 0.5394 - val_accuracy: 0.7330\n",
            "Epoch 11/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5241 - accuracy: 0.7406WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 33s 4s/step - loss: 0.5241 - accuracy: 0.7406 - val_loss: 0.5412 - val_accuracy: 0.7319\n",
            "Epoch 12/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5219 - accuracy: 0.7444WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 34s 4s/step - loss: 0.5219 - accuracy: 0.7444 - val_loss: 0.5391 - val_accuracy: 0.7330\n",
            "Epoch 13/50\n",
            "8/9 [=========================>....] - ETA: 3s - loss: 0.5081 - accuracy: 0.7569WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5057 - accuracy: 0.7473WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5057 - accuracy: 0.7473 - val_loss: 0.5288 - val_accuracy: 0.7330\n",
            "Epoch 15/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.5043 - accuracy: 0.7625WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.5043 - accuracy: 0.7625 - val_loss: 0.5139 - val_accuracy: 0.7389\n",
            "Epoch 16/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4979 - accuracy: 0.7593WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 31s 3s/step - loss: 0.4979 - accuracy: 0.7593 - val_loss: 0.5208 - val_accuracy: 0.7377\n",
            "Epoch 17/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4943 - accuracy: 0.7654WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.4943 - accuracy: 0.7654 - val_loss: 0.5211 - val_accuracy: 0.7354\n",
            "Epoch 18/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4909 - accuracy: 0.7613WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 31s 3s/step - loss: 0.4909 - accuracy: 0.7613 - val_loss: 0.5017 - val_accuracy: 0.7482\n",
            "Epoch 19/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4802 - accuracy: 0.7690WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.4802 - accuracy: 0.7690 - val_loss: 0.4929 - val_accuracy: 0.7553\n",
            "Epoch 20/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4818 - accuracy: 0.7687WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 32s 4s/step - loss: 0.4818 - accuracy: 0.7687 - val_loss: 0.4833 - val_accuracy: 0.7576\n",
            "Epoch 21/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4769 - accuracy: 0.7698WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 33s 4s/step - loss: 0.4769 - accuracy: 0.7698 - val_loss: 0.4823 - val_accuracy: 0.7600\n",
            "Epoch 22/50\n",
            "9/9 [==============================] - ETA: 0s - loss: 0.4691 - accuracy: 0.7807WARNING:tensorflow:Early stopping conditioned on metric `acc` which is not available. Available metrics are: loss,accuracy,val_loss,val_accuracy\n",
            "9/9 [==============================] - 31s 3s/step - loss: 0.4691 - accuracy: 0.7807 - val_loss: 0.4763 - val_accuracy: 0.7623\n",
            "Epoch 23/50\n",
            "2/9 [=====>........................] - ETA: 13s - loss: 0.4452 - accuracy: 0.8050"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bneC4Z35qRyo",
        "outputId": "4c44ab8f-dd2c-42de-f102-6db357bb858c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 178
        }
      },
      "source": [
        "score = model_m.evaluate(test_x, test_y,batch_size=BATCH_SIZE, verbose=2)\n",
        "print(\"The test accuracy is\",score[1]*100,\"%\")"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-bfed95282794>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mscore\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel_m\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mevaluate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_x\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest_y\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mbatch_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mBATCH_SIZE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"The test accuracy is\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mscore\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\"%\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'model_m' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o8T0sZspqRyq"
      },
      "source": [
        "### Results\n",
        "\n",
        "The network was succesfully trained to recognize the difference between HDPE and PP. \n",
        "\n",
        "The model has a test accuracy of ~85%. "
      ]
    }
  ]
}