Support Vector Machine-4-Multiclass Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine - Multiclass Classification \n",
    "\n",
    "In this notebook, we use Scikit-Learn's svm.SVC class to perform multi-class classification.\n",
    "\n",
    "By default the SVC class uses <strong>One-versus-One (OvO)</strong> technique when we train it on more than two classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset\n",
    "\n",
    "\n",
    "We will use the iris dataset, which is a multivariate data set. \n",
    "\n",
    "This is a famous dataset that contains the sepal and petal length and width of 150 iris flowers of three different species: Iris-Setosa, Iris-Versicolor, and Iris-Virginica\n",
    "\n",
    "There are 4 features: \n",
    "- sepal length (cm)\n",
    "- sepal width (cm)\n",
    "- petal length (cm)\n",
    "- petal width (cm)\n",
    "\n",
    "Total number of samples: 150\n",
    "\n",
    "The dataset is also known as Fisher's Iris data set as it was introduced by the British statistician and biologist Ronald Fisher in his 1936 paper \"The use of multiple measurements in taxonomic problems as an example of linear discriminant analysis\".\n",
    "\n",
    "\n",
    "<img src=\"https://engineering.unl.edu/images/uploads/IrisFlowers.png\" width=800, height=400>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore The Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Key Values: \n",
      " ['data', 'target', 'target_names', 'DESCR', 'feature_names', 'filename']\n",
      "\n",
      "Feature Names: \n",
      " ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n",
      "\n",
      "Target Names: \n",
      " ['setosa', 'versicolor', 'virginica']\n"
     ]
    }
   ],
   "source": [
    "iris = load_iris()\n",
    "\n",
    "# See the key values\n",
    "print(\"\\nKey Values: \\n\", list(iris.keys()))\n",
    "\n",
    "# The feature names\n",
    "print(\"\\nFeature Names: \\n\", list(iris.feature_names))\n",
    "\n",
    "# The target names\n",
    "print(\"\\nTarget Names: \\n\", list(iris.target_names))\n",
    "\n",
    "# The target values (codes)\n",
    "#print(\"\\nTarget Values: \\n\", list(iris.target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Data Matrix (X) and the Label Array (y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4)\n",
      "(150,)\n",
      "\n",
      "X data type:  float64\n",
      "y data type:  int64\n"
     ]
    }
   ],
   "source": [
    "# For the experimentation we use two features\n",
    "X = iris[\"data\"]\n",
    "\n",
    "# Target Array\n",
    "y = iris[\"target\"]\n",
    "\n",
    "print(X.shape)\n",
    "print(y.shape)\n",
    "\n",
    "print(\"\\nX data type: \", X.dtype)\n",
    "print(\"y data type: \", y.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split Data Into Training and Test Sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standardize the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "\n",
    "# Fit on training set only.\n",
    "scaler.fit(X_train)\n",
    "\n",
    "# Apply transform to both the training set and the test set.\n",
    "X_train = scaler.transform(X_train)\n",
    "X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the SVC Model\n",
    "\n",
    "We don't perform hyperparameter tuning to keep our presentation simple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Accuracy:  0.9666666666666667\n",
      "\n",
      "Train Confusion Matrix:\n",
      "[[36  0  0]\n",
      " [ 0 40  2]\n",
      " [ 0  2 40]]\n"
     ]
    }
   ],
   "source": [
    "svm_clf = SVC()\n",
    "\n",
    "svm_clf.fit(X_train, y_train)\n",
    "y_predicted = svm_clf.predict(X_train)\n",
    "\n",
    "accuracy_score = np.mean(y_predicted == y_train)\n",
    "print(\"\\nTrain Accuracy: \", accuracy_score)\n",
    "\n",
    "print(\"\\nTrain Confusion Matrix:\")\n",
    "print(confusion_matrix(y_train, y_predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate the Model using Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Test Accuracy:  0.9666666666666667\n",
      "\n",
      "Test Confusion Matrix:\n",
      "[[14  0  0]\n",
      " [ 0  7  1]\n",
      " [ 0  0  8]]\n"
     ]
    }
   ],
   "source": [
    "y_test_predicted = svm_clf.predict(X_test)\n",
    "\n",
    "accuracy_score_test = np.mean(y_test_predicted == y_test)\n",
    "print(\"\\nTest Accuracy: \", accuracy_score_test)\n",
    "\n",
    "print(\"\\nTest Confusion Matrix:\")\n",
    "print(confusion_matrix(y_test, y_test_predicted))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}