MLwPython-Maria-Lida.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### About dataset"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets download the dataset"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-03-22 20:35:02--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.07s   \n\n2020-03-22 20:35:03 (311 KB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ],
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Data From CSV File  "
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    },
                    "execution_count": 3,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 10)"
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Convert to date time object "
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Data visualization and pre-processing\n\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Let\u2019s see how many of each class is in our data set "
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['loan_status'].value_counts()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Lets plot some columns to underestand data better:"
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {
                "scrolled": false
            },
            "outputs": [],
            "source": "# notice: installing seaborn might takes a few minutes\n#!conda install -c anaconda seaborn -y # pip install works faster"
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Requirement already satisfied: seaborn in /opt/conda/envs/Python36/lib/python3.6/site-packages (0.10.0)\nRequirement already satisfied: scipy>=1.0.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from seaborn) (1.2.0)\nRequirement already satisfied: matplotlib>=2.1.2 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from seaborn) (3.0.2)\nRequirement already satisfied: pandas>=0.22.0 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from seaborn) (0.24.1)\nRequirement already satisfied: numpy>=1.13.3 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from seaborn) (1.15.4)\nRequirement already satisfied: cycler>=0.10 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from matplotlib>=2.1.2->seaborn) (0.10.0)\nRequirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from matplotlib>=2.1.2->seaborn) (1.0.1)\nRequirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from matplotlib>=2.1.2->seaborn) (2.3.1)\nRequirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from matplotlib>=2.1.2->seaborn) (2.7.5)\nRequirement already satisfied: pytz>=2011k in /opt/conda/envs/Python36/lib/python3.6/site-packages (from pandas>=0.22.0->seaborn) (2018.9)\nRequirement already satisfied: six in /opt/conda/envs/Python36/lib/python3.6/site-packages (from cycler>=0.10->matplotlib>=2.1.2->seaborn) (1.12.0)\nRequirement already satisfied: setuptools in /opt/conda/envs/Python36/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib>=2.1.2->seaborn) (40.8.0)\n"
                }
            ],
            "source": "!pip install seaborn"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {
                "scrolled": false
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Pre-processing:  Feature selection/extraction"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Lets look at the day of the week people get the loan "
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0) #this one accounts Friday as weekend, wrong assumption but follow this as per instructions of the assignement - MLK\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Convert Categorical features to numerical values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets look at gender:"
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                },
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets convert male to 0 and female to 1:\n"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## One Hot Encoding  \n#### How about education?"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Feature befor One Hot Encoding"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Feature selection"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets defind feature sets, X:"
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X = Feature\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "What are our lables?"
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y = df['loan_status'].values\ny[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Normalize Data "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "# from sklearn.model_selection import train_test_split\n# X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\n# print ('Train set:', X_train.shape,  y_train.shape)\n# print ('Test set:', X_test.shape,  y_test.shape)"
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                },
                {
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Classification "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Train set: (276, 8) (276,)\nTest set: (70, 8) (70,)\n"
                }
            ],
            "source": "from sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)\nprint ('Train set:', X_train.shape,  y_train.shape)\nprint ('Test set:', X_test.shape,  y_test.shape)"
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([[ 0.56081126, -0.9677917 ,  0.69858179, -0.40565621, -1.17444044,\n        -0.40565621, -0.90321065,  1.21007674],\n       [ 0.56081126,  0.9437816 , -0.81959596, -0.40565621,  0.85146932,\n        -0.40565621,  1.10716144, -0.82639387],\n       [ 0.56081126,  0.9437816 ,  0.02383612, -0.40565621, -1.17444044,\n        -0.40565621, -0.90321065,  1.21007674],\n       [ 0.56081126, -0.9677917 , -0.98828237, -0.40565621,  0.85146932,\n        -0.40565621, -0.90321065,  1.21007674],\n       [ 0.56081126,  0.9437816 , -0.48222312, -0.40565621, -1.17444044,\n        -0.40565621, -0.90321065,  1.21007674]])"
                    },
                    "execution_count": 23,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X_train= preprocessing.StandardScaler().fit(X_train).transform(X_train.astype(float))#normalize\nX_train[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([[ 0.33474248,  0.83916906, -0.19614926, -0.47756693,  0.74535599,\n        -0.2773501 ,  1.26197963, -1.05887304],\n       [-1.70282047, -0.9301633 , -0.19614926, -0.47756693,  0.74535599,\n        -0.2773501 , -0.79240582,  0.94440028],\n       [ 0.33474248, -0.9301633 , -0.04012144, -0.47756693, -1.34164079,\n        -0.2773501 ,  1.26197963, -1.05887304],\n       [ 0.33474248,  0.83916906, -1.13231619, -0.47756693, -1.34164079,\n        -0.2773501 , -0.79240582,  0.94440028],\n       [ 0.33474248,  0.83916906,  0.42796202, -0.47756693, -1.34164079,\n        -0.2773501 , -0.79240582,  0.94440028]])"
                    },
                    "execution_count": 24,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X_test= preprocessing.StandardScaler().fit(X_test).transform(X_test.astype(float))#nrmalize\nX_test[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics\nfrom sklearn.metrics import jaccard_similarity_score"
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([0.67142857, 0.64285714, 0.72857143, 0.65714286, 0.71428571,\n       0.65714286, 0.74285714, 0.74285714, 0.71428571])"
                    },
                    "execution_count": 27,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Ks = 10\nmean_acc = np.zeros((Ks-1))\nstd_acc = np.zeros((Ks-1))\nConfustionMX = [];\nfor n in range(1,Ks):\n    \n    neigh = KNeighborsClassifier(n_neighbors= n).fit(X_train, y_train)\n    yhat = neigh.predict(X_test)\n    \n    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n    std_acc[n-1] = np.std(yhat == y_test)/np.sqrt(yhat.shape[0])\n    \nmean_acc"
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd8lfXd//HXJ5sAYYaZMBJ2oqywXQicgFUQBzne1l2trdbRWltbb2tt+2u1e3h717YObOthKMOJCNRxDhsZCXuZk7DCCIRA9vf3R064YwjJSXLOuc45+Twfj/OQc3Kd63oTST7n+k4xxqCUUkoFmwirAyillFL10QKllFIqKGmBUkopFZS0QCmllApKWqCUUkoFJS1QSimlgpIWKKWUUkFJC5RSSqmgpAVKKaVUUIqyOoCvdO3a1fTr18/qGEoppRqxcePG48aYxMaOC5sC1a9fPzZs2GB1DKWUUo0QkS+9OU6b+JRSSgUlLVBKKaWCkhYopZRSQSls+qDqU15eTl5eHiUlJVZHCQtxcXEkJSURHR1tdRSlVCsQ1gUqLy+P9u3b069fP0TE6jghzRjDiRMnyMvLo3///lbHUUq1AmHdxFdSUkKXLl20OPmAiNClSxe9G1VKBUxYFyhAi5MP6fdSKRVIYd3Ep5RSAPtO7iP7WLbVMRo1oPMA0rqlWR0jaGiBCoBFixZx0003sWPHDoYMGWJ1HKVanev+fR27T+y2OkajIiUS130uxvYea3WUoKAFKgDefPNNrrjiChwOB88++6zfrlNZWUlkZKTfzq9UKDpy9gi7T+zm+xO/z23pt1kd55LKq8q5Zf4t3LnoTjZ9cxPx0fFWR7JcqylQj334GJuPbPbpOUf0GMEfpv+hwWPOnj2L0+lk1apVzJw58ysF6oUXXuCNN94gIiKCGTNm8Ktf/Yq9e/fy4IMPUlBQQGRkJAsWLMDtdvOb3/yGd999F4CHH36YjIwM7r77bvr168e9997LRx99xMMPP0xRUREvv/wyZWVlDBgwgDfeeIP4+HiOHj3Kgw8+yP79+wF46aWX+OCDD+jatSuPPvooAD/+8Y/p3r07jzzyiE+/T0pZyeV2AXDT0JsY2XOkxWka9tqNrzFl7hSe+vgp/jjjj1bHsZxfB0mIyHQR2SUie0Xkh/V8/fcistnz2C0ihXW+niAi+SLyF3/m9KfFixczffp0Bg0aROfOndm0aRMAH3zwAYsXL2bt2rVs2bKFJ598EoDbb7+dhx56iC1btuByuejZs2ej14iLi+Pzzz/Hbrdz0003sX79erZs2cLQoUP5xz/+AcAjjzzC1VdfzZYtW9i0aRNpaWncd999vP766wBUVVXhcDi4/fbb/fSdUMoazlwnsZGxjOwR3MUJ4Nr+1/LI2Ef407o/sWL/CqvjWM5vd1AiEgm8CEwD8oD1IrLUGLO95hhjzOO1jv8OUPdf0M+AT3yRp7E7HX958803eeyxxwCw2+28+eabjBo1io8//ph77rmH+Pjq2/jOnTtTVFREfn4+s2fPBqoLjzeysrIu/Dk7O5unn36awsJCzp49S2ZmJgArV65k7ty5AERGRtKhQwc6dOhAly5d+OKLLzh69CgjR46kS5cuPvu7KxUMXHkuxvQeQ2xUrNVRvPLLqb/kw30fcs+Se9j2rW10iOtgdSTL+LOJbyyw1xizH0BEHMAsYPsljr8N+EnNExEZDXQHPgQy/JjTb06cOMHKlSvJzs5GRKisrEREeOGFFzDGXDRs2xhT73mioqKoqqq68LzuXKS2bdte+PPdd9/N4sWLGT58OK+99hr/+c9/Gsz4jW98g9dee40jR45w7733NvFvqFRwO19+no2HNvL4+McbPzhIxEfH88bsN5j4j4k8+uGjvHbja1ZHsow/m/h6A+5az/M8r11ERPoC/YGVnucRwG+B7zd0ARF5QEQ2iMiGgoICn4T2pYULF3LnnXfy5ZdfcvDgQdxuN/379+fzzz/HZrPxyiuvcO7cOQBOnjxJQkICSUlJLF68GIDS0lLOnTtH37592b59O6WlpZw+fZoVKy59619UVETPnj0pLy/nX//614XXp0yZwksvvQRUD6Y4c+YMALNnz+bDDz9k/fr1F+62lAoXGw9vpLyqnEl9JlkdpUnG9h7Lj678Ea9veZ3FOxdbHccy/ixQ9c3qrP8WAezAQmNMpef5t4H3jTHuSxxffTJjXjbGZBhjMhITG937KuDefPPNC811NW6++Wb+/e9/M336dGbOnElGRgYjRozgN7/5DQBvvPEGf/rTn7j88suZOHEiR44cITk5mTlz5nD55Zdz++23M3LkpdvSf/aznzFu3DimTZv2lSHtf/zjH1m1ahWXXXYZo0ePJicnB4CYmBgmT57MnDlzdASgCjvOXCcAE5MnWpyk6Z6+6mlG9hjJA+88wLHiY1bHsYRcqlmpxScWmQA8a4zJ9Dx/CsAY88t6jv0CeMgY4/I8/xdwJVAFtANigP8xxlw00KJGRkaGqbth4Y4dOxg6dKhv/kJhqqqqilGjRrFgwQIGDhzY6PH6PVWhZJZjFjuP72TXw7usjtIsOcdyGP3yaKYPmM6irEVhs5qLiGw0xjTadePPO6j1wEAR6S8iMVTfJS2te5CIDAY6AatrXjPG3G6M6WOM6Qc8AcxtqDip5tm+fTsDBgxgypQpXhUnpUKJMQaX28Wk5NBq3qstrVsaP7/25yzZtYS5W+ZaHSfg/DZIwhhTISIPA8uASOAVY0yOiDwHbDDG1BSr2wCH8detnLqkYcOGXZgXpVS42XNyD8fPHQ/J5r3aHh//OEt3LeWRDx9hcv/J9OnQx+pIAePXeVDGmPeNMYOMManGmF94XnumVnHCGPNsQ3dHxpjXjDEPtyBDc9+q6tDvpQolNf1PoXwHBRAZEclrN75GlaniniX3UGWqGn9TmAjr1czj4uI4ceKE/mL1gZr9oLydm6WU1ZxuJ53iOjG462Cro7RYSqcUfmf7HSsPrOTFdS9aHSdgwnqpo6SkJPLy8gjGIeihqGZHXaVCgcvtYmLyRCIkPD6Hf2PUN1i8azFPfvwk01KnMaRr+C88HdYFKjo6Wnd/VaoVOnn+JDuO7+Drl3/d6ig+IyL8/Ya/k/5SOnctvgvnvU6iIsL6V3h4N/EppVqn1e7qQcGh3v9UV8/2PXnpay+xLn8dv/r8V1bH8TstUEqpsON0V99djOk9xuooPjcnbQ63pd/GTz/5KZsOb7I6jl9pgVJKhR2X28XIHiPDdk+lv1z3FxLjE7lj0R2UVJQ0/oYQpQVKKRVWyivLWZe/Luya92rr3KYzr8x6he0F2/nvlf9tdRy/0QKllAorm49s5nzF+ZCfoNuY6QOm8+DoB/nt6t/y6ZefWh3HL7RAKaXCitPtmaAbYiuYN8evbb+mf6f+3LX4LopKi6yO43NaoJRSYcXldtG3Q196te9ldRS/axfTjrk3zuXLwi/53kffszqOz2mBUkqFDWMMTrezVdw91ZjUZxJPTnqSv236G+/tfs/qOD6lBUopFTa+PP0lh4oOMTEpvPuf6vrpNT/lsm6X8Y13vsGJcyesjuMzWqCUUmHD5XYBraP/qbbYqFjmzp7LiXMn+NZ73wqb9Ue1QCmlwoYz10m7mHakd0u3OkrAjegxgmeveZYF2xfgyHZYHccntEAppcKGK8/F+KTxYb9G3aU8OelJxieN56H3HyL/TL7VcVpMC5RSKiwUlRax9ejWVtf/VFtURBRzb5xLaWUp9y29L+Sb+rRAKaXCwtr8tVSZqlbX/1TXwC4DeWHqCyzbt4y/bvyr1XFaRAuU8rmjZ4+G/Cc3FXqcuU4EYXzSeKujWO5bY77FtJRpfO+j77H35F6r4zSbFijlU7mnc0n+fTL/3vZvq6OoVsaV5+Ky7peREJtgdRTLRUgEr8x6heiIaO5afBeVVZVWR2oWLVDKpz45+AnlVeUs2bXE6iiqFamsqmS1e3VYLxDbVEkJSfzlur/gcrv4jes3VsdpFi1Qyqdq5qF8vP/jkP3UpkJPTkEORWVFYb9AbFPdftnt3Dz0Zp75zzNsPbrV6jhNpgVK+ZTT7SQ2MpZTJafYeHij1XFUK+HM9SwQq3dQXyEivPS1l+gY15E7Ft1BaUWp1ZGaRAuU8pnTJafJPpbN/aPuRxCW7V1mdSTVSjjdTnq060G/jv2sjhJ0Etsm8rcb/sbWo1v56Sc/tTpOk2iBUj6zJm8NBsOsIbMY3Ws0H+3/yOpIqpVwuV1MSp6EiFgdJSjNHDyTe0fcy/PO51ntXm11HK9pgVI+43K7iJAIxvUehy3Fxmr3as6UnrE6lgpzh4sOc6DwgPY/NeL3039PckIydy6+k+KyYqvjeEULlPIZp9vJ5d0vp31se2ypNipNJasOrLI6lgpzFxaI1f6nBiXEJvDaja+x9+RefvDxD6yO4xUtUMonKqoqWJO35sIyMxOSJ9Auph3L9mk/lPIvp9tJXFQcI3uOtDpK0Lum3zU8Pv5xXlz/Ih/tC/4meC1Qyie2Hd1GcXnxhWVmYiJjmNxvckj8EKjQ5nK7GNNrDDGRMVZHCQm/uPYXDO06lHuX3Mup86esjtMgLVDKJ5zu6mG+tfsBbKk29p3ax76T+6yKpcLc+fLzbDq8SZv3mqBNdBvmzp7LkbNHeOTDR6yO0yAtUMonXG4Xvdr3om+Hvhdes6XaAFi+f7lVsVSY23BoA+VV5TpAookyemXw9FVP88+t/2Th9oVWx7kkLVDKJ5xuJxOTJ35lmO/AzgPp26Gv9kMpv6m5c5+QPMHiJKHnx1f+mNE9R/Pguw9y5OwRq+PUSwuUarG8M3nkns69qJlFRMhMzWTlgZWUV5ZblE6FM5fbxeAug+ka39XqKCEnOjKaN2a/wdmyszzwzgNBuQOBFijVYjXDfOtrZrGl2jhTeoZ1+esCHUuFOWPMhQm6qnmGJg7ll1N+yTu73+HVza9aHecifi1QIjJdRHaJyF4R+WE9X/+9iGz2PHaLSKHn9REislpEckRkq4hk+TOnahmX20WbqDaM7HHxMN9r+19LhEToaD7lc7tO7OLE+RPa/9RCj45/lKv7Xs2jHz7KwcKDVsf5Cr8VKBGJBF4EZgDDgNtEZFjtY4wxjxtjRhhjRgB/Bt72fOkccKcxJg2YDvxBRDr6K6tqGafbydjeY4mOjL7oa53adGJs77HaD6V87sIE3Va+g25LRUgEr934GoJw9+K7qTJVVke6wJ93UGOBvcaY/caYMsABzGrg+NuANwGMMbuNMXs8fz4EHAMS/ZhVNVNxWTFfHP6iwU+xmamZrD+0npPnTwYwmQp3zlwnndt0ZlCXQVZHCXn9OvbjD9P/wCdffsIf1/zR6jgX+LNA9QbctZ7neV67iIj0BfoDK+v52lggBtDJNEFo/aH1VJrKBvsBbKk2qkwVKw9c9L9XqWZz5bmYmDyRCNGudF+4Z8Q93DDoBp5a8RTbC7ZbHQfwb4Gqb1nhSw0TsQMLjTFf2eFORHoCbwD3GHPxfaeIPCAiG0RkQ0FBQYsDq6araWZpaJjv2N5jSYhN0H4o5TMnzp1g5/GdF5bWUi0nIrx8w8u0i2nHnYvuDIqRt/4sUHlAcq3nScChSxxrx9O8V0NEEoD3gKeNMWvqe5Mx5mVjTIYxJiMxUVsAreB0OxnadSid23S+5DFREVFM6T+FZfuWBeVQVhV6VudVbxmh/U++1aNdD/56/V/ZeHgjv/jsF1bH8WuBWg8MFJH+IhJDdRFaWvcgERkMdAJW13otBlgEzDXGLPBjRtUCVaYKl9vl1SgqW6qN3NO57D6xOwDJVLhz5jqJiohiTK8xVkcJOzcPu5mvX/51fv7pz9lwaIOlWfxWoIwxFcDDwDJgBzDfGJMjIs+JyMxah94GOMxXP1rPAa4C7q41DH2Ev7Kq5tl5fCeFJYVezUPJTM0E0GY+5ROuPBejeo6iTXQbq6OEpT/P+DM92vXgjkV3cL78vGU5/Nq7aIx53xgzyBiTaoz5hee1Z4wxS2sd86wx5od13vdPY0x0zRB0z2OzP7OqpnPmXrxA7KX079SfAZ0H6C67qsXKKstYl79OJ+j6Uce4jrw661V2Ht/Jj1b8yLIcOvxFNZsrz0WXNl28HuZrS7Gx6sAqyirL/JxMhbPNRzZTUlGiE3T9bFrqNB4a8xB/WPsHyzYe1QKlms2Ze/ECsQ2xpdooLi++MPJPqeZoyp27apnnpz7PgM4DuHvJ3ZwpPRPw62uBUs1SUFzAnpN7mtTMMrn/ZKIiorQfSrWI0+2kX8d+9Grfy+ooYa9tTFvm3jiXvDN5PPbhYwG/vhYo1SzNWWYmITaBCUkTtECpZjPG4HQ7tf8pgCYkT+AHk37Aq5tfZemuiwZi+5UWKNUsLreL6IhoRvcc3aT32VJtbDq8iYJinVitmu5g4UGOnD2izXsB9uw1zzK8+3Duf+f+gP7saoFSzeJ0Oxnda3STh/naUm0YDB/v/9hPycKHMSaoFu4MBhfu3PUOKqBiImOYO3suhSWFPPjegwGbcK8FSjVZaUUpGw5taNYyM6N7jqZzm8463NwLf934V3r+tidFpUVWRwkaTreT9jHtSe+WbnWUVufy7pfz3DXP8faOt5mXMy8g14wKyFVUWNl0eBOllaXNWmYmMiKSqSlT+WjfRxhjvB4B2Bq9vPFljhUf453d7/Bfl/2X1XGCgsvtYnzSeCIjIq2O0io9MfEJyqvKmT5gekCup3dQqsmc7pYN87Wl2DhUdIicghxfxgoru0/s5osjXwAE7NNqsDtTeoZtx7Zp856FIiMiefqqp+kYF5jt+bRAqSZzuV2kdEqhR7sezXq/LdUG6LJHDZmXPQ9ByErL4oM9H3Dq/CmrI1lubd5aqkyVDpBoRbRAqSapGebbkl8SyR2SGdp1qBaoSzDG8Gb2m1zZ90q+N+F7lFeVs3jnYqtjWc7pdhIhEYxLGmd1FBUgWqBUk+w/tZ9jxcda3MxiS7XxyZefUFJR4qNk4SP7WDY7ju/AnmYno1cGKZ1ScOQ4rI5lOZfbxWXdLiMhNsHqKCpAtECpJmlp/1MNW6qNkooSPvvyM1/ECiuObAeREsnNw25GpLqZb8X+Fa167lhlVSVr8tZo/1MrowVKNYnL7SIhNoG0xLQWnefqvlcTExmjzXx1GGOYlzOPKSlT6Na2GwD2dDuVppK3drxlcTrrbDu2jaKyIu1/amW0QKkmcbqdTEia0OJhvm1j2nJFnyt0PlQdGw9vZN+pfdjT7Bdeu6zbZQztOhRHdutt5mvO0loq9GmBUl4rLCkk51iOzz7F2lJsbD26lcNFh31yvnDgyHYQHRHN7KGzL7wmItjT7Xz65afkn8m3MJ11nG4nPdv1pG+HvlZHUQGkBUp5bU3eGgzGZ/0ANcPNl+9f7pPzhboqU8W8nHlMHzD9onkmWWlZGAwLti+wKJ21XG4Xk/pM0ondrYwWKOU1l9tFhEQwtvdYn5xveI/hJMYnaj+Uh8vtIu9MHvZ0+0VfG9x1MCN6jGiVzXyHig5xsPBgs5bWUqFNC5TymtPtZHj34bSPbe+T80VIBLZUG8v3L9dFUalu3msT1YaZg2fW+3V7mp21+Ws5cOpAgJNZS/ufWi8tUMorFVUVrM1b6/NRVLZUG8eKj7H16FafnjfUVFRVsGD7Aq4fdD3tYtrVe0xWehYA83PmBzKa5Zy5TtpEtWFkj5FWR1EBpgVKeWXr0a0Ulxf7fB7KtJRpACzbu8yn5w01nxz8hGPFx+pt3qvRr2M/xieNb3WTdl15Lsb0HkN0ZLTVUVSAaYFSXnHm+maCbl092/fksm6Xtfrh5o5sB+1i2jFjwIwGj7On2dl8ZDM7j+8MUDJrnSs/x6bDm3SCbiulBUp5xZXnonf73vTp0Mfn585MzeTz3M8pLiv2+blDQVllGW/teIsbh9zY6AaQt6bdiiDMy24dK5xvOLSBiqoKnaDbSmmBUl5x5lYvEOuPYb62VBtllWV8+uWnPj93KFi+bzmnSk59ZXLupfRq34ur+l6FI8cRsF1NrVRz5z4haYLFSZQVtECpRrlPu3GfcfutmeWKPlcQFxXHsn2tsx9qXs48OsV1YlrqNK+Ot6fb2Xl8J9uObfNzMus53U6GdB1Cl/guVkdRFtACpRrl72G+baLbcFXfq1rlfKjz5edZvHMxNw+9mZjIGK/ec/PQm4mUyLCfE1Vlqlidt1r7n1oxLVCqUS63i/joeIZ3H+63a2SmZrLj+A7cp91+u0Yw+mDvBxSVFV0YQu6NxLaJTE2ZiiM7vJv5dh3fxcnzJ7X/qRXTAqUa5XQ7Gdt7rF+H+bbWZY8c2Q66te3GNf2uadL7stKyOFB4gPWH1vsnWBC4cOeud1CtlhYo1aDismI2H9ns92Vm0hLT6NmuZ6vqhyoqLeLd3e9y67BbiYqIatJ7Zw+dTXREdFg38zndTrq06cKgLoOsjqIsogVKNWhd/joqTaXfl5kREWypNj7e/zGVVZV+vVaweGf3O5yvON/g5NxL6RjXkRkDZzA/Z37YLhPlcrv8NnJUhQYtUKpBNc0s45PG+/1amamZnDx/kk2HN/n9WsFgXs48khKSmt3HYk+zk1+Uf2Eodjg5fu44u07s0ua9Vk4LlGqQ0+1kWOIwOrfp7PdrTU2ZCtAqRvOdOn+KD/Z8QFZaFhHSvB/DGwbfQJuoNmHZzLfavRrw/colKrQ0+pMhIg+LSKdAhFHBpWaYb6C2OUhsm8ionqNaRT/U4p2LKa8qJyvN+9F7dbWLaccNg29gwfYFVFRV+DCd9ZxuJ9ER0WT0yrA6irKQNx/degDrRWS+iEwXbRBuNXYU7KCwpDCg2xzYUmyszlvNmdIzAbumFRw5DlI6pbT4F3BWWhYF5wpYdWCVj5IFB5fbxaieoxpd+kmFt0YLlDHmaWAg8A/gbmCPiPw/EUlt7L2egrZLRPaKyA/r+frvRWSz57FbRAprfe0uEdnjedzVpL+V8gmn2z8LxDYkc0AmFVUV/OfgfwJ2zUArKC5gxf4V2NPsLR4AMGPADNrHtGdeTviszVdWWcb6Q+u1/0l51wdlqmcDHvE8KoBOwEIReeFS7xGRSOBFYAYwDLhNRIbVOe/jxpgRxpgRwJ+Btz3v7Qz8BBgHjAV+os2Mgedyu0iMT2Rg54EBu+aEpAm0jW4b1v1Qb+14i0pT2azRe3W1iW7DjUNu5K0db1FWWeaDdNbbdHgTJRUl2v+kvOqDekRENgIvAE7gMmPMt4DRwM0NvHUssNcYs98YUwY4gFkNHH8b8Kbnz5nAcmPMSWPMKWA5ML3Rv43yKafbfwvEXkpsVCzX9LsmrPuhHNkOhiUOI71buk/OZ0+3U1hSGDZFvWbkqBYo5c0dVFfgJmNMpjFmgTGmHMAYUwVc38D7egO1163J87x2ERHpC/QHVjblvSLygIhsEJENBQUFXvxVlLeOFR9j78m9lvySsKXa2HtyL/tP7Q/4tf0t/0w+n375KVlpWT4r/FNTptK5TeewGc3ndDvp37E/Pdv3tDqKspg3Bep94GTNExFpLyLjAIwxOxp4X30/fZdaOMwOLDTG1MzQ9Oq9xpiXjTEZxpiMxMTEBqKoprJymZnM1EygehuKcLNg+wIMpkWj9+qKiYzhpiE3sWTXEs6Vn/PZea1gjMHldgV0YI4KXt4UqJeAs7WeF3tea0wekFzreRJw6BLH2vm/5r2mvlf5gcvtIiYyhtG9Rgf82oO6DKJPhz5hucuuI9vByB4jGdx1sE/Pa0+3c7bsLO/ved+n5w20A4UHOHL2SMCmNqjg5k2BElNryWRP0543C4etBwaKSH8RiaG6CC296OQig6kedLG61svLAJuIdPIMjrB5XlMB4nQ7Gd1zNHFRcQG/tohgS7GxYv+KsJrfc+DUAdbmr/XJ4Ii6rul3Dd3bdg/50Xz+3tpFhRZvCtR+z0CJaM/jUaDRzgFjTAXwMNWFZQcw3xiTIyLPicjMWofeBjjqFMGTwM+oLnLrgec8r6kAKK0oZcOhDZZ2UttSbZwuPc26/HWWZfC1+TnzAZiTNsfn546MiOTWYbfy7u53KSot8vn5A8WZ6yQhNoG0xDSro6gg4E2BehCYCORT3fQ2DnjAm5MbY943xgwyxqQaY37hee0ZY8zSWsc8a4y5aI6UMeYVY8wAz+NVb66nfGPj4Y2UVZZZOg9lSsoUIiQibEamQfXk3PFJ4+nXsZ9fzm9Pt1NSUcLSXRc1VIQMV56L8UnjiYyItDqKCgLeTNQ9ZoyxG2O6GWO6G2P+yxhzLBDhlDVqFh+18g6qc5vOjOk1JmwK1M7jO9l8ZDP2NN8379WYkDyB5IRkHDmhOZrvdMlpth3dphN01QXezIOKE5GHROR/ROSVmkcgwilruPJcpHZKpXu77pbmsKXaWJu/llPnT1mawxfmZc9DEG5Nu9Vv14iQCOakzWHZ3mUh+T1bm78Wg9H5T+oCb5r43qB6Pb5M4BOqR9SFbiO3apAxBmeuMyh+SdhSbVSZKlYeWNn4wUHMGIMjx8HV/a6mV/tefr2WPd1OeVU5i3Yu8ut1/MGZ6yRCIhjXe5zVUVSQ8KZADTDG/DdQbIx5HfgacJl/Yymr7Du1j4JzBUHRzDKu9zgSYhNCvplv27Ft7Dy+06/NezVG9xxNaqfUkJy063Q7ubz75bSPbW91FBUkvClQ5Z7/FopIOtAB6Oe3RMpSNf1PwTDMNzoymmv7X8uyfcuoNcgz5DiyHURKJDcPa2hlMN8QEezpdlYcWMGx4tDpKq6oqmBt/tqg+GCkgoc3Beplz1ykp6mex7QdeN6vqZRlXG4XHWI7MCxxWOMHB4AtxcaXp79kz8k9VkdpFmMMjmwHU1Om0jW+a0CuaU+3U2WqWLh9YUCu5wvbjm7jbNnZoGhaVsGjwQIlIhHAGWPMKWPMp8aYFM9ovr8GKJ8KMKfbyYTkCc3e5dXXbKk2IHR32V1/aD0HCg/4ZXLupaR3S2dY4rCQauazcmktFbwa/C3kWTXi4QBlURYrLCkkpyAnqJaZSe2cSmqn1JAtUI5sBzGRMdw45MaAXteeZufz3M/JO5MX0Os2l9PtpFf7XvRvm25cAAAgAElEQVTp0MfqKCqIePMxebmIPCEiySLSuebh92Qq4Fa7q1ebCob+p9psqTZWHVwVcvsdVZkq5ufMZ8aAGXSM6xjQa2elZ2EwLMhZENDrNpfL7WJS8qSAbu2igp83Bepe4CHgU2Cj57HBn6GUNVxuF5ESydjeY62O8hW2VBtny85eKKChwpnrJL8oP6DNezUGdRnEqJ6jQmLSbv6ZfL48/aU276mLeLOSRP96HimBCKcCy+l2MrzHcNrFtLM6yldM7jeZSIkMuWY+R7aDNlFtuH5QQ9um+Y89zc66/HVBv6+WblCoLsWblSTurO8RiHAqcMory1mbvzao+p9qdIjrwITkCSG1/UZFVQULti/ghsE3WFbwaxalnZcd3CucO91O2kS1YUSPEVZHUUHGmya+MbUeVwLPAjMbeoMKPVuPbuVc+bmg63+qYUuxsfHQRo6fO251FK+sOrCKgnMFAZmceyl9O/ZlQtKEoG/mc7ldjO09lujIaKujqCDjTRPfd2o97gdGAjH+j6YCyem2foHYhthSbRgMH+//2OooXnFkO2gf054ZA2dYmsOebmfr0a3sKGho82vrnCs/xxdHvtD+J1Wv5kx2OQcM9HUQZS2X20VSQlLQDvPN6JVBp7hOIdEPVVZZxts732b20NmWbPhY263DbkWQoN3IcF3+OiqqKoL2g5Gyljd9UO+IyFLP411gF7DE/9FUIDndzqD+FBsZEcnUlKl8tO+joF/26KN9H1FYUkhWWpbVUejZvifX9LsGR7YjKL9vNQMkJiRPsDiJCkbe3EH9Bvit5/FL4Kr6NhhUoct92k3embyg/xRrS7WRX5TPjuPB2VxVw5HtoHObzkxNmWp1FKC6mW/XiV1sObrF6igXcbqdDO06lM5tdGqlupg3BSoXWGuM+cQY4wROiEg/v6ZSAVXT/xTMd1AA01KmAbBs7zKLk1zaufJzLNm1hJuH3kxMZHB01d409CYiJTLolj6qMlWsdq8O+n93yjreFKgFQFWt55We11SYcLldxEfHc3n3y62O0qC+HfsyuMvgoB5u/v6e9zlbdtaSybmX0jW+K9NSpzEvZ15QNfPtPL6TUyWngv7OXVnHmwIVZYy5sMaM58/B8dFQ+YTT7WRc73EhMcw3MzWTTw5+QklFidVR6jUvZx7d23bn6r5XWx3lK+xpdg4WHmRd/jqro1xwYYHYIJ3aoKznTYEqEJEL855EZBYQGpNRVKPOlp1ly5EtIfMp1pZq43zF+Qv7VgWTotIi3t39LrcOu5XIiEir43zFjUNuJCYyJqia+ZxuJ13juzKwsw4KVvXzpkA9CPxIRHJFJBf4AfBN/8ZSgbIufx2VpjJk+gGu7nc10RHRLNsXfP1QS3ctpaSiJKia92p0iOvAdQOvY17OPCqrKq2OA1TfQU1MnqgLxKpL8mai7j5jzHhgGJBmjJlojNnr/2gqEGqaWcYnjbc4iXfaxbRjUp9JQTkfypHjIDkhOWiHTNvT7Bw+e5jPcz+3OgoFxQXsPrE7ZD4YKWt4Mw/q/4lIR2PMWWNMkYh0EpGfByKc8j+n20laYhqd2nSyOorXMlMz2XJ0C0fOHrE6ygUnz59k2d5lZKVlBc1mj3VdP+h64qPjg2LS7uq86pXpQ6VpWVnDm5+kGcaYwponxphTwHX+i6QCpWaYb6j9kqjZZTeYlj1atGMR5VXlQdm8V6NtTFtuGHQDC7YvoKKqwtIszlwn0RHRZPTKsDSHCm7eFKhIEYmteSIibYDYBo5XIWJ7wXZOl54OuWaWET1G0DW+a1D1Q83LmceAzgMY1XOU1VEaZE+3c/zccVYeWGlpDqfbyeheoy1fCkoFN28K1D+BFSJyn4jcCywH5vo3lgqEmpFwoTbMN0IimJYyjeX7llNlqhp/g58dKz7GigMryErLCvoO/+kDppMQm2DpaL7SilI2HNoQch+MVOB5M0jiBeDnwFAgDfiZMeZ5fwdT/ufKc5EYn0hqp1SrozRZZmomR4uPsu3oNqujsHD7QqpMVVA379WIi4pj9pDZvL3jbUorSi3JsOnwJkorS0OuaVkFnle9ucaYD40xTxhjvgecFZEX/ZxLBYAz18mkPpOC/lN/faalVi97FAyj+RzZDtIS00jvlm51FK9kpWVxuvS0Zd873UFXecurAiUiI0TkeRE5SPXd1E6/plJ+d/TsUfad2heUO+h6o1f7XqR3S7e8HyrvTB6f5X4WEndPNaamTKVzm86WbWTodDtJ6ZRCj3Y9LLm+Ch2XLFAiMkhEnhGRHcBfgDxAjDGTjTF/DlhC5RfhsMyMLcXGZ7mfca78nGUZFuRUL0sZDFtreCs6Mppbht7Ckp1LAv69M8bgcru0/0l5paE7qJ3AFOAGY8wVnqIUHFPQVYu53C5iImOCftRZQzIHZFJWWcanX35qWQZHjoNRPUcxsEtoLddjT7dTXF7Me7vfC+h195/az9Hio1qglFcaKlA3A0eAVSLyNxGZAoReZ4Wql9PtJKNXRkgP872yz5XERsZa1pey/9R+1uWvw54WOs17Na7qexU92vUIeDOf9j+pprhkgTLGLDLGZAFDgP8AjwPdReQlEbF5c3IRmS4iu0Rkr4jUu8mhiMwRke0ikiMi/671+gue13aIyJ8kFHvyg1RJRQkbD28M2f6nGm2i23BV36ss64eal129IsOctDmWXL8lIiMimTNsDu/tfo8zpWcCdl2n20lCbAJp3dICdk0VurwZZl5sjPmXMeZ6IAnYDDS6o66IRAIvAjOoXsfvNhEZVueYgcBTwCRjTBrwmOf1icAk4HIgHRgDBNf+BSFs46GNlFWWhXT/Uw1bqo3tBdvJO5MX8Gs7chxMTJ5I3459A35tX8hKz6K0spSlu5YG7Jout4sJSROCdjkoFVya9K/EGHPSGPNXY8y1Xhw+FthrjNnv2UPKAcyqc8z9wIue5ZMwxhyruRQQR/W+U7FANHC0KVnVpdXsoDshKTgXNW2KzNRMAJbvWx7Q6+4o2MHWo1tDsnmvxvik8fTp0Cdgk3YLSwrJPpat/U/Ka/78GNMbcNd6nud5rbZBwCARcYrIGhGZDmCMWQ2sAg57HsuMMTvqXkBEHhCRDSKyoaCgwC9/iXDkcrsY0HkA3dt1tzpKi6V3S6dHux4B32V3Xs48BOGWYbcE9Lq+FCERZKVlsWzfMk6eP+n3663JW4PBaP+T8po/C1R9fUZ195uOAgYC1wC3AX8XkY4iMoDqlSuSqC5q14rIVRedzJiXjTEZxpiMxMREn4YPV+E2zFdEsKXaWL5vecD2OTLG4Mh2cE2/a+jZvmdArukv9nQ7FVUVvL3jbb9fy+V2ESERjEsa5/drqfDgzwKVByTXep4EHKrnmCXGmHJjzAFgF9UFazawxrPFx1ngAyA0NiwKcntP7qXgXEFYfYq1pdg4cf4EXxz5IiDX23J0C7tO7AqpybmXMrLHSAZ2HhiQZj6n28nw7sNpF9PO79dS4cGfBWo9MFBE+otIDGAH6vbGLgYmA4hIV6qb/PYDucDVIhIlItFUD5C4qIlPNV1N/1O43EFB4Jc9cmQ7iIqI4qahNwXkev4kImSlZbHq4CqOnvVfN29FVQVr89aG1b875X9+K1DGmArgYWAZ1cVlvjEmR0SeE5GZnsOWASdEZDvVfU7fN8acABYC+4BtwBZgizHmHX9lbU1cbhcd4zoyNHGo1VF8plvbbozsMTIgBcoYw7yceUxLmUbX+K5+v14g2NPtVJkqFm5f6LdrbD26leLy4rC6c1f+59exnsaY940xg4wxqcaYX3hee8YYs9TzZ2OM+a4xZpgx5jJjjMPzeqUx5pvGmKGer33XnzlbE6fbGZbDfG2pNpxuJ0WlRX69zrr8dRwsPBhSSxs1Jq1b9UK3/py0Gw5La6nAC6/fUqpBp86fYnvB9rD8FGtLtVFRVcF/Dv7Hr9dxZDuIiYzhxiE3+vU6gWZPs/N57ue4T7sbP7gZnG4nSQlJ9OnQxy/nV+FJC1QrsjpvNRBe/U81JiVPIj463q/NfJVVlczLmcd1A6+jQ1wHv13HClnp1XeE83Pm++X8LrcrLD8YKf/SAtWKuNwuIiWSsb3HWh3F52KjYrmm3zV+nQ/1ee7nHD57OKQn517KgM4DyOiVwbyceT4/d96ZPHJP54blByPlX1qgWhGn28mIHiNoG9PW6ih+YUuxsfvEbg4WHvTL+R3ZDuKj47l+0PV+Ob/VstKyWH9oPftO7vPpeXWBWNVcWqBaifLKctbmrQ3rXxK21Oo1jP3RzFdRVcHCHQuZOXhm2Bb4mkVvfX0X5cx1Eh8dz/Duw316XhX+tEC1EluObuF8xfmwbmYZ0nUISQlJfilQKw+s5Pi542E1eq+uPh36MCl5ks8n7TrdTsb2Hkt0ZLRPz6vCnxaoVsKZ65mgG8bDfEWEzNRMVhxYQUVVhU/P7ch2kBCbwPQB03163mBjT7ez7dg2co7l+OR8xWXFbD6yOaw/GCn/0QLVSrjyXCQnJJOUkGR1FL+ypdooLClkff56n52ztKKUt3e8zewhs0N6g0dv3DLsFiIkwmfNfOvy11FpKsO6aVn5jxaoVsAYgzPXGdZ3TzWm9J+CID5t5lu2bxmnS0+Hxdp7jenRrgeT+01mXs48jKm7tnPT1QyQCIetXVTgaYFqBdxn3OQX5Yf8Drre6BLfhYxeGT4dbu7IdtClTRem9J/is3MGs6y0LHaf2M3mI5tbfC6n28mwxGF0atPJB8lUa6MFqhZffGIMRq2h/6m2zNRM1uatpbCksMXnOld+jqW7lnLz0JtbTSf/TUNvIioiqsWDJapMFavzVmv/k2o2LVBU/yB9d9l3eXrl01ZH8QuX20Xb6LZc3v1yq6MEhC3VRqWpZOWBlS0+13u736O4vLhVNO/V6BLfBVuqDUeOo0Uf2nYU7KCwpFALlGo2LVBU7yxaVFrEr5y/utBmHk6cbifjksYRFRFldZSAGJ80nnYx7XzSD+XIcdCjXQ+u6nvRfplhzZ5mJ/d0Lmvy1jT7HDpBV7WUFiiP32X+jj4d+nDnojspLiu2Oo7PnC07y5ajW1pF/1ON6Mhoru1/Lcv2LWvRHcCZ0jO8t/s95gybQ2REpA8TBr9ZQ2YRGxnbotF8TreTxPhEBnQe4MNkqjXRAuXRPrY9r816jf2n9vPk8ietjuMza/PWUmWqWk3/U43M1EwOFh5k36nmL9uzZOcSSitLW1XzXo2E2ASuG3gd83PmU1lV2axz1CwQKyI+TqdaCy1QtVzd72oeH/84/7PhfwK2O6u/udwuBGF80nirowRUzbJHy/Yua/Y55uXMo0+HPq3ue1fDnm7n8NnDfJb7WZPfe6z4GHtO7tH+J9UiWqDq+MWUXzAscRj3LrmXU+dPWR2nxZxuJ2nd0ugY19HqKAGV2imV/h37N3u4+cnzJ1m2bxlZaVmt9g7gawO/Rtvots0azbfaXb21i/Y/qZbQAlVHXFQcc2+cy9Hio3zng+9YHadFKqsqW+0wXxHBlmpj5YGVlFeWN/n9b+94m4qqilbZvFejbUxbZg6eycLtC5v8PXS6ncRExjC612g/pVOtgRaoeozuNZr/vuq/+de2f7Fw+0Kr4zTb9oLtnCk902o/xWamZnK27GyzRqI5sh0M7DyQkT1G+iFZ6LCn2zlx/kSTh+w73U5G9xwd9ktDKf/SAnUJT13xFGN6jeHBdx/kyNkjVsdpFqfbM0G3Fd5BAUzuP5lIiWTZvqb1Qx05e4RVB1dhT7e32ua9GpmpmXSI7YAjx/tmvtKKUjYc2tBq/90p39ECdQnRkdHMnT2X4vJi7n/n/pBcZcLldtGtbTdSOqVYHcUSHeM6Mi5pXJMHvCzcvpAqU9Wqm/dqxEbFMnvobN7e8TalFaVevWfj4Y2UVZa12jt35TtaoBowpOsQfjXlV7y7+11e+eIVq+M0mdPtZFLypFZ9F2BLsbHh0AZOnDvh9Xvm5cwjvVs6wxKH+TFZ6LCn2TlTeoYP937o1fE6QVf5ihaoRnxn3HeY3G8yjy17jAOnDlgdx2tHzh5h/6n9rf6XROaATAyGFQdWeHW8+7Sbz3M/x56md081ru1/LV3ju3rdzOd0OxnQeQDd23X3czIV7rRANSJCInh11qsIwt1L7qbKVFkdySs1n2Jbez9ARq8MOsZ19Ho+1Pyc+QBkpYfvzrlNFR0ZzS1Db2HprqWNrrJijLkwQVepltIC5YW+Hfvypxl/4tMvP+UPa/5gdRyvuNwuYiNjGdVzlNVRLBUVEcWU/lP4aP9HXvUjOnIcZPTK0OV56rCn2zlXfo739rzX4HH7Tu3jWPGxVv/BSPmGFigv3TX8LmYNnsWPVvyI7QXbrY7TKKfbSUavDGKjYq2OYjlbqo28M3nsPL6zweP2ntzLhkMbtHmvHlf0uYKe7Xo2OmlX+5+UL2mB8pKI8PINL9M+tj13LLqjWZM/A6WkooSNhzbqLwmPmmWPGhvNV9O8Nydtjt8zhZrIiEjmpM3h/T3vc7rk9CWPc+Y66RDbQQeYKJ/QAtUE3dp24+XrX2bT4U38/NOfWx3nkjYc2kB5Vbk2s3j069iPQV0GNTofypHtYFLyJJI7JAcoWWixp9sprSxlya4llzzGlediQvIEIkR/taiW039FTTR76GzuHH4nv/jsF6zPX291nHrV7KCrd1D/x5Zi4z8H/3PJuTw5x3LYdmybzn1qwLje4+jboe8lm/kKSwrJOZajH4yUz2iBaoY/Tv8jPdv35I5Fd3C+/LzVcS7iynMxsPNAEtsmWh0laNhSbZyvOH9hdY265uXMI0IiuGXYLQFOFjpEBHu6neX7l9c7r2y1ezUGox+MlM9ogWqGjnEdeXXWq+w6sYunVjxldZyvqBnm29r2f2rM5P6TiY6IrrcfyhiDI9vB5H6T6dGuhwXpQoc93U5FVQVv73j7oq+53C4iJZKxvcdakEyFIy1QzTQ1ZSoPj3mYP679I6sOrLI6zgV7Tu7h+LnjrWoHXW+0i2nHxOSJ9fZDbT6ymT0n95CVpnOfGjO8+3AGdRlU76Rdp9vJ8B7DaRfTzoJkKhxpgWqB56c9z6Aug7h7yd0NjmwKpJr+J72Dupgt1cbmI5s5evboV153ZDuIiojipqE3WZQsdIgI9jQ7qw6s4nDR4QuvV1RVsDZ/rfY/KZ/ya4ESkekisktE9orIDy9xzBwR2S4iOSLy71qv9xGRj0Rkh+fr/fyZtTnio+OZe+Nc8s7k8fiyx62OA1Q3s3SM68iQrkOsjhJ0aoabf7z/4wuvGWNw5DiwpdroEt/FqmghJSs9C4P5ylY0W45s4Vz5OS1Qyqf8VqBEJBJ4EZgBDANuE5FhdY4ZCDwFTDLGpAGP1fryXODXxpihwFjgmL+ytsS4pHE8dcVTvLr5VZbsvPTw20Bxup1MTJ6ow3zrMarnKLq06fKVZr41eWvIPZ2rk3ObYFjiMC7vfjnzcuZdeE0n6Cp/8OdvsbHAXmPMfmNMGeAAZtU55n7gRWPMKQBjzDEATyGLMsYs97x+1hhzzo9ZW+SZq59hRI8RPPDuAxQUF1iW4+T5k+w4vkP7ny4hQiKYljqNj/b937JHjmwHsZGxzBpS95+maog9zY7T7ST3dC5Q/cEoOSFZ55Apn/JngeoNuGs9z/O8VtsgYJCIOEVkjYhMr/V6oYi8LSJfiMivPXdkXyEiD4jIBhHZUFBgXWGIiYzhjdlvUFhSyDff/aZle0etdq8GtP+pIbYUG0eLj7Lt2DYqqyqZv30+1w28joTYBKujhZSaxXRrVt/QBWKVP/izQNW3CVHd39xRwEDgGuA24O8i0tHz+pXAE8AYIAW4+6KTGfOyMSbDGJORmGjtnJ/0bun8fPLPWbRzEf/c+k9LMtQM8x3Ta4wl1w8F01KnAdXLHn2W+xlHzh7RybnNkNIphTG9xuDIduA+7cZ9xq39T8rn/Fmg8oDa9/tJwKF6jllijCk3xhwAdlFdsPKALzzNgxXAYiDol+X+7oTvckWfK3j4g4dxn3Y3/gYfc7qdjOw5krYxbQN+7VCRlJBEWmIay/Ytw5HtoG10W7428GtWxwpJ9nQ7Gw9vZO6WuYD2Pynf82eBWg8MFJH+IhID2IGldY5ZDEwGEJGuVDft7fe8t5OI1NwWXQsE/RLikRGRvH7j61RWVXLPknsCundUeWU56/LX6adYL9hSbXz25Wcs2L6AmYNnakFvpppFdX/5+S+Jj45neI/hFidS4cZvBcpz5/MwsAzYAcw3xuSIyHMiMtNz2DLghIhsB1YB3zfGnDDGVFLdvLdCRLZR3Vz4N39l9aWUTin8LvN3rDiwgv9Z/z8Bu+7mI5s5X3FeP8V6wZZqo7SylJPnT2rzXgskJSRxZZ8rKS4vZlzvcURFRFkdSYUZv/6LMsa8D7xf57Vnav3ZAN/1POq+dzlwuT/z+cv9o+5n8c7FPLn8SWypNgZ1GeT3a9asMacFqnFX9b2K2MhY4qLiyEzNtDpOSLOn2/ks9zO9c1d+oZNl/EBE+PvMvxMXFcedi+6koqrC79d0uV306dCHpIQkv18r1MVHx/OtjG/xxMQndEPHFspKy2JC0gRdZFf5hRYoP+nVvhcvfe0l1uav5fnPn/frtYwxON1O/RTbBL+f/nuevuppq2OEvC7xXXDd59L+J+UXWqD8KCs9C3u6nWc/eZYvDn/ht+vkns7lUNEhbd5TSoUVLVB+9uJ1L5IYn8idi++kpKLEL9eo6X/SOyilVDjRAuVnndt05h8z/0H2sWyeWfVM429oBpfbRdvotlzW/TK/nF8ppaygBSoAZgycwQOjHuA3rt/w2Zef+fz8TreT8UnjdZivUiqsaIEKkN9m/pb+nfpz1+K7KCot8tl5i0qL2Hp0q/Y/KaXCjhaoAGkX047Xb3ydg4UHeeKjJ3x23rX5a6kyVdr/pJQKO1qgAuiKPlfwxMQneHnTy3yw5wOfnNPldiEI45PG++R8SikVLLRABdhzk58jvVs69y29jxPnTrT4fE63k/Ru6XSI6+CDdEopFTy0QAVYXFQcb8x+g+PnjvPQ+w+16FyVVZWsdq/W5j2lVFjSAmWBET1G8JOrf8K8nHk4sh3NPk9OQQ5FZUU6QEIpFZa0QFnkB1f8gHG9x/Ht977NoaK622R5x5nrmaCrO+gqpcKQFiiLREVEMXf2XEoqSrhv6X3N2ibeleeie9vu9O/Y3w8JlVLKWlqgLDSoyyBemPYCH+79kL9tavp2V85cJ5P6TEJE/JBOKaWspQXKYt8e822mpkzlu8u+y76T+7x+3+GiwxwoPMDEJO1/UkqFJy1QFouQCF6Z+QpREVHctfguKqsqvXqfy+0CtP9JKRW+tEAFgeQOyfx5xp9xup38bvXvvHqPy+0iNjKWkT1G+jmdUkpZQwtUkPj65V/npqE38fSqp9l2dFujxzvdTsb0HqM7wiqlwpYWqCAhIvzv1/6XjnEduWPRHZRVll3y2PPl59l0eJNO0FVKhTUtUEEksW0iL1//MluObuG5T5675HEbDm2gvKpcJ+gqpcKaFqggM2vILO4ecTe//PyXrMlbU+8xNTvoaoFSSoUzLVBB6A+ZfyApIYk7F91JcVnxRV93uV0M6jKIrvFdLUinlFKBoQUqCHWI68Brs15jz8k9/PDjH37la8YYXG6X9j8ppcKeFqggNbn/ZB4d9yh/Wf8XPt7/8YXXd5/YzYnzJ7R5TykV9rRABbFfTvklQ7oO4Z4l91BYUgj8X/+T3kEppcKdFqgg1ia6DXNvnMvhosM88sEjQHX/U6e4TgzuOtjidEop5V9aoILcmN5j+PGVP+aNrW/w9o63cbqdTEyeSITo/zqlVHjT33Ih4OmrnmZUz1Hc/8797Dy+U/uflFKtghaoEBAdGc0bs9+4MORc+5+UUq2BFqgQMSxxGL/L/B39OvZjTO8xVsdRSim/k+bs5BqMMjIyzIYNG6yOoZRSqhEistEYk9HYcXoHpZRSKij5tUCJyHQR2SUie0Xkh5c4Zo6IbBeRHBH5d52vJYhIvoj8xZ85lVJKBZ8of51YRCKBF4FpQB6wXkSWGmO21zpmIPAUMMkYc0pEutU5zc+AT/yVUSmlVPDy5x3UWGCvMWa/MaYMcACz6hxzP/CiMeYUgDHmWM0XRGQ00B34yI8ZlVJKBSl/FqjegLvW8zzPa7UNAgaJiFNE1ojIdAARiQB+C3y/oQuIyAMiskFENhQUFPgwulJKKav5s0BJPa/VHTIYBQwErgFuA/4uIh2BbwPvG2PcNMAY87IxJsMYk5GYmOiDyEoppYKF3/qgqL5jSq71PAk4VM8xa4wx5cABEdlFdcGaAFwpIt8G2gExInLWGFPvQAullFLhx593UOuBgSLSX0RiADuwtM4xi4HJACLSleomv/3GmNuNMX2MMf2AJ4C5WpyUUqp18VuBMsZUAA8Dy4AdwHxjTI6IPCciMz2HLQNOiMh2YBXwfWPMCX9lUkopFTrCZiUJESkAvmzhaboCx30Qx59CISOERk7N6DuhkFMz+k5Lc/Y1xjQ6cCBsCpQviMgGb5bfsFIoZITQyKkZfScUcmpG3wlUTl3qSCmlVFDSAqWUUiooaYH6qpetDuCFUMgIoZFTM/pOKOTUjL4TkJzaB6WUUioo6R2UUkqpoKQFSimlVFDSAgWIyCsickxEsq3Ocikikiwiq0Rkh2fvrEetzlSXiMSJyDoR2eLJ+FOrM12KiESKyBci8q7VWS5FRA6KyDYR2SwiQbldtIh0FJGFIrLT829zgtWZ6hKRwZ7vYc3jjIg8ZnWuukTkcc/PTbaIvCkicVZnqktEHvXkywnE91D7oAARuQo4S/WSSulW56mPiPQEehpjNolIe2AjcGPt/bWsJiICtDXGnBWRaOBz4FFjzBqLo11ERL4LZAAJxpjrrc5THxE5CGQYY4J24pTTaY8AAAeBSURBVKaIvA58Zoz5u2dJs3hjTKHVuS7Fs09dPjDOGNPSif0+IyK9qf55GWaMOS8i86leMPs1a5P9HxFJp3rbpLFAGfAh8C1jzB5/XVPvoABjzKfASatzNMQYc9gYs8nz5yKql4+qu32JpUy1s56n0Z5H0H0CEpEk4GvA363OEspEJAG4CvgHgDGmLJiLk8cUYF8wFadaooA2IhIFxHPx4tpWG0r14t7nPEvZfQLM9ucFtUCFIBHpB4wE1lqb5GKeprPNwDFguTEm6DICfwCeBKqsDtIIA3wkIhtF5AGrw9QjBSgAXvU0l/5dRNpaHaoRduBNq0PUZYzJB34D5AKHgdPGmGDbrDUbuEpEuohIPHAdX92xwue0QIUYEWkHvAU8Zow5Y3WeuowxlcaYEVRvrzLW0ywQNETkeuCYMWaj1Vm8MMkYMwqYATzkaYoOJlHAKOAlY8xIoBgI2l0HPE2QM4EFVmepS0Q6Ub3jeH+gF9BWRL5ubaqvMsbsAJ4HllPdvLcFqPDnNbVAhRBPv85bwL+MMW9bnachnqae/wDTLY5S1yRgpqd/xwFcKyL/tDZS/Ywxhzz/PQYsorrtP5jkAXm17pIXUl2wgtUMYJMx5qjVQeoxFThgjCnw7I/3NjDR4kwXMcb8wxgzyhhzFdXdIn7rfwItUCHDMwDhH8AOY8zvrM5THxFJ9OyIjIi0ofqHbqe1qb7KGPOUMSbJs9eYHVhpjAmqT6oAItLWMxgGT7OZjeomlqBhjDkCuEVksOelKUDQDNqpx20EYfOeRy4wXkTiPT/rU6juZw4qItLN898+wE34+fvpzx11Q4aIvEn1tvNdRSQP+Ikx5h/WprrIJOAOYJunjwfgR8aY9y3MVFdP4HXPSKkIqvcAC9ph3EGuO7Co+ncVUcC/jTEfWhupXt8B/uVpPtsP3GNxnnp5+kymAd+0Okt9jDFrRWQhsInqZrMvCM5lj94SkS5AOfCQMeaUPy+mw8yVUkoFJW3iU0opFZS0QCmllApKWqCUUkoFJS1QSimlgpIWKKWUUkFJC5QKGyJiROS3tZ4/ISLP+ujcr4nILb44VyPXudWzKviqOq/38/z9vlPrtb+IyN2NnC9QuReKSIrnzwdFpKvnz6NF5ICIjBSR64N5hXsVfLRAqXBSCtxU88sxWHjmhXnrPuDbxpjJ9XztGPCoZ86R33kWLfXmuDQg0hizv87rl1O9ukSWMeYL4D2qV/GI93lYFZa0QKlwUkH15MbH636h7p2EiJz1/PcaEflEROaLyG4R+ZWI3C7V+1ptE5HUWqeZKiKfeY673vP+SBH5tYisF5GtIvLNWuddJSL/BrbVk+c2z/mzReR5z2vPAFcA/ysiv67n71cArADuqud893sybBGRt+oUgfpyx4nIq54MX4jIZM/rd4v8//buJjSqMwrj+P/oxk1xoSB2UU3R6WeIUEaR1o/ahcsI0kUNtS1WaPFrISIiLioobgRdKBU3ftBFLYhBShELFunGKFZQbF20tm4E96JizOninKl3bu5kEoRyHZ4fhExe3pn7DiQ58957OY/9YGbniUa1c83sskWO0i0zW1axriFguDT2FnAO+NTdRyC63RPtr2oZbyL1owIlveYIMGRmM6fwnAFgG9BPdOtouPtiIo5jS2HefGAFEdXxrUWg3Aai83QTaAIbzawv5y8Gdrv728WDmdmrRNPNVcAioGlma9x9L3ANGHL3HR3WegDYXrErO+vuTXcfIFrkbOiy7k0A7t5PtAA6ac8D8pYCn7n7KmAdcCEbAA8ANxjvfSKfrGgY2Ozuv5bGrwFVRU5kHBUo6SnZ4f0UsHUKT7uaeVtPgD+BVszBTeKfe8sZdx/LgLa/gDeJHnnrs/3UFWAWsDDnj7j73YrjNYFfsjHoKPAdkas0mfd3FxghCkfRu7lLuknsaN7psu4PgNP5mn8A/wCNnH/R3Vv5aFeBL/JaXn9mkZXNJXZ3RT8DX1YU0gdEt26RrlSgpBcdInYQxWyiUfL3PZtxFq/jPCk8Hiv8PEZ7v8pyXzAHDNji7ovyq6+Q4/Oww/pssm+kg/3ATtr/fk8QO5Z+4BugGBfead2d/LfuDPNcTqTQnjaz9RXzH5WOB7A5vx8tjc/I+SJdqUBJz8lP/2doP831N/BePh4k0n6n6mMzm5bXpV4H7gAXgK8tolAws4Z1D+27Aqwws9m5w/iESCedlNzx3Kb9Ws4rwP1cx9Ak1n25Nc/MGsBrOd7GzOYR+VnHiW76VXEavwMLSmNj+b7eMLO9hfEGNevKLvWlAiW96iBQvJvvOFEURoAldN7dTOQOUUh+Ar5y98fEdarbwHUzuwUco0tKgLvfB3YBl4jQt+vuXr7JoJt9RChkyx6i8F1kfMRJ1bqPAtPzlOD3wOd5irNsJXDDzH4D1gKHK+b8mPPa5OsNEnfubcrhD3O+SFfqZi4iL8Qi++sSkQD8bIJ5c4jYkI/+t8XJS00FSkRemJmtJsI0700wpwk8dfeqOwFFxlGBEhGRWtI1KBERqSUVKBERqSUVKBERqSUVKBERqSUVKBERqaV/AS+kd47zQJtbAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "plt.plot(range(1,Ks), mean_acc, 'g')\nplt.legend(('Accuracy ', '+/- 1xstd'))\nplt.ylabel('Accuracy ')\nplt.xlabel('Number of Nabors (K)')\nplt.tight_layout()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "The best accuracy was with 0.7428571428571429 with k= 7\n"
                }
            ],
            "source": "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n           metric_params=None, n_jobs=None, n_neighbors=7, p=2,\n           weights='uniform')"
                    },
                    "execution_count": 30,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "KNN = KNeighborsClassifier(n_neighbors = 7).fit(X_train, y_train)\nKNN"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {
                "scrolled": false
            },
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    },
                    "execution_count": 33,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.tree import DecisionTreeClassifier\n# prepare DT setting\nDT = DecisionTreeClassifier(criterion=\"entropy\", max_depth=4)\n# perform the test\nDT.fit(X_train, y_train)\nDT"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Support Vector Machine"
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n  shrinking=True, tol=0.001, verbose=False)"
                    },
                    "execution_count": 36,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn import svm\nsvm_Loan = svm.SVC(kernel='rbf')\nsvm_Loan.fit(X_train,y_train)\nsvm_Loan"
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {
                "scrolled": true
            },
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [],
            "source": "# The code was removed by Watson Studio for sharing."
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "LogisticRegression(C=0.001, class_weight=None, dual=False, fit_intercept=True,\n          intercept_scaling=1, max_iter=100, multi_class='warn',\n          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',\n          tol=0.0001, verbose=0, warm_start=False)"
                    },
                    "execution_count": 40,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.linear_model import LogisticRegression\nLR_Loan = LogisticRegression(C=0.001, solver='liblinear').fit(X_train, y_train)\nLR_Loan"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Model Evaluation using Test set"
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "First, download and load the test set:"
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-03-22 20:35:08--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2020-03-22 20:35:09 (315 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ],
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Test set for evaluation "
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                },
                "scrolled": false
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30       9/8/2016   \n1           5             5     PAIDOFF        300      7       9/9/2016   \n2          21            21     PAIDOFF       1000     30      9/10/2016   \n3          24            24     PAIDOFF       1000     30      9/10/2016   \n4          35            35     PAIDOFF        800     15      9/11/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   50              Bechalor  female  \n1  9/15/2016   35       Master or Above    male  \n2  10/9/2016   43  High School or Below  female  \n3  10/9/2016   26               college    male  \n4  9/25/2016   29              Bechalor    male  "
                    },
                    "execution_count": 43,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df = pd.read_csv('loan_test.csv')\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [],
            "source": "#similar preprocessing to the training set\ntest_df['due_date'] = pd.to_datetime(test_df['due_date'])\ntest_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\ntest_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\ntest_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ntest_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ntest_Feature = test_df[['Principal','terms','age','Gender','weekend']]\ntest_Feature = pd.concat([test_Feature,pd.get_dummies(test_df['education'])], axis=1)\ntest_Feature.drop(['Master or Above'], axis = 1,inplace=True)"
        },
        {
            "cell_type": "code",
            "execution_count": 45,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        1         0                     0   \n2       1000     30   43       1        1         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    },
                    "execution_count": 45,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "#Assign X similar to the train dataset\nX_loan_test = test_Feature\nX_loan_test[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 46,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y_loan_test = test_df['loan_status'].values\ny_loan_test[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 47,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                }
            ],
            "source": "X_loan_test = preprocessing.StandardScaler().fit(X_loan_test).transform(X_loan_test) #normalise"
        },
        {
            "cell_type": "code",
            "execution_count": 48,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "(54, 8)\n(54,)\n"
                }
            ],
            "source": "print (X_loan_test.shape)\nprint (y_loan_test.shape)"
        },
        {
            "cell_type": "code",
            "execution_count": 49,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "[0.67, 0.72, 0.8, 0.78]"
                    },
                    "execution_count": 49,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "#from sklearn.metrics import jaccard_similarity_score - already imported for Jaccard\nneigh_yhat = KNN.predict(X_loan_test)\njc_knn = round(jaccard_similarity_score(y_loan_test, neigh_yhat), 2)\n# evaluate Decision Trees\ndt_yhat = DT.predict(X_loan_test)\njc_dt = round(jaccard_similarity_score(y_loan_test, dt_yhat), 2)\n#evaluate SVM\nsvm_yhat = svm_Loan.predict(X_loan_test)\njc_svm = round(jaccard_similarity_score(y_loan_test, svm_yhat), 2)\n# evaluate Logistic Regression\nlr_yhat = LR_Loan.predict(X_loan_test)\njc_lr = round(jaccard_similarity_score(y_loan_test, lr_yhat), 2)\n\nlist_jc = [jc_knn, jc_dt, jc_svm, jc_lr]\nlist_jc"
        },
        {
            "cell_type": "code",
            "execution_count": 50,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "[0.63, 0.74, 0.76, 0.73]"
                    },
                    "execution_count": 50,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "#from sklearn.metrics import f1_score for f1score\nfs_knn = round(f1_score(y_loan_test,neigh_yhat,average='weighted'),2)\nfs_dt = round(f1_score(y_loan_test,dt_yhat,average='weighted'),2)\nfs_svm = round(f1_score(y_loan_test,svm_yhat,average='weighted'),2)\nfs_lr = round(f1_score(y_loan_test,lr_yhat,average='weighted'),2)\nlist_fs = [fs_knn,fs_dt,fs_svm,fs_lr]\nlist_fs "
        },
        {
            "cell_type": "code",
            "execution_count": 51,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "['NA', 'NA', 'NA', 0.67]"
                    },
                    "execution_count": 51,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# Logloss\nfrom sklearn.metrics import log_loss\nlr_prob = LR_Loan.predict_proba(X_loan_test)\nlist_ll = ['NA','NA','NA',round(log_loss(y_loan_test,lr_prob),2)]\nlist_ll"
        },
        {
            "cell_type": "code",
            "execution_count": 52,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th>Algorithm</th>\n      <th>Jaccard</th>\n      <th>F1-score</th>\n      <th>LogLoss</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>KNN</th>\n      <td>0.67</td>\n      <td>0.63</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>Decision Tree</th>\n      <td>0.72</td>\n      <td>0.74</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>SVM</th>\n      <td>0.80</td>\n      <td>0.76</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>Logistic Regression</th>\n      <td>0.78</td>\n      <td>0.73</td>\n      <td>0.67</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "Algorithm            Jaccard  F1-score LogLoss\nKNN                     0.67      0.63      NA\nDecision Tree           0.72      0.74      NA\nSVM                     0.80      0.76      NA\nLogistic Regression     0.78      0.73    0.67"
                    },
                    "execution_count": 52,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.DataFrame(list_jc, index=['KNN','Decision Tree','SVM','Logistic Regression']) #create the table\ndf.columns = ['Jaccard']\ndf.insert(loc=1, column='F1-score', value=list_fs)\ndf.insert(loc=2, column='LogLoss', value=list_ll)\ndf.columns.name = 'Algorithm'\ndf"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}