hw-03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Тема “Обучение с учителем”\n",
    "## Задание 1\n",
    "Импортируйте библиотеки pandas и numpy.\n",
    "\n",
    "Загрузите \"Boston House Prices dataset\" из встроенных наборов данных библиотеки sklearn. Создайте датафреймы X и y из этих данных.\n",
    "\n",
    "Разбейте эти датафреймы на тренировочные **(X_train, y_train)** и тестовые **(X_test, y_test)** с помощью функции train_test_split так, чтобы размер тестовой выборки\n",
    "составлял 30% от всех данных, при этом аргумент **random_state** должен быть равен 42.\n",
    "\n",
    "Создайте модель линейной регрессии под названием lr с помощью класса LinearRegression из модуля **sklearn.linear_model**.\n",
    "\n",
    "Обучите модель на тренировочных данных (используйте все признаки) и сделайте предсказание на тестовых.\n",
    "\n",
    "Вычислите R2 полученных предсказаний с помощью **r2_score** из модуля sklearn.metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Загрузим данные"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\_code\\pyenv\\math\\lib\\site-packages\\sklearn\\utils\\deprecation.py:87: FutureWarning: Function load_boston is deprecated; `load_boston` is deprecated in 1.0 and will be removed in 1.2.\n",
      "\n",
      "    The Boston housing prices dataset has an ethical problem. You can refer to\n",
      "    the documentation of this function for further details.\n",
      "\n",
      "    The scikit-learn maintainers therefore strongly discourage the use of this\n",
      "    dataset unless the purpose of the code is to study and educate about\n",
      "    ethical issues in data science and machine learning.\n",
      "\n",
      "    In this special case, you can fetch the dataset from the original\n",
      "    source::\n",
      "\n",
      "        import pandas as pd\n",
      "        import numpy as np\n",
      "\n",
      "        data_url = \"http://lib.stat.cmu.edu/datasets/boston\"\n",
      "        raw_df = pd.read_csv(data_url, sep=\"\\s+\", skiprows=22, header=None)\n",
      "        data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])\n",
      "        target = raw_df.values[1::2, 2]\n",
      "\n",
      "    Alternative datasets include the California housing dataset (i.e.\n",
      "    :func:`~sklearn.datasets.fetch_california_housing`) and the Ames housing\n",
      "    dataset. You can load the datasets as follows::\n",
      "\n",
      "        from sklearn.datasets import fetch_california_housing\n",
      "        housing = fetch_california_housing()\n",
      "\n",
      "    for the California housing dataset and::\n",
      "\n",
      "        from sklearn.datasets import fetch_openml\n",
      "        housing = fetch_openml(name=\"house_prices\", as_frame=True)\n",
      "\n",
      "    for the Ames housing dataset.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# Загрузим данные датасета\n",
    "from sklearn.datasets import load_boston\n",
    "boston = load_boston()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Загрузим фичи для анализа\n",
    "df = pd.DataFrame(boston.data, columns=boston.feature_names)\n",
    "# Добавим целевую переменную - цены на недвижимость\n",
    "df['PRICE'] = boston.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _boston_dataset:\n",
      "\n",
      "Boston house prices dataset\n",
      "---------------------------\n",
      "\n",
      "**Data Set Characteristics:**  \n",
      "\n",
      "    :Number of Instances: 506 \n",
      "\n",
      "    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.\n",
      "\n",
      "    :Attribute Information (in order):\n",
      "        - CRIM     per capita crime rate by town\n",
      "        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
      "        - INDUS    proportion of non-retail business acres per town\n",
      "        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
      "        - NOX      nitric oxides concentration (parts per 10 million)\n",
      "        - RM       average number of rooms per dwelling\n",
      "        - AGE      proportion of owner-occupied units built prior to 1940\n",
      "        - DIS      weighted distances to five Boston employment centres\n",
      "        - RAD      index of accessibility to radial highways\n",
      "        - TAX      full-value property-tax rate per $10,000\n",
      "        - PTRATIO  pupil-teacher ratio by town\n",
      "        - B        1000(Bk - 0.63)^2 where Bk is the proportion of black people by town\n",
      "        - LSTAT    % lower status of the population\n",
      "        - MEDV     Median value of owner-occupied homes in $1000's\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "    :Creator: Harrison, D. and Rubinfeld, D.L.\n",
      "\n",
      "This is a copy of UCI ML housing dataset.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/housing/\n",
      "\n",
      "\n",
      "This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.\n",
      "\n",
      "The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic\n",
      "prices and the demand for clean air', J. Environ. Economics & Management,\n",
      "vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics\n",
      "...', Wiley, 1980.   N.B. Various transformations are used in the table on\n",
      "pages 244-261 of the latter.\n",
      "\n",
      "The Boston house-price data has been used in many machine learning papers that address regression\n",
      "problems.   \n",
      "     \n",
      ".. topic:: References\n",
      "\n",
      "   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.\n",
      "   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Посмотрим его описание\n",
    "print(boston.DESCR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "      crim    zn  indus  chas    nox     rm   age     dis  rad    tax  \\\n0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n\n   ptratio       b  lstat  price  \n0     15.3  396.90   4.98   24.0  \n1     17.8  396.90   9.14   21.6  \n2     17.8  392.83   4.03   34.7  \n3     18.7  394.63   2.94   33.4  \n4     18.7  396.90   5.33   36.2  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>crim</th>\n      <th>zn</th>\n      <th>indus</th>\n      <th>chas</th>\n      <th>nox</th>\n      <th>rm</th>\n      <th>age</th>\n      <th>dis</th>\n      <th>rad</th>\n      <th>tax</th>\n      <th>ptratio</th>\n      <th>b</th>\n      <th>lstat</th>\n      <th>price</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.00632</td>\n      <td>18.0</td>\n      <td>2.31</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>6.575</td>\n      <td>65.2</td>\n      <td>4.0900</td>\n      <td>1.0</td>\n      <td>296.0</td>\n      <td>15.3</td>\n      <td>396.90</td>\n      <td>4.98</td>\n      <td>24.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.02731</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>6.421</td>\n      <td>78.9</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>396.90</td>\n      <td>9.14</td>\n      <td>21.6</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.02729</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>7.185</td>\n      <td>61.1</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>392.83</td>\n      <td>4.03</td>\n      <td>34.7</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.03237</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>6.998</td>\n      <td>45.8</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>394.63</td>\n      <td>2.94</td>\n      <td>33.4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.06905</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>7.147</td>\n      <td>54.2</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>396.90</td>\n      <td>5.33</td>\n      <td>36.2</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Посмотрим на данные\n",
    "df.columns = map(str.lower, df.columns)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Обработаем данные"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 506 entries, 0 to 505\n",
      "Data columns (total 14 columns):\n",
      " #   Column   Non-Null Count  Dtype  \n",
      "---  ------   --------------  -----  \n",
      " 0   crim     506 non-null    float64\n",
      " 1   zn       506 non-null    float64\n",
      " 2   indus    506 non-null    float64\n",
      " 3   chas     506 non-null    float64\n",
      " 4   nox      506 non-null    float64\n",
      " 5   rm       506 non-null    float64\n",
      " 6   age      506 non-null    float64\n",
      " 7   dis      506 non-null    float64\n",
      " 8   rad      506 non-null    float64\n",
      " 9   tax      506 non-null    float64\n",
      " 10  ptratio  506 non-null    float64\n",
      " 11  b        506 non-null    float64\n",
      " 12  lstat    506 non-null    float64\n",
      " 13  price    506 non-null    float64\n",
      "dtypes: float64(14)\n",
      "memory usage: 55.5 KB\n"
     ]
    }
   ],
   "source": [
    "# Посмотрим на типы переменных и количество данных\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Стандартные характеристики распределение (медиана, мода, минимальное значение, максимальное)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "         count        mean         std        min         25%        50%  \\\ncrim     506.0    3.613524    8.601545    0.00632    0.082045    0.25651   \nzn       506.0   11.363636   23.322453    0.00000    0.000000    0.00000   \nindus    506.0   11.136779    6.860353    0.46000    5.190000    9.69000   \nchas     506.0    0.069170    0.253994    0.00000    0.000000    0.00000   \nnox      506.0    0.554695    0.115878    0.38500    0.449000    0.53800   \nrm       506.0    6.284634    0.702617    3.56100    5.885500    6.20850   \nage      506.0   68.574901   28.148861    2.90000   45.025000   77.50000   \ndis      506.0    3.795043    2.105710    1.12960    2.100175    3.20745   \nrad      506.0    9.549407    8.707259    1.00000    4.000000    5.00000   \ntax      506.0  408.237154  168.537116  187.00000  279.000000  330.00000   \nptratio  506.0   18.455534    2.164946   12.60000   17.400000   19.05000   \nb        506.0  356.674032   91.294864    0.32000  375.377500  391.44000   \nlstat    506.0   12.653063    7.141062    1.73000    6.950000   11.36000   \nprice    506.0   22.532806    9.197104    5.00000   17.025000   21.20000   \n\n                75%       max  \ncrim       3.677083   88.9762  \nzn        12.500000  100.0000  \nindus     18.100000   27.7400  \nchas       0.000000    1.0000  \nnox        0.624000    0.8710  \nrm         6.623500    8.7800  \nage       94.075000  100.0000  \ndis        5.188425   12.1265  \nrad       24.000000   24.0000  \ntax      666.000000  711.0000  \nptratio   20.200000   22.0000  \nb        396.225000  396.9000  \nlstat     16.955000   37.9700  \nprice     25.000000   50.0000  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>count</th>\n      <th>mean</th>\n      <th>std</th>\n      <th>min</th>\n      <th>25%</th>\n      <th>50%</th>\n      <th>75%</th>\n      <th>max</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>crim</th>\n      <td>506.0</td>\n      <td>3.613524</td>\n      <td>8.601545</td>\n      <td>0.00632</td>\n      <td>0.082045</td>\n      <td>0.25651</td>\n      <td>3.677083</td>\n      <td>88.9762</td>\n    </tr>\n    <tr>\n      <th>zn</th>\n      <td>506.0</td>\n      <td>11.363636</td>\n      <td>23.322453</td>\n      <td>0.00000</td>\n      <td>0.000000</td>\n      <td>0.00000</td>\n      <td>12.500000</td>\n      <td>100.0000</td>\n    </tr>\n    <tr>\n      <th>indus</th>\n      <td>506.0</td>\n      <td>11.136779</td>\n      <td>6.860353</td>\n      <td>0.46000</td>\n      <td>5.190000</td>\n      <td>9.69000</td>\n      <td>18.100000</td>\n      <td>27.7400</td>\n    </tr>\n    <tr>\n      <th>chas</th>\n      <td>506.0</td>\n      <td>0.069170</td>\n      <td>0.253994</td>\n      <td>0.00000</td>\n      <td>0.000000</td>\n      <td>0.00000</td>\n      <td>0.000000</td>\n      <td>1.0000</td>\n    </tr>\n    <tr>\n      <th>nox</th>\n      <td>506.0</td>\n      <td>0.554695</td>\n      <td>0.115878</td>\n      <td>0.38500</td>\n      <td>0.449000</td>\n      <td>0.53800</td>\n      <td>0.624000</td>\n      <td>0.8710</td>\n    </tr>\n    <tr>\n      <th>rm</th>\n      <td>506.0</td>\n      <td>6.284634</td>\n      <td>0.702617</td>\n      <td>3.56100</td>\n      <td>5.885500</td>\n      <td>6.20850</td>\n      <td>6.623500</td>\n      <td>8.7800</td>\n    </tr>\n    <tr>\n      <th>age</th>\n      <td>506.0</td>\n      <td>68.574901</td>\n      <td>28.148861</td>\n      <td>2.90000</td>\n      <td>45.025000</td>\n      <td>77.50000</td>\n      <td>94.075000</td>\n      <td>100.0000</td>\n    </tr>\n    <tr>\n      <th>dis</th>\n      <td>506.0</td>\n      <td>3.795043</td>\n      <td>2.105710</td>\n      <td>1.12960</td>\n      <td>2.100175</td>\n      <td>3.20745</td>\n      <td>5.188425</td>\n      <td>12.1265</td>\n    </tr>\n    <tr>\n      <th>rad</th>\n      <td>506.0</td>\n      <td>9.549407</td>\n      <td>8.707259</td>\n      <td>1.00000</td>\n      <td>4.000000</td>\n      <td>5.00000</td>\n      <td>24.000000</td>\n      <td>24.0000</td>\n    </tr>\n    <tr>\n      <th>tax</th>\n      <td>506.0</td>\n      <td>408.237154</td>\n      <td>168.537116</td>\n      <td>187.00000</td>\n      <td>279.000000</td>\n      <td>330.00000</td>\n      <td>666.000000</td>\n      <td>711.0000</td>\n    </tr>\n    <tr>\n      <th>ptratio</th>\n      <td>506.0</td>\n      <td>18.455534</td>\n      <td>2.164946</td>\n      <td>12.60000</td>\n      <td>17.400000</td>\n      <td>19.05000</td>\n      <td>20.200000</td>\n      <td>22.0000</td>\n    </tr>\n    <tr>\n      <th>b</th>\n      <td>506.0</td>\n      <td>356.674032</td>\n      <td>91.294864</td>\n      <td>0.32000</td>\n      <td>375.377500</td>\n      <td>391.44000</td>\n      <td>396.225000</td>\n      <td>396.9000</td>\n    </tr>\n    <tr>\n      <th>lstat</th>\n      <td>506.0</td>\n      <td>12.653063</td>\n      <td>7.141062</td>\n      <td>1.73000</td>\n      <td>6.950000</td>\n      <td>11.36000</td>\n      <td>16.955000</td>\n      <td>37.9700</td>\n    </tr>\n    <tr>\n      <th>price</th>\n      <td>506.0</td>\n      <td>22.532806</td>\n      <td>9.197104</td>\n      <td>5.00000</td>\n      <td>17.025000</td>\n      <td>21.20000</td>\n      <td>25.000000</td>\n      <td>50.0000</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Разделим данные"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "Index(['crim', 'zn', 'indus', 'chas', 'nox', 'rm', 'age', 'dis', 'rad', 'tax',\n       'ptratio', 'b', 'lstat', 'price'],\n      dtype='object')"
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Разделим колонки с признаками и колонку с целевой переменной"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "X = df[['crim', 'zn', 'indus', 'chas', 'nox', 'rm', 'age', 'dis', 'rad', 'tax',\n",
    "       'ptratio', 'b', 'lstat']]\n",
    "y = df['price']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "        crim    zn  indus  chas    nox     rm   age     dis  rad    tax  \\\n0    0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n1    0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n2    0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n3    0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n4    0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n..       ...   ...    ...   ...    ...    ...   ...     ...  ...    ...   \n501  0.06263   0.0  11.93   0.0  0.573  6.593  69.1  2.4786  1.0  273.0   \n502  0.04527   0.0  11.93   0.0  0.573  6.120  76.7  2.2875  1.0  273.0   \n503  0.06076   0.0  11.93   0.0  0.573  6.976  91.0  2.1675  1.0  273.0   \n504  0.10959   0.0  11.93   0.0  0.573  6.794  89.3  2.3889  1.0  273.0   \n505  0.04741   0.0  11.93   0.0  0.573  6.030  80.8  2.5050  1.0  273.0   \n\n     ptratio       b  lstat  \n0       15.3  396.90   4.98  \n1       17.8  396.90   9.14  \n2       17.8  392.83   4.03  \n3       18.7  394.63   2.94  \n4       18.7  396.90   5.33  \n..       ...     ...    ...  \n501     21.0  391.99   9.67  \n502     21.0  396.90   9.08  \n503     21.0  396.90   5.64  \n504     21.0  393.45   6.48  \n505     21.0  396.90   7.88  \n\n[506 rows x 13 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>crim</th>\n      <th>zn</th>\n      <th>indus</th>\n      <th>chas</th>\n      <th>nox</th>\n      <th>rm</th>\n      <th>age</th>\n      <th>dis</th>\n      <th>rad</th>\n      <th>tax</th>\n      <th>ptratio</th>\n      <th>b</th>\n      <th>lstat</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.00632</td>\n      <td>18.0</td>\n      <td>2.31</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>6.575</td>\n      <td>65.2</td>\n      <td>4.0900</td>\n      <td>1.0</td>\n      <td>296.0</td>\n      <td>15.3</td>\n      <td>396.90</td>\n      <td>4.98</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.02731</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>6.421</td>\n      <td>78.9</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>396.90</td>\n      <td>9.14</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.02729</td>\n      <td>0.0</td>\n      <td>7.07</td>\n      <td>0.0</td>\n      <td>0.469</td>\n      <td>7.185</td>\n      <td>61.1</td>\n      <td>4.9671</td>\n      <td>2.0</td>\n      <td>242.0</td>\n      <td>17.8</td>\n      <td>392.83</td>\n      <td>4.03</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.03237</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>6.998</td>\n      <td>45.8</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>394.63</td>\n      <td>2.94</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.06905</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>7.147</td>\n      <td>54.2</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>396.90</td>\n      <td>5.33</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>501</th>\n      <td>0.06263</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.593</td>\n      <td>69.1</td>\n      <td>2.4786</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>391.99</td>\n      <td>9.67</td>\n    </tr>\n    <tr>\n      <th>502</th>\n      <td>0.04527</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.120</td>\n      <td>76.7</td>\n      <td>2.2875</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>9.08</td>\n    </tr>\n    <tr>\n      <th>503</th>\n      <td>0.06076</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.976</td>\n      <td>91.0</td>\n      <td>2.1675</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>5.64</td>\n    </tr>\n    <tr>\n      <th>504</th>\n      <td>0.10959</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.794</td>\n      <td>89.3</td>\n      <td>2.3889</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>393.45</td>\n      <td>6.48</td>\n    </tr>\n    <tr>\n      <th>505</th>\n      <td>0.04741</td>\n      <td>0.0</td>\n      <td>11.93</td>\n      <td>0.0</td>\n      <td>0.573</td>\n      <td>6.030</td>\n      <td>80.8</td>\n      <td>2.5050</td>\n      <td>1.0</td>\n      <td>273.0</td>\n      <td>21.0</td>\n      <td>396.90</td>\n      <td>7.88</td>\n    </tr>\n  </tbody>\n</table>\n<p>506 rows × 13 columns</p>\n</div>"
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0      24.0\n1      21.6\n2      34.7\n3      33.4\n4      36.2\n       ... \n501    22.4\n502    20.6\n503    23.9\n504    22.0\n505    11.9\nName: price, Length: 506, dtype: float64"
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Pазобъём выборку на обучающую и тестовую. Качество модели нужно оценивать по выборке, которая не участвовала в обучении."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X.copy(), y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "         crim    zn  indus  chas    nox     rm   age     dis   rad    tax  \\\n5     0.02985   0.0   2.18   0.0  0.458  6.430  58.7  6.0622   3.0  222.0   \n116   0.13158   0.0  10.01   0.0  0.547  6.176  72.5  2.7301   6.0  432.0   \n45    0.17142   0.0   6.91   0.0  0.448  5.682  33.8  5.1004   3.0  233.0   \n16    1.05393   0.0   8.14   0.0  0.538  5.935  29.3  4.4986   4.0  307.0   \n468  15.57570   0.0  18.10   0.0  0.580  5.926  71.0  2.9084  24.0  666.0   \n..        ...   ...    ...   ...    ...    ...   ...     ...   ...    ...   \n106   0.17120   0.0   8.56   0.0  0.520  5.836  91.9  2.2110   5.0  384.0   \n270   0.29916  20.0   6.96   0.0  0.464  5.856  42.1  4.4290   3.0  223.0   \n348   0.01501  80.0   2.01   0.0  0.435  6.635  29.7  8.3440   4.0  280.0   \n435  11.16040   0.0  18.10   0.0  0.740  6.629  94.6  2.1247  24.0  666.0   \n102   0.22876   0.0   8.56   0.0  0.520  6.405  85.4  2.7147   5.0  384.0   \n\n     ptratio       b  lstat  \n5       18.7  394.12   5.21  \n116     17.8  393.30  12.04  \n45      17.9  396.90  10.21  \n16      21.0  386.85   6.58  \n468     20.2  368.74  18.13  \n..       ...     ...    ...  \n106     20.9  395.67  18.66  \n270     18.6  388.65  13.00  \n348     17.0  390.94   5.99  \n435     20.2  109.85  23.27  \n102     20.9   70.80  10.63  \n\n[354 rows x 13 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>crim</th>\n      <th>zn</th>\n      <th>indus</th>\n      <th>chas</th>\n      <th>nox</th>\n      <th>rm</th>\n      <th>age</th>\n      <th>dis</th>\n      <th>rad</th>\n      <th>tax</th>\n      <th>ptratio</th>\n      <th>b</th>\n      <th>lstat</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>5</th>\n      <td>0.02985</td>\n      <td>0.0</td>\n      <td>2.18</td>\n      <td>0.0</td>\n      <td>0.458</td>\n      <td>6.430</td>\n      <td>58.7</td>\n      <td>6.0622</td>\n      <td>3.0</td>\n      <td>222.0</td>\n      <td>18.7</td>\n      <td>394.12</td>\n      <td>5.21</td>\n    </tr>\n    <tr>\n      <th>116</th>\n      <td>0.13158</td>\n      <td>0.0</td>\n      <td>10.01</td>\n      <td>0.0</td>\n      <td>0.547</td>\n      <td>6.176</td>\n      <td>72.5</td>\n      <td>2.7301</td>\n      <td>6.0</td>\n      <td>432.0</td>\n      <td>17.8</td>\n      <td>393.30</td>\n      <td>12.04</td>\n    </tr>\n    <tr>\n      <th>45</th>\n      <td>0.17142</td>\n      <td>0.0</td>\n      <td>6.91</td>\n      <td>0.0</td>\n      <td>0.448</td>\n      <td>5.682</td>\n      <td>33.8</td>\n      <td>5.1004</td>\n      <td>3.0</td>\n      <td>233.0</td>\n      <td>17.9</td>\n      <td>396.90</td>\n      <td>10.21</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>1.05393</td>\n      <td>0.0</td>\n      <td>8.14</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>5.935</td>\n      <td>29.3</td>\n      <td>4.4986</td>\n      <td>4.0</td>\n      <td>307.0</td>\n      <td>21.0</td>\n      <td>386.85</td>\n      <td>6.58</td>\n    </tr>\n    <tr>\n      <th>468</th>\n      <td>15.57570</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.580</td>\n      <td>5.926</td>\n      <td>71.0</td>\n      <td>2.9084</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>368.74</td>\n      <td>18.13</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>106</th>\n      <td>0.17120</td>\n      <td>0.0</td>\n      <td>8.56</td>\n      <td>0.0</td>\n      <td>0.520</td>\n      <td>5.836</td>\n      <td>91.9</td>\n      <td>2.2110</td>\n      <td>5.0</td>\n      <td>384.0</td>\n      <td>20.9</td>\n      <td>395.67</td>\n      <td>18.66</td>\n    </tr>\n    <tr>\n      <th>270</th>\n      <td>0.29916</td>\n      <td>20.0</td>\n      <td>6.96</td>\n      <td>0.0</td>\n      <td>0.464</td>\n      <td>5.856</td>\n      <td>42.1</td>\n      <td>4.4290</td>\n      <td>3.0</td>\n      <td>223.0</td>\n      <td>18.6</td>\n      <td>388.65</td>\n      <td>13.00</td>\n    </tr>\n    <tr>\n      <th>348</th>\n      <td>0.01501</td>\n      <td>80.0</td>\n      <td>2.01</td>\n      <td>0.0</td>\n      <td>0.435</td>\n      <td>6.635</td>\n      <td>29.7</td>\n      <td>8.3440</td>\n      <td>4.0</td>\n      <td>280.0</td>\n      <td>17.0</td>\n      <td>390.94</td>\n      <td>5.99</td>\n    </tr>\n    <tr>\n      <th>435</th>\n      <td>11.16040</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.740</td>\n      <td>6.629</td>\n      <td>94.6</td>\n      <td>2.1247</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>109.85</td>\n      <td>23.27</td>\n    </tr>\n    <tr>\n      <th>102</th>\n      <td>0.22876</td>\n      <td>0.0</td>\n      <td>8.56</td>\n      <td>0.0</td>\n      <td>0.520</td>\n      <td>6.405</td>\n      <td>85.4</td>\n      <td>2.7147</td>\n      <td>5.0</td>\n      <td>384.0</td>\n      <td>20.9</td>\n      <td>70.80</td>\n      <td>10.63</td>\n    </tr>\n  </tbody>\n</table>\n<p>354 rows × 13 columns</p>\n</div>"
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "        crim    zn  indus  chas    nox     rm    age     dis   rad    tax  \\\n173  0.09178   0.0   4.05   0.0  0.510  6.416   84.1  2.6463   5.0  296.0   \n274  0.05644  40.0   6.41   1.0  0.447  6.758   32.9  4.0776   4.0  254.0   \n491  0.10574   0.0  27.74   0.0  0.609  5.983   98.8  1.8681   4.0  711.0   \n72   0.09164   0.0  10.81   0.0  0.413  6.065    7.8  5.2873   4.0  305.0   \n452  5.09017   0.0  18.10   0.0  0.713  6.297   91.8  2.3682  24.0  666.0   \n..       ...   ...    ...   ...    ...    ...    ...     ...   ...    ...   \n441  9.72418   0.0  18.10   0.0  0.740  6.406   97.2  2.0651  24.0  666.0   \n23   0.98843   0.0   8.14   0.0  0.538  5.813  100.0  4.0952   4.0  307.0   \n225  0.52693   0.0   6.20   0.0  0.504  8.725   83.0  2.8944   8.0  307.0   \n433  5.58107   0.0  18.10   0.0  0.713  6.436   87.9  2.3158  24.0  666.0   \n447  9.92485   0.0  18.10   0.0  0.740  6.251   96.6  2.1980  24.0  666.0   \n\n     ptratio       b  lstat  \n173     16.6  395.50   9.04  \n274     17.6  396.90   3.53  \n491     20.1  390.11  18.07  \n72      19.2  390.91   5.52  \n452     20.2  385.09  17.27  \n..       ...     ...    ...  \n441     20.2  385.96  19.52  \n23      21.0  394.54  19.88  \n225     17.4  382.00   4.63  \n433     20.2  100.19  16.22  \n447     20.2  388.52  16.44  \n\n[152 rows x 13 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>crim</th>\n      <th>zn</th>\n      <th>indus</th>\n      <th>chas</th>\n      <th>nox</th>\n      <th>rm</th>\n      <th>age</th>\n      <th>dis</th>\n      <th>rad</th>\n      <th>tax</th>\n      <th>ptratio</th>\n      <th>b</th>\n      <th>lstat</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>173</th>\n      <td>0.09178</td>\n      <td>0.0</td>\n      <td>4.05</td>\n      <td>0.0</td>\n      <td>0.510</td>\n      <td>6.416</td>\n      <td>84.1</td>\n      <td>2.6463</td>\n      <td>5.0</td>\n      <td>296.0</td>\n      <td>16.6</td>\n      <td>395.50</td>\n      <td>9.04</td>\n    </tr>\n    <tr>\n      <th>274</th>\n      <td>0.05644</td>\n      <td>40.0</td>\n      <td>6.41</td>\n      <td>1.0</td>\n      <td>0.447</td>\n      <td>6.758</td>\n      <td>32.9</td>\n      <td>4.0776</td>\n      <td>4.0</td>\n      <td>254.0</td>\n      <td>17.6</td>\n      <td>396.90</td>\n      <td>3.53</td>\n    </tr>\n    <tr>\n      <th>491</th>\n      <td>0.10574</td>\n      <td>0.0</td>\n      <td>27.74</td>\n      <td>0.0</td>\n      <td>0.609</td>\n      <td>5.983</td>\n      <td>98.8</td>\n      <td>1.8681</td>\n      <td>4.0</td>\n      <td>711.0</td>\n      <td>20.1</td>\n      <td>390.11</td>\n      <td>18.07</td>\n    </tr>\n    <tr>\n      <th>72</th>\n      <td>0.09164</td>\n      <td>0.0</td>\n      <td>10.81</td>\n      <td>0.0</td>\n      <td>0.413</td>\n      <td>6.065</td>\n      <td>7.8</td>\n      <td>5.2873</td>\n      <td>4.0</td>\n      <td>305.0</td>\n      <td>19.2</td>\n      <td>390.91</td>\n      <td>5.52</td>\n    </tr>\n    <tr>\n      <th>452</th>\n      <td>5.09017</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.713</td>\n      <td>6.297</td>\n      <td>91.8</td>\n      <td>2.3682</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>385.09</td>\n      <td>17.27</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>441</th>\n      <td>9.72418</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.740</td>\n      <td>6.406</td>\n      <td>97.2</td>\n      <td>2.0651</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>385.96</td>\n      <td>19.52</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>0.98843</td>\n      <td>0.0</td>\n      <td>8.14</td>\n      <td>0.0</td>\n      <td>0.538</td>\n      <td>5.813</td>\n      <td>100.0</td>\n      <td>4.0952</td>\n      <td>4.0</td>\n      <td>307.0</td>\n      <td>21.0</td>\n      <td>394.54</td>\n      <td>19.88</td>\n    </tr>\n    <tr>\n      <th>225</th>\n      <td>0.52693</td>\n      <td>0.0</td>\n      <td>6.20</td>\n      <td>0.0</td>\n      <td>0.504</td>\n      <td>8.725</td>\n      <td>83.0</td>\n      <td>2.8944</td>\n      <td>8.0</td>\n      <td>307.0</td>\n      <td>17.4</td>\n      <td>382.00</td>\n      <td>4.63</td>\n    </tr>\n    <tr>\n      <th>433</th>\n      <td>5.58107</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.713</td>\n      <td>6.436</td>\n      <td>87.9</td>\n      <td>2.3158</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>100.19</td>\n      <td>16.22</td>\n    </tr>\n    <tr>\n      <th>447</th>\n      <td>9.92485</td>\n      <td>0.0</td>\n      <td>18.10</td>\n      <td>0.0</td>\n      <td>0.740</td>\n      <td>6.251</td>\n      <td>96.6</td>\n      <td>2.1980</td>\n      <td>24.0</td>\n      <td>666.0</td>\n      <td>20.2</td>\n      <td>388.52</td>\n      <td>16.44</td>\n    </tr>\n  </tbody>\n</table>\n<p>152 rows × 13 columns</p>\n</div>"
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "5      28.7\n116    21.2\n45     19.3\n16     23.1\n468    19.1\n       ... \n106    19.5\n270    21.1\n348    24.5\n435    13.4\n102    18.6\nName: price, Length: 354, dtype: float64"
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "173    23.6\n274    32.4\n491    13.6\n72     22.8\n452    16.1\n       ... \n441    17.1\n23     14.5\n225    50.0\n433    14.3\n447    12.6\nName: price, Length: 152, dtype: float64"
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Моделирование"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "LinearRegression()",
      "text/html": "<style>#sk-container-id-7 {color: black;background-color: white;}#sk-container-id-7 pre{padding: 0;}#sk-container-id-7 div.sk-toggleable {background-color: white;}#sk-container-id-7 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-7 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-7 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-7 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-7 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-7 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-7 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-7 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-7 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-7 div.sk-item {position: relative;z-index: 1;}#sk-container-id-7 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-7 div.sk-item::before, #sk-container-id-7 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-7 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-7 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-7 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-7 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-7 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-7 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-7 div.sk-label-container {text-align: center;}#sk-container-id-7 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-7 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-7\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" checked><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lr = LinearRegression()\n",
    "lr.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Оценка качества модели на отложенных данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import r2_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "В задаче мы используем метрику R_squared.\n",
    "\n",
    "$$R^2 = 1 - \\frac{\\sum_{i=1}^{n} (y^i - y_{pred}^i)^2}{\\sum_{i=1}^{n} (y^i - y_{mean})^2}$$\n",
    "\n",
    "Это доля объясненной вариации. Максимальное значение = 1, когда у нас есть идеальный предсказатель. Значение меньше 0 означает, что наша модель хуже, чем константная модель, которая выдает просто среднее по таргету.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficient of determination: 0.71\n"
     ]
    }
   ],
   "source": [
    "y_pred = lr.predict(X_test)\n",
    "print('Coefficient of determination: %.2f' % r2_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Оценим на глаз**"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "outputs": [
    {
     "data": {
      "text/plain": "        0          1\n0    23.6  28.648960\n1    32.4  36.495014\n2    13.6  15.411193\n3    22.8  25.403213\n4    16.1  18.855280\n..    ...        ...\n147  17.1  17.403672\n148  14.5  13.385941\n149  50.0  39.983425\n150  14.3  16.682863\n151  12.6  18.285618\n\n[152 rows x 2 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>0</th>\n      <th>1</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>23.6</td>\n      <td>28.648960</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>32.4</td>\n      <td>36.495014</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>13.6</td>\n      <td>15.411193</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>22.8</td>\n      <td>25.403213</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>16.1</td>\n      <td>18.855280</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>147</th>\n      <td>17.1</td>\n      <td>17.403672</td>\n    </tr>\n    <tr>\n      <th>148</th>\n      <td>14.5</td>\n      <td>13.385941</td>\n    </tr>\n    <tr>\n      <th>149</th>\n      <td>50.0</td>\n      <td>39.983425</td>\n    </tr>\n    <tr>\n      <th>150</th>\n      <td>14.3</td>\n      <td>16.682863</td>\n    </tr>\n    <tr>\n      <th>151</th>\n      <td>12.6</td>\n      <td>18.285618</td>\n    </tr>\n  </tbody>\n</table>\n<p>152 rows × 2 columns</p>\n</div>"
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([y_test.values, y_pred]).T"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Задание 2\n",
    "Создайте модель под названием model с помощью RandomForestRegressor из модуля sklearn.ensemble.\n",
    "Сделайте агрумент **n_estimators** равным 1000, **max_depth** должен быть равен 12 и **random_state** сделайте равным 42.\n",
    "\n",
    "Обучите модель на тренировочных данных аналогично тому, как вы обучали модель LinearRegression,\n",
    "но при этом в метод fit вместо датафрейма **y_train** поставьте **y_train.values[:, 0]**,\n",
    "чтобы получить из датафрейма одномерный массив Numpy,\n",
    "так как для класса RandomForestRegressor в данном методе для аргумента y предпочтительно применение массивов вместо датафрейма.\n",
    "\n",
    "Сделайте предсказание на тестовых данных и посчитайте R2. Сравните с результатом из предыдущего задания.\n",
    "Напишите в комментариях к коду, какая модель в данном случае работает лучше.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "RandomForestRegressor(max_depth=12, n_estimators=1000, random_state=42)",
      "text/html": "<style>#sk-container-id-8 {color: black;background-color: white;}#sk-container-id-8 pre{padding: 0;}#sk-container-id-8 div.sk-toggleable {background-color: white;}#sk-container-id-8 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-8 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-8 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-8 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-8 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-8 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-8 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-8 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-8 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-8 div.sk-item {position: relative;z-index: 1;}#sk-container-id-8 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-8 div.sk-item::before, #sk-container-id-8 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-8 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-8 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-8 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-8 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-8 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-8 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-8 div.sk-label-container {text-align: center;}#sk-container-id-8 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-8 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestRegressor(max_depth=12, n_estimators=1000, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" checked><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestRegressor</label><div class=\"sk-toggleable__content\"><pre>RandomForestRegressor(max_depth=12, n_estimators=1000, random_state=42)</pre></div></div></div></div></div>"
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## https://dyakonov.org/2016/11/14/%D1%81%D0%BB%D1%83%D1%87%D0%B0%D0%B9%D0%BD%D1%8B%D0%B9-%D0%BB%D0%B5%D1%81-random-forest/ Случайный лес (Random Forest)\n",
    "\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "rf = RandomForestRegressor(n_estimators=1000, max_depth=12, random_state=42)\n",
    "rf.fit(X_train, y_train.to_frame().values[:, 0])  # в моем случае можно rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficient of determination: 0.87\n"
     ]
    }
   ],
   "source": [
    "y_pred_rf = rf.predict(X_test)\n",
    "print('Coefficient of determination: %.2f' % r2_score(y_test, y_pred_rf))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Как видим, метод, основанный на линейной регрессии дал результат хуже, чем метод, основанные на лесах.\n",
    "Возможно игра с гиперпараметрами поможет улучшить предсказания моделей."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## *Задание 3\n",
    "Вызовите документацию для класса **RandomForestRegressor**,\n",
    "найдите информацию об атрибуте **feature_importances_**.\n",
    "С помощью этого атрибута найдите сумму всех показателей важности,\n",
    "установите, какие два признака показывают наибольшую важность."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class RandomForestRegressor in module sklearn.ensemble._forest:\n",
      "\n",
      "class RandomForestRegressor(ForestRegressor)\n",
      " |  RandomForestRegressor(n_estimators=100, *, criterion='squared_error', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=1.0, max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=True, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False, ccp_alpha=0.0, max_samples=None)\n",
      " |  \n",
      " |  A random forest regressor.\n",
      " |  \n",
      " |  A random forest is a meta estimator that fits a number of classifying\n",
      " |  decision trees on various sub-samples of the dataset and uses averaging\n",
      " |  to improve the predictive accuracy and control over-fitting.\n",
      " |  The sub-sample size is controlled with the `max_samples` parameter if\n",
      " |  `bootstrap=True` (default), otherwise the whole dataset is used to build\n",
      " |  each tree.\n",
      " |  \n",
      " |  Read more in the :ref:`User Guide <forest>`.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  n_estimators : int, default=100\n",
      " |      The number of trees in the forest.\n",
      " |  \n",
      " |      .. versionchanged:: 0.22\n",
      " |         The default value of ``n_estimators`` changed from 10 to 100\n",
      " |         in 0.22.\n",
      " |  \n",
      " |  criterion : {\"squared_error\", \"absolute_error\", \"poisson\"},             default=\"squared_error\"\n",
      " |      The function to measure the quality of a split. Supported criteria\n",
      " |      are \"squared_error\" for the mean squared error, which is equal to\n",
      " |      variance reduction as feature selection criterion, \"absolute_error\"\n",
      " |      for the mean absolute error, and \"poisson\" which uses reduction in\n",
      " |      Poisson deviance to find splits.\n",
      " |      Training using \"absolute_error\" is significantly slower\n",
      " |      than when using \"squared_error\".\n",
      " |  \n",
      " |      .. versionadded:: 0.18\n",
      " |         Mean Absolute Error (MAE) criterion.\n",
      " |  \n",
      " |      .. versionadded:: 1.0\n",
      " |         Poisson criterion.\n",
      " |  \n",
      " |      .. deprecated:: 1.0\n",
      " |          Criterion \"mse\" was deprecated in v1.0 and will be removed in\n",
      " |          version 1.2. Use `criterion=\"squared_error\"` which is equivalent.\n",
      " |  \n",
      " |      .. deprecated:: 1.0\n",
      " |          Criterion \"mae\" was deprecated in v1.0 and will be removed in\n",
      " |          version 1.2. Use `criterion=\"absolute_error\"` which is equivalent.\n",
      " |  \n",
      " |  max_depth : int, default=None\n",
      " |      The maximum depth of the tree. If None, then nodes are expanded until\n",
      " |      all leaves are pure or until all leaves contain less than\n",
      " |      min_samples_split samples.\n",
      " |  \n",
      " |  min_samples_split : int or float, default=2\n",
      " |      The minimum number of samples required to split an internal node:\n",
      " |  \n",
      " |      - If int, then consider `min_samples_split` as the minimum number.\n",
      " |      - If float, then `min_samples_split` is a fraction and\n",
      " |        `ceil(min_samples_split * n_samples)` are the minimum\n",
      " |        number of samples for each split.\n",
      " |  \n",
      " |      .. versionchanged:: 0.18\n",
      " |         Added float values for fractions.\n",
      " |  \n",
      " |  min_samples_leaf : int or float, default=1\n",
      " |      The minimum number of samples required to be at a leaf node.\n",
      " |      A split point at any depth will only be considered if it leaves at\n",
      " |      least ``min_samples_leaf`` training samples in each of the left and\n",
      " |      right branches.  This may have the effect of smoothing the model,\n",
      " |      especially in regression.\n",
      " |  \n",
      " |      - If int, then consider `min_samples_leaf` as the minimum number.\n",
      " |      - If float, then `min_samples_leaf` is a fraction and\n",
      " |        `ceil(min_samples_leaf * n_samples)` are the minimum\n",
      " |        number of samples for each node.\n",
      " |  \n",
      " |      .. versionchanged:: 0.18\n",
      " |         Added float values for fractions.\n",
      " |  \n",
      " |  min_weight_fraction_leaf : float, default=0.0\n",
      " |      The minimum weighted fraction of the sum total of weights (of all\n",
      " |      the input samples) required to be at a leaf node. Samples have\n",
      " |      equal weight when sample_weight is not provided.\n",
      " |  \n",
      " |  max_features : {\"sqrt\", \"log2\", None}, int or float, default=1.0\n",
      " |      The number of features to consider when looking for the best split:\n",
      " |  \n",
      " |      - If int, then consider `max_features` features at each split.\n",
      " |      - If float, then `max_features` is a fraction and\n",
      " |        `round(max_features * n_features)` features are considered at each\n",
      " |        split.\n",
      " |      - If \"auto\", then `max_features=n_features`.\n",
      " |      - If \"sqrt\", then `max_features=sqrt(n_features)`.\n",
      " |      - If \"log2\", then `max_features=log2(n_features)`.\n",
      " |      - If None or 1.0, then `max_features=n_features`.\n",
      " |  \n",
      " |      .. note::\n",
      " |          The default of 1.0 is equivalent to bagged trees and more\n",
      " |          randomness can be achieved by setting smaller values, e.g. 0.3.\n",
      " |  \n",
      " |      .. versionchanged:: 1.1\n",
      " |          The default of `max_features` changed from `\"auto\"` to 1.0.\n",
      " |  \n",
      " |      .. deprecated:: 1.1\n",
      " |          The `\"auto\"` option was deprecated in 1.1 and will be removed\n",
      " |          in 1.3.\n",
      " |  \n",
      " |      Note: the search for a split does not stop until at least one\n",
      " |      valid partition of the node samples is found, even if it requires to\n",
      " |      effectively inspect more than ``max_features`` features.\n",
      " |  \n",
      " |  max_leaf_nodes : int, default=None\n",
      " |      Grow trees with ``max_leaf_nodes`` in best-first fashion.\n",
      " |      Best nodes are defined as relative reduction in impurity.\n",
      " |      If None then unlimited number of leaf nodes.\n",
      " |  \n",
      " |  min_impurity_decrease : float, default=0.0\n",
      " |      A node will be split if this split induces a decrease of the impurity\n",
      " |      greater than or equal to this value.\n",
      " |  \n",
      " |      The weighted impurity decrease equation is the following::\n",
      " |  \n",
      " |          N_t / N * (impurity - N_t_R / N_t * right_impurity\n",
      " |                              - N_t_L / N_t * left_impurity)\n",
      " |  \n",
      " |      where ``N`` is the total number of samples, ``N_t`` is the number of\n",
      " |      samples at the current node, ``N_t_L`` is the number of samples in the\n",
      " |      left child, and ``N_t_R`` is the number of samples in the right child.\n",
      " |  \n",
      " |      ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum,\n",
      " |      if ``sample_weight`` is passed.\n",
      " |  \n",
      " |      .. versionadded:: 0.19\n",
      " |  \n",
      " |  bootstrap : bool, default=True\n",
      " |      Whether bootstrap samples are used when building trees. If False, the\n",
      " |      whole dataset is used to build each tree.\n",
      " |  \n",
      " |  oob_score : bool, default=False\n",
      " |      Whether to use out-of-bag samples to estimate the generalization score.\n",
      " |      Only available if bootstrap=True.\n",
      " |  \n",
      " |  n_jobs : int, default=None\n",
      " |      The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`,\n",
      " |      :meth:`decision_path` and :meth:`apply` are all parallelized over the\n",
      " |      trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend`\n",
      " |      context. ``-1`` means using all processors. See :term:`Glossary\n",
      " |      <n_jobs>` for more details.\n",
      " |  \n",
      " |  random_state : int, RandomState instance or None, default=None\n",
      " |      Controls both the randomness of the bootstrapping of the samples used\n",
      " |      when building trees (if ``bootstrap=True``) and the sampling of the\n",
      " |      features to consider when looking for the best split at each node\n",
      " |      (if ``max_features < n_features``).\n",
      " |      See :term:`Glossary <random_state>` for details.\n",
      " |  \n",
      " |  verbose : int, default=0\n",
      " |      Controls the verbosity when fitting and predicting.\n",
      " |  \n",
      " |  warm_start : bool, default=False\n",
      " |      When set to ``True``, reuse the solution of the previous call to fit\n",
      " |      and add more estimators to the ensemble, otherwise, just fit a whole\n",
      " |      new forest. See :term:`the Glossary <warm_start>`.\n",
      " |  \n",
      " |  ccp_alpha : non-negative float, default=0.0\n",
      " |      Complexity parameter used for Minimal Cost-Complexity Pruning. The\n",
      " |      subtree with the largest cost complexity that is smaller than\n",
      " |      ``ccp_alpha`` will be chosen. By default, no pruning is performed. See\n",
      " |      :ref:`minimal_cost_complexity_pruning` for details.\n",
      " |  \n",
      " |      .. versionadded:: 0.22\n",
      " |  \n",
      " |  max_samples : int or float, default=None\n",
      " |      If bootstrap is True, the number of samples to draw from X\n",
      " |      to train each base estimator.\n",
      " |  \n",
      " |      - If None (default), then draw `X.shape[0]` samples.\n",
      " |      - If int, then draw `max_samples` samples.\n",
      " |      - If float, then draw `max_samples * X.shape[0]` samples. Thus,\n",
      " |        `max_samples` should be in the interval `(0.0, 1.0]`.\n",
      " |  \n",
      " |      .. versionadded:: 0.22\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  base_estimator_ : DecisionTreeRegressor\n",
      " |      The child estimator template used to create the collection of fitted\n",
      " |      sub-estimators.\n",
      " |  \n",
      " |  estimators_ : list of DecisionTreeRegressor\n",
      " |      The collection of fitted sub-estimators.\n",
      " |  \n",
      " |  feature_importances_ : ndarray of shape (n_features,)\n",
      " |      The impurity-based feature importances.\n",
      " |      The higher, the more important the feature.\n",
      " |      The importance of a feature is computed as the (normalized)\n",
      " |      total reduction of the criterion brought by that feature.  It is also\n",
      " |      known as the Gini importance.\n",
      " |  \n",
      " |      Warning: impurity-based feature importances can be misleading for\n",
      " |      high cardinality features (many unique values). See\n",
      " |      :func:`sklearn.inspection.permutation_importance` as an alternative.\n",
      " |  \n",
      " |  n_features_ : int\n",
      " |      The number of features when ``fit`` is performed.\n",
      " |  \n",
      " |      .. deprecated:: 1.0\n",
      " |          Attribute `n_features_` was deprecated in version 1.0 and will be\n",
      " |          removed in 1.2. Use `n_features_in_` instead.\n",
      " |  \n",
      " |  n_features_in_ : int\n",
      " |      Number of features seen during :term:`fit`.\n",
      " |  \n",
      " |      .. versionadded:: 0.24\n",
      " |  \n",
      " |  feature_names_in_ : ndarray of shape (`n_features_in_`,)\n",
      " |      Names of features seen during :term:`fit`. Defined only when `X`\n",
      " |      has feature names that are all strings.\n",
      " |  \n",
      " |      .. versionadded:: 1.0\n",
      " |  \n",
      " |  n_outputs_ : int\n",
      " |      The number of outputs when ``fit`` is performed.\n",
      " |  \n",
      " |  oob_score_ : float\n",
      " |      Score of the training dataset obtained using an out-of-bag estimate.\n",
      " |      This attribute exists only when ``oob_score`` is True.\n",
      " |  \n",
      " |  oob_prediction_ : ndarray of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |      Prediction computed with out-of-bag estimate on the training set.\n",
      " |      This attribute exists only when ``oob_score`` is True.\n",
      " |  \n",
      " |  See Also\n",
      " |  --------\n",
      " |  sklearn.tree.DecisionTreeRegressor : A decision tree regressor.\n",
      " |  sklearn.ensemble.ExtraTreesRegressor : Ensemble of extremely randomized\n",
      " |      tree regressors.\n",
      " |  \n",
      " |  Notes\n",
      " |  -----\n",
      " |  The default values for the parameters controlling the size of the trees\n",
      " |  (e.g. ``max_depth``, ``min_samples_leaf``, etc.) lead to fully grown and\n",
      " |  unpruned trees which can potentially be very large on some data sets. To\n",
      " |  reduce memory consumption, the complexity and size of the trees should be\n",
      " |  controlled by setting those parameter values.\n",
      " |  \n",
      " |  The features are always randomly permuted at each split. Therefore,\n",
      " |  the best found split may vary, even with the same training data,\n",
      " |  ``max_features=n_features`` and ``bootstrap=False``, if the improvement\n",
      " |  of the criterion is identical for several splits enumerated during the\n",
      " |  search of the best split. To obtain a deterministic behaviour during\n",
      " |  fitting, ``random_state`` has to be fixed.\n",
      " |  \n",
      " |  The default value ``max_features=\"auto\"`` uses ``n_features``\n",
      " |  rather than ``n_features / 3``. The latter was originally suggested in\n",
      " |  [1], whereas the former was more recently justified empirically in [2].\n",
      " |  \n",
      " |  References\n",
      " |  ----------\n",
      " |  .. [1] L. Breiman, \"Random Forests\", Machine Learning, 45(1), 5-32, 2001.\n",
      " |  \n",
      " |  .. [2] P. Geurts, D. Ernst., and L. Wehenkel, \"Extremely randomized\n",
      " |         trees\", Machine Learning, 63(1), 3-42, 2006.\n",
      " |  \n",
      " |  Examples\n",
      " |  --------\n",
      " |  >>> from sklearn.ensemble import RandomForestRegressor\n",
      " |  >>> from sklearn.datasets import make_regression\n",
      " |  >>> X, y = make_regression(n_features=4, n_informative=2,\n",
      " |  ...                        random_state=0, shuffle=False)\n",
      " |  >>> regr = RandomForestRegressor(max_depth=2, random_state=0)\n",
      " |  >>> regr.fit(X, y)\n",
      " |  RandomForestRegressor(...)\n",
      " |  >>> print(regr.predict([[0, 0, 0, 0]]))\n",
      " |  [-8.32987858]\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      RandomForestRegressor\n",
      " |      ForestRegressor\n",
      " |      sklearn.base.RegressorMixin\n",
      " |      BaseForest\n",
      " |      sklearn.base.MultiOutputMixin\n",
      " |      sklearn.ensemble._base.BaseEnsemble\n",
      " |      sklearn.base.MetaEstimatorMixin\n",
      " |      sklearn.base.BaseEstimator\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, n_estimators=100, *, criterion='squared_error', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=1.0, max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=True, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False, ccp_alpha=0.0, max_samples=None)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data and other attributes defined here:\n",
      " |  \n",
      " |  __abstractmethods__ = frozenset()\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from ForestRegressor:\n",
      " |  \n",
      " |  predict(self, X)\n",
      " |      Predict regression target for X.\n",
      " |      \n",
      " |      The predicted regression target of an input sample is computed as the\n",
      " |      mean predicted regression targets of the trees in the forest.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features)\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      y : ndarray of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |          The predicted values.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.RegressorMixin:\n",
      " |  \n",
      " |  score(self, X, y, sample_weight=None)\n",
      " |      Return the coefficient of determination of the prediction.\n",
      " |      \n",
      " |      The coefficient of determination :math:`R^2` is defined as\n",
      " |      :math:`(1 - \\frac{u}{v})`, where :math:`u` is the residual\n",
      " |      sum of squares ``((y_true - y_pred)** 2).sum()`` and :math:`v`\n",
      " |      is the total sum of squares ``((y_true - y_true.mean()) ** 2).sum()``.\n",
      " |      The best possible score is 1.0 and it can be negative (because the\n",
      " |      model can be arbitrarily worse). A constant model that always predicts\n",
      " |      the expected value of `y`, disregarding the input features, would get\n",
      " |      a :math:`R^2` score of 0.0.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : array-like of shape (n_samples, n_features)\n",
      " |          Test samples. For some estimators this may be a precomputed\n",
      " |          kernel matrix or a list of generic objects instead with shape\n",
      " |          ``(n_samples, n_samples_fitted)``, where ``n_samples_fitted``\n",
      " |          is the number of samples used in the fitting for the estimator.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |          True values for `X`.\n",
      " |      \n",
      " |      sample_weight : array-like of shape (n_samples,), default=None\n",
      " |          Sample weights.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      score : float\n",
      " |          :math:`R^2` of ``self.predict(X)`` wrt. `y`.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The :math:`R^2` score used when calling ``score`` on a regressor uses\n",
      " |      ``multioutput='uniform_average'`` from version 0.23 to keep consistent\n",
      " |      with default value of :func:`~sklearn.metrics.r2_score`.\n",
      " |      This influences the ``score`` method of all the multioutput\n",
      " |      regressors (except for\n",
      " |      :class:`~sklearn.multioutput.MultiOutputRegressor`).\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from sklearn.base.RegressorMixin:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from BaseForest:\n",
      " |  \n",
      " |  apply(self, X)\n",
      " |      Apply trees in the forest to X, return leaf indices.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features)\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      X_leaves : ndarray of shape (n_samples, n_estimators)\n",
      " |          For each datapoint x in X and for each tree in the forest,\n",
      " |          return the index of the leaf x ends up in.\n",
      " |  \n",
      " |  decision_path(self, X)\n",
      " |      Return the decision path in the forest.\n",
      " |      \n",
      " |      .. versionadded:: 0.18\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features)\n",
      " |          The input samples. Internally, its dtype will be converted to\n",
      " |          ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csr_matrix``.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      indicator : sparse matrix of shape (n_samples, n_nodes)\n",
      " |          Return a node indicator matrix where non zero elements indicates\n",
      " |          that the samples goes through the nodes. The matrix is of CSR\n",
      " |          format.\n",
      " |      \n",
      " |      n_nodes_ptr : ndarray of shape (n_estimators + 1,)\n",
      " |          The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]]\n",
      " |          gives the indicator value for the i-th estimator.\n",
      " |  \n",
      " |  fit(self, X, y, sample_weight=None)\n",
      " |      Build a forest of trees from the training set (X, y).\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      X : {array-like, sparse matrix} of shape (n_samples, n_features)\n",
      " |          The training input samples. Internally, its dtype will be converted\n",
      " |          to ``dtype=np.float32``. If a sparse matrix is provided, it will be\n",
      " |          converted into a sparse ``csc_matrix``.\n",
      " |      \n",
      " |      y : array-like of shape (n_samples,) or (n_samples, n_outputs)\n",
      " |          The target values (class labels in classification, real numbers in\n",
      " |          regression).\n",
      " |      \n",
      " |      sample_weight : array-like of shape (n_samples,), default=None\n",
      " |          Sample weights. If None, then samples are equally weighted. Splits\n",
      " |          that would create child nodes with net zero or negative weight are\n",
      " |          ignored while searching for a split in each node. In the case of\n",
      " |          classification, splits are also ignored if they would result in any\n",
      " |          single class carrying a negative weight in either child node.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : object\n",
      " |          Fitted estimator.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Readonly properties inherited from BaseForest:\n",
      " |  \n",
      " |  feature_importances_\n",
      " |      The impurity-based feature importances.\n",
      " |      \n",
      " |      The higher, the more important the feature.\n",
      " |      The importance of a feature is computed as the (normalized)\n",
      " |      total reduction of the criterion brought by that feature.  It is also\n",
      " |      known as the Gini importance.\n",
      " |      \n",
      " |      Warning: impurity-based feature importances can be misleading for\n",
      " |      high cardinality features (many unique values). See\n",
      " |      :func:`sklearn.inspection.permutation_importance` as an alternative.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      feature_importances_ : ndarray of shape (n_features,)\n",
      " |          The values of this array sum to 1, unless all trees are single node\n",
      " |          trees consisting of only the root node, in which case it will be an\n",
      " |          array of zeros.\n",
      " |  \n",
      " |  n_features_\n",
      " |      DEPRECATED: Attribute `n_features_` was deprecated in version 1.0 and will be removed in 1.2. Use `n_features_in_` instead.\n",
      " |      \n",
      " |      Number of features when fitting the estimator.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.ensemble._base.BaseEnsemble:\n",
      " |  \n",
      " |  __getitem__(self, index)\n",
      " |      Return the index'th estimator in the ensemble.\n",
      " |  \n",
      " |  __iter__(self)\n",
      " |      Return iterator over estimators in the ensemble.\n",
      " |  \n",
      " |  __len__(self)\n",
      " |      Return the number of estimators in the ensemble.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data and other attributes inherited from sklearn.ensemble._base.BaseEnsemble:\n",
      " |  \n",
      " |  __annotations__ = {'_required_parameters': typing.List[str]}\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from sklearn.base.BaseEstimator:\n",
      " |  \n",
      " |  __getstate__(self)\n",
      " |  \n",
      " |  __repr__(self, N_CHAR_MAX=700)\n",
      " |      Return repr(self).\n",
      " |  \n",
      " |  __setstate__(self, state)\n",
      " |  \n",
      " |  get_params(self, deep=True)\n",
      " |      Get parameters for this estimator.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      deep : bool, default=True\n",
      " |          If True, will return the parameters for this estimator and\n",
      " |          contained subobjects that are estimators.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      params : dict\n",
      " |          Parameter names mapped to their values.\n",
      " |  \n",
      " |  set_params(self, **params)\n",
      " |      Set the parameters of this estimator.\n",
      " |      \n",
      " |      The method works on simple estimators as well as on nested objects\n",
      " |      (such as :class:`~sklearn.pipeline.Pipeline`). The latter have\n",
      " |      parameters of the form ``<component>__<parameter>`` so that it's\n",
      " |      possible to update each component of a nested object.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      **params : dict\n",
      " |          Estimator parameters.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      self : estimator instance\n",
      " |          Estimator instance.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# print(RandomForestRegressor.__doc__)\n",
    "# RandomForestRegressor??\n",
    "help(RandomForestRegressor)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "fet_ind = np.argsort(rf.feature_importances_)[::-1]\n",
    "fet_imp = rf.feature_importances_[fet_ind]"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "outputs": [
    {
     "data": {
      "text/plain": "array([12,  5,  7,  0, 10,  6,  4, 11,  9,  2,  8,  1,  3], dtype=int64)"
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fet_ind"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "outputs": [
    {
     "data": {
      "text/plain": "0.9999999999999999"
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(fet_imp)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "outputs": [
    {
     "data": {
      "text/plain": "Text(0.5, 1.0, 'Features importance')"
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": "<Figure size 864x360 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAFeCAYAAAB3voKLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAogUlEQVR4nO3dfZxkZX3n/c/XQdQEVJQJ0eFpxEkUxVt0RHeNTxF0vE0YkmiEuAaMBt2FCCGaYOKNBpdVcRWjIaskEqMbMyomZLwdg09A1gd0BkF0MOgwIsyocRTEKIgO/PaPcxqKYz9Ud1d3dVV/3q9XvTjnOudU/U4N3f2tq65znVQVkiRJku5yj2EXIEmSJC01hmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSNEKSHJjkh0lWDLsWSRpnhmRJYyfJdUlubcPkxOPBA3jOIwdV41xV1fVVtVdV3T7sWpIcnKSS7DHsWiRp0AzJksbVr7dhcuLxzWEWM25BctzOR5K6DMmSlo0k90vyziTfSrIzyX+fGLaQ5JAkn0zyvSTfTfL3Se7fbnsPcCDwobZX+o+TPDXJjs7z39nbnOQ1SS5I8r+T/AA4YYbXf2iSS5Pc3L7++6Y4h7v13ia5pH2ez7S1fSjJA9v6f5Bkc5KDe46vJC9Lsr19nTcmuUe77R5JXpXkG0m+k+TdSe7Xed0XJbke+CTwr+3Tfr997f803fvY8x69PMlV7bm+L8m9e7avT3JlW/u1SdbN9G8nSQvBkCxpOXkXsBt4KHA48Azgxe22AK8DHgw8HDgAeA1AVb0AuJ67eqfP7vP11gMXAPcH/n6G138t8FFgH2B/4G2zOK9jgRcAq4BDgM8Cfws8APgK8OrO/r8BrAUe09b4e237Ce3jacBDgL2Av+wc+xSa9+eZwJPbtvu378tnmeZ97PHbwDpgNfCo9jVJcgTwbuAVNO/Zk4Hr2mPexdTvnSQNnCFZ0ri6MMn328eFSfYD/l/g1Kr6UVV9BziHJmBSVduq6mNVdVtV7QLeTBMI5+OzVXVhVd0B3He61wd+ChwEPLiqflxVn5rF6/xtVV1bVTcDHwGuraqPV9Vu4AM0obLXG6rqxqq6HngLcFzb/nzgzVW1vap+CLwSOLYztOI1bf23TlZIn+/jW6vqm1V1I/Ah4NFt+4uA89vj76iqnVX1bzP920nSQnBMmaRxdUxVfXxipe2lvCfwrSQTzfcAbmi37wf8BfAkYO92203zrOGGnuWDpnt94I9pepM/n+Qm4E1VdX6fr/PvPcu3TrK+1zR1fYOm15f2v9/obNsD2G+KY39Gn+/jt3uWb+l5/QOATZM87UzvnSQNnCFZ0nJxA3AbsG/bw9r1P4ACDquqG5Mcw92HGlRn/x8BPzex0o6PXdnZp/eYaV+/qr4N/H77XL8CfDzJv1bVtj7ObbYOALa2ywcCExc1fpMmkNKzbTdN6N5/otTesid57pnex+ncQDNcZLL26f7tJGngHG4haVmoqm/RjPl9U5L7thepHZJkYijA3sAPgZuTrKIZF9vr32nG6U74KnDvJM9Ock/gVcC95vr6SZ6bZCKI3kQTNO+Y10lP7RVJ9klyAHAKMHGR4D8Af5hkdZK9aALv+6YJprvaGnvfl5nex+m8E3hhkqe378+qJA/r499OkgbOkCxpOfldYE/gapogegHwoHbbn9NcyHYz8GHgHzvHvg54VTvG+eXt+N//BvwNsJOmZ3kH05vu9R8HfC7JD4GNwClVtX2O5zmTfwYuB66kOdd3tu3nA++hmbXi68CPgT+Y6kmq6hbgLODT7fvyBGZ+H6dUVZ8HXkgz3vhm4FLu6tme7r2TpIFL1WTflkmSxlGSAtYs0DAOSRob9iRLkiRJHYZkSZIkqcPhFpIkSVKHPcmSJElShyFZkiRJ6lhyNxPZd9996+CDDx52GZIkSRpzl19++XerqnsjKGAJhuSDDz6YLVu2DLsMSZIkjbkk35hqm8MtJEmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjr2GHYBg3Dw6R9elNe57vXPXpTXkSRJ0nDZkyxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVJHXyE5ybok1yTZluT0afb7rSSVZG1P2yvb465J8sxBFC1JkiQtpBmngEuyAjgXOArYAWxOsrGqru7stzdwCvC5nrZDgWOBRwAPBj6e5Jeq6vbBnYIkSZI0WP3Mk3wEsK2qtgMk2QCsB67u7Pda4A3AK3ra1gMbquo24OtJtrXP99n5Fj7OnPdZkiRpuPoZbrEKuKFnfUfbdqckjwEOqKpuupvxWEmSJGmpmfeFe0nuAbwZ+KN5PMeJSbYk2bJr1675liRJkiTNSz8heSdwQM/6/m3bhL2BRwKXJLkOeAKwsb14b6ZjAaiq86pqbVWtXbly5ezOQJIkSRqwfkLyZmBNktVJ9qS5EG/jxMaqurmq9q2qg6vqYOAy4Oiq2tLud2ySeyVZDawBPj/ws5AkSZIGaMYL96pqd5KTgYuAFcD5VbU1yZnAlqraOM2xW5O8n+Yiv93ASc5sIUmSpKWun9ktqKpNwKZO2xlT7PvUzvpZwFlzrE+SJEladN5xT5IkSeowJEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVKHIVmSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLU0VdITrIuyTVJtiU5fZLtL03ypSRXJvlUkkPb9oOT3Nq2X5nk7YM+AUmSJGnQ9phphyQrgHOBo4AdwOYkG6vq6p7d3ltVb2/3Pxp4M7Cu3XZtVT16oFVLkiRJC6ifnuQjgG1Vtb2qfgJsANb37lBVP+hZ/XmgBleiJEmStLj6CcmrgBt61ne0bXeT5KQk1wJnAy/r2bQ6yRVJLk3ypHlVK0mSJC2CgV24V1XnVtUhwJ8Ar2qbvwUcWFWHA6cB701y3+6xSU5MsiXJll27dg2qJEmSJGlO+gnJO4EDetb3b9umsgE4BqCqbquq77XLlwPXAr/UPaCqzquqtVW1duXKlX2WLkmSJC2MfkLyZmBNktVJ9gSOBTb27pBkTc/qs4Gvte0r2wv/SPIQYA2wfRCFS5IkSQtlxtktqmp3kpOBi4AVwPlVtTXJmcCWqtoInJzkSOCnwE3A8e3hTwbOTPJT4A7gpVV140KciCRJkjQoM4ZkgKraBGzqtJ3Rs3zKFMd9EPjgfAqUJEmSFpt33JMkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVKHIVmSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLUYUiWJEmSOgzJkiRJUochWZIkSeowJEuSJEkdhmRJkiSpw5AsSZIkdfQVkpOsS3JNkm1JTp9k+0uTfCnJlUk+leTQnm2vbI+7JskzB1m8JEmStBBmDMlJVgDnAs8CDgWO6w3BrfdW1WFV9WjgbODN7bGHAscCjwDWAX/VPp8kSZK0ZPXTk3wEsK2qtlfVT4ANwPreHarqBz2rPw9Uu7we2FBVt1XV14Ft7fNJkiRJS9YefeyzCrihZ30H8PjuTklOAk4D9gR+tefYyzrHrppTpZIkSdIiGdiFe1V1blUdAvwJ8KrZHJvkxCRbkmzZtWvXoEqSJEmS5qSfkLwTOKBnff+2bSobgGNmc2xVnVdVa6tq7cqVK/soSZIkSVo4/YTkzcCaJKuT7ElzId7G3h2SrOlZfTbwtXZ5I3BsknslWQ2sAT4//7IlSZKkhTPjmOSq2p3kZOAiYAVwflVtTXImsKWqNgInJzkS+ClwE3B8e+zWJO8HrgZ2AydV1e0LdC6SJEnSQPRz4R5VtQnY1Gk7o2f5lGmOPQs4a64FSpIkSYvNO+5JkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLUYUiWJEmSOgzJkiRJUochWZIkSeowJEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjr6CslJ1iW5Jsm2JKdPsv20JFcnuSrJJ5Ic1LPt9iRXto+NgyxekiRJWgh7zLRDkhXAucBRwA5gc5KNVXV1z25XAGur6pYk/xU4G3heu+3Wqnr0YMuWJEmSFk4/PclHANuqantV/QTYAKzv3aGqLq6qW9rVy4D9B1umJEmStHj6CcmrgBt61ne0bVN5EfCRnvV7J9mS5LIkx8y+REmSJGlxzTjcYjaS/BdgLfCUnuaDqmpnkocAn0zypaq6tnPcicCJAAceeOAgS5IkSZJmrZ+e5J3AAT3r+7dtd5PkSODPgKOr6raJ9qra2f53O3AJcHj32Ko6r6rWVtXalStXzuoEJEmSpEHrJyRvBtYkWZ1kT+BY4G6zVCQ5HHgHTUD+Tk/7Pknu1S7vCzwR6L3gT5IkSVpyZhxuUVW7k5wMXASsAM6vqq1JzgS2VNVG4I3AXsAHkgBcX1VHAw8H3pHkDppA/vrOrBiSJEnSktPXmOSq2gRs6rSd0bN85BTHfQY4bD4FSpIkSYvNO+5JkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLUYUiWJEmSOgzJkiRJUochWZIkSeowJEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVJHXyE5ybok1yTZluT0SbafluTqJFcl+USSg3q2HZ/ka+3j+EEWL0mSJC2EGUNykhXAucCzgEOB45Ic2tntCmBtVT0KuAA4uz32AcCrgccDRwCvTrLP4MqXJEmSBq+fnuQjgG1Vtb2qfgJsANb37lBVF1fVLe3qZcD+7fIzgY9V1Y1VdRPwMWDdYEqXJEmSFkY/IXkVcEPP+o62bSovAj4ym2OTnJhkS5Itu3bt6qMkSZIkaeEM9MK9JP8FWAu8cTbHVdV5VbW2qtauXLlykCVJkiRJs9ZPSN4JHNCzvn/bdjdJjgT+DDi6qm6bzbGSJEnSUtJPSN4MrEmyOsmewLHAxt4dkhwOvIMmIH+nZ9NFwDOS7NNesPeMtk2SJElasvaYaYeq2p3kZJpwuwI4v6q2JjkT2FJVG2mGV+wFfCAJwPVVdXRV3ZjktTRBG+DMqrpxQc5EkiRJGpAZQzJAVW0CNnXazuhZPnKaY88Hzp9rgZIkSdJi8457kiRJUochWZIkSeowJEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVKHIVmSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqSOvkJyknVJrkmyLcnpk2x/cpIvJNmd5DmdbbcnubJ9bBxU4ZIkSdJC2WOmHZKsAM4FjgJ2AJuTbKyqq3t2ux44AXj5JE9xa1U9ev6lSpIkSYtjxpAMHAFsq6rtAEk2AOuBO0NyVV3XbrtjAWqUJEmSFlU/wy1WATf0rO9o2/p17yRbklyW5JjJdkhyYrvPll27ds3iqSVJkqTBW4wL9w6qqrXA7wBvSXJId4eqOq+q1lbV2pUrVy5CSZIkSdLU+gnJO4EDetb3b9v6UlU72/9uBy4BDp9FfZIkSdKi6yckbwbWJFmdZE/gWKCvWSqS7JPkXu3yvsAT6RnLLEmSJC1FM4bkqtoNnAxcBHwFeH9VbU1yZpKjAZI8LskO4LnAO5JsbQ9/OLAlyReBi4HXd2bFkCRJkpacfma3oKo2AZs6bWf0LG+mGYbRPe4zwGHzrFGSJElaVN5xT5IkSeowJEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVKHIVmSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLU0VdITrIuyTVJtiU5fZLtT07yhSS7kzyns+34JF9rH8cPqnBJkiRpocwYkpOsAM4FngUcChyX5NDObtcDJwDv7Rz7AODVwOOBI4BXJ9ln/mVLkiRJC6efnuQjgG1Vtb2qfgJsANb37lBV11XVVcAdnWOfCXysqm6sqpuAjwHrBlC3JEmStGD6CcmrgBt61ne0bf3o69gkJybZkmTLrl27+nxqSZIkaWEsiQv3quq8qlpbVWtXrlw57HIkSZK0zPUTkncCB/Ss79+29WM+x0qSJElD0U9I3gysSbI6yZ7AscDGPp//IuAZSfZpL9h7RtsmSZIkLVkzhuSq2g2cTBNuvwK8v6q2JjkzydEASR6XZAfwXOAdSba2x94IvJYmaG8GzmzbJEmSpCVrj352qqpNwKZO2xk9y5tphlJMduz5wPnzqFGSJElaVEviwj1JkiRpKTEkS5IkSR2GZEmSJKnDkCxJkiR1GJIlSZKkDkOyJEmS1GFIliRJkjoMyZIkSVKHIVmSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqQOQ7IkSZLUYUiWJEmSOgzJkiRJUochWZIkSerYY9gFaLwdfPqHF+V1rnv9sxfldSRJ0vJgT7IkSZLUYUiWJEmSOgzJkiRJUochWZIkSeroKyQnWZfkmiTbkpw+yfZ7JXlfu/1zSQ5u2w9OcmuSK9vH2wdcvyRJkjRwM85ukWQFcC5wFLAD2JxkY1Vd3bPbi4CbquqhSY4F3gA8r912bVU9erBlS5IkSQunn57kI4BtVbW9qn4CbADWd/ZZD/xdu3wB8PQkGVyZkiRJ0uLpJySvAm7oWd/Rtk26T1XtBm4GHthuW53kiiSXJnnSZC+Q5MQkW5Js2bVr16xOQJIkSRq0hb5w71vAgVV1OHAa8N4k9+3uVFXnVdXaqlq7cuXKBS5JkiRJml4/IXkncEDP+v5t26T7JNkDuB/wvaq6raq+B1BVlwPXAr8036IlSZKkhdRPSN4MrEmyOsmewLHAxs4+G4Hj2+XnAJ+sqkqysr3wjyQPAdYA2wdTuiRJkrQwZpzdoqp2JzkZuAhYAZxfVVuTnAlsqaqNwDuB9yTZBtxIE6QBngycmeSnwB3AS6vqxoU4EUmSJGlQZgzJAFW1CdjUaTujZ/nHwHMnOe6DwAfnWaMkSZK0qLzjniRJktRhSJYkSZI6DMmSJElShyFZkiRJ6jAkS5IkSR2GZEmSJKmjryngJDUOPv3Di/I6173+2YvyOpIkaXL2JEuSJEkdhmRJkiSpw5AsSZIkdRiSJUmSpA5DsiRJktRhSJYkSZI6DMmSJElSh/MkS8uUcz5LkjQ1e5IlSZKkDkOyJEmS1OFwC0ljYZyGj4zTuUjSqDIkS5IW1DiF/nE6F0nTc7iFJEmS1GFIliRJkjocbiFJ0jLl8BFpan2F5CTrgL8AVgB/U1Wv72y/F/Bu4LHA94DnVdV17bZXAi8CbgdeVlUXDax6SZIkDPwavBmHWyRZAZwLPAs4FDguyaGd3V4E3FRVDwXOAd7QHnsocCzwCGAd8Fft80mSJElLVj9jko8AtlXV9qr6CbABWN/ZZz3wd+3yBcDTk6Rt31BVt1XV14Ft7fNJkiRJS1Y/wy1WATf0rO8AHj/VPlW1O8nNwAPb9ss6x66ac7WSJEnLwDgNHxnVc0lVTb9D8hxgXVW9uF1/AfD4qjq5Z58vt/vsaNevpQnSrwEuq6r/3ba/E/hIVV3QeY0TgRPb1V8Grpn/qc1oX+C7i/A6i2WczmeczgXG63zG6VxgvM5nnM4Fxut8xulcYLzOZ5zOBcbrfBbrXA6qqpWTbeinJ3kncEDP+v5t22T77EiyB3A/mgv4+jmWqjoPOK+PWgYmyZaqWruYr7mQxul8xulcYLzOZ5zOBcbrfMbpXGC8zmeczgXG63zG6VxgvM5nKZxLP2OSNwNrkqxOsifNhXgbO/tsBI5vl58DfLKaLuqNwLFJ7pVkNbAG+PxgSpckSZIWxow9ye0Y45OBi2imgDu/qrYmORPYUlUbgXcC70myDbiRJkjT7vd+4GpgN3BSVd2+QOciSZIkDURf8yRX1SZgU6ftjJ7lHwPPneLYs4Cz5lHjQlnU4R2LYJzOZ5zOBcbrfMbpXGC8zmeczgXG63zG6VxgvM5nnM4Fxut8hn4uM164J0mSJC03/YxJliRJkpYVQ7IkSZLUsSxCcpIDh12DJC2GdhpODUGSx02z7QWLWYuk+VsWY5KTfKGqHjPsOgYlyVun215VL1usWgYtyQOBJwPXV9Xlw65nNpL85nTbq+ofF6uWQUnyC8BJwCPapq3AX1XVvw+vqtlLct+q+sEU2w6squsXu6b5SPIh4OSq+kan/UjgLVX1yOFUNjhJ9gW+VyP0RyrJVcCngVdW1ffbtkcCfwXcWFXHDK+6wUhyT+CRwM6q+s6w65mt9kPks4CHtU1fAf6lqnYPr6rZS/K7022vqncvVi2DkOT3gUuq6mtJApwP/BZwHXBCVX1hGHUti55kIMMuYMBeCvwK8E1gC3B55zEykvz/7R8RkjwI+DLwezRTCp46zNrm4ALgVcCvtY9f73n82hDrmpMkT6SZJx3g3e0D4HPttlFyycRCkk90tl24qJUMxgbg4iR/luSeSR7cTrd5FnfNWT8ykjwhySVJ/jHJ4e1dXL8M/HuSdcOubxYeA1wPXJHk95KcA3wQOHtUA3KStyd5RLt8P+CLNL8Lrkhy3FCLm6Ukq2g+6P8R8GBgFfAKYGuSBw+ztjl43BSP19IEzFFzCk0gBjgOeBSwGjgN+Ish1bRsepK/Q/NHZVKj1vPa9rY+F3gezfzT7wMumOi5GCVJtlbVxC/gPwUeVlW/m2Rv4NNV9ajhVti/JMfQzBH+UOCfgX+oqm1DLWoeklwG/NequqLT/mjgHVX1+KEUNgdJrqiqw7vLk62PijawnA08HbgnTUD+61HqeZ2QZAvwpzR3az0PeFZVXZbkYTQ/RyP175PkFcAbaDoyjqiqbw65pDnr/I4+FXhqVR2T5BeBj4zSv02SdwFXVtVbOu0vAx5bVSP3AROg7Xl9PvAnNPelOKuqrhpuVbOT5MqqenS7/F7gc1X1F+360EYDLJee5Fv52d7Wkex5Baiq71XV26vqacALgfsDV4/omLef9iw/nXY+7qr6D+COoVQ0R1V1YVUdCzwFuBZ4U5JPJXnKkEubq/t2AzJAVV0J7L345cxLTbE82fqoOBQ4guYuprcB+9Hn3PdL0B5V9dGq+gDw7aq6DKCq/m3Idc1KkkOS/Avwq8DDgf8J/GuSFw63snn5Sc/yUbTfvFTVt4dSzfw8oRuQAarqrcATFr+c+UmyR5IX0wwZORJ4TlU9b9QCcuuOJA9Kcm+aLPDxnm33GVJNI/sLdba+V1V/N+wiBi3JY2i+ljgK+AgjGPiBG5L8AbCD5qvKfwFIch+a3rFR9GPgZuAHwEHAvYdbzpwlyT5VdVOn8QGM3gfsX0hyGs3Qq4ll2vWVwytrbpK8Ezgc+G9V9dkkPw/8OfDFJKdW1UeHW+Gs9X4gvrWzbZQ+xFwEnF5VF7Tr17TDYN6c5MVVNWrDlAC+n+TXgJ3AE4EXwZ1je4cWXuao+/9Wr1sWrYoBSHISzRCFTwDrquq64VY0b2fQDB9dAWysqq0AbSfT9mEVtVyGW1xWVT/zKTHJrwDHVdVJQyhrztpbgj+b5tPjBkbwooMJ7YVhZwIPAs6d+OOe5Gk0X3/9z2HWNxtJfpVmuMURNJ+CN1TVluFWNXdJTgR+H3g5MHHRxGNpvkY+v6reMazaZivJq6fbXlV/vli1DEKSPwTeWlW3d9oPo7mw8knDqWxuktwO/IjmQ8t9uCuwBLh3VY3EB+Yke1XVD6fYdmRVfXyybUtZkl8C3gr8Is1Foe9q258JPKOq/miI5c1Kku00v89+ZhPNuPFDFrmkOUtyB/AdYBd3/yAZoEZpqOKE9oPX3r0dM0l+DljRfru8+DUth5DcK8nhwO/QjOn9OvCPVfW24VY1O+0Px9e56w/JxD/iyP5wjIP23+Uq4FM0/yZ3++EatbHvAG0P0h9z99kt3lhVHxpeVYLxmXlk3CU5hOZvzrETY3s1HEn+drrtVTUyw2KSvJTmm9fJQtzzqursRS5poNpx1r9K87Pza1W131DqWA4huf0kfFz7+C7NhW4vr6qDhlrYHCWZtu7utFBLWTuV1ZT/E1bV0YtYzrwkOYHpz2XshvyMivbq/EOqamO7fg7NRWIAfzms6YXmqp1d5L3Au7hrmNVjaWa2eH5VfXpIpQloZ0p4Hs0f+MOA19F0yHxpqIXNQZK38bNj+r8LXFxVnxpOVYOXZL9R+oDZfvtyKfCCqtrZ2Tay094meQLNz80xwANoOgI2dof9LVo9yyQk3wH8H+BFE7MNJNleVQ8ZbmWDleQeNMNH/n7YtfRrpovaqurSxapFdzfJH8e7GaWe8fbD2Ouq6jPt+tXA/wf8HPBbozY91zjNPDJO2iFKx9FMLfb+9vHPVbV6qIXNQ5LJZnx4APDbwPsmuxBuVCS5P81cvL8DPLyqRmYauCRX0My/fQbwhz3j4Edyxp4k/4PmG/7rgX8A/gnYMuyfneVy4d5v0owVvbi98ngDIzx3cpL70ny6WgVsBD4GnEwz9+MXgZEJyb0hOMnKtm3X8Cqau3HqFW+N7HjqSTxoIiC3flBVHwRI8pIh1TQfU8480k6fqOH4S+CzwO9MXI+QZKR7oqb6BizJ24HPAG9Z1ILmqb0ofD1NMD6cZqaeY4B/HWJZc1FV9ddJLgX+PsmzgZOq6hZG62LXCS8Gvgr8L+BDVXXbUvjZWRYhuaouBC5srwBfD5xKc4X7/wL+aQSvBH8PcBPNL+MX08wvGuCYdnqukdJeVPUHNDMmJMlu4G1VdeZwK5u1kbnIsE+/XFV/OuwiBuRuwbFzIe8vLHItgzBOM4+Mk1U0PZNvaucRfj+jO0vPtKrq1mbY6Oho5999EvBR4G3AJ4FtVXXJMOuaj6r6apL/BPx3mhu8THsnviXsQTQzdR0HvCXJxcB9kuwxzIkJlsUv03YCcarqR1X13qr6dWB/4AqaybdHzUOq6oR2doHjaOZLfeaIBuTTaO4e+LiqekBV7QM8HnhiewX/yKiqS6d60Nz0ZdSM0p3OZvLNJD8zBKEd/zaKN3o4B/hokqck2bt9PJVmKshzhlrZ8nZRO4f9U2jmev0+zV0Dv9J+nTwW2vl5X0gzdecoOZSmg+krwFfa2WGG3ls5R3d+Qqmq3VV1OvASmqEKa4ZW1RxV1e1V9S/V3NDlEJr5uD8N7Gw/3AzFchmTPLKD2CfTPZ9RPr92XNVRVfXdTvtK4KOjNK4qyQqacXqraKbl+3I7O8SfAvcZpXMBSPJF4KlMMTSpqm5c1ILmIckRNBfsvou7T2d3PM2V4J8fUmlz5swjS89UY0GTrKG5XmTUvh0jyX/QBMne3wO30Fw0dmqN2N0E09zF8TiaCyu/C/wy8MhRumgPIMkx7bfk3fZ9gJdU1esXv6rBa4eP/UZVvXsor79MQvK/0fxQTPXHftSubJ+YUxTuPq/oxBRw9x1WbbOV5MtV9cjZbluK2m8sDqC5A9rjaXoo19LcXODC4VU2N0luo7mBwGQ/NzVqF74m2Y+7pkwrmlD5cZqpuUZqrnQtTUl2AG+eantVTblNiy/JY7lrStgdVfWfh1zSspW7bvA0qWH97CyLMck0PXtvYoo/9jRz8Y2Mqlox7BoG6Cdz3LYUrQUeVVV3pLm15rdpph373pDrmqurR633ezptT9EZuetOlccDTwY+ONTC5iDJGdNsrqp67aIVo14rgL2Y+m/NSGpv8vAs4GFt09U0Q0tGcRjZnarqcuDyJKcDrxx2Pctc73UjLwF6b1Y1tJ+d5dKTPHLToSwXnV7xu21ihO60BeM1DAam/7kZwTlFx22u9MnucvbzNLcMfmBV7bXIJYnR/5mfTJJVNBe4fYvmOp7QzArxi8DTRmm4xRQzQ51EMzPUVVW1fojlqbWUMpshWRqQJLcA2yZWaS4+mFinRuxOiElOqPYWtO36/RndOUXHdq70dszeKTQB+f3Am6rqO8Otankax7817TCyK7vzISd5GfDY9kKrkZDkn7lrZqin08xsE+CUUbzwfVwtpQ+by2W4xd1msEhyT+CRwE7/mGiA/h9gP+CGTvsBNEMvRkpVvWuM5hQdq7nS4c7p3k4Dng/8HfCY7pRwWnRPH3YBC+AJVXVCt7Gq3prkmiHUMx8PqarDAJL8DU3v+IFV9ePhlqWlallMAQf8ZntbWpLcj+aGG++mmVPwuKFWpnFyDnBzVX2j9wHczAhOy9VOu/NVmrkr3wYcDNxUVZdU1R3DrG22qurCqjqWZkzlxfTMlZ7kGUMtbg6SvBHYDPwHcFhVvcaAPHyjNOPLLNw6zbZbFq2KwfjpxEI7/dsOA/LSkORLSa5KchXwsInlnrbh1LVMhltsraqJkHwq8NSqOqad7P0j4/b1mIYjyeaqetwU27400YMxKpJcSfNB+t3AhqraMS5DFODOqZKeSzMF3Ej1ALbDR26jmX+795f4yM1wo6UtyXbg5ZNtAs6uqkMWuaQ5G6eZocZNO03ilN/ETgyTW2zLZbhF7ywJRwEfAKiqb4/aHYO0pN1/mm33WawiBqWqHt0zp+jHk3wX2HvULtqbStvzel77GClVtVy+BdTwXQr8+hTbRmrY1ZjNDDVuzgFe2X77eqf2YstzmPr/wQW1XHqSL6aZAm4nzVetD2sD8h7Al6vqYdM+gdSHJP8AfLKq/rrT/mKaG6Y8bziVDYZzikqSFsJS/SZ2ufQkvwR4K82UNadW1cRFVE8HPjy0qjRuTgX+KcnzgcvbtrXAnsBvDKuoQXFOUWn5SfK702yuqnrPohWjcXb/abYN7ZvYZRGSq+qrwLpJ2i9K8vAhlKQx1A5B+M9JnkYzewrAh6vqk0Msa85mmlN0iKVJWjyT9u4BR9P8bjAkaxC2JPn9Kb6JvXyKYxbcshhuMZ0k11fVgcOuQ1pqnFNUUq80F/E8n2Za1auBs6rKD8yatyT7Af9Ecw3Zz3wT2zMCYHHrMiTnhqo6YNh1SEtN7ziwJCtwTlFpWWqv3zmBZpaLy4DXVdWozZGsEdD5JnbrsL+JXRbDLWawvD8lSFO725yiSZxTVFpmkpxEc0fHTwDrquq64VakcVZVF9NMsLAkLIue5CT/weRhOMB9qsoPC1KHc4pKaufk/g6wi8nn5H7UUAqTFsGyCIdVtfewa5BG0L2r6qcz7yZpjK0edgHSsCyLnmRJs5fkC1X1mGHXIUnSMCyLnmRJc+LtKKVlbobhig670lizJ1nSpJLsAN481faqmnKbJEmjzp5kSVNZAeyFPcqSpGXInmRJk3JMsiRpObvHsAuQtGTZgyxJWrbsSZY0qSQPBn4beCjwJeCdVbV7uFVJkrQ4DMmSJpXkfTR33fs/wLOAb1TVKcOtSpKkxWFIljSpJF+qqsPa5T2AzztGWZK0XDgmWdJU7rzbnsMsJEnLjT3JkiaV5HbgRxOrwH2AW/AmApKkZcCQLEmSJHU43EKSJEnqMCRLkiRJHYZkSZIkqcOQLEmSJHUYkiVJkqSO/wuR/KZ/d1DEuQAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "fig, ax = plt.subplots(1, 1, figsize=(12, 5))\n",
    "labels = boston['feature_names'][fet_ind]\n",
    "pd.Series(fet_imp, index=labels).plot(kind='bar', ax=ax)\n",
    "ax.set_title('Features importance')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "**LSTAT** и **RM** имеют наибольшую важность."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## *Задание 4\n",
    "\n",
    "В этом задании мы будем работать с датасетом, с которым мы уже\n",
    "знакомы по домашнему заданию по библиотеке Matplotlib, это датасет *Credit Card Fraud Detection*. Для этого датасета мы будем решать задачу классификации - будем определять, какие из транзакциции по кредитной карте являются мошенническими. Данный датасет сильно несбалансирован (так как случаи мошенничества относительно редки), так что применение метрики accuracy не принесет пользы и не поможет выбрать лучшую модель. Мы будем вычислять AUC, то есть площадь под кривой ROC.\n",
    "\n",
    "Импортируйте из соответствующих модулей **RandomForestClassifier**, **GridSearchCV** и **train_test_split**.\n",
    "\n",
    "Загрузите датасет **creditcard.csv** и создайте датафрейм df.\n",
    "\n",
    "С помощью метода **value_counts** с аргументом **normalize=True** убедитесь в том, что выборка несбалансирована. Используя метод **info**, проверьте, все ли столбцы содержат числовые данные и нет ли в них пропусков. Примените следующую настройку, чтобы можно было просматривать все столбцы датафрейма:\n",
    "**pd.options.display.max_columns = 100**.\n",
    "\n",
    "Просмотрите первые 10 строк датафрейма df.\n",
    "\n",
    "Создайте датафрейм X из датафрейма df, исключив столбец Class. Создайте объект Series под названием y из столбца Class.\n",
    "\n",
    "Разбейте X и y на тренировочный и тестовый наборы данных при помощи функции train_test_split, используя аргументы: **test_size=0.3, random_state=100, stratify=y**.\n",
    "У вас должны получиться объекты **X_train, X_test, y_train** и **y_test**.\n",
    "Просмотрите информацию о их форме.\n",
    "\n",
    "Для поиска по сетке параметров задайте такие параметры:\n",
    "**parameters = [{'n_estimators': [10, 15],\n",
    "'max_features': np.arange(3, 5),\n",
    "'max_depth': np.arange(4, 7)}]**\n",
    "\n",
    "Создайте модель GridSearchCV со следующими аргументами:\n",
    "**estimator=RandomForestClassifier(random_state=100),\n",
    "param_grid=parameters,\n",
    "scoring='roc_auc',\n",
    "cv=3.**\n",
    "\n",
    "Обучите модель на тренировочном наборе данных (может занять несколько минут).\n",
    "\n",
    "Просмотрите параметры лучшей модели с помощью атрибута **best_params_**.\n",
    "\n",
    "Предскажите вероятности классов с помощью полученной модели и метода **predict_proba**.\n",
    "\n",
    "Из полученного результата (массив Numpy) выберите столбец с индексом 1 (вероятность класса 1) и запишите в массив **y_pred_proba**. Из модуля **sklearn.metrics** импортируйте метрику **roc_auc_score**.\n",
    "\n",
    "Вычислите AUC на тестовых данных и сравните с результатом,\n",
    "полученным на тренировочных данных, используя в качестве аргументов\n",
    "массивы **y_test** и **y_pred_proba**."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Подключаем библиотеки"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "outputs": [],
   "source": [
    "import os\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import roc_auc_score"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Считываем набор данных"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "outputs": [],
   "source": [
    "PATH_TO_DATA = 'data/'\n",
    "df = pd.read_csv(os.path.join(PATH_TO_DATA, 'creditcard.csv'))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "outputs": [
    {
     "data": {
      "text/plain": "0    0.998273\n1    0.001727\nName: Class, dtype: float64"
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Class'].value_counts(normalize=True)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Заметен дисбаланс в распределении целевой переменной\n",
    "\n",
    "#### Исследуем набор данных"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 284807 entries, 0 to 284806\n",
      "Data columns (total 31 columns):\n",
      " #   Column  Non-Null Count   Dtype  \n",
      "---  ------  --------------   -----  \n",
      " 0   Time    284807 non-null  float64\n",
      " 1   V1      284807 non-null  float64\n",
      " 2   V2      284807 non-null  float64\n",
      " 3   V3      284807 non-null  float64\n",
      " 4   V4      284807 non-null  float64\n",
      " 5   V5      284807 non-null  float64\n",
      " 6   V6      284807 non-null  float64\n",
      " 7   V7      284807 non-null  float64\n",
      " 8   V8      284807 non-null  float64\n",
      " 9   V9      284807 non-null  float64\n",
      " 10  V10     284807 non-null  float64\n",
      " 11  V11     284807 non-null  float64\n",
      " 12  V12     284807 non-null  float64\n",
      " 13  V13     284807 non-null  float64\n",
      " 14  V14     284807 non-null  float64\n",
      " 15  V15     284807 non-null  float64\n",
      " 16  V16     284807 non-null  float64\n",
      " 17  V17     284807 non-null  float64\n",
      " 18  V18     284807 non-null  float64\n",
      " 19  V19     284807 non-null  float64\n",
      " 20  V20     284807 non-null  float64\n",
      " 21  V21     284807 non-null  float64\n",
      " 22  V22     284807 non-null  float64\n",
      " 23  V23     284807 non-null  float64\n",
      " 24  V24     284807 non-null  float64\n",
      " 25  V25     284807 non-null  float64\n",
      " 26  V26     284807 non-null  float64\n",
      " 27  V27     284807 non-null  float64\n",
      " 28  V28     284807 non-null  float64\n",
      " 29  Amount  284807 non-null  float64\n",
      " 30  Class   284807 non-null  int64  \n",
      "dtypes: float64(30), int64(1)\n",
      "memory usage: 67.4 MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "1. Видим, что структурно данные состоят из 284,807 строк и 31 столбца.\n",
    "2. Все признаки вещественные, кроме целевой целочисленной переменной\n",
    "3. В данных нет пропусков.\n",
    "4. Время также представленно в виде вещественного числа и представляет время в секундах с начала первой транзакции"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Предобработка данных"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "outputs": [
    {
     "data": {
      "text/plain": "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n5   2.0 -0.425966  0.960523  1.141109 -0.168252  0.420987 -0.029728  0.476201   \n6   4.0  1.229658  0.141004  0.045371  1.202613  0.191881  0.272708 -0.005159   \n7   7.0 -0.644269  1.417964  1.074380 -0.492199  0.948934  0.428118  1.120631   \n8   7.0 -0.894286  0.286157 -0.113192 -0.271526  2.669599  3.721818  0.370145   \n9   9.0 -0.338262  1.119593  1.044367 -0.222187  0.499361 -0.246761  0.651583   \n\n         V8        V9       V10       V11       V12       V13       V14  \\\n0  0.098698  0.363787  0.090794 -0.551600 -0.617801 -0.991390 -0.311169   \n1  0.085102 -0.255425 -0.166974  1.612727  1.065235  0.489095 -0.143772   \n2  0.247676 -1.514654  0.207643  0.624501  0.066084  0.717293 -0.165946   \n3  0.377436 -1.387024 -0.054952 -0.226487  0.178228  0.507757 -0.287924   \n4 -0.270533  0.817739  0.753074 -0.822843  0.538196  1.345852 -1.119670   \n5  0.260314 -0.568671 -0.371407  1.341262  0.359894 -0.358091 -0.137134   \n6  0.081213  0.464960 -0.099254 -1.416907 -0.153826 -0.751063  0.167372   \n7 -3.807864  0.615375  1.249376 -0.619468  0.291474  1.757964 -1.323865   \n8  0.851084 -0.392048 -0.410430 -0.705117 -0.110452 -0.286254  0.074355   \n9  0.069539 -0.736727 -0.366846  1.017614  0.836390  1.006844 -0.443523   \n\n        V15       V16       V17       V18       V19       V20       V21  \\\n0  1.468177 -0.470401  0.207971  0.025791  0.403993  0.251412 -0.018307   \n1  0.635558  0.463917 -0.114805 -0.183361 -0.145783 -0.069083 -0.225775   \n2  2.345865 -2.890083  1.109969 -0.121359 -2.261857  0.524980  0.247998   \n3 -0.631418 -1.059647 -0.684093  1.965775 -1.232622 -0.208038 -0.108300   \n4  0.175121 -0.451449 -0.237033 -0.038195  0.803487  0.408542 -0.009431   \n5  0.517617  0.401726 -0.058133  0.068653 -0.033194  0.084968 -0.208254   \n6  0.050144 -0.443587  0.002821 -0.611987 -0.045575 -0.219633 -0.167716   \n7  0.686133 -0.076127 -1.222127 -0.358222  0.324505 -0.156742  1.943465   \n8 -0.328783 -0.210077 -0.499768  0.118765  0.570328  0.052736 -0.073425   \n9  0.150219  0.739453 -0.540980  0.476677  0.451773  0.203711 -0.246914   \n\n        V22       V23       V24       V25       V26       V27       V28  \\\n0  0.277838 -0.110474  0.066928  0.128539 -0.189115  0.133558 -0.021053   \n1 -0.638672  0.101288 -0.339846  0.167170  0.125895 -0.008983  0.014724   \n2  0.771679  0.909412 -0.689281 -0.327642 -0.139097 -0.055353 -0.059752   \n3  0.005274 -0.190321 -1.175575  0.647376 -0.221929  0.062723  0.061458   \n4  0.798278 -0.137458  0.141267 -0.206010  0.502292  0.219422  0.215153   \n5 -0.559825 -0.026398 -0.371427 -0.232794  0.105915  0.253844  0.081080   \n6 -0.270710 -0.154104 -0.780055  0.750137 -0.257237  0.034507  0.005168   \n7 -1.015455  0.057504 -0.649709 -0.415267 -0.051634 -1.206921 -1.085339   \n8 -0.268092 -0.204233  1.011592  0.373205 -0.384157  0.011747  0.142404   \n9 -0.633753 -0.120794 -0.385050 -0.069733  0.094199  0.246219  0.083076   \n\n   Amount  Class  \n0  149.62      0  \n1    2.69      0  \n2  378.66      0  \n3  123.50      0  \n4   69.99      0  \n5    3.67      0  \n6    4.99      0  \n7   40.80      0  \n8   93.20      0  \n9    3.68      0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Time</th>\n      <th>V1</th>\n      <th>V2</th>\n      <th>V3</th>\n      <th>V4</th>\n      <th>V5</th>\n      <th>V6</th>\n      <th>V7</th>\n      <th>V8</th>\n      <th>V9</th>\n      <th>V10</th>\n      <th>V11</th>\n      <th>V12</th>\n      <th>V13</th>\n      <th>V14</th>\n      <th>V15</th>\n      <th>V16</th>\n      <th>V17</th>\n      <th>V18</th>\n      <th>V19</th>\n      <th>V20</th>\n      <th>V21</th>\n      <th>V22</th>\n      <th>V23</th>\n      <th>V24</th>\n      <th>V25</th>\n      <th>V26</th>\n      <th>V27</th>\n      <th>V28</th>\n      <th>Amount</th>\n      <th>Class</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.0</td>\n      <td>-1.359807</td>\n      <td>-0.072781</td>\n      <td>2.536347</td>\n      <td>1.378155</td>\n      <td>-0.338321</td>\n      <td>0.462388</td>\n      <td>0.239599</td>\n      <td>0.098698</td>\n      <td>0.363787</td>\n      <td>0.090794</td>\n      <td>-0.551600</td>\n      <td>-0.617801</td>\n      <td>-0.991390</td>\n      <td>-0.311169</td>\n      <td>1.468177</td>\n      <td>-0.470401</td>\n      <td>0.207971</td>\n      <td>0.025791</td>\n      <td>0.403993</td>\n      <td>0.251412</td>\n      <td>-0.018307</td>\n      <td>0.277838</td>\n      <td>-0.110474</td>\n      <td>0.066928</td>\n      <td>0.128539</td>\n      <td>-0.189115</td>\n      <td>0.133558</td>\n      <td>-0.021053</td>\n      <td>149.62</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.0</td>\n      <td>1.191857</td>\n      <td>0.266151</td>\n      <td>0.166480</td>\n      <td>0.448154</td>\n      <td>0.060018</td>\n      <td>-0.082361</td>\n      <td>-0.078803</td>\n      <td>0.085102</td>\n      <td>-0.255425</td>\n      <td>-0.166974</td>\n      <td>1.612727</td>\n      <td>1.065235</td>\n      <td>0.489095</td>\n      <td>-0.143772</td>\n      <td>0.635558</td>\n      <td>0.463917</td>\n      <td>-0.114805</td>\n      <td>-0.183361</td>\n      <td>-0.145783</td>\n      <td>-0.069083</td>\n      <td>-0.225775</td>\n      <td>-0.638672</td>\n      <td>0.101288</td>\n      <td>-0.339846</td>\n      <td>0.167170</td>\n      <td>0.125895</td>\n      <td>-0.008983</td>\n      <td>0.014724</td>\n      <td>2.69</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1.0</td>\n      <td>-1.358354</td>\n      <td>-1.340163</td>\n      <td>1.773209</td>\n      <td>0.379780</td>\n      <td>-0.503198</td>\n      <td>1.800499</td>\n      <td>0.791461</td>\n      <td>0.247676</td>\n      <td>-1.514654</td>\n      <td>0.207643</td>\n      <td>0.624501</td>\n      <td>0.066084</td>\n      <td>0.717293</td>\n      <td>-0.165946</td>\n      <td>2.345865</td>\n      <td>-2.890083</td>\n      <td>1.109969</td>\n      <td>-0.121359</td>\n      <td>-2.261857</td>\n      <td>0.524980</td>\n      <td>0.247998</td>\n      <td>0.771679</td>\n      <td>0.909412</td>\n      <td>-0.689281</td>\n      <td>-0.327642</td>\n      <td>-0.139097</td>\n      <td>-0.055353</td>\n      <td>-0.059752</td>\n      <td>378.66</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1.0</td>\n      <td>-0.966272</td>\n      <td>-0.185226</td>\n      <td>1.792993</td>\n      <td>-0.863291</td>\n      <td>-0.010309</td>\n      <td>1.247203</td>\n      <td>0.237609</td>\n      <td>0.377436</td>\n      <td>-1.387024</td>\n      <td>-0.054952</td>\n      <td>-0.226487</td>\n      <td>0.178228</td>\n      <td>0.507757</td>\n      <td>-0.287924</td>\n      <td>-0.631418</td>\n      <td>-1.059647</td>\n      <td>-0.684093</td>\n      <td>1.965775</td>\n      <td>-1.232622</td>\n      <td>-0.208038</td>\n      <td>-0.108300</td>\n      <td>0.005274</td>\n      <td>-0.190321</td>\n      <td>-1.175575</td>\n      <td>0.647376</td>\n      <td>-0.221929</td>\n      <td>0.062723</td>\n      <td>0.061458</td>\n      <td>123.50</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2.0</td>\n      <td>-1.158233</td>\n      <td>0.877737</td>\n      <td>1.548718</td>\n      <td>0.403034</td>\n      <td>-0.407193</td>\n      <td>0.095921</td>\n      <td>0.592941</td>\n      <td>-0.270533</td>\n      <td>0.817739</td>\n      <td>0.753074</td>\n      <td>-0.822843</td>\n      <td>0.538196</td>\n      <td>1.345852</td>\n      <td>-1.119670</td>\n      <td>0.175121</td>\n      <td>-0.451449</td>\n      <td>-0.237033</td>\n      <td>-0.038195</td>\n      <td>0.803487</td>\n      <td>0.408542</td>\n      <td>-0.009431</td>\n      <td>0.798278</td>\n      <td>-0.137458</td>\n      <td>0.141267</td>\n      <td>-0.206010</td>\n      <td>0.502292</td>\n      <td>0.219422</td>\n      <td>0.215153</td>\n      <td>69.99</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>2.0</td>\n      <td>-0.425966</td>\n      <td>0.960523</td>\n      <td>1.141109</td>\n      <td>-0.168252</td>\n      <td>0.420987</td>\n      <td>-0.029728</td>\n      <td>0.476201</td>\n      <td>0.260314</td>\n      <td>-0.568671</td>\n      <td>-0.371407</td>\n      <td>1.341262</td>\n      <td>0.359894</td>\n      <td>-0.358091</td>\n      <td>-0.137134</td>\n      <td>0.517617</td>\n      <td>0.401726</td>\n      <td>-0.058133</td>\n      <td>0.068653</td>\n      <td>-0.033194</td>\n      <td>0.084968</td>\n      <td>-0.208254</td>\n      <td>-0.559825</td>\n      <td>-0.026398</td>\n      <td>-0.371427</td>\n      <td>-0.232794</td>\n      <td>0.105915</td>\n      <td>0.253844</td>\n      <td>0.081080</td>\n      <td>3.67</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>4.0</td>\n      <td>1.229658</td>\n      <td>0.141004</td>\n      <td>0.045371</td>\n      <td>1.202613</td>\n      <td>0.191881</td>\n      <td>0.272708</td>\n      <td>-0.005159</td>\n      <td>0.081213</td>\n      <td>0.464960</td>\n      <td>-0.099254</td>\n      <td>-1.416907</td>\n      <td>-0.153826</td>\n      <td>-0.751063</td>\n      <td>0.167372</td>\n      <td>0.050144</td>\n      <td>-0.443587</td>\n      <td>0.002821</td>\n      <td>-0.611987</td>\n      <td>-0.045575</td>\n      <td>-0.219633</td>\n      <td>-0.167716</td>\n      <td>-0.270710</td>\n      <td>-0.154104</td>\n      <td>-0.780055</td>\n      <td>0.750137</td>\n      <td>-0.257237</td>\n      <td>0.034507</td>\n      <td>0.005168</td>\n      <td>4.99</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>7.0</td>\n      <td>-0.644269</td>\n      <td>1.417964</td>\n      <td>1.074380</td>\n      <td>-0.492199</td>\n      <td>0.948934</td>\n      <td>0.428118</td>\n      <td>1.120631</td>\n      <td>-3.807864</td>\n      <td>0.615375</td>\n      <td>1.249376</td>\n      <td>-0.619468</td>\n      <td>0.291474</td>\n      <td>1.757964</td>\n      <td>-1.323865</td>\n      <td>0.686133</td>\n      <td>-0.076127</td>\n      <td>-1.222127</td>\n      <td>-0.358222</td>\n      <td>0.324505</td>\n      <td>-0.156742</td>\n      <td>1.943465</td>\n      <td>-1.015455</td>\n      <td>0.057504</td>\n      <td>-0.649709</td>\n      <td>-0.415267</td>\n      <td>-0.051634</td>\n      <td>-1.206921</td>\n      <td>-1.085339</td>\n      <td>40.80</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>7.0</td>\n      <td>-0.894286</td>\n      <td>0.286157</td>\n      <td>-0.113192</td>\n      <td>-0.271526</td>\n      <td>2.669599</td>\n      <td>3.721818</td>\n      <td>0.370145</td>\n      <td>0.851084</td>\n      <td>-0.392048</td>\n      <td>-0.410430</td>\n      <td>-0.705117</td>\n      <td>-0.110452</td>\n      <td>-0.286254</td>\n      <td>0.074355</td>\n      <td>-0.328783</td>\n      <td>-0.210077</td>\n      <td>-0.499768</td>\n      <td>0.118765</td>\n      <td>0.570328</td>\n      <td>0.052736</td>\n      <td>-0.073425</td>\n      <td>-0.268092</td>\n      <td>-0.204233</td>\n      <td>1.011592</td>\n      <td>0.373205</td>\n      <td>-0.384157</td>\n      <td>0.011747</td>\n      <td>0.142404</td>\n      <td>93.20</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>9.0</td>\n      <td>-0.338262</td>\n      <td>1.119593</td>\n      <td>1.044367</td>\n      <td>-0.222187</td>\n      <td>0.499361</td>\n      <td>-0.246761</td>\n      <td>0.651583</td>\n      <td>0.069539</td>\n      <td>-0.736727</td>\n      <td>-0.366846</td>\n      <td>1.017614</td>\n      <td>0.836390</td>\n      <td>1.006844</td>\n      <td>-0.443523</td>\n      <td>0.150219</td>\n      <td>0.739453</td>\n      <td>-0.540980</td>\n      <td>0.476677</td>\n      <td>0.451773</td>\n      <td>0.203711</td>\n      <td>-0.246914</td>\n      <td>-0.633753</td>\n      <td>-0.120794</td>\n      <td>-0.385050</td>\n      <td>-0.069733</td>\n      <td>0.094199</td>\n      <td>0.246219</td>\n      <td>0.083076</td>\n      <td>3.68</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.options.display.max_columns = 100\n",
    "df.head(10)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "outputs": [],
   "source": [
    "X, y = pd.DataFrame(df).drop(['Class'], axis=1), df['Class']\n",
    "# X.head(10)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                   test_size=0.3,\n",
    "                                                   random_state=100,\n",
    "                                                   stratify=y)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(199364, 30) (85443, 30) (199364,) (85443,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Кросс-валидация и настройка гиперпараметров модели\n",
    "[link](https://neerc.ifmo.ru/wiki/index.php?title=%D0%9D%D0%B0%D1%81%D1%82%D1%80%D0%BE%D0%B9%D0%BA%D0%B0_%D0%B3%D0%B8%D0%BF%D0%B5%D1%80%D0%BF%D0%B0%D1%80%D0%B0%D0%BC%D0%B5%D1%82%D1%80%D0%BE%D0%B2)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "parameters = [{'n_estimators': [10, 15],\n",
    "               'max_features': np.arange(3, 5),\n",
    "               'max_depth': np.arange(4, 7)}]\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": 156,
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "outputs": [],
   "source": [
    "model = GridSearchCV(estimator=RandomForestClassifier(random_state=100),\n",
    "                   param_grid=parameters,\n",
    "                   scoring='roc_auc',\n",
    "                   cv=3)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "outputs": [
    {
     "data": {
      "text/plain": "GridSearchCV(cv=3, estimator=RandomForestClassifier(random_state=100),\n             param_grid=[{'max_depth': array([4, 5, 6]),\n                          'max_features': array([3, 4]),\n                          'n_estimators': [10, 15]}],\n             scoring='roc_auc')",
      "text/html": "<style>#sk-container-id-9 {color: black;background-color: white;}#sk-container-id-9 pre{padding: 0;}#sk-container-id-9 div.sk-toggleable {background-color: white;}#sk-container-id-9 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-9 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-9 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-9 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-9 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-9 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-9 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-9 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-9 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-9 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-9 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-9 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-9 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-9 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-9 div.sk-item {position: relative;z-index: 1;}#sk-container-id-9 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-9 div.sk-item::before, #sk-container-id-9 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-9 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-9 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-9 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-9 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-9 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-9 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-9 div.sk-label-container {text-align: center;}#sk-container-id-9 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-9 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-9\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=3, estimator=RandomForestClassifier(random_state=100),\n             param_grid=[{&#x27;max_depth&#x27;: array([4, 5, 6]),\n                          &#x27;max_features&#x27;: array([3, 4]),\n                          &#x27;n_estimators&#x27;: [10, 15]}],\n             scoring=&#x27;roc_auc&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" ><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GridSearchCV</label><div class=\"sk-toggleable__content\"><pre>GridSearchCV(cv=3, estimator=RandomForestClassifier(random_state=100),\n             param_grid=[{&#x27;max_depth&#x27;: array([4, 5, 6]),\n                          &#x27;max_features&#x27;: array([3, 4]),\n                          &#x27;n_estimators&#x27;: [10, 15]}],\n             scoring=&#x27;roc_auc&#x27;)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" ><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(random_state=100)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" ><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(random_state=100)</pre></div></div></div></div></div></div></div></div></div></div>"
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, y_train)"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "outputs": [
    {
     "data": {
      "text/plain": "{'max_depth': 6, 'max_features': 3, 'n_estimators': 15}"
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.best_params_"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "outputs": [
    {
     "data": {
      "text/plain": "array([[9.99070828e-01, 9.29171738e-04],\n       [9.99704794e-01, 2.95206364e-04],\n       [9.99717846e-01, 2.82154033e-04],\n       ...,\n       [9.99717846e-01, 2.82154033e-04],\n       [9.99317795e-01, 6.82204754e-04],\n       [9.87539019e-01, 1.24609813e-02]])"
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_proba = model.predict_proba(X_test)\n",
    "y_pred_proba"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "outputs": [
    {
     "data": {
      "text/plain": "array([0.00092917, 0.00029521, 0.00028215, ..., 0.00028215, 0.0006822 ,\n       0.01246098])"
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred_proba = y_pred_proba[:, 1]\n",
    "y_pred_proba"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "outputs": [
    {
     "data": {
      "text/plain": "array([[9.99717846e-01, 2.82154033e-04],\n       [9.99717846e-01, 2.82154033e-04],\n       [9.99717846e-01, 2.82154033e-04],\n       ...,\n       [9.99717846e-01, 2.82154033e-04],\n       [9.99717846e-01, 2.82154033e-04],\n       [9.99737336e-01, 2.62663815e-04]])"
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_proba = model.predict_proba(X_train)\n",
    "y_train_proba"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "outputs": [
    {
     "data": {
      "text/plain": "array([0.00028215, 0.00028215, 0.00028215, ..., 0.00028215, 0.00028215,\n       0.00026266])"
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_proba = y_train_proba[:, 1]\n",
    "y_train_proba"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y_test sampling ROC AUC Score\n",
      "0.9462664156037156\n",
      "y_train sampling ROC AUC Score\n",
      "0.9703527882554751\n"
     ]
    }
   ],
   "source": [
    "print (\"y_test sampling ROC AUC Score\")\n",
    "print (roc_auc_score(y_true=y_test, y_score=y_pred_proba))\n",
    "print (\"y_train sampling ROC AUC Score\")\n",
    "print (roc_auc_score(y_true=y_train, y_score=y_train_proba))\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Дополнительные задания"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "1. Загрузите датасет Wine из встроенных датасетов sklearn.datasets с помощью функции load_wine в переменную data."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "data = load_wine()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "2. Полученный датасет не является датафреймом. Это структура данных, имеющая ключи аналогично словарю. Просмотрите тип данных этой структуры данных и создайте список data_keys, содержащий ее ключи.\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Тип data: <class 'sklearn.utils._bunch.Bunch'>\n"
     ]
    }
   ],
   "source": [
    "print(\"Тип data: {}\".format(type(data)))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ключи data: \n",
      "dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names'])\n"
     ]
    }
   ],
   "source": [
    "print(\"Ключи data: \\n{}\".format(data.keys()))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "3. Просмотрите данные, описание и названия признаков в датасете. Описание нужно вывести в виде привычного, аккуратно оформленного текста, без обозначений переноса строки, но с самими переносами и т.д."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _wine_dataset:\n",
      "\n",
      "Wine recognition dataset\n",
      "------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 178 (50 in each of three classes)\n",
      "    :Number of Attributes: 13 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      " \t\t- Alcohol\n",
      " \t\t- Malic acid\n",
      " \t\t- Ash\n",
      "\t\t- Alcalinity of ash  \n",
      " \t\t- Magnesium\n",
      "\t\t- Total phenols\n",
      " \t\t- Flavanoids\n",
      " \t\t- Nonflavanoid phenols\n",
      " \t\t- Proanthocyanins\n",
      "\t\t- Color intensity\n",
      " \t\t- Hue\n",
      " \t\t- OD280/OD315 of diluted wines\n",
      " \t\t- Proline\n",
      "\n",
      "    - class:\n",
      "            - class_0\n",
      "            - class_1\n",
      "            - class_2\n",
      "\t\t\n",
      "    :Summary Statistics:\n",
      "    \n",
      "    ============================= ==== ===== ======= =====\n",
      "                                   Min   Max   Mean     SD\n",
      "    ============================= ==== ===== ======= =====\n",
      "    Alcohol:                      11.0  14.8    13.0   0.8\n",
      "    Malic Acid:                   0.74  5.80    2.34  1.12\n",
      "    Ash:                          1.36  3.23    2.36  0.27\n",
      "    Alcalinity of Ash:            10.6  30.0    19.5   3.3\n",
      "    Magnesium:                    70.0 162.0    99.7  14.3\n",
      "    Total Phenols:                0.98  3.88    2.29  0.63\n",
      "    Flavanoids:                   0.34  5.08    2.03  1.00\n",
      "    Nonflavanoid Phenols:         0.13  0.66    0.36  0.12\n",
      "    Proanthocyanins:              0.41  3.58    1.59  0.57\n",
      "    Colour Intensity:              1.3  13.0     5.1   2.3\n",
      "    Hue:                          0.48  1.71    0.96  0.23\n",
      "    OD280/OD315 of diluted wines: 1.27  4.00    2.61  0.71\n",
      "    Proline:                       278  1680     746   315\n",
      "    ============================= ==== ===== ======= =====\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: class_0 (59), class_1 (71), class_2 (48)\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML Wine recognition datasets.\n",
      "https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data\n",
      "\n",
      "The data is the results of a chemical analysis of wines grown in the same\n",
      "region in Italy by three different cultivators. There are thirteen different\n",
      "measurements taken for different constituents found in the three types of\n",
      "wine.\n",
      "\n",
      "Original Owners: \n",
      "\n",
      "Forina, M. et al, PARVUS - \n",
      "An Extendible Package for Data Exploration, Classification and Correlation. \n",
      "Institute of Pharmaceutical and Food Analysis and Technologies,\n",
      "Via Brigata Salerno, 16147 Genoa, Italy.\n",
      "\n",
      "Citation:\n",
      "\n",
      "Lichman, M. (2013). UCI Machine Learning Repository\n",
      "[https://archive.ics.uci.edu/ml]. Irvine, CA: University of California,\n",
      "School of Information and Computer Science. \n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "  (1) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  Comparison of Classifiers in High Dimensional Settings, \n",
      "  Tech. Rep. no. 92-02, (1992), Dept. of Computer Science and Dept. of  \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Technometrics). \n",
      "\n",
      "  The data was used with many others for comparing various \n",
      "  classifiers. The classes are separable, though only RDA \n",
      "  has achieved 100% correct classification. \n",
      "  (RDA : 100%, QDA 99.4%, LDA 98.9%, 1NN 96.1% (z-transformed data)) \n",
      "  (All results using the leave-one-out technique) \n",
      "\n",
      "  (2) S. Aeberhard, D. Coomans and O. de Vel, \n",
      "  \"THE CLASSIFICATION PERFORMANCE OF RDA\" \n",
      "  Tech. Rep. no. 92-01, (1992), Dept. of Computer Science and Dept. of \n",
      "  Mathematics and Statistics, James Cook University of North Queensland. \n",
      "  (Also submitted to Journal of Chemometrics).\n",
      "\n",
      "...\n"
     ]
    }
   ],
   "source": [
    "print(data.DESCR[:5000] + '\\n...')"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "4. Сколько классов содержит целевая переменная датасета? Выведите названия классов."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "outputs": [
    {
     "data": {
      "text/plain": "['class_0', 'class_1', 'class_2']"
     },
     "execution_count": 237,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[x for x in data.target_names]"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "5. На основе данных датасета (они содержатся в двумерном массиве Numpy) и названий признаков создайте датафрейм под названием X."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "X = pd.DataFrame(data.data.copy(), columns=data.feature_names)\n",
    "X.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": 259,
   "outputs": [
    {
     "data": {
      "text/plain": "   alcohol  malic_acid   ash  alcalinity_of_ash  magnesium  total_phenols  \\\n0    14.23        1.71  2.43               15.6      127.0           2.80   \n1    13.20        1.78  2.14               11.2      100.0           2.65   \n2    13.16        2.36  2.67               18.6      101.0           2.80   \n3    14.37        1.95  2.50               16.8      113.0           3.85   \n4    13.24        2.59  2.87               21.0      118.0           2.80   \n\n   flavanoids  nonflavanoid_phenols  proanthocyanins  color_intensity   hue  \\\n0        3.06                  0.28             2.29             5.64  1.04   \n1        2.76                  0.26             1.28             4.38  1.05   \n2        3.24                  0.30             2.81             5.68  1.03   \n3        3.49                  0.24             2.18             7.80  0.86   \n4        2.69                  0.39             1.82             4.32  1.04   \n\n   od280/od315_of_diluted_wines  proline  \n0                          3.92   1065.0  \n1                          3.40   1050.0  \n2                          3.17   1185.0  \n3                          3.45   1480.0  \n4                          2.93    735.0  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>alcohol</th>\n      <th>malic_acid</th>\n      <th>ash</th>\n      <th>alcalinity_of_ash</th>\n      <th>magnesium</th>\n      <th>total_phenols</th>\n      <th>flavanoids</th>\n      <th>nonflavanoid_phenols</th>\n      <th>proanthocyanins</th>\n      <th>color_intensity</th>\n      <th>hue</th>\n      <th>od280/od315_of_diluted_wines</th>\n      <th>proline</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>14.23</td>\n      <td>1.71</td>\n      <td>2.43</td>\n      <td>15.6</td>\n      <td>127.0</td>\n      <td>2.80</td>\n      <td>3.06</td>\n      <td>0.28</td>\n      <td>2.29</td>\n      <td>5.64</td>\n      <td>1.04</td>\n      <td>3.92</td>\n      <td>1065.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>13.20</td>\n      <td>1.78</td>\n      <td>2.14</td>\n      <td>11.2</td>\n      <td>100.0</td>\n      <td>2.65</td>\n      <td>2.76</td>\n      <td>0.26</td>\n      <td>1.28</td>\n      <td>4.38</td>\n      <td>1.05</td>\n      <td>3.40</td>\n      <td>1050.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>13.16</td>\n      <td>2.36</td>\n      <td>2.67</td>\n      <td>18.6</td>\n      <td>101.0</td>\n      <td>2.80</td>\n      <td>3.24</td>\n      <td>0.30</td>\n      <td>2.81</td>\n      <td>5.68</td>\n      <td>1.03</td>\n      <td>3.17</td>\n      <td>1185.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>14.37</td>\n      <td>1.95</td>\n      <td>2.50</td>\n      <td>16.8</td>\n      <td>113.0</td>\n      <td>3.85</td>\n      <td>3.49</td>\n      <td>0.24</td>\n      <td>2.18</td>\n      <td>7.80</td>\n      <td>0.86</td>\n      <td>3.45</td>\n      <td>1480.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>13.24</td>\n      <td>2.59</td>\n      <td>2.87</td>\n      <td>21.0</td>\n      <td>118.0</td>\n      <td>2.80</td>\n      <td>2.69</td>\n      <td>0.39</td>\n      <td>1.82</td>\n      <td>4.32</td>\n      <td>1.04</td>\n      <td>2.93</td>\n      <td>735.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 259,
     "metadata": {},
     "output_type": "execute_result"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "6. Выясните размер датафрейма X и установите, имеются ли в нем пропущенные значения."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 178 entries, 0 to 177\n",
      "Data columns (total 13 columns):\n",
      " #   Column                        Non-Null Count  Dtype  \n",
      "---  ------                        --------------  -----  \n",
      " 0   alcohol                       178 non-null    float64\n",
      " 1   malic_acid                    178 non-null    float64\n",
      " 2   ash                           178 non-null    float64\n",
      " 3   alcalinity_of_ash             178 non-null    float64\n",
      " 4   magnesium                     178 non-null    float64\n",
      " 5   total_phenols                 178 non-null    float64\n",
      " 6   flavanoids                    178 non-null    float64\n",
      " 7   nonflavanoid_phenols          178 non-null    float64\n",
      " 8   proanthocyanins               178 non-null    float64\n",
      " 9   color_intensity               178 non-null    float64\n",
      " 10  hue                           178 non-null    float64\n",
      " 11  od280/od315_of_diluted_wines  178 non-null    float64\n",
      " 12  proline                       178 non-null    float64\n",
      "dtypes: float64(13)\n",
      "memory usage: 18.2 KB\n"
     ]
    },
    {
     "data": {
      "text/plain": "(178, 13)"
     },
     "execution_count": 239,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.info()\n",
    "X.shape"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "7. Добавьте в датафрейм поле с классами вин в виде чисел, имеющих тип данных numpy.int64. Название поля - 'target'."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 178 entries, 0 to 177\n",
      "Data columns (total 14 columns):\n",
      " #   Column                        Non-Null Count  Dtype  \n",
      "---  ------                        --------------  -----  \n",
      " 0   alcohol                       178 non-null    float64\n",
      " 1   malic_acid                    178 non-null    float64\n",
      " 2   ash                           178 non-null    float64\n",
      " 3   alcalinity_of_ash             178 non-null    float64\n",
      " 4   magnesium                     178 non-null    float64\n",
      " 5   total_phenols                 178 non-null    float64\n",
      " 6   flavanoids                    178 non-null    float64\n",
      " 7   nonflavanoid_phenols          178 non-null    float64\n",
      " 8   proanthocyanins               178 non-null    float64\n",
      " 9   color_intensity               178 non-null    float64\n",
      " 10  hue                           178 non-null    float64\n",
      " 11  od280/od315_of_diluted_wines  178 non-null    float64\n",
      " 12  proline                       178 non-null    float64\n",
      " 13  target                        178 non-null    int64  \n",
      "dtypes: float64(13), int64(1)\n",
      "memory usage: 19.6 KB\n"
     ]
    }
   ],
   "source": [
    "X['target'] = data.target.astype('int64')\n",
    "X.info()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "8. Постройте матрицу корреляций для всех полей X. Дайте полученному датафрейму название X_corr.\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "outputs": [
    {
     "data": {
      "text/plain": "                               alcohol  malic_acid       ash  \\\nalcohol                       1.000000    0.094397  0.211545   \nmalic_acid                    0.094397    1.000000  0.164045   \nash                           0.211545    0.164045  1.000000   \nalcalinity_of_ash            -0.310235    0.288500  0.443367   \nmagnesium                     0.270798   -0.054575  0.286587   \ntotal_phenols                 0.289101   -0.335167  0.128980   \nflavanoids                    0.236815   -0.411007  0.115077   \nnonflavanoid_phenols         -0.155929    0.292977  0.186230   \nproanthocyanins               0.136698   -0.220746  0.009652   \ncolor_intensity               0.546364    0.248985  0.258887   \nhue                          -0.071747   -0.561296 -0.074667   \nod280/od315_of_diluted_wines  0.072343   -0.368710  0.003911   \nproline                       0.643720   -0.192011  0.223626   \ntarget                       -0.328222    0.437776 -0.049643   \n\n                              alcalinity_of_ash  magnesium  total_phenols  \\\nalcohol                               -0.310235   0.270798       0.289101   \nmalic_acid                             0.288500  -0.054575      -0.335167   \nash                                    0.443367   0.286587       0.128980   \nalcalinity_of_ash                      1.000000  -0.083333      -0.321113   \nmagnesium                             -0.083333   1.000000       0.214401   \ntotal_phenols                         -0.321113   0.214401       1.000000   \nflavanoids                            -0.351370   0.195784       0.864564   \nnonflavanoid_phenols                   0.361922  -0.256294      -0.449935   \nproanthocyanins                       -0.197327   0.236441       0.612413   \ncolor_intensity                        0.018732   0.199950      -0.055136   \nhue                                   -0.273955   0.055398       0.433681   \nod280/od315_of_diluted_wines          -0.276769   0.066004       0.699949   \nproline                               -0.440597   0.393351       0.498115   \ntarget                                 0.517859  -0.209179      -0.719163   \n\n                              flavanoids  nonflavanoid_phenols  \\\nalcohol                         0.236815             -0.155929   \nmalic_acid                     -0.411007              0.292977   \nash                             0.115077              0.186230   \nalcalinity_of_ash              -0.351370              0.361922   \nmagnesium                       0.195784             -0.256294   \ntotal_phenols                   0.864564             -0.449935   \nflavanoids                      1.000000             -0.537900   \nnonflavanoid_phenols           -0.537900              1.000000   \nproanthocyanins                 0.652692             -0.365845   \ncolor_intensity                -0.172379              0.139057   \nhue                             0.543479             -0.262640   \nod280/od315_of_diluted_wines    0.787194             -0.503270   \nproline                         0.494193             -0.311385   \ntarget                         -0.847498              0.489109   \n\n                              proanthocyanins  color_intensity       hue  \\\nalcohol                              0.136698         0.546364 -0.071747   \nmalic_acid                          -0.220746         0.248985 -0.561296   \nash                                  0.009652         0.258887 -0.074667   \nalcalinity_of_ash                   -0.197327         0.018732 -0.273955   \nmagnesium                            0.236441         0.199950  0.055398   \ntotal_phenols                        0.612413        -0.055136  0.433681   \nflavanoids                           0.652692        -0.172379  0.543479   \nnonflavanoid_phenols                -0.365845         0.139057 -0.262640   \nproanthocyanins                      1.000000        -0.025250  0.295544   \ncolor_intensity                     -0.025250         1.000000 -0.521813   \nhue                                  0.295544        -0.521813  1.000000   \nod280/od315_of_diluted_wines         0.519067        -0.428815  0.565468   \nproline                              0.330417         0.316100  0.236183   \ntarget                              -0.499130         0.265668 -0.617369   \n\n                              od280/od315_of_diluted_wines   proline    target  \nalcohol                                           0.072343  0.643720 -0.328222  \nmalic_acid                                       -0.368710 -0.192011  0.437776  \nash                                               0.003911  0.223626 -0.049643  \nalcalinity_of_ash                                -0.276769 -0.440597  0.517859  \nmagnesium                                         0.066004  0.393351 -0.209179  \ntotal_phenols                                     0.699949  0.498115 -0.719163  \nflavanoids                                        0.787194  0.494193 -0.847498  \nnonflavanoid_phenols                             -0.503270 -0.311385  0.489109  \nproanthocyanins                                   0.519067  0.330417 -0.499130  \ncolor_intensity                                  -0.428815  0.316100  0.265668  \nhue                                               0.565468  0.236183 -0.617369  \nod280/od315_of_diluted_wines                      1.000000  0.312761 -0.788230  \nproline                                           0.312761  1.000000 -0.633717  \ntarget                                           -0.788230 -0.633717  1.000000  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>alcohol</th>\n      <th>malic_acid</th>\n      <th>ash</th>\n      <th>alcalinity_of_ash</th>\n      <th>magnesium</th>\n      <th>total_phenols</th>\n      <th>flavanoids</th>\n      <th>nonflavanoid_phenols</th>\n      <th>proanthocyanins</th>\n      <th>color_intensity</th>\n      <th>hue</th>\n      <th>od280/od315_of_diluted_wines</th>\n      <th>proline</th>\n      <th>target</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>alcohol</th>\n      <td>1.000000</td>\n      <td>0.094397</td>\n      <td>0.211545</td>\n      <td>-0.310235</td>\n      <td>0.270798</td>\n      <td>0.289101</td>\n      <td>0.236815</td>\n      <td>-0.155929</td>\n      <td>0.136698</td>\n      <td>0.546364</td>\n      <td>-0.071747</td>\n      <td>0.072343</td>\n      <td>0.643720</td>\n      <td>-0.328222</td>\n    </tr>\n    <tr>\n      <th>malic_acid</th>\n      <td>0.094397</td>\n      <td>1.000000</td>\n      <td>0.164045</td>\n      <td>0.288500</td>\n      <td>-0.054575</td>\n      <td>-0.335167</td>\n      <td>-0.411007</td>\n      <td>0.292977</td>\n      <td>-0.220746</td>\n      <td>0.248985</td>\n      <td>-0.561296</td>\n      <td>-0.368710</td>\n      <td>-0.192011</td>\n      <td>0.437776</td>\n    </tr>\n    <tr>\n      <th>ash</th>\n      <td>0.211545</td>\n      <td>0.164045</td>\n      <td>1.000000</td>\n      <td>0.443367</td>\n      <td>0.286587</td>\n      <td>0.128980</td>\n      <td>0.115077</td>\n      <td>0.186230</td>\n      <td>0.009652</td>\n      <td>0.258887</td>\n      <td>-0.074667</td>\n      <td>0.003911</td>\n      <td>0.223626</td>\n      <td>-0.049643</td>\n    </tr>\n    <tr>\n      <th>alcalinity_of_ash</th>\n      <td>-0.310235</td>\n      <td>0.288500</td>\n      <td>0.443367</td>\n      <td>1.000000</td>\n      <td>-0.083333</td>\n      <td>-0.321113</td>\n      <td>-0.351370</td>\n      <td>0.361922</td>\n      <td>-0.197327</td>\n      <td>0.018732</td>\n      <td>-0.273955</td>\n      <td>-0.276769</td>\n      <td>-0.440597</td>\n      <td>0.517859</td>\n    </tr>\n    <tr>\n      <th>magnesium</th>\n      <td>0.270798</td>\n      <td>-0.054575</td>\n      <td>0.286587</td>\n      <td>-0.083333</td>\n      <td>1.000000</td>\n      <td>0.214401</td>\n      <td>0.195784</td>\n      <td>-0.256294</td>\n      <td>0.236441</td>\n      <td>0.199950</td>\n      <td>0.055398</td>\n      <td>0.066004</td>\n      <td>0.393351</td>\n      <td>-0.209179</td>\n    </tr>\n    <tr>\n      <th>total_phenols</th>\n      <td>0.289101</td>\n      <td>-0.335167</td>\n      <td>0.128980</td>\n      <td>-0.321113</td>\n      <td>0.214401</td>\n      <td>1.000000</td>\n      <td>0.864564</td>\n      <td>-0.449935</td>\n      <td>0.612413</td>\n      <td>-0.055136</td>\n      <td>0.433681</td>\n      <td>0.699949</td>\n      <td>0.498115</td>\n      <td>-0.719163</td>\n    </tr>\n    <tr>\n      <th>flavanoids</th>\n      <td>0.236815</td>\n      <td>-0.411007</td>\n      <td>0.115077</td>\n      <td>-0.351370</td>\n      <td>0.195784</td>\n      <td>0.864564</td>\n      <td>1.000000</td>\n      <td>-0.537900</td>\n      <td>0.652692</td>\n      <td>-0.172379</td>\n      <td>0.543479</td>\n      <td>0.787194</td>\n      <td>0.494193</td>\n      <td>-0.847498</td>\n    </tr>\n    <tr>\n      <th>nonflavanoid_phenols</th>\n      <td>-0.155929</td>\n      <td>0.292977</td>\n      <td>0.186230</td>\n      <td>0.361922</td>\n      <td>-0.256294</td>\n      <td>-0.449935</td>\n      <td>-0.537900</td>\n      <td>1.000000</td>\n      <td>-0.365845</td>\n      <td>0.139057</td>\n      <td>-0.262640</td>\n      <td>-0.503270</td>\n      <td>-0.311385</td>\n      <td>0.489109</td>\n    </tr>\n    <tr>\n      <th>proanthocyanins</th>\n      <td>0.136698</td>\n      <td>-0.220746</td>\n      <td>0.009652</td>\n      <td>-0.197327</td>\n      <td>0.236441</td>\n      <td>0.612413</td>\n      <td>0.652692</td>\n      <td>-0.365845</td>\n      <td>1.000000</td>\n      <td>-0.025250</td>\n      <td>0.295544</td>\n      <td>0.519067</td>\n      <td>0.330417</td>\n      <td>-0.499130</td>\n    </tr>\n    <tr>\n      <th>color_intensity</th>\n      <td>0.546364</td>\n      <td>0.248985</td>\n      <td>0.258887</td>\n      <td>0.018732</td>\n      <td>0.199950</td>\n      <td>-0.055136</td>\n      <td>-0.172379</td>\n      <td>0.139057</td>\n      <td>-0.025250</td>\n      <td>1.000000</td>\n      <td>-0.521813</td>\n      <td>-0.428815</td>\n      <td>0.316100</td>\n      <td>0.265668</td>\n    </tr>\n    <tr>\n      <th>hue</th>\n      <td>-0.071747</td>\n      <td>-0.561296</td>\n      <td>-0.074667</td>\n      <td>-0.273955</td>\n      <td>0.055398</td>\n      <td>0.433681</td>\n      <td>0.543479</td>\n      <td>-0.262640</td>\n      <td>0.295544</td>\n      <td>-0.521813</td>\n      <td>1.000000</td>\n      <td>0.565468</td>\n      <td>0.236183</td>\n      <td>-0.617369</td>\n    </tr>\n    <tr>\n      <th>od280/od315_of_diluted_wines</th>\n      <td>0.072343</td>\n      <td>-0.368710</td>\n      <td>0.003911</td>\n      <td>-0.276769</td>\n      <td>0.066004</td>\n      <td>0.699949</td>\n      <td>0.787194</td>\n      <td>-0.503270</td>\n      <td>0.519067</td>\n      <td>-0.428815</td>\n      <td>0.565468</td>\n      <td>1.000000</td>\n      <td>0.312761</td>\n      <td>-0.788230</td>\n    </tr>\n    <tr>\n      <th>proline</th>\n      <td>0.643720</td>\n      <td>-0.192011</td>\n      <td>0.223626</td>\n      <td>-0.440597</td>\n      <td>0.393351</td>\n      <td>0.498115</td>\n      <td>0.494193</td>\n      <td>-0.311385</td>\n      <td>0.330417</td>\n      <td>0.316100</td>\n      <td>0.236183</td>\n      <td>0.312761</td>\n      <td>1.000000</td>\n      <td>-0.633717</td>\n    </tr>\n    <tr>\n      <th>target</th>\n      <td>-0.328222</td>\n      <td>0.437776</td>\n      <td>-0.049643</td>\n      <td>0.517859</td>\n      <td>-0.209179</td>\n      <td>-0.719163</td>\n      <td>-0.847498</td>\n      <td>0.489109</td>\n      <td>-0.499130</td>\n      <td>0.265668</td>\n      <td>-0.617369</td>\n      <td>-0.788230</td>\n      <td>-0.633717</td>\n      <td>1.000000</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_corr = X.corr()\n",
    "X_corr"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "9. Создайте список high_corr из признаков, корреляция которых с полем target по абсолютному значению превышает 0.5 (причем, само поле target не должно входить в этот список)."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "high_corr = X_corr.drop('target')[X_corr.abs() > 0.5].target\n",
    "high_corr = high_corr.dropna().index\n",
    "high_corr\n",
    "#"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "execution_count": 278,
   "outputs": [
    {
     "data": {
      "text/plain": "Index(['alcalinity_of_ash', 'total_phenols', 'flavanoids', 'hue',\n       'od280/od315_of_diluted_wines', 'proline'],\n      dtype='object')"
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "10. Удалите из датафрейма X поле с целевой переменной. Для всех признаков, названия которых содержатся в списке high_corr, вычислите квадрат их значений и добавьте в датафрейм X соответствующие поля с суффиксом '\\_2', добавленного к первоначальному названию признака. Итоговый датафрейм должен содержать все поля, которые, были в нем изначально, а также поля с признаками из списка high_corr, возведенными в квадрат. Выведите описание полей датафрейма X с помощью метода describe.\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 337,
   "outputs": [
    {
     "data": {
      "text/plain": "                                count           mean            std  \\\nalcohol                         178.0      13.000618       0.811827   \nmalic_acid                      178.0       2.336348       1.117146   \nash                             178.0       2.366517       0.274344   \nalcalinity_of_ash               178.0      19.494944       3.339564   \nmagnesium                       178.0      99.741573      14.282484   \ntotal_phenols                   178.0       2.295112       0.625851   \nflavanoids                      178.0       2.029270       0.998859   \nnonflavanoid_phenols            178.0       0.361854       0.124453   \nproanthocyanins                 178.0       1.590899       0.572359   \ncolor_intensity                 178.0       5.058090       2.318286   \nhue                             178.0       0.957449       0.228572   \nod280/od315_of_diluted_wines    178.0       2.611685       0.709990   \nproline                         178.0     746.893258     314.907474   \nalcalinity_of_ash_2             178.0     391.142865     133.671775   \ntotal_phenols_2                 178.0       5.657030       2.936294   \nflavanoids_2                    178.0       5.110049       4.211441   \nhue_2                           178.0       0.968661       0.443798   \nod280/od315_of_diluted_wines_2  178.0       7.322155       3.584316   \nproline_2                       178.0  656459.140449  555859.139636   \n\n                                       min            25%           50%  \\\nalcohol                            11.0300      12.362500      13.05000   \nmalic_acid                          0.7400       1.602500       1.86500   \nash                                 1.3600       2.210000       2.36000   \nalcalinity_of_ash                  10.6000      17.200000      19.50000   \nmagnesium                          70.0000      88.000000      98.00000   \ntotal_phenols                       0.9800       1.742500       2.35500   \nflavanoids                          0.3400       1.205000       2.13500   \nnonflavanoid_phenols                0.1300       0.270000       0.34000   \nproanthocyanins                     0.4100       1.250000       1.55500   \ncolor_intensity                     1.2800       3.220000       4.69000   \nhue                                 0.4800       0.782500       0.96500   \nod280/od315_of_diluted_wines        1.2700       1.937500       2.78000   \nproline                           278.0000     500.500000     673.50000   \nalcalinity_of_ash_2               112.3600     295.840000     380.25000   \ntotal_phenols_2                     0.9604       3.036325       5.54605   \nflavanoids_2                        0.1156       1.452100       4.55825   \nhue_2                               0.2304       0.612325       0.93125   \nod280/od315_of_diluted_wines_2      1.6129       3.754075       7.72840   \nproline_2                       77284.0000  250501.000000  453604.50000   \n\n                                        75%           max  \nalcohol                             13.6775  1.483000e+01  \nmalic_acid                           3.0825  5.800000e+00  \nash                                  2.5575  3.230000e+00  \nalcalinity_of_ash                   21.5000  3.000000e+01  \nmagnesium                          107.0000  1.620000e+02  \ntotal_phenols                        2.8000  3.880000e+00  \nflavanoids                           2.8750  5.080000e+00  \nnonflavanoid_phenols                 0.4375  6.600000e-01  \nproanthocyanins                      1.9500  3.580000e+00  \ncolor_intensity                      6.2000  1.300000e+01  \nhue                                  1.1200  1.710000e+00  \nod280/od315_of_diluted_wines         3.1700  4.000000e+00  \nproline                            985.0000  1.680000e+03  \nalcalinity_of_ash_2                462.2500  9.000000e+02  \ntotal_phenols_2                      7.8400  1.505440e+01  \nflavanoids_2                         8.2657  2.580640e+01  \nhue_2                                1.2544  2.924100e+00  \nod280/od315_of_diluted_wines_2      10.0489  1.600000e+01  \nproline_2                       970225.0000  2.822400e+06  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>count</th>\n      <th>mean</th>\n      <th>std</th>\n      <th>min</th>\n      <th>25%</th>\n      <th>50%</th>\n      <th>75%</th>\n      <th>max</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>alcohol</th>\n      <td>178.0</td>\n      <td>13.000618</td>\n      <td>0.811827</td>\n      <td>11.0300</td>\n      <td>12.362500</td>\n      <td>13.05000</td>\n      <td>13.6775</td>\n      <td>1.483000e+01</td>\n    </tr>\n    <tr>\n      <th>malic_acid</th>\n      <td>178.0</td>\n      <td>2.336348</td>\n      <td>1.117146</td>\n      <td>0.7400</td>\n      <td>1.602500</td>\n      <td>1.86500</td>\n      <td>3.0825</td>\n      <td>5.800000e+00</td>\n    </tr>\n    <tr>\n      <th>ash</th>\n      <td>178.0</td>\n      <td>2.366517</td>\n      <td>0.274344</td>\n      <td>1.3600</td>\n      <td>2.210000</td>\n      <td>2.36000</td>\n      <td>2.5575</td>\n      <td>3.230000e+00</td>\n    </tr>\n    <tr>\n      <th>alcalinity_of_ash</th>\n      <td>178.0</td>\n      <td>19.494944</td>\n      <td>3.339564</td>\n      <td>10.6000</td>\n      <td>17.200000</td>\n      <td>19.50000</td>\n      <td>21.5000</td>\n      <td>3.000000e+01</td>\n    </tr>\n    <tr>\n      <th>magnesium</th>\n      <td>178.0</td>\n      <td>99.741573</td>\n      <td>14.282484</td>\n      <td>70.0000</td>\n      <td>88.000000</td>\n      <td>98.00000</td>\n      <td>107.0000</td>\n      <td>1.620000e+02</td>\n    </tr>\n    <tr>\n      <th>total_phenols</th>\n      <td>178.0</td>\n      <td>2.295112</td>\n      <td>0.625851</td>\n      <td>0.9800</td>\n      <td>1.742500</td>\n      <td>2.35500</td>\n      <td>2.8000</td>\n      <td>3.880000e+00</td>\n    </tr>\n    <tr>\n      <th>flavanoids</th>\n      <td>178.0</td>\n      <td>2.029270</td>\n      <td>0.998859</td>\n      <td>0.3400</td>\n      <td>1.205000</td>\n      <td>2.13500</td>\n      <td>2.8750</td>\n      <td>5.080000e+00</td>\n    </tr>\n    <tr>\n      <th>nonflavanoid_phenols</th>\n      <td>178.0</td>\n      <td>0.361854</td>\n      <td>0.124453</td>\n      <td>0.1300</td>\n      <td>0.270000</td>\n      <td>0.34000</td>\n      <td>0.4375</td>\n      <td>6.600000e-01</td>\n    </tr>\n    <tr>\n      <th>proanthocyanins</th>\n      <td>178.0</td>\n      <td>1.590899</td>\n      <td>0.572359</td>\n      <td>0.4100</td>\n      <td>1.250000</td>\n      <td>1.55500</td>\n      <td>1.9500</td>\n      <td>3.580000e+00</td>\n    </tr>\n    <tr>\n      <th>color_intensity</th>\n      <td>178.0</td>\n      <td>5.058090</td>\n      <td>2.318286</td>\n      <td>1.2800</td>\n      <td>3.220000</td>\n      <td>4.69000</td>\n      <td>6.2000</td>\n      <td>1.300000e+01</td>\n    </tr>\n    <tr>\n      <th>hue</th>\n      <td>178.0</td>\n      <td>0.957449</td>\n      <td>0.228572</td>\n      <td>0.4800</td>\n      <td>0.782500</td>\n      <td>0.96500</td>\n      <td>1.1200</td>\n      <td>1.710000e+00</td>\n    </tr>\n    <tr>\n      <th>od280/od315_of_diluted_wines</th>\n      <td>178.0</td>\n      <td>2.611685</td>\n      <td>0.709990</td>\n      <td>1.2700</td>\n      <td>1.937500</td>\n      <td>2.78000</td>\n      <td>3.1700</td>\n      <td>4.000000e+00</td>\n    </tr>\n    <tr>\n      <th>proline</th>\n      <td>178.0</td>\n      <td>746.893258</td>\n      <td>314.907474</td>\n      <td>278.0000</td>\n      <td>500.500000</td>\n      <td>673.50000</td>\n      <td>985.0000</td>\n      <td>1.680000e+03</td>\n    </tr>\n    <tr>\n      <th>alcalinity_of_ash_2</th>\n      <td>178.0</td>\n      <td>391.142865</td>\n      <td>133.671775</td>\n      <td>112.3600</td>\n      <td>295.840000</td>\n      <td>380.25000</td>\n      <td>462.2500</td>\n      <td>9.000000e+02</td>\n    </tr>\n    <tr>\n      <th>total_phenols_2</th>\n      <td>178.0</td>\n      <td>5.657030</td>\n      <td>2.936294</td>\n      <td>0.9604</td>\n      <td>3.036325</td>\n      <td>5.54605</td>\n      <td>7.8400</td>\n      <td>1.505440e+01</td>\n    </tr>\n    <tr>\n      <th>flavanoids_2</th>\n      <td>178.0</td>\n      <td>5.110049</td>\n      <td>4.211441</td>\n      <td>0.1156</td>\n      <td>1.452100</td>\n      <td>4.55825</td>\n      <td>8.2657</td>\n      <td>2.580640e+01</td>\n    </tr>\n    <tr>\n      <th>hue_2</th>\n      <td>178.0</td>\n      <td>0.968661</td>\n      <td>0.443798</td>\n      <td>0.2304</td>\n      <td>0.612325</td>\n      <td>0.93125</td>\n      <td>1.2544</td>\n      <td>2.924100e+00</td>\n    </tr>\n    <tr>\n      <th>od280/od315_of_diluted_wines_2</th>\n      <td>178.0</td>\n      <td>7.322155</td>\n      <td>3.584316</td>\n      <td>1.6129</td>\n      <td>3.754075</td>\n      <td>7.72840</td>\n      <td>10.0489</td>\n      <td>1.600000e+01</td>\n    </tr>\n    <tr>\n      <th>proline_2</th>\n      <td>178.0</td>\n      <td>656459.140449</td>\n      <td>555859.139636</td>\n      <td>77284.0000</td>\n      <td>250501.000000</td>\n      <td>453604.50000</td>\n      <td>970225.0000</td>\n      <td>2.822400e+06</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 337,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = pd.DataFrame(data.data.copy(), columns=data.feature_names)\n",
    "X['target'] = data.target.astype('int64')\n",
    "X, X_2 = X.drop('target', axis=1), pd.DataFrame(np.square(X[high_corr].values), columns=list(map(lambda x: x+'_2', high_corr)))\n",
    "X = pd.concat([X, X_2], axis=1).describe().T\n",
    "X\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "colab": {
   "name": "hw-03.ipynb",
   "provenance": []
  },
  "interpreter": {
   "hash": "9e07c548dbda38a8b955cdf4e6f77a649b5c75e4e51e9adacbc198047690b300"
  },
  "kernel_info": {
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "nteract": {
   "version": "0.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}