Chapter2 Training Machine Learning Algorithms for Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 概要\n",
    "教師あり学習の線形分類の基本的な考え方を学ぶ。そのためにパーセプトロンを実装してみる。\n",
    "\n",
    "## パーセプトロン\n",
    "1個のニューロンを模したアルゴリズムで、線形分離可能なデータセットに対して2つに分類することができる。軸索での他ニューロンからのインプットを\n",
    "\n",
    "重みベクトル・入力データベクトル^(i) + 定数（重みの0番目と考える）\n",
    "\n",
    "で現し、活性化を２値の階段関数（-1が定常, 1が発火）で考え、データを分類することを考える。\n",
    "\n",
    "教師あり学習を考えているので、データセットとしては[入力データベクトル^(0),入力データベクトル^(1), ... ]に対して[答え[0], 答え[1], ...]を想定している\n",
    "\n",
    "## パーセプトロンのルール\n",
    "1. 重みベクトルを0 or 小さいランダムな値で初期化\n",
    "2. 各データセット(i)ごとに\n",
    "    1. 階段関数(重みベクトル・入力データベクトル(i))を計算して推測値を出す\n",
    "    2. 推測値と教師と比べて、重みベクトルを更新\n",
    "    \n",
    "更新は、\n",
    "\n",
    "- 重み＿( j )    += 学習率 ・（答え - 推測値）・入力データベクトル^(i)＿(j)\n",
    "- 重み＿(0)   += 学習率 ・（答え - 推測値）\n",
    "\n",
    "で行う。（目的関数を（答え-推測値)^2 として、各重みで偏微分）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "入力値:0.0\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "# 例えば2次元の\n",
    "input_vecs = np.array([[1,2],[1,3], [-1,1], [-1,2]]) # データーセット\n",
    "awnser_value = [1,1,-1,-1] # 答え\n",
    "# に対して、重みは3次元で用意し、\n",
    "weight = np.zeros(3)\n",
    "# を考えインプットは\n",
    "input_value = np.dot(input_vecs[0], weight[1:]) + weight[0]\n",
    "print(\"入力値:\" +str(input_value))\n",
    "# を考える"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "予測値: -1\n"
     ]
    }
   ],
   "source": [
    "# 活性化関数は\n",
    "predict_value = np.where(input_value >0.0, 1, -1)\n",
    "print(\"予測値:\" , str(predict_value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.2  0.2  0.4]\n"
     ]
    }
   ],
   "source": [
    "# 学習は学習率で、\n",
    "learning_rate = 0.1\n",
    "# 学習率・（答え - 推測）は\n",
    "update = learning_rate * (awnser_value[0] - predict_value)\n",
    "# 重みの更新は\n",
    "weight[1:] += update * input_vecs[0]\n",
    "weight[0] += update\n",
    "print(weight)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "これを全データセットに対して行い、また繰り返し行うことで、精度を上げていく。\n",
    "\n",
    "使いやすいように、本通りにクラスで実装してみる。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Perceptron(object):\n",
    "    \"\"\"\n",
    "    eta : 学習率η\n",
    "    n_iter :  学習を何回繰り返すか\n",
    "    X : 入力のデータセット\n",
    "    y : 答えのデータセット\n",
    "    w_ : 重みベクトル\n",
    "    \"\"\"\n",
    "    def __init__(self, eta=0.01, n_iter=10):\n",
    "        self.eta = eta\n",
    "        self.n_iter = n_iter\n",
    "    \n",
    "    def fit(self, X , y):\n",
    "        \"\"\"\n",
    "        学習メソッド\n",
    "        X : np.array\n",
    "        y : np.array\n",
    "        \"\"\"\n",
    "        self.w_ = np.zeros(1 + X.shape[1]) # 入力データセット + 1で重み初期化\n",
    "        self.errors_ = [] #あとで学習の結果を見るため\n",
    "        for _ in range(self.n_iter):\n",
    "            error = 0\n",
    "            for xi, yi in zip(X, y):\n",
    "                update = self.eta * (yi - self.predict(xi))\n",
    "                self.w_[1:] += update * xi\n",
    "                self.w_[0] += update\n",
    "                error += int(update != 0.0)\n",
    "            self.errors_.append(error)\n",
    "        return self\n",
    "    \n",
    "    def net_input(self, X):\n",
    "        \"\"\"\n",
    "        活性化関数に入れる前の入力の値を決める。predictから使われる\n",
    "        return: 重みベクトル・入力データベクトル^(i) + 定数（重みの0番目と考える）\n",
    "        \"\"\"\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "    \n",
    "    def predict(self, X):\n",
    "        \"\"\"\n",
    "        予想値を出力する\n",
    "        return: 階段関数（入力値）\n",
    "        \"\"\"\n",
    "        return np.where(self.net_input(X) >=0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 実際にirisデータで学習してみる"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     0    1    2    3            4\n",
      "0  5.1  3.5  1.4  0.2  Iris-setosa\n",
      "1  4.9  3.0  1.4  0.2  Iris-setosa\n",
      "2  4.7  3.2  1.3  0.2  Iris-setosa\n",
      "3  4.6  3.1  1.5  0.2  Iris-setosa\n",
      "4  5.0  3.6  1.4  0.2  Iris-setosa\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "df = pd.io.parsers.read_csv(\n",
    "    filepath_or_buffer='data/iris.data',\n",
    "    header=None,\n",
    "    sep=',',\n",
    "    )\n",
    "\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "答え:[-1 -1 -1 -1 -1]\n"
     ]
    }
   ],
   "source": [
    "y = df.iloc[:100, 4].values # 答えを抽出\n",
    "y = np.where(y==\"Iris-setosa\", -1, 1)  # 答えを階段関数と同じ形式にする\n",
    "print(\"\\n答え:\" +str(y[:5]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.1,  1.4],\n",
       "       [ 4.9,  1.4],\n",
       "       [ 4.7,  1.3],\n",
       "       [ 4.6,  1.5],\n",
       "       [ 5. ,  1.4]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = df.iloc[:100, [0,2]].values # 入力ベクトルを抽出\n",
    "X[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEPCAYAAABFpK+YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGgVJREFUeJzt3XuwHOV55/HfTzoIjkwhR+DgCxiCyyZFjBZrA9iFKQ5m\nBY4dc5HNerFrDa5Udisxy02VjWPs4mSNnHWyRNycZXFcSoi92URErJ3YSLBZTTj2LthcZK7GLJRj\nYbNUfCMmYBbQs390H53RaObMO3O6e3qmv5+qLk339On3Oe+RHvV53nfedkQIANAcy0YdAACgWiR+\nAGgYEj8ANAyJHwAahsQPAA1D4geAhik98dteZXuL7UdsP2T7xLLbBAD0NlVBG9dI+kpEnGt7StLK\nCtoEAPTgMj/AZfsgSfdFxBtKawQAMJCySz2/IOkHtjfbvtf2jbanS24TALCIshP/lKS1kj4TEWsl\nPSfpoyW3CQBYRNk1/icl7YqIu/P9myX9dudJtlkwCAAGFBEe5utKveOPiKcl7bL9pvzQaZIe7nHu\nWG5XXHHFyGMg/tHHQfzjuY1z/EtRxayeiyR9wfZ+kp6Q9OEK2gQA9FB64o+Ib0o6vux2AABp+OTu\nEs3MzIw6hCUh/tEi/tEa9/iHVeo8/uQg7KhDHAAwLmwr6ji4CwCoHxI/ADQMiR8AGobEDwANQ+IH\ngIYh8QNAw5D4AaBhSPwA0DAkfgBoGBI/ADQMiR8AGobEDwANQ+IHgIYh8QNAw5D4AaBhSPwA0DAk\nfgBoGBI/ADQMiR8AGobEDwANQ+IHgIYh8QNAw5D4AaBhSPwA0DAkfqBiu3ZJc3ML+3Nz2TGgKiR+\noGLf+Y60fr3UamXb+vXZMaAqjohyG7C/I+kZSbslvRgRJ3Q5J8qOA6iTVks69dTs9Y4d0szMKKPB\nOLKtiPAwXztVdDBd7JY0ExE/rqAtAEAfVZR6XFE7wFiYm5POPTe709+xI3vdXvNvEsY7RqOKO/6Q\ndLvtlyXdGBGfraBNoLaOPFLaulU6+eRsf+vW7FgTzY93bNmS7Z97btYfhx8+0rAmXhU1/tdExFO2\nXyXpdkkXRsRXO86hxg80FOMdw6l1jT8insr//Afbt0g6QdJXO8+bnZ3d83pmZkYz/PQBYI9Wq6VW\nq1XItUq947e9UtKyiHjW9isk3SbpdyPito7zuOMHSrRrV1ZWmS8vzc1l5aVRl1Tm5rqXeubjHERd\nv8ey1PmO/1BJt9iOvK0vdCZ9AOWray29yPGOun6PdVR6jT8pCO74gdI1oZbehO9x3lLu+JlmCRSE\nqYnloF+LR+IHClLnpRjG+bMDqf06zt9j1Sj1AAWqa6lh3Ac+U/p13L/HQdV5cBdADRx++N4JcJhZ\nM3XXhO+xKJR6gIIUWWq46y7p+usX9q+/Pjs2rLrWyVPiKrqEU9e+qBJ3/EBBipya+I1vSBddJL30\nUrZ/2WXStddKJ5443PXqOtUxJa6il7ioa19UiRo/UFNXXy1demn2etMm6ZJLlna9uo4/jCKuuvbF\nIJjOCQBIRuIHauiTn8zu9jdtyrZLL82ODavIOnlKjTy1jj6KKZhM+6TGD9TSz/+8dOCB0nHHZfsH\nHpgdG1bVSyOk1tFHsUQ1y2JT4wdqq8516JTY6hz/JKDGj4lQ9DS7IksSRSpyqmaR8TPNsUEiYuRb\nFgaa7o47Ig45JGLHjmw75JDsWJnXK7rNFNddF2FHbNqUbXZ2bJi4ioy/yDZH0a9Nk+fN4XLusF9Y\n5Ebix7wdO7K/lVL2uorrFd1mik2bFtrctGnf97/73b0T5R13ZMe6KTL+lGulxDZI/BjOUhI/g7tA\nDdV5+YGU2OocP8QdP+ojtTyQejdZ15JESqknVZF9VmRfpP6M+M1geKLUg0lQZEJPvd4oEs+dd+6d\n6K+7Ljs2jCL7rMi+GMUYRdMsJfEznRNjiamCg6u6z1Lb42c5HKZzAgCSkfgxdvjI/YIil0Yoch5/\n6s+In+VoUOrB2Gnak5YWMzfXfWmEzlk0KX2Weq0UqT8jfpbDW0qph8QPjLkia+TU28cHNX4AQDIS\nPyZWSs26ruvmpCqyRk69vTn45C4mVsrSwEU+4nAUj/QrcolhlituDmr8mGgpNesiH3FIjRxVocYP\ndLFrl3T//Qv799/ffVnmxx9f2H/88e7lGZYsxiSh1IOJ9cUvZnfvmzZl+5dcIi1bJl144cI5V12V\n1fU/8pFs//rrpeXLs98C2qWUcdpr5O3nsEAZ6obEj4l11lnS7t3SxRdn+9dckx1rd9552SLE116b\n7V90UXas08knZ0m/vYzTmdCpkWNcVJL4bS+TdLekJyPizCraBA4/XFqzZmF/zZp9B1pPPFF6/vmF\nxH/OOcMN7M63x1LEGAdV1fgvlvRwRW1hwqVOwUyZnpg6hfHmm6Wzz1447+yzs2PDqOsjIdEcpd/x\n2z5M0rskbZR0WdntYfKlTsFMKb2MojyTMl4wiqmhaI7Sp3Pa3qIs6a+StKFbqYfpnBhUkVMwU1W9\nNAJTQ7GYpUznLPWO3/a7JT0dETttz0jqGeTs7Oye1zMzM5rhbzkA7NFqtdRqtYq52LBPcEnZJH1K\n0nclPSHpKUnPSrqpy3lLfhoNRmMUT7BKfXRhkbFt2RKxatXCk6JWrcqODaOuj4TEeFFdH7YeER+T\n9DFJsn2KslLPh8psE9UaRS36+OOzmv78fPypqexYHWJLUdexBzRHZUs2tCV+avwTps61aJYsxqQa\niyUbIuLvuiV9YFBMdQSWhrV6sCSjWMp3voTTamXb+vXZsTJjY8liTBJW58SSjOrReSlllyJj4xGB\nqBsevYjGod6OphuLGj9QlCKXTwCaqO90TtsnSZqVdER+vpXNHz2q3NAAAGXoW+qx/S1Jl0q6R9LL\n88cj4oeFBUGpBwOi1IOmK3vJhmci4tZhLg4AqJ+ed/y21+Yv/6Wk5ZK2Snph/v2IuLewILjjxwDm\n5rp/Irdz/Xtm4mCSlTKrx/aORb4uIuIdwzTYoy0SP5KlJvTU/yCAcVTqdE7bR0XEE/2OLQWJH2Vh\nLACTquzpnN0mym0ZpjEAwOj1HNy1/YuSfknSKtvr2946SNIBZQeG8jSl9t2+zILUvdTTlL4A2i02\nq+doSb8q6ZWS3tN2/KeSfr3MoFCuui5XXLSUpY2b0hdAu5Qa/9si4n+XGgQ1/spR+15AX2AclT2P\n/wO2z+s49oykuyPii8M0CgAYnZTB3f0lHSfpsXxbI+kwSb9m++oSY0NJWGJ4AX2BJkop9dwp6aSI\neDnfn5I0J+ntkh6IiGOWHASlnkoxoLmAvsC4Knse/6OSToiIZ/L9VZK+HhFH274vIt4yTMMdbZD4\nAWAAZc/j/31JO21vtv0nku6T9Ae2XyHpfwzTKMqT8lhCHl0INFvSg1hsv0bSCfnuNyLi+4UGwR1/\nYVKWKWApA2D8lf4ELtuv08J6/JKkiLhjmAZ7XJ/EX6CU6YlMYQTGW6nTOW1/WtL7JT0kaXd+OCQV\nlvgBANVJmcd/tqSjI+KFvmdi5FKWKUg5B8DkSpnVc6ukcyPi2dKCoNRTmJTpiUxhBMZf2dM5/0rS\nP5P0t9r7QSwXDdNgjzZI/AAwgLKXbPhSvgEAJkDqrJ5pSa+PiEdLCYI7fgAYSKkf4LL9Hkk7JW3L\n94+zzW8AADCmUj65O6vsw1s/kaSI2CnpqJSL297f9l2277P9kO1PDR0pAKAQKYn/xfl1etrs7npm\nh3wK6Kn5ej5rJL3D9kkDxogRYWkHYDKlJP6HbH9A0nLbb7R9naT/ldpARDyXv9w/b+/Hg4eJUZh/\nOlWrlW3r12fHAIy3lOmcKyVdLul0SZa0XdInI+JnSQ3YyyTdI+kNkm6IiH/f5RwGd2uKpR2Aeip1\nOmd+x355vg0sInZLeovtgyTdZvuUiPi7zvNmZ2f3vJ6ZmdEMGQYA9mi1Wmq1WoVcq+cdv+2/VrYm\nT1cRcebAjdmfkPRcRFzVcZw7/hpiFU+gvsq64/9PQ8azh+1DlA8O558FWCfpd5d6XVTjyCP3TvRb\nt2bHAIy3pA9wDX1x+1hJf6psbGCZpD+LiH3+Q+GOHwAGU/p6/GUj8QPAYMp+9CIAYIKQ+AGgYXoO\n7pYxqwcAMHqlzuoBANQPg7sAMIbKftj6GyX9nqRjJB0wfzwiklboBADUS8rg7mZJ/1nSS5JOlXST\npM+XGRQAoDwpiX86Iv5WWVno7yNiVtK7yw0LAFCWlGfuvpCvsPmY7QslfU/SgeWGBQAoS8qyzMdL\nekTSKyV9UtIqSb8fEXcWFgSDuwAwkEqWbMiXVY6I+OkwDfW5NokfAAZQ9sPWf9n2A5Lul/SA7W/a\n/ufDNAYAGL2UUs/9kj4SEXP5/tsl/VFErCksCO74AWAgZS/S9vJ80pekiPiqsqmdAIAxlHLHf7Wk\naUl/rmztnvdL+pnyufwRce+Sg+COHwAGUurgru0di7wdEfGOYRruaIPEDwAD4EEsANAwZc/qOdT2\n52zfmu8fY/vXhmkMADB6KYO7fyJpu6TX5vvflnRJWQEBAMqVkvgPiYi/lLRbkiLiJUkvlxoVAKA0\nKYn/n2wfrPxpXLbfKumZUqMCAJQmZZG2yyR9SdIbbH9N0qskva/UqAAApUma1WN7StLRkizp0Yh4\nsdAgmNUDAAMpe1bPucrW5H9I0tmS/sL22mEaAwCMXkqN/xMR8dN8jZ7TJH1O2RO5AABjKGmtnvzP\nd0v6bER8WdKK8kICAJQpJfF/z/Z/UbZGz1ds75/4dQCAGkpZq2elpHdKeiAiHrP9GknHRsRthQXB\n4C4ADKS2a/XYPkzSTZIOVfYBsM9GxLVdziPxA8AA6pz4Xy3p1RGx0/aBku6RdFZEfKvjPBI/AAyg\n7AexDC0i/m9E7MxfP6vsoe2vK7NNAMDiKhuktX2kpOMk3VVVmwCAfaUs2bBkeZnnZkkX53f++5id\nnd3zemZmRjMzM1WEBgBjodVqqdVqFXKt0h/Eki/38DeSbo2Ia3qcQ40/xfbt0lVXZa83bJDOOGMy\n2wTQV20HdyXJ9k2SfhARly1yDom/n+3bpXPOkZ5/PtufnpZuuaXcRDyKNgEkqe3gru2TJH1Q0jts\n32f7XtvvLLPNiXXVVQsJWMpez9+JT1KbAEpXao0/Ir4maXmZbQAABsPSC+Niw4as1DJvejo7Nmlt\nAihd6TX+pCCo8adhcBdArtaDu0lBkPgBYCC1HdwFANQPiR+L275dOv30bNu+vfz2Nm6UDj442zZu\nLL89qfrvERgxSj3orep5/Bs3Sh//+N7HrrxSuvzyctqT+KwCxhY1fpTj9NOl22/f+9i6ddJthT2K\nYW8HHyz96Ed7H1u9WvrhD8tpT6r+ewQKQo1/EqSUG4osg1xwgbTfftl2wQVLuxaA8RIRI9+yMBps\n27aI6ekIKdump7Nj7a68cuH9+e3KK4dr7/zz973W+efve16Rbaaour2ItL4HaijPm0PlXEo9dZBS\nbiiyDLLfftJLL+19bGpKevHFweMq2saN0h/+Yfb6ssvKre/P47MKGENLKfVUsiwzkOzyy6tJ9u3O\nOINkj0ahxl+2lNp9ytIIl3VZ3LTbMal//f6DH0w7tmGDtLxtqaXly7sv2ZA69lD1OEYqpnOiaYat\nERW5aVJr/IPUj7dti1i3Ltt6nXPllRGrV2dbr9p3Sv1+27aIZcsW3l+2rHubKddKrctXPY6Riho/\nxpSo8dfUKGrkKfX71LhSrpU69lD1OEYqpnNiTDGdEwCQjMTfTVE13w0bsjvkeVNTS1vWOKX+nVK/\nT40r5VqpYw9Fj2MU+TNi6Wk0zbA1oiI31anGX2TNt8ia9SC19H71+9RrpZy3bVvE1NTC+1NT5Y9j\nFF2XT4kLqBktocY/8qQfdUv869btm+zWrRvuWqtX73ut1avLvVZK/KnXSjmvyP5KNYo2gZpZSuKn\n1DOsoqcAMqUwQz8ApSPxd3rta/sfm1/R8fbbs+2cc7onqdSadcr1iqylv+c9+35dt2MpbRZZI0/t\n11NOSTsGoLthf1UoclOdSj1FlzdSatap10u5VkT/mnXR8RdVI0+Ni1IPsKRSD0s2lK3IJQhSr1Xk\nEgQpbbLkATBWKPV0qrq8Ucb1UtpbsWJhf8WKekxhTO2HUU3BZPwBk2LYXxWK3FSnUk9EteWNsq7X\nr60VKxbKJCtW1GcaY2o/VD0Fk6UdUDNiyQYMhGUKBkefoWZYsgEAkKxZiZ8abYZlCgZHn2GCNKfU\nMz9H/Pnns/3paemWW5o7G4WnTg2OPkONLKXUU2rit/05Sb8q6emIWLPIeeUnfmq0ACZInWv8myVx\nWwQANVJq4o+Ir0r6cZltJGPuNwBIqqDGb/sISX898lKPVH2NlnEFACWpbY1fqlnirxrjCgBKspTE\nX5u1emZnZ/e8npmZ0czMzMhiAYC6abVaarVahVyrijv+I5Xd8R+7yDmTecdPqQdASWpb6rH9XyXN\nSDpY0tOSroiIzV3Om8zELzH3G0Apapv4k4OY5MQPACWo8zx+AEDNkPgBoGFI/ADQMCR+AGgYEj8A\nNAyJHwAahsQPAA1D4geAhiHxA0DDkPgBoGFI/ADQMCR+AGgYEj8ANAyJHwAahsQPAA1D4geAhiHx\nA0DDkPgBoGFI/ADQMCR+AGgYEj8ANAyJHwAahsQPAA1D4geAhiHxA0DDkPgBoGFI/ADQMCR+AGiY\n0hO/7Xfa/pbtb9v+7bLbAwAsrtTEb3uZpOslnSHplySdZ/sXy2yzaq1Wa9QhLAnxjxbxj9a4xz+s\nsu/4T5D0WET8fUS8KOm/STqr5DYrNe5/cYh/tIh/tMY9/mGVnfhfJ2lX2/6T+TEAwIgwuAsADeOI\nKO/i9lslzUbEO/P9j0qKiPh0x3nlBQEAEyoiPMzXlZ34l0t6VNJpkp6S9HVJ50XEI6U1CgBY1FSZ\nF4+Il21fKOk2ZWWlz5H0AWC0Sr3jBwDUT6WDu7aX2b7X9pd6vH+t7cds77R9XJWxpVgsftun2P5J\n/v69tj8+ihh7sf0d29+0fZ/tr/c4p7b93y/+Mej/Vba32H7E9kO2T+xyTi37v1/sde5722/K/87c\nm//5jO2LupxX177vG/8w/V9qqaeLiyU9LOmgzjds/4qkN0TEG/O/WDdIemvF8fXTM/7cHRFxZoXx\nDGK3pJmI+HG3N8eg/xeNP1fn/r9G0lci4lzbU5JWtr9Z8/5fNPZcLfs+Ir4t6S3Sng+UPinplvZz\n6tz3KfHnBur/yu74bR8m6V2S/rjHKWdJukmSIuIuSatsH1pReH0lxC9JQ42wV8Ra/Odd6/5X//jn\nz6kd2wdJOjkiNktSRLwUEf/YcVot+z8xdqmmfd/hX0h6PCJ2dRyvZd930St+acD+r7LUs0nSb0nq\nNajQ+WGv76leH/bqF78kvS3/VfHLto+pKK5UIel229+w/etd3q97//eLX6pv//+CpB/Y3pz/Kn6j\n7emOc+ra/ymxS/Xt+3bvl/TnXY7Xte879YpfGrD/K0n8tt8t6emI2Knsf6ZxuDvYIzH+eyS9PiKO\nU7Y+0X+vMMQUJ0XEWmW/tXzE9ttHHdCA+sVf5/6fkrRW0mfy7+E5SR8dbUjJUmKvc99LkmzvJ+lM\nSVtGHcsw+sQ/cP9Xdcd/kqQzbT+h7H+sU23f1HHO9yQd3rZ/WH6sDvrGHxHPRsRz+etbJe1ne3X1\noXYXEU/lf/6DshrhCR2n1Ln/+8Zf8/5/UtKuiLg7379ZWTJtV9f+7xt7zft+3q9Iuif/+9Oprn3f\nrmf8w/R/JYk/Ij4WEa+PiKMk/StJ/zMiPtRx2pckfUja84nfn0TE01XE109K/O01QdsnKJsq+6OK\nQ+3K9krbB+avXyHpdEkPdpxW2/5Pib/O/Z/34y7bb8oPnaZskkC7WvZ/Sux17vs256l3maSWfd+h\nZ/zD9H/Vs3r2YvvfKlvC4caI+Irtd9n+P5L+SdKHRxlbivb4Jb3P9m9IelHS88rqcXVxqKRbnC2N\nMSXpCxFx2xj1f9/4Ve/+l6SLJH0h/5X9CUkfHqP+XzR21bzvba9UNjD6b9qOjUvf941fQ/Q/H+AC\ngIZhdU4AaBgSPwA0DIkfABqGxA8ADUPiB4CGIfEDQMOQ+DGxbJ9v+9UJ5222vT71eAFx/U7b6yNs\nP1B0G8BiSPyYZBeonottfaxjnw/ToFIkfoyF/M74Eduft/2w7b+0fUD+3lrbrXzlzlttv9r2eyX9\nsqTP56tK7m/7E7bvsn2/7RsGbL+zjUPz4zts/8f8ut+yfVJ+fNr2X9h+0PZW23fm1/g9SdN5TH+W\nX34qX/XyQdvbbO9fXM8B+yLxY5wcLen6iDhG0k8l/aazB4NcJ+m9EXG8pM2SNkbEX0m6W9IHImJt\nRLwg6bqIODEi1khama+62lePNj7VdsryiDhR0qWSZvNjvynpRxHxZkmfUL6wWUT8jqTn8pj+dX7u\nG/PY3izpGUnvHbxrgHQjXasHGNB3I+LO/PXnJf07SdslvVnZWv3zD2v5ftvXtC+hfZrt31L2BKmf\nU7bQ25cT2j26Txtb8z/vkXRE/vrtkq6WpIh4yPb9i1z/iYiYr/PfI+nIhJiAoZH4Mc5CWWJ/MCJO\nWuzEvHzyGUlrI+L7tq+QdEBiO/3aeCH/82X1/jflHq/bv37+GqlxAUOh1INx8novPOj7A5LmJD0q\n6VX5crqyPeWFJxD9oxaej3yAsv8ofpgv8fy+AdpdrI1evqZ8lcT83GPb3vt/tpe37Y/Vg4kw/kj8\nGCePKnv61sOSXinphoh4UVkS/7TtnZLuk/S2/Pw/lXSD7Xsl/UzZ85IfknSrpK+3XbfXrJqQpD5t\n9PraP5J0iO0HJf0HZWWlZ/L3bpT0QNvgLrN6UCmWZcZYsH2EpL+JiGP7nlwDtpdJ2i8iXrB9lKTb\nJR0dES+NODSAGj/GyjjdpayUtCN/eIkk/QZJH3XBHT8ANAw1fgBoGBI/ADQMiR8AGobEDwANQ+IH\ngIYh8QNAw/x/pYxYtiY4m2oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1086bab00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#プロットしてみる\n",
    "for xi, yi in zip(X, y):\n",
    "    if yi == -1:\n",
    "        plt.scatter(xi[0], xi[1], color=\"red\", marker=\"o\", label=\"setosa\")\n",
    "    else:\n",
    "        plt.scatter(xi[0], xi[1], color=\"blue\", marker=\"x\", label=\"versicolor\")\n",
    "plt.xlabel(\"petal length\")\n",
    "plt.ylabel(\"sepal length\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGQxJREFUeJzt3X2UZHV54PHv0w6ihCMvRlEZATGwMSQ4Y3MILoRpQ7IC\nyZHdVSO4HjeQTUaEQGNejBxxxj0bzvrHrrxlA8w6IFlcUd6WNw0qdty4K4GeaoEBHJKQCcLMaAKD\nR3ARup/9o+6F7qZ7+mXq1r1V9f2cU2eqqm/XPFMz07/+3lu3OjITSZJKQ3UPIElqFhcGSdIMLgyS\npBlcGCRJM7gwSJJmcGGQJM1Q6cIQEXtGxN0R0YqIzRFx4TzbXRIRj0TERESsqnImSdKurajywTPz\nuYh4V2Y+GxGvAL4dEcdm5rfLbSLiJOCtmXlYRPwycDlwTJVzSZLmV/mupMx8tri6Z/H7PTVrk1OA\na4pt7wb2iYgDqp5LkjS3yheGiBiKiBawHRjLzAdnbXIg8Ni0248X90mSatCNYpjKzNXASuD4iFhT\n9e8pSVq+So8xTJeZP4qI24GjgL+a9qHHgTdPu72yuG+GiPBNnSRpGTIzlrJ91a9K+tmI2Ke4/mrg\n14GJWZvdAny42OYYYGdm7pjr8TKzUZd169bVPkNmMjk5yZFH/j4wCSSwDphk1arfZ3Jysvb5mvRc\nOVN/zNTUuZo403JUXQxvBD4fEUF7EfqLzPxGRKwFMjOvzMw7IuLkiPhb4Bng9Ipn6jtDQ0OcfPJa\nHn10lMnJNfzkJw+y337nsnHjRxga8lQVSUtT9ctV7wfeMcf9V8y6fXaVc/S7556Da689gjvvvIg9\n9mhxySX7cuutF7Pvvi4Kkpaua8cY+tHIyEjdIwCwcSP80i/BMccMAcOcfvoHOeigIS68EDZsqHu6\ntqY8V9M50+I0cSZo5lxNnGk5Yrn7oLotIrJXZu2m556Dww6D66+Ho49+6f4nn2zff++98Ja31Def\npHpFBNmkg8+qXlkL0xcFgP33h49+FC6c801IJGl+FkMPm68WSlaDJIthwMxXCyWrQdJyWAw9aqFa\nKFkN0mCzGAbIQrVQshokLZXF0IMWWwslq0EaXBbDgFhsLZSsBklLYTH0mKXWQslqkAaTxTAAlloL\nJatB0mJZDD1kubVQshqkwWMx9Lnl1kLJapC0GBZDj9jdWihZDdJgsRj62O7WQslqkLQQi6EHdKoW\nSlaDNDgshj7VqVooWQ2SdsViaLhO10LJapAGg8XQhzpdCyWrQdJ8LIYGq6oWSlaD1P8shj5TVS2U\nrAZJc7EYGqrqWihZDVJ/sxj6SNW1ULIaJM1mMTRQt2qhZDVI/cti6BPdqoWS1SBpOouhYbpdCyWr\nQepPFkMf6HYtlKwGSSWLoUHqqoWS1SD1n8YVQ0SsjIi7ImJzRNwfEefMsc2aiNgZEZuKyyernKnJ\n6qqFktUgCSouhoh4A/CGzJyIiL2BceCUzHx42jZrgD/IzPcs8Fh9XQx110LJapD6S+OKITO3Z+ZE\ncf3HwEPAgXNsuqSh+1HdtVCyGiR17RhDRBwCjAG/WCwS5f1rgBuA7wOPA3+UmQ/O8fl9WwxNqYWS\n1SD1j8YVQ6nYjXQ9cO70RaEwDhyUmauAy4CbuzFTkzSlFkpWgzTYKi+GiFgB3AZ8JTMvXsT2jwLD\nmfnkrPtz3bp1L94eGRlhZGSkw9N2X9NqoWQ1SL1pbGyMsbGxF29/+tOfXnIxdGNhuAb4p8z82Dwf\nPyAzdxTXjwa+lJmHzLFdX+5K+vM/h9tug9tvr3uSl7vgAti+HTZsqHsSScu1nF1JVb8q6VjgW8D9\nQBaX84GDgczMKyPiLOBM4HngJ8B5mXn3HI/VdwtDU2uhZDVIva9xC0Mn9ePC0ORaKFkNUm9zYegh\nTa+FktUg9bbGvipJL9e0VyLNx1coSYPHYqhBr9RCyWqQepfF0CN6pRZKVoM0WCyGLuu1WihZDVJv\nshh6QK/VQslqkAaHxdBFvVoLJatB6j0WQ8P1ai2UrAZpMFgMXdLrtVCyGqTeYjE0WK/XQslqkPqf\nxdAF/VILJatB6h0WQ0P1Sy2UrAapv1kMFeu3WihZDVJvsBgaqN9qoWQ1SP3LYqhQv9ZCyWqQms9i\naJh+rYWS1SD1J4uhIv1eCyWrQWo2i6FB+r0WSlaD1H8shgoMSi2UrAapuSyGhhiUWihZDVJ/sRg6\nbNBqoWQ1SM1kMTTAoNVCyWqQ+ofF0EGDWgslq0FqHouhZoNaCyWrQeoPFkOHDHotlKwGqVkshhoN\nei2UrAap91kMHWAtzGQ1SM1hMdTEWpjJapB6m8Wwm6yFuVkNUjMspxgqXRgiYiVwDXAAMAVsyMxL\n5tjuEuAk4BngtzNzYo5tcnJykqGh+iNnamqKVqsFwHe+s5o77hji9ttrHqqBLrgAtm2b4swz28/V\n6tWrG/H3Jw2SJu5KegH4WGYeAbwTOCsifn76BhFxEvDWzDwMWAtcPt+DDQ+P0mptrnLeBbVamxke\nHuX447dy/PFbOe+8UU49td6ZmuqEEzZz9dWjHHdc+7lqwt+fpIV1dVdSRNwMXJqZ35h23+XANzPz\nuuL2Q8BIZu6Y9bkJk6xaNcr4+EW1fOc5NTXF8PAoExMX8dKaOlXrTE3lcyU1w3KKYUVVw8wWEYcA\nq4C7Z33oQOCxabcfL+7bwcsM8cADa/jIR1q86U3Dlcy5K0880eKBB0aYGVpDbNmyhlarxfBw92dq\nqlarxZYtI/hcSb2nKwtDROwNXA+cm5k/Xv4jrWdy8kHGx/flpz/9IIccMtKhCSWpP4yNjTE2NrZb\nj1H5rqSIWAHcBnwlMy+e4+OzdyU9DKxxV1Jv87mSmqGJB58BNgIPzrUoFG4BPgwQEccAO2cvCqW3\nv/1cNm5cW9sXlaGhITZuXMuqVaPstdcN7LXXDbXP1FSzn6sVK25g//19rqReUPXLVY8FvgXcD2Rx\nOR84GMjMvLLY7jLgRNovVz09MzfN8ViNfLmqL8HctfK5evppeN/7VjM+PuR5DVIXNe48hk5q6glu\nWrwLLoDt22HDhronkQaHC4MazbOhpe5r6jEGCfA9lKReYTGoq6wGqbssBjWe1SA1n8WgrrMapO6x\nGNQTrAap2SwG1cJqkLrDYlDPsBqk5rIYVBurQaqexaCeYjVIzWQxqFZWg1Qti0E9x2qQmsdiUO2s\nBqk6FoN6ktUgNYvFoEawGqRqWAzqWVaD1BwWgxrDapA6z2JQT7MapGawGNQoVoPUWRaDep7VINXP\nYlDjWA1S51gM6gtWg1SvJRdDROwHvDkz76tmpHl/X4thgFgNUmdUVgwRMRYRr4mI/YFNwIaI+K/L\nGVJaDKtBqs+iiiEiWpm5OiL+A+1aWBcR92XmkdWP+OIMFsOAsRqk3VflMYYVEfFG4LeA25Y8mbQM\nVoNUj8UuDP8R+Evg7zLznog4FHikurGktvPOgxtvhEcfrXsSaXD4clU13gUXwPbtsGFD3ZNIvafK\ng88rI+KmiPhBcbkhIlYu4vM+FxE7ImLOVzBFxJqI2BkRm4rLJ5cyvAaD1SB112J3JV0F3AK8qbjc\nWty3mM979wLbfCsz31Fc/tMi59EA8ViD1F2LXRhel5lXZeYLxeVq4HULfVJm/jXw1AKbLSlxNJis\nBql7Frsw/HNEfCgiXlFcPgT8c4dmeGdETETE7RHxCx16TPUZq0HqnhWL3O4M4FLgs0AC/wc4vQO/\n/zhwUGY+GxEnATcDh8+38fr161+8PjIywsjISAdGUK8477z2eQ3nn+95DdJ8xsbGGBsb263HWPBV\nSRHxCuCczPzssn6DiIOBWxdzMlxEPAoMZ+aTc3zMVyXJVyhJS1TJq5IycxI4bdlTtY8hzDlURBww\n7frRtBeqly0KUsljDVL1FvuWGJ8F9gCuA54p78/MTQt83heAEeC1wA5gHfDK9qfmlRFxFnAm8Dzw\nE+C8zLx7nseyGARYDdJSLKcYFrswfHOOuzMzf3Upv9nucGFQyfdQkhavkoUhIoaA92Xml3ZnuN3l\nwqDprAZpcaoshnsz86hlT9YBLgyazmqQFqfKd1f9ekT8YUS8OSL2Ly/LmFHqCM9rkKqz2GKY6zUg\nmZmHdn6keWewGDSD1SAtrLJdSU3gwqC5eKxB2rWO70qKiD+edv39sz5mxKt2ntcgdd5CxxhOnXb9\nE7M+dmKHZ5GWzGMNUucttDDEPNfnui3VwmqQOmuhhSHnuT7XbakWVoPUWbs8+BwRk7TfAiOAVwPP\nlh8CXpWZe1Q+4UuzePBZ8/IVStLcfFWSBpqvUJJezoVBA81qkF6uyjOfpcbzWIPUGRaD+orVIM1k\nMWjgWQ3S7rMY1HesBuklFoOE1SDtLotBfclqkNosBqlgNUjLZzGob1kNksUgzWA1SMtjMaivWQ0a\ndBaDNIvVIC2dxaC+ZzVokFkM0hysBmlpLAYNBKtBg8pikOZhNUiLZzFoYFgNGkSNK4aI+FxE7IiI\n+3axzSUR8UhETETEqirn0WArq+FP/3SK8fFxxsfHmZqaqnssqXGq3pV0FfDu+T4YEScBb83Mw4C1\nwOUVz6MBd8IJm7n66lGOO24rxx+/leHhUVqtzXWPJTVK5buSIuJg4NbMPHKOj10OfDMzrytuPwSM\nZOaOObZ1V5J2y9TUFMPDo0xMXMRL3xNNsWrVKOPjFzE05CE39Z/G7UpahAOBx6bdfry4T+q4VqvF\nli0jzPxnP8SWLWtotVo1TSU1z4q6B1iK9evXv3h9ZGSEkZGR2maRpCYaGxtjbGxstx6jabuSHgbW\nuCtJVXBXkgZRU3clRXGZyy3AhwEi4hhg51yLgtQJQ0NDbNy4llWrRtlrrxtYseIG9t//XDZuXOui\nIE1TaTFExBeAEeC1wA5gHfBKIDPzymKby4ATgWeA0zNz0zyPZTGoI6ampmi1Wjz9NLz//au5994h\nz2tQ31pOMXiCmwbapz4F27bBhg11TyJVw4VBWqInn4TDD4d77vFsaPWnph5jkBrL91CSXs5i0MCz\nGtTPLAZpGawGaSaLQcJqUP+yGKRlshqkl1gMUsFqUD+yGKTdYDVIbRaDNI3VoH5jMUi7yWqQLAbp\nZawG9ROLQeoAq0GDzmKQ5mA1qF9YDFKHWA0aZBaDNA+rQf3AYpA6yGrQoLIYpF2wGtTrLAapw6wG\nDSKLQVqA1aBeZjFIFbAaNGgsBmkRrAb1KotBqojVoEFiMUiLZDWoF1kMUoWsBg0Ki0FaAqtBvcZi\nkCpmNWgQWAzSElkN6iUWg9QFVoP6XeULQ0ScGBEPR8SWiPj4HB9fExE7I2JTcflk1TNJu2t0FG66\nCR59tO5JpM6rdFdSRAwBW4ATgCeAe4BTM/PhadusAf4gM9+zwGO5K0mN8qlPwbZtsGFD3ZNI82vi\nrqSjgUcyc2tmPg98EThlju2WNLTUBFaD+lXVC8OBwGPTbn+/uG+2d0bERETcHhG/UPFMUkd4rEH9\nqgkHn8eBgzJzFXAZcHPN80iLZjWoH62o+PEfBw6adntlcd+LMvPH065/JSL+W0Tsn5lPzn6w9evX\nv3h9ZGSEkZGRTs8rLcn0avBYg5pgbGyMsbGx3XqMqg8+vwL4Hu2Dz9uAvwFOy8yHpm1zQGbuKK4f\nDXwpMw+Z47E8+KxG8rwGNVnjDj5n5iRwNnAnsBn4YmY+FBFrI+L3is3eFxEPREQLuAj4QJUzSZ3m\nsQb1G898ljrAalBTNa4YpEFhNaifWAxSh1gNaiKLQaqR1aB+YTFIHWQ1qGksBqlmVoP6gcUgdZjV\noCaxGKQGsBrU6ywGqQJWg5rCYpAawmpQL7MYpIpYDWoCi0FqEKtBvcpikCpkNahuFoPUMFaDepHF\nIFXMalCdLAapgawG9RqLQeoCq0F1sRikhrIa1EssBqlLrAbVwWKQGsxqUK+wGKQushrUbRaD1HBW\ng3qBxSB1mdWgbrIYpB5gNajpLAapBlaDusVikHqE1aAmsxikmlgN6gaLQeohVoOaymKQamQ1qGqN\nLIaIODEiHo6ILRHx8Xm2uSQiHomIiYhYVfVMUlNYDWqiSheGiBgCLgPeDRwBnBYRPz9rm5OAt2bm\nYcBa4PIqZ+qksbGxukeYUxPncqb5jY7CjTdOccst41xxxRVMTU3VPRIAU1NTjI83ayZo5lxNnmk5\nqi6Go4FHMnNrZj4PfBE4ZdY2pwDXAGTm3cA+EXFAxXN1RFO+sMzWxLmcaX5bt25mjz1Gee97t3L2\n2XcxPDxKq7W51plarc0MD49y/PHNmampczV9puWoemE4EHhs2u3vF/ftapvH59hG6ktTU1OcccYV\n7NhxES+88G954YW3MTFxEWecUd93nuVMExMX8eyzzZipqXP1wkzLsaLDM0laglarxZYtI8z8Hm2I\n++5bw5o1Lfbdd7jrM+3c2eK++5o1U1Pn6p2ZlqbSVyVFxDHA+sw8sbj9J0Bm5membXM58M3MvK64\n/TCwJjN3zHosX5IkScuw1FclVV0M9wA/FxEHA9uAU4HTZm1zC3AWcF2xkOycvSjA0v9gkqTlqXRh\nyMzJiDgbuJN213wuMx+KiLXtD+eVmXlHRJwcEX8LPAOcXuVMkqRd65kT3CRJ3dH4t8SIiM9FxI6I\nuK/uWUoRsTIi7oqIzRFxf0Sc04CZ9oyIuyOiVczVmFOmImIoIjZFxC11z1KKiH+IiO8Wz9ff1D0P\nQETsExFfjoiHir/DX655nsOL52dT8evTDfm3/oni+bkvIq6NiFc2YKZzi68FtX49mOvrZUTsFxF3\nRsT3IuIvI2KfhR6n8QsDcBXtE+Sa5AXgY5l5BPBO4KzZJ+51W2Y+B7wrM1cDRwK/GhHH1jnTNOcC\nD9Y9xCxTwEhmrs7Mo+sepnAxcEdmvg14O/BQncNk5pbi+XkHMEx7V+9Ndc5UHK/8XWB1Zh5Je3f4\nqTXPdATwO8BRwCrgNyPi0JrGmevr5Z8AX8/MfwHcBXxioQdp/MKQmX8NPFX3HNNl5vbMnCiu/5j2\nf+Daz73IzGeLq3vS/rut/XmLiJXAycB/r3uWWYIG/fuPiNcAv5KZVwFk5guZ+aOax5ru14C/y8zH\nFtyyWj8Cfgr8TESsAPYCnqh3JN4G3J2Zz2XmJPAtYHknEOymeb5engJ8vrj+eeBfL/Q4jfmP0asi\n4hDa3yXcXe8kL+6yaQHbgbHMbMJ36Z8F/gho2sGsBL4WEfdExO/WPQzwFuCfIuKqYtfNlRHx6rqH\nmuYDwP+se4jMfAr4L8A/0j4Zdmdmfr3eqXgA+JVil81etL8RenPNM033+vKVnpm5HXj9Qp/gwrAb\nImJv4Hrg3KIcapWZU8WupJXA8RGxps55IuI3gB1FXUVxaYpji10kJ9PeFXhczfOsAN4B/Fkx17O0\ndwHULiL2AN4DfLkBsxwKnAccDLwJ2DsiPljnTJn5MPAZ4GvAHUALmKxzpgUs+E2aC8MyFRl7PfAX\nmfm/6p5numIXxO2093nW6VjgPRHx97S/23xXRFxT80wAZOa24tcf0t5vXvdxhu8Dj2XmvcXt62kv\nFE1wEjBePFd1Owr4dmY+Wey2uRH4lzXPRGZelZlHZeYIsBPYUvNI0+0o338uIt4A/GChT+iVhaFp\n320CbAQezMyL6x4EICJ+tny1QbEL4teBiTpnyszzM/OgzDyU9gHCuzLzw3XOBBARexW1R0T8DPCv\naO8OqE2R+o9FxOHFXSfQnAP2p9GA3UiF7wHHRMSrIiJoP0+1HqQHiIjXFb8eBPwb4At1jsPMr5e3\nAL9dXP/3wILfyDb+vZIi4gvACPDaiPhHYF15gK7GmY4F/h1wf7FPP4HzM/OrNY71RuDzxX+WIdol\n840a52myA4CbirdZWQFcm5l31jwTwDnAtcWum7+nASd7FvvMfw34vbpnAcjM7xbVOU57d00LuLLe\nqQC4ISL2B54HPlrXCwfm+noJ/GfgyxFxBrAV+K0FH8cT3CRJ0/XKriRJUpe4MEiSZnBhkCTN4MIg\nSZrBhUGSNIMLgyRpBhcGDbyImJz21tKbIuKPO/jYB0fE/Z16PKkbGn+Cm9QFzxTvT1QVTxZST7EY\npHnebiUiHo2IzxQ/EOY75XvsFxXwjYiYiIivFW8tTkS8PiJuLO5vRftnmAOsKN4t9YGI+GpE7Fls\nf07xA2cmijNWpUZwYZDg1bN2Jb1/2seeKn4gzJ/R/kE6AJcCV2XmKtrviXNpcf8ltN/ufBXtN8Db\nXNx/GHBpZv4i8DTw3uL+jwOriu0/UtUfTloq3xJDAy8ifpSZr5nj/kdp/1S8fyjeTXdbZr4uIn4I\nvCEzJ4v7n8jM10fED4ADM/P5aY9xMHBn8dOzKI5frMjMCyPiDto/Fe1m4ObMfKb6P620MItB2rWc\n5/pSPDft+iQvHdv7DeAy2nVxT0T4/1GN4D9Eaddv6f6B4tdTgf9bXP827beiBvgQ8L+L618HPgov\n/jS9skLme/yDMvOvaP9AntcAey99dKnzfFWSBK+KiE20v4An8NXMPL/42H4R8V3g//HSYnAOcFVE\n/CHwQ156e+xR4MqI+B3gBeBM2j9m9WWlUeyC+h/F4hHAxQ37Gc8aYB5jkOZRHGMYzswn655F6iZ3\nJUnz87smDSSLQZI0g8UgSZrBhUGSNIMLgyRpBhcGSdIMLgySpBlcGCRJM/x//EauJIMzkaAAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1086c5630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 学習してみる\n",
    "ppn = Perceptron(eta=0.1, n_iter=10)\n",
    "ppn.fit(X, y)\n",
    "plt.plot(range(1, len(ppn.errors_)+1), ppn.errors_, marker=\"o\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Erros\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "エラーがー少なくなっていることがわかる（予想できた）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adaptive Linear Neuron\n",
    "パーセプトロンとの違いは、活性化関数を恒等写像にし、分類はQuantizerを使うことである。\n",
    "\n",
    "- 入力ベクトル→重みベクトル→活性化関数（階段関数）\n",
    "- 入力ベクトル→重みベクトル→活性化関数（恒等写像）→Quantizer（階段関数）\n",
    "\n",
    "学習は活性化関数で行い、目的関数（ここでは二乗誤差関数）を最小化することを考える。最小化のアルゴリズムには最急降下法を用いる。\n",
    "\n",
    "二乗誤差関数は\n",
    "\n",
    "```python\n",
    "def cost_fucntion(X,Y):\n",
    "    \"\"\"擬似コードです\"\"\"\n",
    "    costs = 0\n",
    "        for xi, yi in zip(X, Y):\n",
    "            costs += np.power(yi - (np.dot(xi, w[1:])+w[0],2)/2        \n",
    "    return costs\n",
    "```\n",
    "\n",
    "で、すべてのデータセットに対しての和を考えているところがパーセプトロンとの違いです。そのため、バッチ学習と呼ばれたりもします。\n",
    "\n",
    "これを各重みで偏微分して、学習率をかけて目的関数を最小化する。\n",
    "\n",
    "```python\n",
    "def update_weight(X, Y, learning_rate)\n",
    "    \"\"\"擬似コードです\"\"\"\n",
    "    output = self.activate_func(X) # 活性化での出力（ベクトル）\n",
    "    erros = (Y - output)\n",
    "    w[1:] += learning_rate * X.T.dot(erros) # Xを転置して、内積\n",
    "    w[0]  += learning_rate * erros.sum()\n",
    "```\n",
    "\n",
    "パーセプトロンとは違い、学習がすべてのデータセットで行っていることがわかる。\n",
    "\n",
    "ただ、データセットが多いと計算量が問題になったり、また重みの変更が1イテレーションごとに1回なので、効率が悪い。そのため\n",
    "\n",
    "- 確率的勾配降下法\n",
    "- ミニバッチ\n",
    "\n",
    "などの手法が考えられている。\n",
    "\n",
    "\n",
    "# 確率的勾配降下法\n",
    "最急降下法から、各イテレーションごとの学習のデータセットをランダムに取得し、また重みの学習も各データセットごとに置き換えたもの。\n",
    "\n",
    "それにより、学習のスピードがはやく、大量のデータに対しても計算量が多くなり過ぎない。\n",
    "\n",
    "また、次々にデータが増えていくデータセットに対しても学習することができ、オンライン学習のうちの一つと言われる。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}