HW-cifar10-calssification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training a CIFAR10 classifier\n",
    "\n",
    "## Q1 定義神經網路模型(30/100)\n",
    "\n",
    "- 請建構一個用於分類cifar10資料的神經網路模型\n",
    "- 於1個batch data測試正常後，得30分\n",
    "\n",
    "## Q2 訓練模型並畫出 learning curve(50/100)\n",
    "\n",
    "- 請訓練上題所定義的神經網路\n",
    "- 畫出訓練誤差的學習曲線，得50分\n",
    "\n",
    "## Q3 於測試資料上的正確率(20/100)\n",
    "\n",
    "- 請於測試資料上評估模型的好壞\n",
    "- 給出模型於測試資料上的正確率，得20分\n",
    "\n",
    "## 作業繳交規則\n",
    "\n",
    "- 請將.ipynb檔名\"HW-cifar10-calssification\"加上\"-學號\"並匯出成html格式\n",
    "- 將.ipynb與html上傳至e3以利批改\n",
    "\n",
    "## 參考資料\n",
    "\n",
    "- [Pytorch-cifar10-tutorial](http://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py)\n",
    "- [PyTorch documentation](http://pytorch.org/docs/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.autograd import Variable\n",
    "\n",
    "import torchvision\n",
    "from torchvision import datasets, transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 載入CIFAR10資料\n",
    "\n",
    "- 原始資料會下載在\"./data\"上一層的目錄底下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                               transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "batch_size = 32\n",
    "\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    datasets.CIFAR10('./data', train=True, download=True,\n",
    "                   transform=transform),\n",
    "    batch_size=batch_size)\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    datasets.CIFAR10('./data', train=False, transform=transform),\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 取出一部分資料來視覺化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def show_grid_img(img_tensor):\n",
    "    img_tensor = img_tensor/2 +0.5\n",
    "    i = torchvision.utils.make_grid(img_tensor).numpy()\n",
    "    i = np.transpose(i,(1,2,0))\n",
    "    plt.imshow(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data,target = next(iter(train_loader))\n",
    "show_grid_img(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 每一個batch的資料維度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q1 定義神經網路模型(40/100)\n",
    "\n",
    "## 請根據cifar10資料特性設計神經網路架構\n",
    "\n",
    "- 可以優先考慮使用CNN\n",
    "- 對於沒有GPU環境，請先嘗試較為簡單的模型，不然計算時間會太久"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        '''\n",
    "        \n",
    "        your code\n",
    "        \n",
    "        '''\n",
    "    def forward(self,x):\n",
    "        '''\n",
    "        \n",
    "        your code\n",
    "        \n",
    "        '''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 測試模型能否正常運作\n",
    "\n",
    "- 輸入 [batch_size , 3 , 32 , 32 ]\n",
    "- 輸出 [batch_size , 10 ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net = Net()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "predict = net(Variable(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "predict.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 測試能否計算loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "F.cross_entropy(predict,Variable(target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q2 訓練模型並畫出 learning curve\n",
    "\n",
    "## 調整 batch_size\n",
    "\n",
    "預設的 batch_size為32，可以嘗試64,128"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                               transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "batch_size = 32 # you can change batch size\n",
    "\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    datasets.CIFAR10('./data', train=True, download=False,\n",
    "                   transform=transform),\n",
    "    batch_size=batch_size)\n",
    "\n",
    "test_loader = torch.utils.data.DataLoader(\n",
    "    datasets.CIFAR10('./data', train=False, transform=transform),\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 選擇優化演算法\n",
    "\n",
    "- 建議使用Adam\n",
    "- 如想使用SGD需要仔細調整 learning_rate，或加入momentum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_op = optim.Adam(net.parameters())\n",
    "#train_op = optim.SGD(net.parameters(),lr=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 訓練神經網路\n",
    "\n",
    "- n_epochs 代表要把整個訓練資料循環過幾次\n",
    "- train_loss_history 用來保存每個epoch的平均 batch_train_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "n_epochs = 10 #you can change here\n",
    "train_loss_history = []\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    batch_train_loss_ = []\n",
    "    \n",
    "    for _, (data, target) in enumerate(train_loader):\n",
    "        \n",
    "        '''\n",
    "        \n",
    "        your code\n",
    "        \n",
    "        '''\n",
    "        \n",
    "        batch_train_loss_.append(batch_train_loss.data[0])\n",
    "    \n",
    "    batch_train_loss = np.mean(batch_train_loss_)\n",
    "    train_loss_history.append(batch_train_loss)\n",
    "    print \"epoch:%s , train_loss:%s\" % (epoch , batch_train_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 畫出每個 epoch對 train_loss learning curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.plot(range(0,epoch+1) , train_loss_history,'o--')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Q3 於測試資料上的正確率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "correct = 0\n",
    "n_testsets = 10000.0\n",
    "\n",
    "for _, (data, target) in enumerate(test_loader):\n",
    "    '''\n",
    "    \n",
    "    your code\n",
    "    \n",
    "    '''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "correct / n_testsets"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}