TF的套路.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TF的套路\n",
    "\n",
    "别误会，并非取自“套路太深”，而是习武之人都懂的武术的“套路”和“散打”两大类的“套路”，表示“一整套规定的动作”，如华山基本剑法：白云出岫、有凤来仪、天绅倒悬、白虹贯日、苍松迎客、金雁横空、无边落木、青山隐隐、古柏森森、无双无对、金玉满堂。\n",
    "\n",
    "操作TensorFlow也必须明白一些术语和规程：\n",
    "\n",
    "- 图（Graph）：表示计算任务\n",
    "- 会话（Session）：表示执行图的上下文\n",
    "- 张量（Tensor）：表示数据\n",
    "- 变量（Variable）：维护状态\n",
    "- Feed/Fetch：为任意操作赋值或获取其中的数据\n",
    "\n",
    "## 综述\n",
    "\n",
    "总之，TF是一套变成系统，用图来表示计算任务，图中的节点成为OP（OPeration）。一个OP有**0**个或多个张量，执行计算产出**0**个或多个张量。每个张量是一种类型化的多维数组。\n",
    "\n",
    "举个例子，某组图像集表示为一个四维浮点数组： $$[batch, height, width, channels]$$ 。\n",
    "\n",
    "而TF描绘了计算过程。\n",
    "\n",
    "为了进行计算，图必须在会话中被启动。\n",
    "\n",
    "会话把图的OP分发到CPU/GPU等设备上，同时提供执行OP的方法。\n",
    "\n",
    "这些方法执行完成，将产生新的张量并返回。\n",
    "\n",
    "- Python中返回的是numpy的ndarray对象\n",
    "- C/C++中返回的是tensorflow::Tensor实例\n",
    "\n",
    "### 计算图\n",
    "\n",
    "通常分阶段组织程序。\n",
    "\n",
    "在构建阶段，OP的执行步骤被描述成一个图。\n",
    "\n",
    "在执行阶段，在会话中执行图中的OP。\n",
    "\n",
    "举个例子，一般在构建阶段创建一个图来表示和训练神经网络，接着在执行阶段反复执行图中的训练OP。\n",
    "\n",
    "TF目前支持C/C++/Python/JS等编程语言，而Python是最早被支持且支持德最全面。\n",
    "\n",
    "*会话库是一致的*。\n",
    "\n",
    "### 构建图\n",
    "\n",
    "- 第一步：创建源OP\n",
    "\n",
    "    源OP不需要任何输入，如常量（Constant）。\n",
    "    \n",
    "    源OP的输出被传递给别的OP做计算。\n",
    "    \n",
    "    Python库中，OP构造器的返回至代表构造出的OP的输出，可以传递给别的OP的构造器作为输入。\n",
    "    \n",
    "    Python库中，有个默认图，OP构造器可为其添加节点，该默认图可满足大多程序，参考Graph类了解[TF如何管理多个图](./TF如何管理多个图.ipynb)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 创建一个常量OP、产出一个1*2的矩阵，该OP作为一个节点\n",
    "# 把该OP添加到默认图中\n",
    "\n",
    "# 构造器的返回值代表该常量OP的返回值\n",
    "\n",
    "matrix_1_2 = tf.constant([[3., 3.]]) # 一行两列\n",
    "\n",
    "matrix_2_1 = tf.constant([[2.], [2.]]) # 两行一列\n",
    "\n",
    "# 创建矩阵乘积OP，将matrix_1_2和matrix_2_1作为输入、返回值就是矩阵乘积的结果\n",
    "\n",
    "product = tf.matmul(matrix_1_2, matrix_2_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此时的默认图中有三个节点：\n",
    "\n",
    "- constant OP: 2\n",
    "- matmul OP: 1\n",
    "\n",
    "为了真正地进行矩阵乘积计算并得到结果，就必须在会话中启动这个图。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 在一个会话中启动图\n",
    "\n",
    "构造阶段完成后方可启动图。\n",
    "\n",
    "- 第一步：创建一个Session对象\n",
    "\n",
    "    如无任何创建参数，会话构造器将启动默认图。\n",
    "    \n",
    "    参考Session类了解完整的会话API。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 12.]]\n"
     ]
    }
   ],
   "source": [
    "# 启动默认图\n",
    "\n",
    "sess = tf.Session()\n",
    "\n",
    "# 调用sess的run()方法来执行矩阵乘积OP，传入product作为该方法的参数\n",
    "# 这里提到的product代表矩阵乘积OP的输出，传入它是想向方法表明：希望取回矩阵乘积OP的输出\n",
    "\n",
    "# 整个执行过程是自动的，会话负责传递OP所需的全部输入\n",
    "# OP的执行通常是并发的\n",
    "\n",
    "# 函数调用run(product)触发图中的三个OP的执行\n",
    "\n",
    "# 返回值result是个numpy的ndarray对象\n",
    "\n",
    "result = sess.run(product)\n",
    "print(result)\n",
    "\n",
    "# 任务完成关闭会话\n",
    "\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "关闭Session对象以释放资源，除了显式调用close之外，也可以用with代码块。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([[ 12.]], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    result = sess.run([product])\n",
    "    print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在实现上，TF将图定义转换成分布式执行的OP簇，以充分利用可用的计算资源（CPU/GPU）。\n",
    "\n",
    "一般不需要显式指定CPU或GPU，TF能自动检测。\n",
    "\n",
    "若检测到GPU，TF会尽可能利用找到的第一块GPU来执行OP。\n",
    "\n",
    "若机器上有多个GPU，除了第一块之外，别的GPU默认不参与计算。\n",
    "\n",
    "为了让TF能利用这些GPU，必须将OP明确指派执行设备，通过with...Device实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"MatMul_1:0\", shape=(1, 1), dtype=float32, device=/device:CPU:0)\n"
     ]
    }
   ],
   "source": [
    "DN = 0\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # with tf.device(\"/gpu:%d\" % DN): # TODO: 为何本地有个NVIDIA-GeForce-GT-705的显卡却报错找不到第一块显卡\n",
    "    with tf.device(\"/cpu:%d\" % DN):\n",
    "        matrix_1_2 = tf.constant([[3., 3.]])\n",
    "        matrix_2_1 = tf.constant([[2.],[2.]])\n",
    "        product = tf.matmul(matrix_1_2, matrix_2_1)\n",
    "        sess.run(product)\n",
    "        print(product)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设备采用串码标识，目前支持的设备：\n",
    "\n",
    "- /cpu:0 标识机器上第一块CPU\n",
    "- /gpu:0 标识机器上第一块GPU\n",
    "\n",
    "*以此类推*。\n",
    "\n",
    "参考[TF使用GPU](./TF使用GPU.ipynb)了解tensorflow-gpu的更多详情。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 交互式\n",
    "\n",
    "文档中的Python示例使用一个会话Session来启动图（Graph），并调用Session.run()方法来执行每个OP。\n",
    "\n",
    "为了便于应用如IPython之类的Python交互式环境，可以：\n",
    "\n",
    "- 使用InteractiveSession代替Session类\n",
    "- 使用Tensor.eval()和Operation.run()代替Session.run\n",
    "\n",
    "能避免通过一个变量来持有会话。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-2. -1.]\n"
     ]
    }
   ],
   "source": [
    "# 进入一个交互式TF会话\n",
    "\n",
    "sess = tf.InteractiveSession()\n",
    "\n",
    "x = tf.Variable([1.0, 2.0]) # 变量、一行两列\n",
    "a = tf.constant([3.0, 3.0]) # 常量、一行两列\n",
    "\n",
    "# 采用初始化器（initializer）OP的run()方法初始化x\n",
    "\n",
    "x.initializer.run()\n",
    "\n",
    "# 添加一个减法（sub）OP、从x减去a、运行该OP，输出结果\n",
    "\n",
    "sub = tf.subtract(x, a)\n",
    "print(sub.eval())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 张量（Tensor）\n",
    "\n",
    "这是TF中的统一数据结构，操作之间传递的都是张量。\n",
    "\n",
    "可以看作是N维数组。\n",
    "\n",
    "一个张量包含：\n",
    "\n",
    "- 阶rank\n",
    "- 数据类型type\n",
    "- 形状shape\n",
    "\n",
    "详见[张量的阶、数据类型和形状](./TF中张量的阶、数据类型和形状.ipynb)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 变量\n",
    "\n",
    "详见Variable类，参考[TF的变量](./TF的变量.ipynb)。\n",
    "\n",
    "变量来维护图执行过程中的状态。\n",
    "\n",
    "看一个的计数器的例程："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "# 创建一个变量，初始化为零（标量）\n",
    "\n",
    "state = tf.Variable(0, name=\"counter\")\n",
    "\n",
    "# 创建一个让state加一的OP\n",
    "\n",
    "one = tf.constant(1)\n",
    "\n",
    "egg = tf.add(state, one)\n",
    "\n",
    "update = tf.assign(state, egg)\n",
    "\n",
    "# 启动默认图之前，变量必须初始化（通过初始化OP实现）\n",
    "\n",
    "# 添加初始化OP到图中\n",
    "\n",
    "iOP = tf.global_variables_initializer()\n",
    "\n",
    "# 启动图，运行所有OP\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(iOP)\n",
    "    print(sess.run(state))\n",
    "    for _ in range(3):\n",
    "        sess.run(update)\n",
    "        print(sess.run(state))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "代码中的assign()操作时图所描绘的表达式的一部分，和add()操作一样。因此在调用run()执行表达式之前，并不会真正执行赋值操作。\n",
    "\n",
    "通常会将一个统计模型中的参数表示为一组变量。举个例子，可以将一个神经网络的权重作为某个变量存储在一个张量中，在训练过程中，通过迭代执行训练图更新这个张量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fetch\n",
    "\n",
    "为了去出操作的输出内容，可以在调用Session对象的run()方法执行图计算时，传入一些张量来保存结果。上述例程中只取回了单个节点（state），也可以取回多个张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9.0, 3.0]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "ia = tf.constant(3.0)\n",
    "ib = tf.constant(2.0)\n",
    "ic = tf.constant(1.0)\n",
    "\n",
    "ib_plus_ic = tf.add(ib, ic)\n",
    "product = tf.multiply(ia, ib_plus_ic)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    result = sess.run([product, ib_plus_ic])\n",
    "    print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feed\n",
    "\n",
    "上述例程引入了张量，常量或变量的形式存储。\n",
    "\n",
    "由TF提供的Feed机制可以临时替代图中任意操作的张量、可以对图中任何操作提交补丁（例如直接插入一个张量）。\n",
    "\n",
    "Feed用一个张量值临时替换一个操作的输出结果，把Feed数据作为run()的参数，Feed只在调用它的方法内有效，调用结束Feed即刻消失。\n",
    "\n",
    "最常见的用例是将某些特殊的操作指定为Feed操作，通过tf.placeholder()为这些操作创建占位符莱标记。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([ 63.], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "ia = tf.placeholder(tf.float32)\n",
    "ib = tf.placeholder(tf.float32)\n",
    "out = tf.multiply(ia, ib)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    print(sess.run([out], feed_dict={ia:[7.0], ib:[9.0]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "若没有正确提供Feed，placeholder()操作将会产生错误。\n",
    "\n",
    "参考[TF-MNIST](./TF-MNIST.ipynb)给出的更大规模的Feed例程。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}