Three City Example.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:b029ec66a9b852ba5ca1765657ce0e8bbd94d65256d17b0c0d7286f5ff432e54"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import pandas as pd\n",
      "import pandana as pdna"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Nodes are \"places\" in the network.  In this case we'll use three cities in the Bay Area.  Networks in Pandana are not required to be transportation networks, but for our applications they will be."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "nodes = []\n",
      "nodes.append([\"Berkeley\",      37.8717, 122.2728, 116768])\n",
      "nodes.append([\"Oakland\",       37.8044, 122.2708, 406253])\n",
      "nodes.append([\"San Francisco\", 37.783,  122.4167, 837442])\n",
      "nodes = pd.DataFrame(nodes, columns=[\"name\", \"lat\", \"long\", \"population\"])\n",
      "nodes"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>name</th>\n",
        "      <th>lat</th>\n",
        "      <th>long</th>\n",
        "      <th>population</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td>      Berkeley</td>\n",
        "      <td> 37.8717</td>\n",
        "      <td> 122.2728</td>\n",
        "      <td> 116768</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td>       Oakland</td>\n",
        "      <td> 37.8044</td>\n",
        "      <td> 122.2708</td>\n",
        "      <td> 406253</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td> San Francisco</td>\n",
        "      <td> 37.7830</td>\n",
        "      <td> 122.4167</td>\n",
        "      <td> 837442</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>3 rows \u00d7 4 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "            name      lat      long  population\n",
        "0       Berkeley  37.8717  122.2728      116768\n",
        "1        Oakland  37.8044  122.2708      406253\n",
        "2  San Francisco  37.7830  122.4167      837442\n",
        "\n",
        "[3 rows x 4 columns]"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Edges should connect the nodes - here I went to Google Maps and looked up the distances (in miles) between the three cities - by default edges are bidirectional, but real street networks have one-way streets.  Pandana supports both."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "edges = []\n",
      "edges.append([\"Berkeley\", \"Oakland\",       5.2])\n",
      "edges.append([\"Berkeley\", \"San Francisco\", 14.4])\n",
      "edges.append([\"Oakland\",  \"San Francisco\", 11.3])\n",
      "edges = pd.DataFrame(edges, columns=[\"from\", \"to\", \"weight\"])\n",
      "edges"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>from</th>\n",
        "      <th>to</th>\n",
        "      <th>weight</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td> Berkeley</td>\n",
        "      <td>       Oakland</td>\n",
        "      <td>  5.2</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td> Berkeley</td>\n",
        "      <td> San Francisco</td>\n",
        "      <td> 14.4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td>  Oakland</td>\n",
        "      <td> San Francisco</td>\n",
        "      <td> 11.3</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>3 rows \u00d7 3 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "       from             to  weight\n",
        "0  Berkeley        Oakland     5.2\n",
        "1  Berkeley  San Francisco    14.4\n",
        "2   Oakland  San Francisco    11.3\n",
        "\n",
        "[3 rows x 3 columns]"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "This is a small isue in Pandana, which expects ids as integers - so we have to convert the names to ids.  This is not really a problem when using OpenStreetMap or something similar since nodes don't really have names in that context.  These two lines of code add the integer ids associated with the names."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "edges[\"from_id\"] = nodes.reset_index().set_index(\"name\").loc[edges[\"from\"]][\"index\"].values\n",
      "edges[\"to_id\"] = nodes.reset_index().set_index(\"name\").loc[edges[\"to\"]][\"index\"].values\n",
      "edges"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
        "<table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
        "      <th>from</th>\n",
        "      <th>to</th>\n",
        "      <th>weight</th>\n",
        "      <th>from_id</th>\n",
        "      <th>to_id</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
        "      <td> Berkeley</td>\n",
        "      <td>       Oakland</td>\n",
        "      <td>  5.2</td>\n",
        "      <td> 0</td>\n",
        "      <td> 1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
        "      <td> Berkeley</td>\n",
        "      <td> San Francisco</td>\n",
        "      <td> 14.4</td>\n",
        "      <td> 0</td>\n",
        "      <td> 2</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
        "      <td>  Oakland</td>\n",
        "      <td> San Francisco</td>\n",
        "      <td> 11.3</td>\n",
        "      <td> 1</td>\n",
        "      <td> 2</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "<p>3 rows \u00d7 5 columns</p>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 3,
       "text": [
        "       from             to  weight  from_id  to_id\n",
        "0  Berkeley        Oakland     5.2        0      1\n",
        "1  Berkeley  San Francisco    14.4        0      2\n",
        "2   Oakland  San Francisco    11.3        1      2\n",
        "\n",
        "[3 rows x 5 columns]"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Initialize the network with nodes and edges.  Pandana is a C Extension so does NOT play nice with the Notebook.  If you run this line twice in the notebook you will get an error, but Pandana does support multiple networks if you have multiple networks you want to analyze.  Note that we set the node_ids back to the names which will return all the aggregations with the city names below."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net=pdna.Network(nodes[\"long\"], \n",
      "                 nodes[\"lat\"], \n",
      "                 edges[\"from_id\"], \n",
      "                 edges[\"to_id\"], \n",
      "                 edges[[\"weight\"]])\n",
      "net.node_ids = nodes.name"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "We need to set a variable on the network.  We are aggregating something on the network in all of these queries and this sets the thing to aggregate.  You can set multiple variables on the network at the same time.  Think of this as a Pandas DataFrame where the index is the nodes available on the network.  You are now in \"node space\" on the network rather than aggregating in shapes as in traditional GIS.  If you don't specify a name below, a default name will be used and all aggregation queries will use the variable that was most recently \"set\"."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.set(nodes.index, nodes.population)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "The easiest to understand (and least accurate) aggregation to perform has a \"flat\" decay which means no decay.  The following query will sum the population within 6 miles.  The Berkeley and Oakland Populations get summed, but the East Bay Cities can't reach SF and vice versa."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "*It is important to note that Pandana is performing a network aggregation for ALL origin nodes at the same time - in the C programming language, with multiple threads - so that the computations are very efficient.  If you do something very intensive, Pandana can grind your machine to a halt for a few seconds using all the cores available to it (and has even been known to crash my MacBook).  Rest assured there are ways to control this but they are not enabled by default.*"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=6, decay='flat')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "name\n",
        "Berkeley         523021\n",
        "Oakland          523021\n",
        "San Francisco    837442\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Extend the distance to 12 miles and Oakland and SF can reach each other."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, decay='flat')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "name\n",
        "Berkeley          523021\n",
        "Oakland          1360463\n",
        "San Francisco    1243695\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "At 15 miles all cities can reach all others and the summation is equivalent.  But is accessibility really equal in the three locations.  No!!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=15, decay='flat')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "name\n",
        "Berkeley         1360463\n",
        "Oakland          1360463\n",
        "San Francisco    1360463\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "This is the point of having a decay, since having more things closer to the source is very indicative of high accessibility.  A linear decay is easy to apply and understand - a max distance of 20 means population 20 miles away and further will be weighted zero in the summation while population 10 miles away will be weighted .5, items at the source will be weighted 1.0, etc. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=20, decay='linear')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "name\n",
        "Berkeley         415767.21875\n",
        "Oakland          689118.25000\n",
        "San Francisco    942329.56250\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Pandana is intended to be a general network aggregation tool.  For instance, you can compute average population in cities inside the horizon distance.  This is a spatially weighted average ALONG THE NETWORK."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, decay=\"flat\", type=\"AVE\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "name\n",
        "Berkeley         261510.50000\n",
        "Oakland          453487.65625\n",
        "San Francisco    621847.50000\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Although it is a difficult metric to understand, you can also run an average query with a decay."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, decay=\"linear\", type=\"AVE\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "name\n",
        "Berkeley         173505.953125\n",
        "Oakland          173760.671875\n",
        "San Francisco    430570.031250\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Exponential decay is also supported, which is a somewhat stronger decay than the linear decay.  Other decays may be added in the future (and exponential decay could be parameterized.)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, decay=\"exp\", type=\"sum\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "name\n",
        "Berkeley         380181.78125\n",
        "Oakland          808548.56250\n",
        "San Francisco    995871.43750\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Other aggregations include count of nodes, which would be similar to \"size\" in Pandas, but with a \"groupby\" ALONG THE NETWORK."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, type=\"count\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "name\n",
        "Berkeley         2\n",
        "Oakland          3\n",
        "San Francisco    2\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Standard Deviation is also an available aggregation - this is a spatially weighted standard deviation ALONG THE NETWORK."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "net.aggregate(distance=12, type=\"std\", decay=\"flat\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "name\n",
        "Berkeley         144742.50000\n",
        "Oakland          296103.68750\n",
        "San Francisco    215594.53125\n",
        "dtype: float32"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Now the fun begins, as we go beyond the 3 city example to do aggregations for the whole Bay Area in under a second."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}