09b Sentiment Analysis with LSTM (inference).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sentiment Analysis of Movie Reviews\n",
    "===================================\n",
    "\n",
    "In this tutorial, we will load a trained model and perform inference on a new movie review.\n",
    "\n",
    "Setup\n",
    "-----\n",
    "\n",
    "As before, we first create a computational backend to tell neon on what device to execute the computation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from neon.backends import gen_backend\n",
    "\n",
    "be = gen_backend(backend='gpu', batch_size=1)\n",
    "print be"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also define a few parameters, and the load the vocabulary. The vocab is a 1:1 mapping of words to numbers. The file `imdb.vocab` can be downloaded from [https://s3-us-west-1.amazonaws.com/nervana-course/imdb.vocab](https://s3-us-west-1.amazonaws.com/nervana-course/imdb.vocab) and placed in the `data` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pickle as pkl\n",
    "\n",
    "sentence_length = 128\n",
    "vocab_size = 20000\n",
    "\n",
    "# we have some special codes\n",
    "pad_char = 0  # padding character\n",
    "start = 1  # marker for start of review\n",
    "oov = 2  # when the word is out of the vocab\n",
    "index_from = 3  # index of first word in vocab\n",
    "\n",
    "# load the vocab\n",
    "vocab, rev_vocab = pkl.load(open('data/imdb.vocab', 'rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load Model\n",
    "----------\n",
    "\n",
    "To load the model, we just pass in the saved model file. neon will automatically generate the layers specified in the model file and load the corresponding weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from neon.models import Model\n",
    "\n",
    "model = Model('imdb_lstm.pkl')\n",
    "\n",
    "# we initialize the model, passing in the size of the input data.\n",
    "model.initialize(dataset=(sentence_length, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Inference\n",
    "---------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first generate some buffers on both the host (CPU) and the device (GPU) to hold the input data that we would like to pass to the model for inference. Below the variable `be` is the backend that we creater with `gen_backend` earlier in the code. Our backend supports numpy-like functions for allocating buffers on the compute device."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "input_device = be.zeros((sentence_length, 1), dtype=np.int32)  # `be` is the backend that we created earlier in the code.\n",
    "input_numpy = np.zeros((sentence_length, 1), dtype=np.int32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we write our new movie review. We've included a sample here, but feel free to write your own and see how well the model responds.\n",
    "\n",
    "POSITIVE:\n",
    "> \"The pace is steady and constant, the characters full and engaging, the relationships and interactions natural showing that you do not need floods of tears to show emotion, screams to show fear, shouting to show dispute or violence to show anger. Naturally Joyce's short story lends the film a ready made structure as perfect as a polished diamond, but the small changes Huston makes such as the inclusion of the poem fit in neatly. It is truly a masterpiece of tact, subtlety and overwhelming beauty.\"\n",
    "\n",
    "NEGATIVE:\n",
    "\n",
    "> \"Beautiful attracts excellent idea, but ruined with a bad selection of the actors. The main character is a loser and his woman friend and his friend upset viewers. Apart from the first episode all the other become more boring and boring. First, it considers it illogical behavior. No one normal would not behave the way the main character behaves. It all represents a typical Halmark way to endear viewers to the reduced amount of intelligence. Does such a scenario, or the casting director and destroy this question is on Halmark producers. Cat is the main character is wonderful. The main character behaves according to his friend selfish.\"\n",
    "\n",
    "NEUTRAL:\n",
    "\n",
    "> \"The characters voices were very good. I was only really bothered by Kanga. The music, however, was twice as loud in parts than the dialog, and incongruous to the film. As for the story, it was a bit preachy and militant in tone. Overall, I was disappointed, but I would go again just to see the same excitement on my child's face. I liked Lumpy's laugh...\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "line = \"\"\"Beautiful attracts excellent idea, but ruined with a bad selection of the actors. The main character is\n",
    "          a loser and his woman friend and his friend upset viewers. Apart from the first episode all the other become \n",
    "          more boring and boring. First, it considers it illogical behavior. No one normal would not behave the way the \n",
    "          main character behaves. It all represents a typical Halmark way to endear viewers to the reduced amount of \n",
    "          intelligence. Does such a scenario, or the casting director and destroy this question is on Halmark \n",
    "          producers. Cat is the main character is wonderful. The main character behaves according to \n",
    "          his friend selfish.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we send the data to the model, we need to convert the string to a sequence of numbers, with each number representing a word, using the vocab that we loaded earlier in the code. If a word is not in our vocab, we use a special out-of-vocab number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from neon.data.text_preprocessing import clean_string\n",
    "\n",
    "tokens = clean_string(line).strip().split()\n",
    "\n",
    "sent = [len(vocab) + 1 if t not in vocab else vocab[t] for t in tokens]\n",
    "sent = [start] + [w + index_from for w in sent]\n",
    "sent = [oov if w >= vocab_size else w for w in sent]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The text data is now converted to a list of integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print sent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We truncate the input to `sentence_length=128` words. If the text is less than 128 words, we pad with zeros. The text is then loaded into the numpy array named `input_host`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "trunc = sent[-sentence_length:]  # take the last sentence_length words\n",
    "\n",
    "input_numpy[:] = 0  # fill with zeros\n",
    "input_numpy[-len(trunc):, 0] = trunc   # place the input into the numpy array\n",
    "print input_numpy.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "input_device.set(input_numpy)  # copy the numpy array to device\n",
    "y_pred = model.fprop(input_device, inference=True)  # run the forward pass through the model\n",
    "\n",
    "print(\"Predicted sentiment: {}\".format(y_pred.get()[1]))  # print the estimated sentiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Experimentation\n",
    "---------------\n",
    "\n",
    "To make it easy for you to experiment with the model inference, below we wrap all the text above into a single function that you can call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sentiment(sent, model):\n",
    "    input_device = be.zeros((sentence_length, 1), dtype=np.int32)\n",
    "    input_numpy = np.zeros((sentence_length, 1), dtype=np.int32) \n",
    "    tokens = clean_string(line).strip().split()\n",
    "\n",
    "    sent = [len(vocab) + 1 if t not in vocab else vocab[t] for t in tokens]\n",
    "    sent = [start] + [w + index_from for w in sent]\n",
    "    sent = [oov if w >= vocab_size else w for w in sent]\n",
    "    \n",
    "    trunc = sent[-sentence_length:]  # take the last sentence_length words\n",
    "\n",
    "    input_numpy[:] = 0  # fill with zeros\n",
    "    input_numpy[-len(trunc):, 0] = trunc   # place the input into the numpy array\n",
    "    input_device.set(input_numpy)  # copy the numpy array to device\n",
    "    y_pred = model.fprop(input_device, inference=True)  # run the forward pass through the model\n",
    "    \n",
    "    return y_pred.get()[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can easily enter your own review and get the result. Here we included a more neutral review below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "line = \"\"\"The characters voices were very good. I was only really bothered by Kanga. The music, however, was twice \n",
    "          as loud in parts than the dialog, and incongruous to the film. As for the story, it was a bit preachy and \n",
    "          militant in tone. Overall, I was disappointed, but I would go again just to see the same excitement on my \n",
    "          child's face. I liked Lumpy's laugh...\"\"\"\n",
    "\n",
    "result = sentiment(line, model)\n",
    "print(\"Sentiment: {}\".format(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}