20160807Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## jieba 分詞"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "「 ‪ 精靈 寶可夢 」 （ Pok é mon   GO ） 台灣 於 今天 上線 ， 許多 玩家 紛紛 出門 「 抓 寶 」 ， 不過 卻 有 實況 主兔頭 哥邊 直播 騎單 車邊 抓寶可夢 ， 還說 「 單車 不用 停紅綠燈 」 ！ 實況 主兔 頭哥 直播 邊騎 單車邊 抓寶可夢 的 影片 ， 影片 中 他 在 新北市 樹林區 馬 路上 左手 單手 騎 單車 ， 右手 單手 拿手 機玩寶可夢 ， 還一邊 和 網友 聊天 ， 甚至 切換 瀏覽 聊天室 畫面 ， 險象 環生 。 他 在 山佳 火車 站 時 ， 1 名 網友 騎 機車 去 那邊 勸 說 「 我載 你 啦 ， 你 這樣 危險 ！ 」 實況 主 仍 堅持 獨自騎 單車 抓寶可夢 。\n"
     ]
    }
   ],
   "source": [
    "import jieba # 引用jieba 套件\n",
    "article = '「‪精靈寶可夢」（Pokémon GO）台灣於今天上線，許多玩家紛紛出門「抓寶」，不過卻有實況主兔頭哥邊直播騎單車邊抓寶可夢，還說「單車不用停紅綠燈」！實況主兔頭哥直播邊騎單車邊抓寶可夢的影片，影片中他在新北市樹林區馬路上左手單手騎單車，右手單手拿手機玩寶可夢，還一邊和網友聊天，甚至切換瀏覽聊天室畫面，險象環生。他在山佳火車站時，1名網友騎機車去那邊勸說「我載你啦，你這樣危險！」實況主仍堅持獨自騎單車抓寶可夢。'\n",
    "for w in jieba.cut(article):\n",
    "    print w,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "「‪精靈寶可夢」（Pokémon GO）台灣於今天上線\n",
      "許多玩家紛紛出門「抓寶」\n",
      "不過卻有實況主兔頭哥邊直播騎單車邊抓寶可夢\n",
      "還說「單車不用停紅綠燈」\n",
      "實況主兔頭哥直播邊騎單車邊抓寶可夢的影片\n",
      "影片中他在新北市樹林區馬路上左手單手騎單車\n",
      "右手單手拿手機玩寶可夢\n",
      "還一邊和網友聊天\n",
      "甚至切換瀏覽聊天室畫面\n",
      "險象環生\n",
      "他在山佳火車站時\n",
      "1名網友騎機車去那邊勸說「我載你啦\n",
      "你這樣危險\n",
      "」實況主仍堅持獨自騎單車抓寶可夢\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "for s in re.split( '，|。|！', article):\n",
    "    print s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## jIeba 字典\n",
    "- C:\\Users\\Student\\Anaconda2\\Lib\\site-packages\\jieba"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Jieba HMM 模型\n",
    "- C:\\Users\\Student\\Anaconda2\\Lib\\site-packages\\jieba\\posseg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?jieba.cut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "大\n",
      "巨蛋\n",
      "案對\n",
      "市府\n",
      "同仁\n",
      "下\n",
      "封口令\n",
      "？\n",
      "　\n",
      "柯\n",
      "P\n",
      "否認\n"
     ]
    }
   ],
   "source": [
    "seg_list = jieba.cut(\"大巨蛋案對市府同仁下封口令？　柯P否認\")\n",
    "for w in seg_list:\n",
    "    print w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Full Mode: 大/ 巨蛋/ 案/ 對/ 市府/ 同仁/ 下/ 封口/ 封口令/ 口令/ / / / 柯/ P/ 否/ 認\n",
      "Default Mode: 大/ 巨蛋/ 案對/ 市府/ 同仁/ 下/ 封口令/ ？/ 　/ 柯/ P/ 否認\n"
     ]
    }
   ],
   "source": [
    "seg_list = jieba.cut(\"大巨蛋案對市府同仁下封口令？　柯P否認\", cut_all=True)\n",
    "print \"Full Mode:\", \"/ \".join(seg_list) \n",
    "\n",
    "\n",
    "seg_list = jieba.cut(\"大巨蛋案對市府同仁下封口令？　柯P否認\", cut_all=False)\n",
    "print \"Default Mode:\", \"/ \".join(seg_list) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 增添自己的字典"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "jieba.load_userdict(\"userdict.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Default Mode: 大巨蛋/ 案對/ 市府/ 同仁/ 下/ 封口令/ ？/ 　/ 柯P/ 否認\n"
     ]
    }
   ],
   "source": [
    "import jieba\n",
    "seg_list = jieba.cut(\"大巨蛋案對市府同仁下封口令？　柯P否認\", cut_all=False)\n",
    "print \"Default Mode:\", \"/ \".join(seg_list) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Default Mode: 「/ ‪/ 精靈/ 寶可夢/ 」/ （/ Pok/ é/ mon/  / GO/ ）/ 台灣/ 於/ 今天/ 上線/ ，/ 許多/ 玩家/ 紛紛/ 出門/ 「/ 抓/ 寶/ 」/ ，/ 不過/ 卻/ 有/ 實況主/ 兔頭/ 哥邊/ 直播/ 騎單/ 車邊/ 抓/ 寶可夢/ ，/ 還說/ 「/ 單車/ 不用/ 停紅綠燈/ 」/ ！/ 實況主/ 兔頭哥/ 直播/ 邊騎/ 單車邊/ 抓/ 寶可夢/ 的/ 影片/ ，/ 影片/ 中/ 他/ 在/ 新北市/ 樹林區/ 馬/ 路上/ 左手/ 單手/ 騎/ 單車/ ，/ 右手/ 單手/ 拿/ 手機/ 玩/ 寶可夢/ ，/ 還一邊/ 和/ 網友/ 聊天/ ，/ 甚至/ 切換/ 瀏覽/ 聊天室/ 畫面/ ，/ 險象/ 環生/ 。/ 他/ 在/ 山佳/ 火車站/ 時/ ，/ 1/ 名/ 網友/ 騎/ 機車/ 去/ 那邊/ 勸/ 說/ 「/ 我載/ 你/ 啦/ ，/ 你/ 這樣/ 危險/ ！/ 」/ 實況主/ 仍/ 堅持/ 獨自騎/ 單車/ 抓/ 寶可夢/ 。\n"
     ]
    }
   ],
   "source": [
    "jieba.add_word('寶可夢') # 暫時將新詞放入\n",
    "jieba.add_word('實況主')\n",
    "jieba.add_word('手機')\n",
    "jieba.add_word('Pokémon')\n",
    "jieba.add_word('火車站')\n",
    "article = '「‪精靈寶可夢」（Pokémon GO）台灣於今天上線，許多玩家紛紛出門「抓寶」，不過卻有實況主兔頭哥邊直播騎單車邊抓寶可夢，還說「單車不用停紅綠燈」！實況主兔頭哥直播邊騎單車邊抓寶可夢的影片，影片中他在新北市樹林區馬路上左手單手騎單車，右手單手拿手機玩寶可夢，還一邊和網友聊天，甚至切換瀏覽聊天室畫面，險象環生。他在山佳火車站時，1名網友騎機車去那邊勸說「我載你啦，你這樣危險！」實況主仍堅持獨自騎單車抓寶可夢。'\n",
    "seg_list = jieba.cut(article)\n",
    "print \"Default Mode:\", \"/ \".join(seg_list) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "大巨蛋 ns\n",
      "案 ng\n",
      "對 p\n",
      "市府 n\n",
      "同仁 nr\n",
      "下 f\n",
      "封口令 n\n",
      "？ x\n",
      "　x\n",
      "柯P n\n",
      "否認 v\n"
     ]
    }
   ],
   "source": [
    "import jieba.posseg as pseg\n",
    "#jieba.add_word('大巨蛋', 100, 'ns')\n",
    "words = pseg.cut(\"大巨蛋案對市府同仁下封口令？　柯P否認\")\n",
    "for w in words:\n",
    "    print w.word, w.flag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "jieba.del_word('大巨蛋') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "巨蛋 1 3\n",
      "大巨蛋 0 3\n",
      "案對 3 5\n",
      "市府 5 7\n",
      "同仁 7 9\n",
      "下 9 10\n",
      "封口 10 12\n",
      "口令 11 13\n",
      "封口令 10 13\n",
      "？ 13 14\n",
      "　14 15\n",
      "柯P 15 17\n",
      "否認 17 19\n"
     ]
    }
   ],
   "source": [
    "words = jieba.tokenize(u\"大巨蛋案對市府同仁下封口令？　柯P否認\", 'utf-8')\n",
    "\n",
    "for tw in words:\n",
    "    print tw[0], tw[1], tw[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "封口令\n"
     ]
    }
   ],
   "source": [
    "import jieba.analyse\n",
    "# 抓出最重要的關鍵詞\n",
    "tags = jieba.analyse.extract_tags(u\"大巨蛋案對市府同仁下封口令？　柯P否認\", 1)\n",
    "print \",\".join(tags)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "同仁\n"
     ]
    }
   ],
   "source": [
    "#allowPOS 決定允許的字詞的詞性\n",
    "tags = jieba.analyse.extract_tags(u\"大巨蛋案對市府同仁下封口令？　柯P否認\", 1, allowPOS = ['nr'])\n",
    "print \",\".join(tags)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "jieba.analyse.set_stop_words(\"stopword.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "根據/歐洲/電商/組織/(/Ecommerce/ /Europe/)/ /最新消息/得知/，/該/集團/積極/讓/歐洲/的/包裹/遞送/服務/與/電商/領域/有/更/緊密/結合/。/如此/一來/，/便/可解/決電商/物流/的/障礙/。/此/組織/總共結合/歐洲/ /16/ /國的/電商/協會/，/目標/整合/歐洲/的/包裹/物流/市場/，/讓/商品/跨境/更為/順暢\n"
     ]
    }
   ],
   "source": [
    "import jieba.analyse\n",
    "sentence = \"根據歐洲電商組織(Ecommerce Europe) 最新消息得知，\\\n",
    "該集團積極讓歐洲的包裹遞送服務與電商領域有更緊密結合。如此一來，\\\n",
    "便可解決電商物流的障礙。此組織總共結合歐洲 16 國的電商協會，\\\n",
    "目標整合歐洲的包裹物流市場，讓商品跨境更為順暢\"\n",
    "s = jieba.cut(sentence)\n",
    "#tags = jieba.analyse.extract_tags(sentence)\n",
    "print '/'.join(s)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 讀取PTT Creditcard 版的資訊"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('C:\\\\Users\\\\Student\\creditcard.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0    3222\n",
       "id            3222\n",
       "url           3222\n",
       "author        3222\n",
       "title         3222\n",
       "content       3222\n",
       "board         3222\n",
       "date          3222\n",
       "dtype: int64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'Unnamed: 0', u'id', u'url', u'author', u'title', u'content', u'board',\n",
       "       u'date'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3222 entries, 0 to 3221\n",
      "Data columns (total 8 columns):\n",
      "Unnamed: 0    3222 non-null int64\n",
      "id            3222 non-null int64\n",
      "url           3222 non-null object\n",
      "author        3222 non-null object\n",
      "title         3222 non-null object\n",
      "content       3222 non-null object\n",
      "board         3222 non-null object\n",
      "date          3222 non-null object\n",
      "dtypes: int64(2), object(6)\n",
      "memory usage: 201.4+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a': 2, 'c': 1, 'b': 2}\n"
     ]
    }
   ],
   "source": [
    "# 使用字典範例\n",
    "s = ['a' , 'b' , 'c', 'a' , 'b']\n",
    "dic = {}\n",
    "for w in s:\n",
    "    if w not in dic:\n",
    "        dic[w] = 1 # 不在字典中\n",
    "    else:\n",
    "        dic[w] = dic[w] + 1 # 詞在字典中\n",
    "print dic        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({'a': 2, 'b': 2, 'c': 1})\n",
      "[('a', 2)]\n"
     ]
    }
   ],
   "source": [
    "# Use Collections 範例\n",
    "from collections import Counter\n",
    "c = Counter()\n",
    "for w in s:\n",
    "    c[w] +=1\n",
    "print c\n",
    "print c.most_common(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 統計最常出現的詞"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "from collections import Counter\n",
    "c = Counter()\n",
    "for content in df.content:\n",
    "    seg = jieba.cut(content)\n",
    "    for w in seg:\n",
    "        if len(w) >=2: #字詞長度>=2\n",
    "            c[w] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#for key, val in c.most_common(100):\n",
    "#    print key, val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "citi = df[df.title.str.contains('玉山') & df.title.str.contains('問題')]\n",
    "#citi.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "c = Counter()\n",
    "for content in citi.content:\n",
    "    seg = jieba.cut(content)\n",
    "    for w in seg:\n",
    "        if len(w) >=2: #字詞長度>=2\n",
    "            c[w] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "玉山 54\n",
      "回饋 31\n",
      "http 25\n",
      "消費 23\n",
      "com 21\n",
      "現金 15\n",
      "tw 14\n",
      "信用卡 14\n",
      "一下 14\n",
      "謝謝 13\n",
      "優惠 13\n",
      "所以 13\n",
      "申請 13\n",
      "icash 12\n",
      "請問 12\n",
      "因為 12\n",
      "可以 12\n",
      "www 12\n",
      "客服 11\n",
      "聯名 11\n",
      "活動 11\n",
      "好像 11\n",
      "大大 11\n",
      "知道 11\n",
      "刷卡 11\n",
      "帳單 11\n",
      "有人 10\n",
      "credit 10\n",
      "大家 10\n",
      "點數 10\n",
      "event 10\n",
      "esunbank 10\n",
      "銀行 10\n",
      "目前 9\n",
      "最近 9\n",
      "還有 9\n",
      "699 9\n",
      "結帳 9\n",
      "JCB 9\n",
      "... 9\n",
      "是否 9\n",
      "各位 8\n",
      "index 8\n",
      "就是 8\n",
      "還是 8\n",
      "11 8\n",
      "小弟 8\n",
      "去年 7\n",
      "一般 7\n",
      "沒有 7\n",
      "imgur 7\n",
      "申辦 7\n",
      "以上 7\n",
      "官網 7\n",
      "部分 7\n",
      "這樣 7\n",
      "應該 7\n",
      "世界 7\n",
      "收到 7\n",
      "使用 7\n",
      "年費 6\n",
      "100 6\n",
      "這個 6\n",
      "悠遊 6\n",
      "核卡 6\n",
      "當期 6\n",
      "01 6\n",
      "真的 6\n",
      "其他 6\n",
      "一張 6\n",
      "500 6\n",
      "jpg 6\n",
      "如果 6\n",
      "html 6\n",
      "發現 6\n",
      "剛剛 6\n",
      "加辦 5\n",
      "htm 5\n",
      "似乎 5\n",
      "手續費 5\n",
      "拿到 5\n",
      "抵用券 5\n",
      "兩筆 5\n",
      "自動 5\n",
      "如下 5\n",
      "只有 5\n",
      "雙幣 5\n",
      "直接 5\n",
      "ibon 5\n",
      "注意 5\n",
      "這張 5\n",
      "顯示 5\n",
      "但是 5\n",
      "現在 5\n",
      "加碼 5\n",
      ".... 5\n",
      "之前 5\n",
      "舊戶 5\n",
      "一樣 4\n",
      "debit 4\n"
     ]
    }
   ],
   "source": [
    "for key, val in c.most_common(100):\n",
    "    print key, val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 計算 tf-idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 0.0 0.0\n"
     ]
    }
   ],
   "source": [
    "# (tfidf(\"a\", a, D))\n",
    "a, abb, abc = [\"a\"], [\"a\", \"b\", \"b\"], [\"a\", \"b\", \"c\"]\n",
    "D = [a, abb, abc]\n",
    "\n",
    "\n",
    "import scipy as sp\n",
    "tf = a.count('a') / a.count('a')\n",
    "idf = sp.log(3 /  len([doc for doc in D if 'a' in doc]))\n",
    "print tf, idf, tf * idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.666666666667\n",
      "0.666666666667 0.405465108108 0.270310072072\n"
     ]
    }
   ],
   "source": [
    "# (tfidf(\"b\", abb, D))\n",
    "a, abb, abc = [\"a\"], [\"a\", \"b\", \"b\"], [\"a\", \"b\", \"c\"]\n",
    "D = [a, abb, abc]\n",
    "\n",
    "tf = abb.count('b') * 1.0 / (abb.count('a') + abb.count('b'))\n",
    "print tf\n",
    "\n",
    "idf = sp.log(3.0 /  len([doc for doc in D if 'b' in doc]))\n",
    "\n",
    "print tf, idf, tf * idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.333333333333\n",
      "0.333333333333 0.0 0.0\n"
     ]
    }
   ],
   "source": [
    "# (tfidf(\"a\", abc, D))\n",
    "a, abb, abc = [\"a\"], [\"a\", \"b\", \"b\"], [\"a\", \"b\", \"c\"]\n",
    "D = [a, abb, abc]\n",
    "\n",
    "tf = abb.count('a') * 1.0 / (abb.count('a') + abb.count('b'))\n",
    "print tf\n",
    "\n",
    "idf = sp.log(3.0 /  len([doc for doc in D if 'a' in doc]))\n",
    "\n",
    "print tf, idf, tf * idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.333333333333\n",
      "0.333333333333 0.405465108108 0.135155036036\n"
     ]
    }
   ],
   "source": [
    "# (tfidf(\"b\", abc, D))\n",
    "\n",
    "tf = abc.count('b') * 1.0 / (abc.count('a') + abc.count('b') + abc.count('c'))\n",
    "print tf\n",
    "\n",
    "\n",
    "idf = sp.log(3.0 /  len([doc for doc in D if 'b' in doc]))\n",
    "\n",
    "print tf, idf, tf * idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.333333333333\n",
      "0.333333333333 1.09861228867 0.366204096223\n"
     ]
    }
   ],
   "source": [
    "# (tfidf(\"c\", abc, D))\n",
    "tf = abc.count('c') * 1.0 / (abc.count('a') + abc.count('b') + abc.count('c'))\n",
    "print tf\n",
    "\n",
    "idf = sp.log(3.0 /  len([doc for doc in D if 'c' in doc]))\n",
    "print tf, idf, tf * idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "0.270310072072\n",
      "0.0\n",
      "0.135155036036\n",
      "0.366204096223\n"
     ]
    }
   ],
   "source": [
    "def tfidf(t, d, D):\n",
    "    tf = float(d.count(t)) / sum(d.count(w) for w in set(d))\n",
    "    idf = sp.log(float(len(D)) / (len([doc for doc in D if t in doc])))\n",
    "    return tf * idf\n",
    "\n",
    "print(tfidf(\"a\", a, D))\n",
    "print(tfidf(\"b\", abb, D))\n",
    "print(tfidf(\"a\", abc, D))\n",
    "print(tfidf(\"b\", abc, D))\n",
    "print(tfidf(\"c\", abc, D))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jibea 的 IDF 字典\n",
    "- C:\\Users\\Student\\Anaconda2\\Lib\\site-packages\\jieba\\analyse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用關鍵字取得詞頻"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "import jieba.analyse\n",
    "\n",
    "c = Counter()\n",
    "for content in df.content:\n",
    "    seg = jieba.analyse.extract_tags(content)\n",
    "    for w in seg:\n",
    "        if len(w) >=2: #字詞長度>=2\n",
    "            c[w] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "申請 836\n",
      "因為 736\n",
      "回饋 699\n",
      "額度 697\n",
      "消費 513\n",
      "卡片 447\n",
      "信用卡 447\n",
      "謝謝 426\n",
      "客服 411\n",
      "聯名 387\n"
     ]
    }
   ],
   "source": [
    "for k, v in c.most_common(10):\n",
    "    print k,v"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "單車 單手 畫面 玩家 上線 切換 直播 兔頭 險象 瀏覽 綠燈 網友 影片 新北市 樹林 區馬 停紅 不用 環生 右手\n",
      "寶可夢 實況主 單車 單手 網友 直播 影片 停紅綠燈 騎單 紛紛 切換 環生 邊騎 機車 險象 還說 這樣 不過 還一邊 台灣\n"
     ]
    }
   ],
   "source": [
    "# Text Rank\n",
    "import jieba.analyse\n",
    "\n",
    "import jieba # 引用jieba 套件\n",
    "article = '「‪精靈寶可夢」（Pokémon GO）台灣於今天上線，許多玩家紛紛出門「抓寶」，不過卻有實況主兔頭哥邊直播騎單車邊抓寶可夢，還說「單車不用停紅綠燈」！實況主兔頭哥直播邊騎單車邊抓寶可夢的影片，影片中他在新北市樹林區馬路上左手單手騎單車，右手單手拿手機玩寶可夢，還一邊和網友聊天，甚至切換瀏覽聊天室畫面，險象環生。他在山佳火車站時，1名網友騎機車去那邊勸說「我載你啦，你這樣危險！」實況主仍堅持獨自騎單車抓寶可夢。'\n",
    "for w in jieba.analyse.textrank(article):\n",
    "    print w,\n",
    "print \n",
    "for w in jieba.analyse.extract_tags(article):\n",
    "    print w,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 建立詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import jieba\n",
    "ary = ['【更新】柯P：洪智坤洩漏公文案還沒看到公文　今處理',\n",
    "       '留洪智坤 柯：殘障求職不易',\n",
    "       '人事處議處洪智坤　柯P：不清楚議處結果']\n",
    "corpus = []\n",
    "for title in ary:\n",
    "    corpus.append(' '.join(jieba.cut(title)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "【 更新 】 柯P ： 洪智坤 洩漏 公文 案還 沒 看到 公文 　 今處理\n"
     ]
    }
   ],
   "source": [
    "print corpus[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "留洪智坤   柯 ： 殘障 求職 不易\n"
     ]
    }
   ],
   "source": [
    "print corpus[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "人事 處議 處 洪智坤 　 柯P ： 不 清楚 議處 結果\n"
     ]
    }
   ],
   "source": [
    "print corpus[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0 0 1 2 1 1 1 0 0 1 1 0 0 1 0 0 0]\n",
      " [1 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0]\n",
      " [0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣\n",
    "print X.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "不易 人事 今處理 公文 更新 柯p 案還 殘障 求職 洩漏 洪智坤 清楚 留洪智坤 看到 結果 處議 議處\n",
      "[[0 0 1 2 1 1 1 0 0 1 1 0 0 1 0 0 0]\n",
      " [1 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0]\n",
      " [0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "word = vectorizer.get_feature_names() # 取得詞頻矩陣的欄位名稱\n",
    "for w in word:\n",
    "    print w,\n",
    "print \n",
    "print X.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 校正斷詞結果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "jieba.add_word('洪智坤')\n",
    "\n",
    "import jieba\n",
    "ary = ['【更新】柯P：洪智坤洩漏公文案還沒看到公文　今處理',\n",
    "       '留洪智坤 柯：殘障求職不易',\n",
    "       '人事處議處洪智坤　柯P：不清楚議處結果']\n",
    "corpus = []\n",
    "for title in ary:\n",
    "    corpus.append(' '.join(jieba.cut(title)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 264,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "不易 人事 今處理 公文 更新 柯p 案還 殘障 求職 洩漏 洪智坤 清楚 看到 結果 處議 議處\n",
      "[[0 0 1 2 1 1 1 0 0 1 1 0 1 0 0 0]\n",
      " [1 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0]\n",
      " [0 1 0 0 0 1 0 0 0 0 1 1 0 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "word = vectorizer.get_feature_names() # 取得詞頻矩陣的欄位名稱\n",
    "for w in word:\n",
    "    print w,\n",
    "print \n",
    "print X.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 0],\n",
       "       [0, 0, 1],\n",
       "       [1, 0, 0],\n",
       "       [2, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       [1, 0, 1],\n",
       "       [1, 0, 0],\n",
       "       [0, 1, 0],\n",
       "       [0, 1, 0],\n",
       "       [1, 0, 0],\n",
       "       [1, 1, 1],\n",
       "       [0, 0, 1],\n",
       "       [1, 0, 0],\n",
       "       [0, 0, 1],\n",
       "       [0, 0, 1],\n",
       "       [0, 0, 1]], dtype=int64)"
      ]
     },
     "execution_count": 266,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.transpose().toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 計算TFIDF 矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "transformer = TfidfTransformer()\n",
    "tfidf = transformer.fit_transform(X) # 根據CountVectorizer 的結果去計算\n",
    "\n",
    "weight = tfidf.toarray() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "不易 人事 今處理 公文 更新 柯p 案還 殘障 求職 洩漏 洪智坤 清楚 看到 結果 處議 議處\n",
      "[[ 0.          0.          0.31738473  0.63476946  0.31738473  0.24137927\n",
      "   0.31738473  0.          0.          0.31738473  0.18745253  0.\n",
      "   0.31738473  0.          0.          0.        ]\n",
      " [ 0.54645401  0.          0.          0.          0.          0.          0.\n",
      "   0.54645401  0.54645401  0.          0.32274454  0.          0.          0.\n",
      "   0.          0.        ]\n",
      " [ 0.          0.41074684  0.          0.          0.          0.31238356\n",
      "   0.          0.          0.          0.          0.2425937   0.41074684\n",
      "   0.          0.41074684  0.41074684  0.41074684]]\n"
     ]
    }
   ],
   "source": [
    "word = vectorizer.get_feature_names() # 取得詞頻矩陣的欄位名稱\n",
    "for w in word:\n",
    "    print w,\n",
    "print \n",
    "print weight"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 計算Cosine Similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.          0.06049928  0.12087772]\n",
      " [ 0.06049928  1.          0.07829579]\n",
      " [ 0.12087772  0.07829579  1.        ]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "cosine_similarities = cosine_similarity(tfidf, tfidf) #取所有文章的相似度\n",
    "print cosine_similarities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.          0.06049928  0.12087772]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "#只取第一篇文章跟其他人的相似度, flatten 是將矩陣打平, 也就是拿掉最外層的 []\n",
    "cosine_similarities = cosine_similarity(tfidf[0], tfidf).flatten() \n",
    "print cosine_similarities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "【更新】柯P：洪智坤洩漏公文案還沒看到公文　今處理\n",
      "人事處議處洪智坤　柯P：不清楚議處結果\n",
      "留洪智坤 柯：殘障求職不易\n"
     ]
    }
   ],
   "source": [
    "ary = ['【更新】柯P：洪智坤洩漏公文案還沒看到公文　今處理',\n",
    "       '留洪智坤 柯：殘障求職不易',\n",
    "       '人事處議處洪智坤　柯P：不清楚議處結果']\n",
    "related_docs_indices = cosine_similarities.argsort()[::-1] # 可以根據相似度作排序\n",
    "related_docs_indices\n",
    "for index in related_docs_indices:\n",
    "    print ary[index]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 讀取PTT Creditcard 版的資訊"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('C:\\\\Users\\\\Student\\creditcard.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import jieba\n",
    "\n",
    "titles = []\n",
    "corpus = []\n",
    "for article in df.iterrows():\n",
    "    #print article[1].title\n",
    "    titles.append(article[1].title)\n",
    "    corpus.append(' '.join(jieba.cut(article[1].content)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 文章相似度查詢"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "transformer = TfidfTransformer()\n",
    "tfidf = transformer.fit_transform(X) # 根據CountVectorizer 的結果去計算\n",
    "\n",
    "weight = tfidf.toarray() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "#只取第一篇文章跟其他人的相似度, flatten 是將矩陣打平, 也就是拿掉最外層的 []\n",
    "cosine_similarities = cosine_similarity(tfidf[6], tfidf).flatten() # 6 => 文章編號"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[問題]元大鑽金Visa卡，出國竟然無法刷！ 1.0\n",
      "Re: [問題]元大鑽金Visa卡，出國竟然無法刷！ 0.181485541162\n",
      "[心得] 元大鑽金卡　感應卡片無法成功刷卡 0.174765336625\n",
      "Re: [問題] 歐洲/韓國刷元大鑽金卡的經驗 0.174564653277\n",
      "Re: [問題] 歐洲刷元大鑽金卡的經驗 0.161463169154\n",
      "Re: [情報] 註冊花旗網銀享35元全家商品券 0.159430473365\n",
      "Re: [問題] 歐洲刷元大鑽金卡的經驗 0.139847681209\n",
      "[心得] 元大一卡通在日本無法刷卡誇張原因 0.138571349547\n",
      "[閒聊] 全家刷卡注意事項 0.133000365568\n",
      "Re: [問題] 元大鑽金3D認證固定密碼 0.127348879925\n",
      "[問題] 關於愛金新卡自動加值問題 0.120929647024\n",
      "[問題] 元大鑽金常常刷不過 0.118471601689\n",
      "[心得] 一銀EZ訂劃位 0.117115744078\n",
      "Re: [問題] 全家店員說不能刷台新 0.115522007952\n",
      "Re: [問題] 現金卡跟里程卡該怎麼選呢? 0.115261181661\n",
      "[問題] 華南悠遊聯名寵愛紅卡消費通知 0.113857606155\n",
      "[問題] 全家刷台新顯示\"無法認證\"? 0.110543313431\n",
      "[閒聊] 華南旅鑽世界盜刷 0.110379147616\n",
      "[問題] 關於郵局的VISA金融卡在美國使用的問題 0.10967519032\n",
      "Re: [情報] 全家便利商店 週末刷台新 0.109016210742\n",
      "Re: [問題] google play商店不能用jcb信用卡??? 0.108370615967\n",
      "[問題] atm繳款銷帳編號？身分證字號？請教 0.107358268939\n",
      "[心得] 國泰世華好事多聯名卡結帳時發生驗證問提 0.106807882141\n",
      "[心得]花旗額度合併Prestige＋法拉利無限卡核卡 0.104929321148\n",
      "[心得] 玉山商務御璽卡核卡 貴賓室討論 0.104675635018\n",
      "Re: [問題] 全家店員說不能刷台新 0.104181687381\n",
      "[閒聊] 自製信用卡優惠網站 CreDiscount 0.103468571875\n",
      "[討論] icash聯名卡...感應時好時壞 0.103327420646\n",
      "Re: [討論] 華南icash網路回饋（韓亞航空） 0.103306130884\n",
      "[問題] 花旗調額 0.103226450398\n",
      "[心得] 台新玫瑰核卡，但收到卡片有個小trouble 0.102656044466\n",
      "[問題] 凱基MB現金點數抵用問題 0.102489523041\n",
      "[閒聊] 花旗語音調額有作用過嗎？ 0.102015207111\n",
      "[心得] AmEx轉卡後第二張申請  SPG credit card 0.10153451562\n",
      "[心得] 渣打銀行繳綜合所得稅無法取消交易!! 0.101089818245\n",
      "[問題] 聯邦偶數日保費2%回饋 0.100999910431\n",
      "[心得] 樂天信用卡 核卡心得 0.100945604802\n",
      "[問題] 關於金融卡刷卡？ 0.100509843824\n",
      "[問題] 中信美國運通被盜刷 0.100356681553\n",
      "[建議] 學生 小白 無學貸 0.100116635188\n"
     ]
    }
   ],
   "source": [
    "related_docs_indices = cosine_similarities.argsort()[::-1] # 可以根據相似度作排序\n",
    "related_docs_indices\n",
    "for index in related_docs_indices:\n",
    "    if cosine_similarities[index] > 0.1:\n",
    "        print titles[index], cosine_similarities[index]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 篩選PTT 資料"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filter_df =  df[df.title.str.contains('問題') & df.title.str.contains('花旗')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#filter_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import jieba.analyse\n",
    "\n",
    "corpus = []\n",
    "titles = []\n",
    "for elem in filter_df.iterrows():\n",
    "    title       = elem[1].title\n",
    "    description = elem[1].content\n",
    "    titles.append(title)\n",
    "    corpus.append(' '.join(jieba.cut(title)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "transformer = TfidfTransformer()\n",
    "tfidf = transformer.fit_transform(X) # 根據CountVectorizer 的結果去計算\n",
    "\n",
    "weight = tfidf.toarray() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 搜尋相關文章"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "def getSimiliarArticle(idx):\n",
    "    print \"[查詢文章]:\" , titles[idx]\n",
    "    cosine_similarities = cosine_similarity(tfidf[idx],tfidf).flatten()\n",
    "    related_docs_indices = cosine_similarities.argsort()[::-1]\n",
    "    for doc in related_docs_indices[1:]:\n",
    "        if cosine_similarities[doc] >= 0.1:\n",
    "            print \"[相似文章]:\", titles[doc], cosine_similarities[doc]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[查詢文章]: [問題] 花旗超級紅利卡代扣公用事業有無回饋？\n",
      "[相似文章]: [問題] 花旗團結衝回饋問題 0.166001856726\n",
      "[相似文章]: [問題] 花旗現金回饋卡的優勢？ 0.130473725562\n",
      "[相似文章]: [問題] 5月體驗天天花旗日活動回饋 0.112508183813\n",
      "[相似文章]: [問題] 信用卡的行李箱 花旗現金回饋卡 0.112508183813\n",
      "[相似文章]: [問題] booking.com花旗回饋金問題(旅館刷卡) 0.105190197395\n",
      "[相似文章]: [問題] 花旗消費回饋之保險專案？ 0.101231104795\n"
     ]
    }
   ],
   "source": [
    "getSimiliarArticle(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 字詞相似度比較"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import jieba\n",
    "\n",
    "titles = []\n",
    "corpus = []\n",
    "for article in df.iterrows():\n",
    "    #print article[1].title\n",
    "    titles.append(article[1].title)\n",
    "    corpus.append(' '.join(jieba.cut(article[1].content)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "transformer = TfidfTransformer()\n",
    "tfidf = transformer.fit_transform(X) # 根據CountVectorizer 的結果去計算\n",
    "\n",
    "weight = tfidf.toarray() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 270,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "XT = weight.transpose()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "word = vectorizer.get_feature_names() # 取得字詞list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29496\n",
      "花旗\n"
     ]
    }
   ],
   "source": [
    "print word.index(u'花旗')\n",
    "print word[29496]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "def getSimiliartTerm(searchTerm, word):\n",
    "    idx = word.index(searchTerm)\n",
    "    #print \"[查詢字詞]:\" , word[idx], 1\n",
    "    print word[idx], 1\n",
    "    cosine_similarities = cosine_similarity(XT[idx],XT).flatten()\n",
    "    related_docs_indices = cosine_similarities.argsort()[::-1]\n",
    "    for doc in related_docs_indices[1:]:\n",
    "        if cosine_similarities[doc] >= 0.3:\n",
    "            #print \"[相似字詞]:\", word[doc], cosine_similarities[doc]\n",
    "            print word[doc], cosine_similarities[doc]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "元大 1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Student\\Anaconda2\\lib\\site-packages\\sklearn\\utils\\validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "鑽金 0.587526015157\n",
      "年資 0.440880203273\n",
      "職業 0.43164626379\n",
      "年齡 0.422996047879\n",
      "貸款 0.407038578652\n",
      "持有 0.399788656602\n",
      "核卡額度 0.399188258944\n",
      "心得 0.390512921008\n",
      "年收入 0.382762732586\n",
      "財力 0.377481610403\n",
      "卡別 0.374299565749\n",
      "證明 0.371473012499\n",
      "富邦 0.359116937815\n",
      "鑽金卡 0.35666283722\n",
      "卡片 0.354909653171\n",
      "提供 0.347555558623\n",
      "核卡 0.346087794623\n",
      "工作 0.343878189645\n",
      "類別 0.343541176903\n",
      "收件 0.333480394205\n",
      "額度 0.328018052703\n",
      "申請 0.318737726582\n",
      "照會 0.317166585629\n",
      "月薪 0.3106538719\n"
     ]
    }
   ],
   "source": [
    "getSimiliartTerm(u'元大', word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 文章分群"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 篩選花旗銀行的資料"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 295,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('C:\\\\Users\\\\Student\\creditcard.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filter_df =  df[df.title.str.contains('問題') & df.title.str.contains('花旗')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 文章切詞"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import jieba.analyse\n",
    "\n",
    "corpus = []\n",
    "titles = []\n",
    "for elem in filter_df.iterrows():\n",
    "    title       = elem[1].title\n",
    "    description = elem[1].content\n",
    "    titles.append(title)\n",
    "    corpus.append(' '.join(jieba.cut(title)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 建立詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 298,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "vectorizer = CountVectorizer() # 宣告詞頻計算\n",
    "X = vectorizer.fit_transform(corpus) #產生詞頻矩陣"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 299,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "transformer = TfidfTransformer()\n",
    "tfidf = transformer.fit_transform(X) # 根據CountVectorizer 的結果去計算\n",
    "\n",
    "weight = tfidf.toarray() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用kmeans 將文章分群"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 300,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import cluster\n",
    "c = cluster.KMeans(n_clusters=10) #將文章分成10群\n",
    "k_data = c.fit_predict(weight)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 表列每一群文章,以了解客戶的問題種類"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 301,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8 8 0 5 0 0 4 2 8 8 8 6 8 4 8 6 3 3 8 5 8 0 2 2 3 9 4 8 8 3 0 4 8 8 0 9 6\n",
      " 8 8 8 8 5 0 7 1 5 8 4 7 4 8 8 8 9 4 8 7 8 6 0 8 0 4]\n"
     ]
    }
   ],
   "source": [
    "print k_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False, False, False, False, False, False, False,  True, False,\n",
       "       False, False, False, False, False, False, False, False, False,\n",
       "       False, False, False, False,  True,  True, False, False, False,\n",
       "       False, False, False, False, False, False, False, False, False,\n",
       "       False, False, False, False, False, False, False, False, False,\n",
       "       False, False, False, False, False, False, False, False, False,\n",
       "       False, False, False, False, False, False, False, False, False], dtype=bool)"
      ]
     },
     "execution_count": 326,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k_data == 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 330,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "516       [問題] 花旗超級紅利卡代扣公用事業有無回饋？\n",
       "881               [問題] 多收到一張花旗信用卡\n",
       "1554    [問題] 現金回饋卡 花旗 國泰 中信 玉山 台新\n",
       "1839             [問題] 花旗現金回饋卡的優勢？\n",
       "2522         [問題] 信用卡的行李箱 花旗現金回饋卡\n",
       "2597            [問題] 花旗消費回饋之保險專案？\n",
       "2849               [問題] 花旗團結衝回饋問題\n",
       "3171            [問題]收到花旗現金回饋無限卡邀請\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 330,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filter_df['title'][k_data==4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用Community Detection 作分群"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 360,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "graph = nx.Graph()\n",
    "\n",
    "for i in range(0, len(titles)):\n",
    "    cosine_similarities = cosine_similarity(tfidf[i],tfidf).flatten()\n",
    "    related_docs_indices = cosine_similarities.argsort()[::-1]\n",
    "    for doc in related_docs_indices[1:]:\n",
    "        if cosine_similarities[doc] >= 0.3:\n",
    "            graph.add_edge(titles[doc], titles[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 381,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAFBCAYAAACrYazjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1YVHXeP/D3QYg5k0/4wHKXt5S6uGVQa4y1WxR123qB\n6Dasv8zwAW0zqDUsR4nNQrKNwFGXu2IuvbJF1LXdFuVeFLPMQjdTBs2FHlYkLVtrGzaQUGZ8gO/v\nD4SVBJyHM3PmDO/XdXF13zhz5gvLzPt8v+dzPl9JCCFAREREqghSewBERER9GYOYiIhIRQxiIiIi\nFTGIiYiIVMQgJiIiUhGDmIiISEUMYiIiIhUxiImIiFTEICYiIlIRg5iIiEhFDGIiIiIVMYiJiIhU\nxCAmIiJSEYOYiIhIRQxiIiIiFTGIiYiIVMQgJiIiUhGDmIiISEUMYiIiIhUxiImIiFTEICYiIlIR\ng5iIiEhFDGIiIiIVMYiJiIhUxCAmIiJSEYOYiIhIRQxiIiIiFTGIiYiIVMQgJiIiUhGDmIiISEUM\nYiIiIhUxiImIiFTEICYiIlIRg5iIiEhFDGIiIiIVMYiJiIhUxCAmIiJSEYOYiIhIRQxiIiIiFTGI\niYiIVMQgJiIiUhGDmIiISEUMYiIiIhUFqz0AIvIfNpsNxUVFqK2uxummJvQfNAhRMTGYM3cuhg8f\nrvbwiAKSJIQQag+CiNRltVpRkJuL7Tt2IBmAweHAAADNACplGVuFwOSEBGRkZcFgMKg8WqLAwiAm\n6uPWWizINpmQabdjjhAI6+YxjQCKJAn5sowcsxnz09N9PUyigMUgJvIif1/qXWuxIM9kws6WFoxx\n4vF1ACbp9chkGBMphkFM5AVaWOq1Wq2YGh+PvU6GcIc6AHF6PcoqKhAbG+ut4RH1GQxiIoVpZal3\nZnIyYktLsdCNj4DVkoRDRiM2lJQoOiZ/X0Eg8gYGMZGCtLLUa7PZMDYyEsccjm5PFK6kAcBonQ61\nJ04oEpBaWEEg8hYGMfVJ3ph5aWmp15yfj0+zs/G6w+H2MebJMsbl5GDR4sUejUUrKwhE3sL7iKlP\n6XXmtWULorKz3Z55FeTmItNudymEAWAMgCV2OwpycxVf6u1JbXU1JngQwgBgsNtxuKbGo2N0rCBc\n6eQlDMCTQmBKSwsmmUwAwDCmgMEZMfUZ3px5qbnUK4RAS0sLvvvuO6e/Gr78Ei9fuIAUN8baYSOA\nv9x7L7a88w6Cglxv0qelFQQib+KMmPoEb8+8iouKYLz4fHcMAWCUJBS9/jrm/frX+Pe//+1SsAYF\nBWHo0KHdfkVGRmL8+PFdvvfCM8+g+c033Rxtu2YAez/8EP3798fo0aMxduxYREVFdfnvkCFDeny+\nllYQiLyJM2IKeL6Yec2fORPjN21CmgfjtAAwAQgNC+sxVC/9GjZsWOf/LcuyS6+l5DXiR9PTcfTo\nUdTW1uLIkSOd/z1y5AiuuuqqbgN6wIABuHnsWL8pFiNSE4OYNMfVQitf3Kbz0JQpmLxtm8dLvdsn\nT8bmbds8OIpzfLGULoTAt99+e1lA19bW4vO6Ovy/tjb80YOPH6WKxYjUxqVp0gx3Cq1sNhu279iB\nl938wJ8jBJ4vL0d9fX2vM6/+gwah2a1X+I9mAAN7WcpVUnh4OCYnJGC9myco6yUJSYmJvf5OJElC\nREQEIiIicNddd3X5t18/9BBiN292+XUvpUSxGJE/4DaIpAlrLRZMjY9HbGkpjjkcWOdwIA1ACoA0\nAK/b7TjmcODW0lJMjY/HWosFgHLXbouLirr997q6OpjNZuz6299Q4eZrdLDKMqKioz08ivMysrKQ\nJ8uoc/F5dQDyZRkZWVluv3ZLczMGuP3sdgMANDc2engUIvUxiMnvXVpotbCHamfgP4VWe1takGcy\nYa3FothtOrUXZ15CCBw8eBBLly7FTTfdhLi4ONTV1eHFl17CW6GhcDcWGgBsFQKzU1M9GqsrDAYD\ncsxmTNLrnQ7jjgYkOWazRxXLSq0gDAhz9xSLyH9waZr8mtVqRbYT1c6XGgNgZ0sL4kwm/HT8eNzt\n4RgGADh+9CgWLFiA0tJSyLIMo9GI1157DRMmTOi8dWfbn//s1aVeb+ioCI8zmbDEbkdqDyc6DWi/\nrWuFQg01omJiUFlSgjQPTpKssoxxPlpBYOtN8ipB5MdSjEaxWpKEAFz+WiVJ4sb//m9hceO5l34V\nAmLE0KHixRdfFJ9++qloa2vrdqyVlZUiQq8XR108/lFAROj1wmq1+vi3+x9Wq1XMTE4Wg3U6MVeW\nRSEgNlz82efKshis04mZycmKjfHbb78Vg3U60eDm/ybfAWKwTidsNpsi4+lJZWWlSDEaxWCdTszT\n6YQFEBsBYbnk95JiNIrKykqvjoMCG4OY/JYSH9Zyv34iVafzKIhTdTphzs93asy/X7lS/JckOR3G\nRwExSq8XawoLvfzbdI7NZhPm/Hwxf9YsMSMpScyfNUuY8/O9EnienmTNTE5WfEyXWlNYKCL0erFa\nknr8G2y4OJYIP/rfkLSHty+RKpxZ6lPiXtcHdTpsO3cOX7W1ef1+VSEEpk2bhob6evzj4EGfLvVq\nkT931tLK5h0UIFQ+EaA+xpWlvkdSUhRZVg4LDharfDDzWrlypYiNjRUOh8PnS71ataawUIxyYTnf\nFysIWr7EQNrEICafcXWpb8JNN4mNHgbxBkBM/PnPRXhoqFc/WPfu3SvCw8PF8ePHu3zfl0u9WtXx\nd7Gql7+L7wCx0kdLwP6+ZE6Bh0FMPuHOzGdEv34iRYEZ8eDQUHHdyJHiv0NCvDLz+te//iWuvfZa\nsX37dh/8JgOTv6wgaKWIjAILrxGT13lyLfA2ADsBuHslcHZICEYuXowXfve7zt2XlLx2e+HCBfzi\nF7/AHXfcgeXLl7s5SupQX1/fXjtQU4PmxkYMCAtDVHQ0Zqem+uQ2IX/ap5n6DgYxeZ0nvZ7NAKwA\n/uTG63ZXaFVVVYWC3FxsKy+HUZJgsNs722RaZRlbhUBSYiIysrKcKgR65plncODAAezcuRP9+vVz\nY5TkT5TavOPwrFlYU1ys1LAowLGhB3mstwpoIYRHvZ7nAXgeQD0AV+dD3TXJiI2NxYaSks6Z1+FL\nZl7joqOR58LMa9u2bSguLsbBgwcZwgHidFMTW2+SzzGIyW3ObMIQNXo07nTz1iGgvdfzZACrAOS6\n8LyOfshlPfRDHj58uEdLh8ePH8fDDz+MrVu3Ijw83O3jkH/xRetNdumiy6h6hZo0y9kKaDMghgFi\njYcFV0MuFlApXWjlDrvdLsaPHy9Wr17tleOTelbk5Ym5HjaAmSvL3TaAYZcu6gmDmFzm1r2fHoTx\nBkAMCQkRYf36iZV+cIvLo48+KqZNm9Zjq0vSLiWqpvv36yf+8Y9/dDkuu3RRbxjE5BKPmh0Awurm\njHjKxIl+cYtLcXGxiIqKEk1NTV57DVKXJ/cRrwDEdeHhYujQoSI7O1ucOnXKL5uWkH9h1TS5xJMK\n6NUADgHY4OLz5soybrrkdhC1bnGpqanBvffei927dyPah/sGk295crvdHaGhCBk2DLfddhtCQ0Ox\nY8cO9DtzBvvPn/e7Np7kR9Q+EyDtUKTZASBsrj7HDxokNDU1iaioKFFcXKzqOMg3PJnFNjc3i8ce\ne0yMGDFC3HHLLWKlm+8XdunqOxjE5DRFClnQXsClpQ+jtrY2MW3aNDF//nxVx0G+5WnrzTfffFPI\nALt00RUFqT0jJ+2ora7GBA86DgGAAUCtk4/tuAUpo4dbkHyloKAAx44dQ0FBgarjIN+an56OsooK\nHDIaMUqnwzxZhgXARrQ37Zgnyxit0+EjoxFlFRWXdWH74tgxTNfpPLp1zyhJKC4q8ujnIP/H+4jJ\naYo1O3DicR3byuWYzapeI/vggw+Qm5uL/fv3Q6fTqTYOUocnDWBqq6txm6cnrnY7DtfUeHQM8n8M\nYnKaUs0OvgLQCPj9Pr02mw0PPvgg1q1bh+uvv161cZD63GkAwy5d5CwuTZPTomJiUOnhrHAPgGND\nh2JUaKjLS32+1NraioceegizZs1CUlKSauMg7fJFly4KDJwRk9Mt92anpmJsdjZWovvZ7JU0ANgR\nGoqjn30GAB73evamZcuWoa2tDc8//7zaQyGNioqJQWVJCdI8WJ62yjLG8Va5gMf7iPuwXntFX9yJ\naHJCAjKysmAwGAB4eB+xJOGQ0YgNJSWK/hxK27FjBx555BEcPHgQP/rRj9QeDmmUzWbD2MhIHHM4\n3D5x/eHuYexTHaDULtsmdbjbcu/tt98WYf36uddZS6/3+sbunvriiy9EeHi42Lt3r9pDoQDgSZcu\nMyBmGo1CCPapDnQM4j7I3WYFsx56SPTv319cFRwsRgQHB1zLPofDIWJjY4XZbFZ7KBQgPGkJOxAQ\nN998s/jf3/+efaoDHIO4j/H0g+HWW28VdXV1Hjc78Efp6ekiOTmZmzmQotw58b1GkkTq7NkiNCRE\n/Bf8Z+cx8g4GcR/jyVLZyh90uaqsrBQTf/YzIUuSeCgoSJVNGJSyceNGMWbMGHHq1Cm1h0IByNUT\n19TZs8XgwYPFUBdWnrR2GYj+g8VafYiSxSNff/01Fi1ahJMnT+K5557DyX/+E0d9vAmDUj755BPE\nx8dj165duPnmm9UeDgWoqqoqFOTmYlt5OYySBIPd3lkcab1YHJmUmIiMrCzExsbiVwkJuOOtt/CU\nG6+llcJIascg7kPM+fn4NDsbr3twO8VcnQ6fRkfjyxMn8Nxzz+GRRx5BSEiIgqP0rebmZhgMBjz9\n9NNITU1VezjUBzize5g3Kq7Jf/E+4j5EiV7RExwOfHzmDI4cOYJBgwYpNDJ1CCHw61//GnFxcQxh\n8hlnunQVFxXBCPfu1we69ql2tSMY+R6DuA9RquXej0eN0nwIA8Arr7yCo0ePYt++fWoPhagLRTZY\nYZ9qzWAQ9yFsufcf+/fvx/Lly7mZA/klJftUswmI/2MQ9yFsudeuvr4eDzzwAF577TWMGjVK7eEQ\nXUapk+a///3vGBsZeXnnvC1bEJWdfVnnPFIHi7UCTG9nv0KIPl8A0traisTERPz0pz/FSy+9pPZw\niLqlRGHlDAB2AH9A99eaG9G+01m+H+x01tcxiAOEs32jGxsbcV9FRUD3iu7NsmXL8P7772PXrl0I\nDuaCEPknJaqmRwE4CuBKp8wde39nMoxVwyAOAGstFmSbTMi02zFHiF7Pfn931VWQhMCH585hjAuv\nUQcgTq9HWUUFYmNjlRm4j7311lt4+OGHcfDgQURERKg9HKJeebLByioAHwHY4OTjA+H9rWUMYo1b\na7Egz2TCzpYWp4K1DsAdISEIFQK7L1xw+jlaP2M+ceIEJkyYgD//+c+466671B4O0RVZrVZMjY/H\nXiff2x3qAMQBKAPgSqQGwoqXVjGINcyTN+rNAOTgYPy2tRVze5hFN6B9Fr1C49eQzp07h7i4OEyb\nNg2LeU8laYg7J9q/APA0gPkuvlYg1IBoVb9ly5YtU3sQ5J7MBQvwwMcfI9HF5w0BIEsSzt59N5p+\n+lM8fvw4akNC8PWFCzgC4AMAFllGenAwBk6dioJ16zDll79U/gfwkYULF0IIgYKCAkiSpPZwiJx2\nq8EAecgQzH7vPfS7cAE/ASB387gGAIWShDkAlsH1EMbF49aGhOCbsDD8/I473B80uYwzYo1SsgUe\ngCu23PNHztwf+cYbb2Dp0qWoqqrC4MGDVR4xkXuc6VN9zfDhmPrVV8j14HUsAA7PmoU1xcXKDJyc\nwrJRjVK6BZ6W2uD1WiF+yf2RU6ZPx4IFC/DOO+8whEnTYmNjsaGkpLNP9eFLTprHRUcjLzUVGfPm\n4aavvvLodTqagJBvMYg1qq+2wLu0Qvzlbq5tp9ntWAngD6WlSCstxdSUFNxyyy1qDJVIcb31qWbn\nPO0KUnsA5B4lW+BpRUfhyt6WFizsocAMaF8leEoIWIXA37ZswVqLxZfDJFJFVEwMKj1s12qVZURp\nvHOeFjGINaqvnf1arVZku1A9CgBjAOxsaUG2yYSqqipvDo9IdbNTU7EV7T0D3NEAYKsQmM2dyHyO\nQaxRfe3styA3F5l2u0u3aQHtYbzEbkdBriclLET+Lzw8HJMTErDezTsD1ksSkhIT/bpAM1Cxalqj\n+tLG4X3pZyXyhEdNQNhZSzWcEWtUXzr7VbJCnCiQGQwG5JjNmKTXo87J53R0zssxmxnCKmEQa1hG\nVhbyZNnpN1yHOgD5soyMrCxvDEtxSlWI12qsQpzIHfPT05FpNiNOr8dqSerxmnEDgFWShDiNt68N\nBAxiDesrZ799sUKcyBPz09NRVlGBQ0YjRul0mCfLsADYiPamHfNkGaN1OnxkNKKsooIhrDLeR6xx\nHW+gOJMJS+x2pAZg3+i+ViFOpARnmoBo4dJUX8AgDgDz09Mx3mBAQW4unu+lBV5SYiLKsrI0MxPu\nEBUTg8qSEqR5sDxtlWWM00iFOJGSemsCQv6BVdMBpuPsV2t9o3vDqmkiCmQMYtKEmcnJuLW0FE+6\n8efKfVaJyJ+xWIs0wZiSgueBgK8QJ6K+h0FMfm/Lli1IT0/H5OnTA75CnIj6HgYx+a3z58/DZDLh\nqaeewvbt27Fx82beH0lEAYfXiMkvffPNN5g+fTquvvpqbNy4EUOHDu38N2c2SU9KTESGBivEiajv\nYRCT36moqMCMGTOQlpaGpUuXIiio+4WbQKwQJ6K+h0FMfkMIgRUrVmDVqlVYv349Jk2apPaQiIi8\njg09yC80NTUhNTUVX3/9NSorKzFy5Ei1h0RE5BMs1iLV/f3vf0dsbCyuvfZa7NmzhyFMRH0Kg5hU\nVVRUhIkTJyInJwevvPIKQkND1R4SEZFPcWmaVOFwOPDEE09gz549eP/99zFu3Di1h0RETrDZbO1F\nktXVON3UhP6DBiEqJgZz5s5lkaSbWKxFPnf8+HFMmzYNo0ePxrp16zBggKebHBKRt1mtVhTk5mL7\njh1IBmBwODpvG6y8eNvg5IQEZGRlwWAwqDxabeHSNPnU9u3bcfvtt2PWrFn405/+xBAm0oC1Fgum\nxscjtrQUxxwOrHM4kAYgBUAagNftdhxzOHBraSmmxsdjrcWi8oi1hTNiPxOoyz6tra3Izs7G+vXr\n8cYbb+COO+5Qe0hE5IS1FgvyTCbsbGnBGCce39Fall3tnMcg9hOBvOxTX1+PGTNmQAiBzZs3Izw8\nXO0hEZETrFYrpsbHY6+TIdyhDkCcXo+yigp2t3MCl6b9QCAv+3z44Ye49dZbcdttt+Htt99mCBNp\nSEFuLjLtdpdCGADGAFhit6MgN9cbwwo4nBGrLFCXfYQQePnll/HCCy9g3bp1mDJlitpDIiIX2Gw2\njI2MxDGHA2FuPL8BwGidDrUnTmj6spovcEasIqvVimwXQhhoP9Pc2dKCbJMJVVVV3hye25qbm/Hg\ngw+iqKgI+/fvZwgTaVBxURGMgFshDABDABglCcVFRcoNKkAxiFUUiMs+n376KSZMmIABAwbggw8+\nwKhRo9QeEhG5oba6GhMcDo+OYbDbUVtTo9CIAheDWCU2mw3bd+zAHDevDMwRAtvKy1FfX6/wyNy3\nefNm3H333ViyZAlee+01yLKs9pCIyE2nm5rg6c2FAwA0N/a0czh1YBCrJJCWfc6dO4cFCxbg2Wef\nxTvvvIO5c+eqPSQi8lD/QYPQ7OExmgEMCHP3U67vYBCrJFCWfb766ivcfffdOHHiBKqqqnDLLbeo\nOh4iUkZUTAwqdTqPjmGVZURFRys0osDFIFZJICz7vPPOOzAYDLj//vuxdetWDB48WLWxEJGyZqem\nYisAdz9hGgBsFQKzU1OVG1SAYhCrRMvLPm1tbVi+fDnmzJmDzZs3IzMzE0FB/FMiCiTh4eGYnJCA\n9ZLk1vPXSxKSEhN565IT+OmpEq0u+3z33XdISkrCzp07UVVVhXvuucenr09EvpORlYU8WUadi8+r\nA5Avy8jIyvLGsAIOg1glWlz2qaqqQmxsLG644Qa89957uOaaa3z22kTkewaDATlmMybp9U6HcUfT\noRyzme0tncQgVomWln2EEFizZg0SEhKwYsUKrFy5EiEhIV5/XSJS3/z0dGSazYjT67FaknqcPDQA\nWCVJiNNA5z9/wxaXKvKkofodOh22793r9TPOlpYWpKen49ChQygpKUFUVJRXX4+I/FNVVRUKcnOx\nrbwcRkmCwW7v3JjGenFjmqTERGRkZXEm7CLOiFXk7rLPxKuugr1fPxw/ftybw8PRo0dx++23o62t\nDfv372cIE/VhsbGx2FBSgtoTJzAuJweHZ83C0wMH4t2EBIzLyUHtiRPYUFLCEHaHINWtKSwUEXq9\nWCVJogEQopuv7wCxUpJEhF4v1hQWikOHDomRI0eKZ599VrS2tio+ppKSEjF8+HBRWFgo2traFD8+\nEWnfPffcI3bt2qX2MDQvWO0TAWq/BjPeYEBBbi6eLS1Fcr9++Nn5890u+5RdsuxTWVmJX/3qV6ip\nqUFxcTEGDLj8zmSbzYbioiLUVlfjdFMT+g8ahKiYGMyZO7fb68vnz59HVlYW/vKXv2D79u2a2/uY\niHxn0KBBOHXqlNrD0DxeI/YjQghERETg1/Pm4d8nT6K5sREDwsIQFR2N2amp3QbnuXPn8Pjjj2P/\n/v34v//7v85NFqxWKwpyc7F9xw4kAzA4HJ3BXnkx2CcnJCAjK6szbL/55htMnz4dV199NTZu3Iih\nQ4f67ocnIk249OT+bxUViBgxAolGY48n93RlDGI/UldXh3vuuQcnTpyA5EI1tRACr776Kl544QVs\n3rwZR//xD2SbTMi02zFHiG77WTcCKJIk5MsycsxmjL3xRsyYMQNpaWlYunQpG3QQURfunNyTcxjE\nfuT111/Hu+++i02bNrn1/N27dyP5l7/EoLNn8e75805VYtcBuDckBN/rdPjTm29i0qRJbr02EQWu\ntRaLyyf3vH3JebxG7Ef27NmDu+66y+3nDxgwALoLF5wOYaB9b+Pd58/jzuBgLkUT0WXWWizIM5mu\neJtlGIAnhcCUlhZMMpkAgGHsJM6I/cj111+P8vJy3HDDDW49f2ZyMmJLS7HQjf9JV0sSDhmN2FBS\n4tZrE1Hg8aTXQZxej7KKCt7O5AQGsUp+WM0sBQfjr2+/jc+PH0d4eLhbxxsbGYljDodbexw3ABit\n06H2xAkWXBARAJ7c+worcnzMarViZnIyxkZG4rPsbIzftAmTt21DXGkppp49i7GRkZiZnAyr1erS\ncYuLimAE3AphABgCwChJKC4qcvMIRBRIbDYbtu/YgTluztXmCIFt5eWor69XeGSBh0HsQ2stFkyN\nj0dsaSmOORxY53AgDUAKgDQAm1pbcczhwK2lpZgaH4+1FovTx66trsYEh8Oj8RnsdtTW1Hh0DCIK\nDDy59x0Wa/mItwseTjc14fJ2Hq4ZAKC50d39oIgokCh1cn+YJ/dXxBmxD1itVmSbTNjpQsHDGAA7\nW1qQbTKhqqqqx8dduHABVVVV+Orrr9Hs4TibAQwIc/f8l4gCCU/ufYdB7AMFubnItNtdqjoE2sN4\nid2Ogtzczu85HA7s2bMHL7zwAiZNmoShQ4ciNTUVrSEh2Ofh1oRWWUZUdLRHxyCiwNB/0CCe3PsI\ng9jLlCh4+GtZGRYuXIg777wTw4YNg8lkwqlTp/DYY4/h2LFj+Pjjj1H617+irF+/HvcKvZIGAFuF\nwOzUVDePQESBJComBpU6nUfH4Mm9c3j7kpeZ8/PxaXY2XvfgWktKUBDq77kHS55+Grfffjv69+/f\n7eN4qwERKYW3RPoOi7W8TImChzvb2nD4mmswceLEXh+XkZWFqTt3IsmNm+/zZRllWVkejZOIAkd4\neDgmJyRg/RVO7m0AigHUAjgNoD+AKAB2AEmJiQxhJ3Bp2st8WfBgMBjw2xdfxN1BQahz8th1ACbp\n9cgxm9kBh4i6yMjKQp4sd/t5YgUwE8BYAJ8BGA9g8sX/fgwgF0BDQ4PLPRH6Igaxl/my4KGtrQ3v\n79mDSIMBcXo9VktSj9eMGwCskiTE6fXIZIN2IuqGwWBAjtmMSXp9lzBeC2AqgFgAxwCsA7r0RCgC\ncBLAxIoKl3si9EVcmvayqJgYVJaUIM2D5WmrLGOcEwUPzz77LGw2G96rqEBNTQ0KcnPxfHk5Es+d\nw51tbZ1bllkvblmWlJiIsqwszoSJqEcdJ+lxJhMWtbSgH4BXAOwFuAmEQlis5WW+KnjYuHEjnnvu\nORw4cKDL42w2G0Zddx2SExNx4exZDAgLQ1R0NGanpvLaDRE5bfny5Xh15Uo4mppQhd5D+Ie4CUTv\nGMQ+4O1q5n379uH+++/He++9h3HjxnX5t2+++QYxMTGw2WyQJMnl1yciEkLglltuQfjVVyNx/348\nyTszFMVrxD7QW8FDbzqqmTN6qWb+4osvMG3aNKxfv/6yEAaA6upqREdHM4SJyG1vvfVWexe/jz5C\nKjeBUByD2Ad6KnjojTPVzM3NzZgyZQoyMzORkJDQ7WNqamoQExPj3sCJiADk5eXhlpgYbgLhJQxi\nH5mfno5Ms1mxaubW1lbMmDEDP//5z/HEE0/0+LodM2IiInccOHAAX3zxBeSgIO7w5iUMYh+an56O\nsooKHDIaMUqnwzxZhgXARgAWAPNkGaN1OnxkNKKsoqLXCsMlS5bAbrfjlVde6XXZmTNiIvJEXl4e\nFi1ahJbvv+cmEF7C25d8LDY2FhtKSlBfX4/ioiIcrqlBc2Mjtr31FpY8/TRq09OvWM382muvoays\nDPv370dILxs9nD9/HkeOHOn22jER0ZUcOXIEf/vb37BhwwbUHDjATSC8hEGskuHDh2PR4sWd///N\nN9+MhMmTrxjC77//Pp555hns3bsXQ4YM6fWxtbW1GDFiBPR6vSJjJqK+ZcWKFXj88cdx9dVX+7Qn\nQl/DpWk/MXr0aHz++ee9Pqaurg4PPvgg/vjHPyIqKuqKx+SyNBG56+uvv8aWLVvwm9/8BgAwOzUV\nWwHu8OaPNzzjAAAWfklEQVQFDGI/caUgPnXqFJKSkpCTk4P/+Z//ceqY1dXVDGIicsvvf/97zJo1\nC0OHDgXQvgnEjWPH4nU3j7dekrgJRA+4NO0HbDYbjn3+Od7auhU1+/ah/6BBiIqJwZy5czF8+HBc\nuHABDzzwACZNmoRHH33U6eNWV1fj4Ycf9uLIiSgQnTp1CuvWrcOhQ4c6v2e1WnHkH//ACgC/hOud\ntV4KDcV27vDWLXbWUpHVakVBbi6279iBX7a14fZz5zr7QVde7Ac9OSEBjqAgnDlzBmVlZQgOdv7c\nKTIyErt378bo0aO99jMQUeB56aWX8Mknn2DDhg2d3+voEKgXAnkAdsK5MK4DcC+Aa2+5BR9+9JF3\nBqxxDGKVrLVYkG0yIdNuxxwhur1JvhHAHyQJywEsM5uR8dRTTh//1KlTGDFiBL7//nsEBfEKBBE5\nx+Fw4Prrr8fbb7/d2YPghz3z1wLIBrAEQCq6b/LRgPZdmFZcfNzzTvTM76v4Ca2CtRYL8kwm7G1p\nwcIeQhho/+N+SghYhcD/PvusS1uJffzxx7jpppsYwkTkkuLiYowfP75LI6DioqIuXbXmAygDcAjA\nKADzgK49EQCMBvDRxcc9CXbV6g2vEfuY1WpF9sUQdvYayxgAO1taEGcyYbzB4NTuJSzUIiJXtba2\nYsWKFVi3bl2X79dWV1/WVSsWwAYA9QCKARzGxfuEAYwDkAfg0rmvwW7HYXbV6haD2McKcnORabe7\nVOgAtIfxErsdBbm5Tu1ewtaWROSqrVu3YtiwYYiLi+vy/dNNTT121RoOYJETx2ZXrZ5x3dKHbDYb\ntu/YgTk+2L2E9xATkSuEEMjLy0NmZuZlbXP7DxrErlpexCD2oR9eZ3GVs7uXCCFQU1PDGTEROe29\n997D6dOnMXXq1Mv+LSomBgd0Oo+Ob5VlRPEzqVsMYh/q7jqLq5zZveTLL7/EwIEDr9gCk4ioQ15e\nHhYvXtxtgefs1FS8ef48u2p5CYPYh3q7zuIsZ66zsFCLiFzx0Ucf4eOPP0ZKSkq3//7VV1/hquBg\ndtXyEgaxD/nqOgsLtYjIFfn5+Vi4cCFCQ0O7fN9ms2FFfj5+lZCAayMi8DyAp9FeKe2sOgD5sowM\ndtXqEYPYh6JiYlDpg+ssLNQiImcdO3YMb7/9dpf2uVarFTOTkzE2MhKfLF2Kp+vr8fSXXyIPwJcA\nfgxgJgDrFY5dB2CSXo8cs9mp2y77KnbW8qEfdqdxVQOA0d10p7HZbCguKkJtdTVONzVh9549SHn4\nYTydlcWlICLq1eOPP46BAwciNzcXgAtd/9DeNSsH7Q0+LtUAoEiSsEKWkWM2Y356ujd/BM1jEPtY\nR7/WhW782ldLEg4ZjZ33EV/aqzoZgMHh6OxVfUCWUXqxV3VGVhYMBoOiPwcRaZ/NZsPYsWPx2Wef\nISIiorPr304nGw7VAbgHwF0AEtD+2WO92Cc/KTERGVlZnAk7gUHsY1arFVPj413qrAW0/8HH6fUo\nq6hAbGys02etRZKEfJ6VElE3nn32WdhsNqxZs8ajz6bb+vVD7M9+huuuvx5R0dGYnZrK1ThXCPK5\nNYWFYpReL44CQjjxdRQQo/R6saawUJHnExE1NzeLYcOGidraWiGEEClGo1gtSU59pvzwa5UkiZnJ\nySr/RNrFGbFKOma0S+x2pPYwo20A8DqAvJAQ/K6gAPPT0xWbURNR37Z69Wrs27cPb775ptfqV8g5\nrJpWyfz0dJRVVOCQ0YhROh3myXLX3UtkGaN1OuydOBHn9XoYp00DoEyvaiLq286dO4dVq1YhMzMT\ngO+6/lH3OCP2A/X19e1VzzU1aG5sxICwsC7XWTIyMuBwOLB8+XKetRKRx4qLi7F+/Xq8++67AID5\nM2di/KZNSPPgmBYAh2fNwpriYkXG2Jdw9yU/MHz4cCxavLjHf1+2bBluuOEG6HU6xc5ae3s9Igpc\nbW1tyM/Px6pVqzq/56uuf9Q9BrEGhIWF4fnnn8eLS5fiaQV6VXNPUKLA98P+Av0HDUJUTAz+69pr\ncdVVV+G+++7rfCx3V1IXrxFrxMMPP4wLZ8/yrJWIenVpV6zPsrMxftMmTN62DeM3bcKny5Zh/qxZ\nGBgcjKqqqs7ncHcldTGINaJfv36Ive02nrUSUY/WWiyYGh+P2NJSHHM4sM7hQBqAFABpAF632/FP\nITC1qgpT4+Ox1mIBwN2V1MalaQ25c+JE/G33bqS1trp9DKssYxzPWokCTkdXrCvd2hgG4CkhMLWl\nBZNMJgDA4KFDIbW14XUAi9x4be6u5BlWTWuIzWZD1MiROH72LKumiaiTR12xgoPRJATuvfde1Hzw\nAXsUqIBL0xoSHh6OpMREFEmSW8/nWStRYPKkv8BvL1zAUFlGeXk5csxmTNLrUefk87m7kjI4I9YY\ndtYioksp0RXr+pAQ1J08ieHDhzvd9Y+7KymHM2KNMRgMyDGbMTE0lGetRKRIV6xfBQd3dsVytuvf\nR0YjyioqGMIKYLGWBs1PT4cAMOE3v8FSITCXZ61EfVZtdTUmKNxfIDY2FhtKSjq7/h2+pOvfuOho\n5HF3JUUxiDXq0fR0DB4yBGmzZ2O5JMEYFASD3d65H/Gle4KWcU9QooDlza5YV+r6R8pgEGvY9OnT\nsW/fPjQ2NmJcdDQ+rKzExx9/jNazZ6HT6TDlxhsRM2ECIiMj1R4qEXkJu2JpH4NY45YtW4bRo0ej\n4Z//xAcffohkAAaHo/0M97PPUFlejqjsbExOSEBGVhYMBoPaQyYiBUXFxKCypARpHixPs7+Aulg1\nrXFrLRY8s3Ahnj53DvPQfcFGI9qvFefzWjFRwOFewtrHqmkN6+ik8+G5c1iEnqsmwwA8KQT2trQg\nz2TqbGtHRNoXHh6OyQkJWM/+AprFGbFG8X5iIurAzwNt44xYozzppLPEbkdBbq43hkVEKjAYDHgu\nPx/xQUE4AMAMYD6Ahy7+1wyg/gfPYX8B/8EZsQbxmhAR/dCDDz6I97dvx/enT+P/AfgZ0Hk7YyWA\nrQAmA0gFUM3+An6FM2INUqKTjlGSOjvpEJG2zUpJwVt//jMyz5zBSQDrga7bHwI4BuCnAKYBePPm\nm9kVy48wiDVIqU46tZd00iEibVr05JN4949/RJUQeLKHLntA+4n7IgBVAGy1tThktfpukNQrBrEG\nebOTDhFpR0lJCdb9/vfYAzhdLzIGwM6WFmSbTKiqqvLi6MhZDGINYicdIjp16hQeS01FtiSxaFPj\nGMQaFBUTg0qdzqNjWGUZUeykQ6RJFy5cwP3334+Wi1sVumOOENhWXo76+h/WU5OvMYg1aHZqKrai\nvWOWOxoAbBUCs1NTlRsUEfnMU089hX998w2mhYSwaDMAMIg1iJ10iPoui8WCXbt24fZbbsFtLNoM\nCAxijcrIykKeLKPOxefVAciXZWRkZXljWETkRe+++y5ycnJQVlaGcy0tLNoMEAxijTIYDMgxmzFJ\nr3c6jNlJh0i7amtr8dBDD+GNN97A6NGjWbQZQBjEGjY/PR2ZZjPi9HqslqQerxk3AFglSYjT65HJ\nTjpEmtPY2IgpU6bgd7/7HeLj4wGwaDOQsMVlAKiqqkJBbi62lZfDKEkw2O2dre2ssoytQiApMREZ\nWVmcCRNpzPnz55GQkICYmBisWrWq8/s2mw1jR47EsbNn2epW44LVHgB5LjY2FhtKSlBfX4/ioiIc\nrqlBc2MjBoSFYVx0NPJSU/lGI9KohQsX4qqrrsKKFSu6fP/QoUPoJ0n4A4Cn3Dguizb9B2fE5BSb\nzYbioiLUVlfjdFMT+g8ahKiYGMyZO5dvZCI3OPOeevXVV1FYWIh9+/Zh0KBBANobeSxatAjvvvsu\nFi9ejBeWLOH2h1oniHpRWVkpUoxGMVinE/N0OmEBxEZAWAAxV5bFYJ1OpBiNorKyUu2hEmmCs++p\nl19+WURERIjPP/+887nl5eVixIgRIi0tTXz//fdCCCHWFBaKUXq9OAoI4cTXUUCM0uvFmsJCtX4F\n9AMMYurRmsJCEaHXi9WSJBp6eFM3AGKVJIkIvrGJrsjZ99RKSRKDALHoySeFEEI0NjaKuXPniuuu\nu07s2rWrx+Ou6uW43108Lt+r/odBTN3iWTaRstx9Ty147DExYsQI8dhjj3XOgrtjtVrFzORkMVin\nE3NlWRQCYgMgCi+Zac9MThZWq9WHPzU5g9eI6TJWqxVT4+N53YlIIZ68p2IlCbmvvop0J2877Cja\nrL2kaDMqOhqzWbTptxjEdJmZycmILS3FQjf+NFZLEg4ZjdhQUuKFkRFpkyfvqVWShI/4ngpoDGLq\nwmazYWxkJI45HLw3kUgBfE/RlbCzFnVRXFQEI8AdXYgUwvcUXQmDmLqora7GBO7oQqQYvqfoShjE\n1MXppibu6EKkIL6n6EoYxNQFd3QhUhbfU3QlDGLqgju6ECmL7ym6ElZNUxes8CRSFt9TdCXcfYm6\nCA8Px+SEBKx3857H1wGEDxuGc+fOOfV4biZBgc7T9xR3SQp8nBHTZTztrJUwfTr++te/IisrC088\n8QRCQkK6fY2C3Fxs37EDyQAMDkfnHsqVF/dQnpyQgIysLBgMBmV+MCKVsFsd9Uqt3prk3zztNV1b\nWysmTZokbrzxRrF79+7Ljs3NJKivYf926gmDmHrk6Y4ubW1tYsuWLWLkyJFixowZ4uTJk/wwoj6N\nuyRRd7g0Tb2qqqpCQW4utpWXwyhJMNjtnUvI1otLyEmJicjIyupx6ezMmTN48cUX8eqrryKkpQUf\nnj/P5TnyS76oWVDiPUWBhUFMTlFiR5df3ncf7t61C0+58frcTIK8SY2aBe6SRB0YxOQTvIWD/NVa\niwXZJhMy7XbMEaLbv89GAEWShHxZRo7ZjPlObklI5Aw29CCfYON78kdrLRbkmUzY29KChT2EMND+\nd/ukENjb0oI8kwlrLRZfDpMCHIOYfIKN78nfWK1WZJtM2OnCLUVjAOxsaUG2yYSqqipvDo/6EDb0\nIJ9Qs/E9m4ZQdwpyc5Fpt7tUOAi0h/ESux0FubmsWSBFMIjJJ9RofN9rAc6WLYjKzmbTkD7KZrNh\n+44deNnNEpk5QuD58nLU19fzZI48xqVp8gklGt9/GBKCyKgopx671mLB1Ph4xJaW4pjDgXUOB9IA\npABIA/C63Y5jDgduLS3F1Ph4XvPrY1izQP6EM2LyidmpqRibnY2VcO/DrwHAltZWBOXno7auDqmp\nqbjrrrsQFHT5ueSlBTi9LTt2FOBMaWnBJJMJAFgN68eUvMSgVM3CYdYskAI4Iyaf6Gx8L0luPX+9\nJMF4//04evQoYmJisGDBAowZMwY5OTk4fvx45+NYgBN4rFYrZiYnY2xkJD7Lzsb4TZsweds2jN+0\nCZ8uW4aokSMxMzkZVqvV6WOqWbNAdBn1mnpRX1NZWSkiXGhveWmbywi9Xlit1s5jtbW1iYMHD4oF\nCxaIYcOGifj4eFFUVCSmT5kiVkuSS8fv+FolSWJmcrKKvyH6IW/1JX8kJUVY3PgbufSrEBDzZ83y\n8m+A+gIGMfmUN3pNOxwO8Ze//EXcd999Qr74wezOB+t3gBis0wmbzaboz/ztt9+KFXl54pGUFDEj\nKUk8kpIiVuTlKf46gcYbfytHjx4VZrNZXH/ddeJBD4N4riwLc36+D38jFKgYxORz3mp8vyIvT6Tq\ndH7z4VpZWSlSjEYxWKcT83Q6YQHERkBYLr7OYJ1OpBiNorKyUpHXCyRKrZ60traKAwcOiN/+9rdi\n3LhxIiIiQsyfP19s2rRJDA4N9buTNuqbGMSkCqvVKmYmJ4vBOp2YK8uiEBAbLi73dYTUzOTkLsvR\nV+JPy42BtNWjGjP6FKPR7UsMKyVJ3HfHHSItLU1cc8014ic/+YnIzMwUH374oWhtbVXkNXgZg5TE\nXtOkKiUb3z80ZQomb9uGFA/GsxFAeVIS/lhW5vYxOqq2nS0YqwMwSa9Hpps9jL3VsESNjRAAZfqS\n/7ckYdHSpUhJScHYsWO7fZzVasXU+PgrVtf/EHcDI8WpfSZApBR/mBErWZDmzGt5a+lbzRn9irw8\nMddHlxi4Pzb5AwYxBQxffoD3xFfLnd4MSrXDydcnVN6qWSByFoOYAsa3334rBut0qhXg+Or1vRmU\nvpzR92RGUpLY6GEQbwDEjKQkp1/TGzULRM5iQw8KGEo0DUlKTHT7uqov2iZ6u2GJEhsheEqNvuSx\nsbHYUFKC2hMnMC4nB4dnzUJ5UhIOz5qFcTk5qD1xAhtKSnhNmLxD7TMBIiWpOaNTakl1zgMPiDNn\nzogLFy5c9hreXPpWe0Whgz9cYiDyJVZNU8DxddVyB6Wqtn8TFAR7cDDOnz+PoKAghIaGIjQ0FCEh\nIWi22XAS7vfrvi44GDn5+Rg2bFjncXU6HUJDQ1Hy5ptoXrcORWfPuj3+ebKMcTk5WLR4sdvHUKJq\nerROh9oTJ7gzEmkCN32ggNMRpnEmE5bY7UgVotsP9AYARZKEFbKMHA9DGFBuSfXayEj86LrrcPLk\nSZw8eRJtbW0ICwtD24ULmBgUhLC2NreOPQTAlLY2rLFYMPxHP4LD4ej8Onv2LE7bbHjRgxAGlNkI\nofMSQ2kpFroxT/D0EgORr/EaMQWk+enpKKuowCGjEaN0OsyTZVjQPuO0oH3mNlqnw0dGI8oqKhTZ\ndUmJrR6tsox56enYvXs3jhw5gtOnT+Nf//oXtm/fjugf/xhxboZwhzvb2nDhzBlERETgpptuQnx8\nPKZNm4ZHH30UY8eM8ZuNEDKyspAny6hz8Xl1APJlGRlZWR6PgchXOCOmgNVRgNPRNOTwJU1DxkVH\nI8+NpiG9UWKrx61CIC81tcv3Bw4ciIEDB6K/LCsSlBPGj8cf33zzsn/78rPP0PzJJx4d39UiqZ4Y\nDAbkmM2Y5MYlhhyzmUVVpCkMYgp4w4cP9+iapbO8vaTq7WriqJgYVJaUIM2DfXqtsoxx0dFuP/9S\nal1iIPI1Lk0TKcibS6pKLX1H9RCUs1NTsRWAuwvLHTP62T+Y0XtCjUsMRL7GqmkihXmratsX1cQz\nk5MR6+aMfrUk4ZDRiA0lJW6M7sqU7EtO5E8YxEResNZiQbYXllS9HZTcCIHI9xjERF5SVVWFgtxc\nbCsvh1GSYLDbO3cvsl7cvSgpMREZWVlOh5cvglKt+7CJ+ioGMZGXKb2k6oug9NaMnoguxyAm0iBf\nBKU3ZvREdDkGMZFG+SooWSRF5F0MYiKNY1ASaRuDmIiISEVs6EFERKQiBjEREZGKGMREREQqYhAT\nERGpiEFMRESkIgYxERGRihjEREREKmIQExERqYhBTEREpCIGMRERkYoYxERERCpiEBMREamIQUxE\nRKQiBjEREZGKGMREREQqYhATERGpiEFMRESkIgYxERGRihjEREREKmIQExERqYhBTEREpCIGMRER\nkYoYxERERCpiEBMREamIQUxERKQiBjEREZGKGMREREQqYhATERGpiEFMRESkIgYxERGRihjERERE\nKmIQExERqYhBTEREpCIGMRERkYoYxERERCpiEBMREamIQUxERKQiBjEREZGKGMREREQqYhATERGp\niEFMRESkIgYxERGRihjEREREKmIQExERqYhBTEREpCIGMRERkYoYxERERCpiEBMREamIQUxERKQi\nBjEREZGKGMREREQq+v/h3/Apcsb4zwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x197c58d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%pylab inline\n",
    "import matplotlib.pyplot as plt\n",
    "import networkx as nx\n",
    "nx.draw(graph,pos=nx.fruchterman_reingold_layout(graph))  # networkx draw()\n",
    "plt.draw()  # pyplot draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## install community module\n",
    " - pip install python-louvain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 361,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import community\n",
    "partition =community.best_partition(graph)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 362,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#print partition\n",
    "from collections import defaultdict\n",
    "# Community Detection\n",
    "v = defaultdict(list)\n",
    "ary = []\n",
    "for key, value in sorted(partition.iteritems()):\n",
    "    ary.append({'title':key, 'cluster':value})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 363,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cluster</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9</td>\n",
       "      <td>Re: [問題] 花旗不能分期？？</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>Re: [問題] 花旗線上繳款問題</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>Re: [問題] 花旗調額</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>Re: [心得] 花旗代繳停車費出問題........</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>[問題] 5月體驗天天花旗日活動回饋</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cluster                        title\n",
       "0        9            Re: [問題] 花旗不能分期？？\n",
       "1        5            Re: [問題] 花旗線上繳款問題\n",
       "2        5                Re: [問題] 花旗調額\n",
       "3        7  Re: [心得] 花旗代繳停車費出問題........\n",
       "4        0           [問題] 5月體驗天天花旗日活動回饋"
      ]
     },
     "execution_count": 363,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "citi = pd.DataFrame(ary)\n",
    "citi.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 376,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cluster</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9</td>\n",
       "      <td>Re: [問題] 花旗不能分期？？</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>9</td>\n",
       "      <td>[問題] booking.com使用花旗訂房問題</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9</td>\n",
       "      <td>[問題] booking.com花旗回饋金問題(旅館刷卡)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>9</td>\n",
       "      <td>[問題] 花旗刷卡後請款的速度</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>9</td>\n",
       "      <td>[問題] 花旗日刷卡金活動分期的問題</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    cluster                          title\n",
       "0         9              Re: [問題] 花旗不能分期？？\n",
       "6         9       [問題] booking.com使用花旗訂房問題\n",
       "7         9  [問題] booking.com花旗回饋金問題(旅館刷卡)\n",
       "19        9                [問題] 花旗刷卡後請款的速度\n",
       "22        9             [問題] 花旗日刷卡金活動分期的問題"
      ]
     },
     "execution_count": 376,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "citi[citi.cluster == 9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}