content/chapter3/ChatGPT使用指南——句词分类.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bbe84b1d",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#ChatGPT使用指南——句词分类\" data-toc-modified-id=\"ChatGPT使用指南——句词分类-1\">ChatGPT使用指南——句词分类</a></span><ul class=\"toc-item\"><li><span><a href=\"#1.-NLU基础\" data-toc-modified-id=\"1.-NLU基础-1.1\">1. NLU基础</a></span><ul class=\"toc-item\"><li><span><a href=\"#1.1-句子级别的分类\" data-toc-modified-id=\"1.1-句子级别的分类-1.1.1\">1.1 句子级别的分类</a></span></li><li><span><a href=\"#1.2-Token级别的分类\" data-toc-modified-id=\"1.2-Token级别的分类-1.1.2\">1.2 Token级别的分类</a></span></li></ul></li><li><span><a href=\"#2.-相关API\" data-toc-modified-id=\"2.-相关API-1.2\">2. 相关API</a></span><ul class=\"toc-item\"><li><span><a href=\"#2.1-ChatGPT-API\" data-toc-modified-id=\"2.1-ChatGPT-API-1.2.1\">2.1 ChatGPT API</a></span></li></ul></li><li><span><a href=\"#3.-NLU应用\" data-toc-modified-id=\"3.-NLU应用-1.3\">3. NLU应用</a></span><ul class=\"toc-item\"><li><span><a href=\"#3.1-文档问答\" data-toc-modified-id=\"3.1-文档问答-1.3.1\">3.1 文档问答</a></span></li><li><span><a href=\"#3.2-分类/实体微调\" data-toc-modified-id=\"3.2-分类/实体微调-1.3.2\">3.2 分类/实体微调</a></span></li><li><span><a href=\"#3.3-智能对话\" data-toc-modified-id=\"3.3-智能对话-1.3.3\">3.3 智能对话</a></span></li></ul></li><li><span><a href=\"#相关文献\" data-toc-modified-id=\"相关文献-1.4\">相关文献</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab6825bd",
   "metadata": {},
   "source": [
    "# ChatGPT使用指南——句词分类"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b6ae6cb",
   "metadata": {},
   "source": [
    "## 1. NLU基础"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0c6d057",
   "metadata": {},
   "source": [
    "&emsp;&emsp;NLU是Natural Language Understanding的简称，即自然语言理解。一直以来都与NLG（Generation）任务并称为NLP两大主流任务。一般意义上的NLU常指与理解给定句子意思相关的意图识别、实体抽取、指代关系等任务，在智能对话中应用比较广泛。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d520324e",
   "metadata": {},
   "source": [
    "&emsp;&emsp;具体点来说，当用户输入一句话时，机器人一般会针对该句话（也可以把历史记录给附加上）进行全方面分析，包括：\n",
    "\n",
    "- 情感倾向分析：简单来说，一般会包括正向、中性、负向三种类型，也可以设计更多的类别。或更复杂的细粒度情感分析，比如针对其中某个实体或属性的情感，而不是整个句子的。\n",
    "- 意图识别：一般都是分类模型，大部分时候都是多分类，但是也有可能是层次分类，或多标签模型。\n",
    "    - 多分类：给定输入文本，输出为一个Label，但Label的总数有多个。比如类型包括询问地址、询问时间、询问价格、闲聊等等。\n",
    "    - 层次分类：给定输入文本，输出为层次的Label，也就是从根节点到最终细粒度类别的路径。比如询问地址/询问家庭地址、询问地址/询问公司地址等等。\n",
    "    - 多标签分类：给定输入文本，输出不定数量的Label，也就是说每个文本可能有多个Label，Label之间是平级关系。\n",
    "- 实体和关系抽取：\n",
    "    - 实体抽取：提取出给定文本中的实体。实体一般指具有特定意义的实词，如人名、地名、作品、品牌等等；很多时候也是业务直接相关的词。\n",
    "    - 关系抽取：实体之间往往有一定的关系，比如「刘亦菲」出演「天龙八部」，其中「刘亦菲」就是人名、「天龙八部」是作品名，其中的关系就是「出演」，一般会和实体作为三元组来表示。\n",
    "\n",
    "&emsp;&emsp;一般经过以上这些分析后，机器人就可以对用户的输入有一个比较清晰的理解，便于接下来据此做出响应。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c029d191",
   "metadata": {},
   "source": [
    "&emsp;&emsp;另外值得一提的是，上面的过程并不一定只用在对话中，只要涉及到用户输入Query需要给出响应的场景，都需要这个NLU的过程，一般也叫Query解析。\n",
    "\n",
    "&emsp;&emsp;上面提到的几个分析，如果从算法的角度看，其实就两种：\n",
    "\n",
    "- 句子级别的分类：如情感分析、意图识别、关系抽取等。也就是给一个句子，给出一个或多个Label。\n",
    "- Token级别的分类：如实体抽取、阅读理解（就是给一段文本和一个问题，然后在文本中找到问题的答案）。也就是给一个句子，给出对应实体的位置。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "445783cf",
   "metadata": {},
   "source": [
    "&emsp;&emsp;Token级别的分类不太好理解，我们举个例子，比如下面这句话：\n",
    "\n",
    "\n",
    "> 刘亦菲出演天龙八部。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4ad97e7",
   "metadata": {},
   "source": [
    "&emsp;&emsp;它在标注的时候是这样的：\n",
    "\n",
    ">刘/B-PER  \n",
    "亦/I-PER  \n",
    "菲/I-PER  \n",
    "出/O  \n",
    "演/O  \n",
    "天/B-WORK  \n",
    "龙/I-WORK  \n",
    "八/I-WORK  \n",
    "部/I-WORK  \n",
    "。/O  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee0c8c55",
   "metadata": {},
   "source": [
    "&emsp;&emsp;在上面这个例子中，每个Token就是每个字，每个Token会对应一个Label（当然也可以多个），Label中的B表示Begin，I表示Internal，O表示Other（也就是非实体）。模型要做的就是学习这种对应关系，当给出新的文本时，能够给出每个Token的Label预测。\n",
    "\n",
    "&emsp;&emsp;可以看到也就是说，它们本质上都是分类任务，只是分类的位置或标准不一样。当然了，实际应用中会有各种不同的变化和设计，但整个思路是差不多的，我们并不需要掌握这些知识，只需要知道大概是怎么一回事就好了。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3853980c",
   "metadata": {},
   "source": [
    "### 1.1 句子级别的分类"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c028680",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来，我们简单介绍一下这些分类具体是怎么做的，先说句子级别的分类。回忆上一章的Embedding，那可以算是整个DeepLearning NLP的基石，我们这部分的内容也会用到Embedding。具体过程如下：\n",
    "\n",
    "- 将给定句子或文本表征成Embedding\n",
    "- 将Embedding传入一个神经网络，计算得到不同Label的概率分布\n",
    "- 将上一步的Label概率分布与真实的分布做比较，并将误差回传，修改神经网络的参数\n",
    "- 得到训练好的神经网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "add819b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.random.seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4297f36",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们举个例子，简单起见，假设Embedding维度为32维（上一章OpenAI返回的维度比较大）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "15c8f3af",
   "metadata": {},
   "outputs": [],
   "source": [
    "emd = np.random.normal(0, 1, (1, 32))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e902d145",
   "metadata": {},
   "source": [
    "&emsp;&emsp;如果我们是三分类，那么最简单的W就是32×3的大小（这个W就被称为模型/参数）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "0ee2dd51",
   "metadata": {},
   "outputs": [],
   "source": [
    "W = np.random.random((32, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "29ee7fe5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6.93930177, 5.96232449, 3.96168115]])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = emd @ W\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc4b1b0",
   "metadata": {},
   "source": [
    "&emsp;&emsp;得到这样的结果，因为我们想要的是概率分布，所以需要对其归一化（也就是变成0-1之间的概率值，并且加起来为1）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "50d82aa4",
   "metadata": {},
   "outputs": [],
   "source": [
    "def norm(z):\n",
    "    exp = np.exp(z)\n",
    "    return exp / np.sum(exp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "65f9b0d7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.70059356, 0.26373654, 0.0356699 ]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = norm(z)\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "8d31f664",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999999999999999"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10cf4953",
   "metadata": {},
   "source": [
    "&emsp;&emsp;根据给出的y，知道这个结果告诉我们预测的Label是0，如果真实的Label是1，那0位置的这个概率分布下次就会变小，1位置的就会变大。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10c261f2",
   "metadata": {},
   "source": [
    "&emsp;&emsp;实际中，W往往更加复杂，可以包含任意的数组，只要最后输出变成1×3的大小即可，比如我们弄个复杂点的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "296d4c25",
   "metadata": {},
   "outputs": [],
   "source": [
    "w1 = np.random.random((32, 100))\n",
    "w2 = np.random.random((100, 32))\n",
    "w3 = np.random.random((32, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "02373e15",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.32940147, 0.34281657, 0.32778196]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = norm(norm(norm(emd @ w1) @ w2) @ w3)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59b7627",
   "metadata": {},
   "source": [
    "&emsp;&emsp;只是参数（模型）更复杂了些，其他都是一样的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "091f1cd3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;稍微复杂点的是多标签分类和层次分类，这俩因为输出的都是多个标签，处理起来要麻烦一些，不过它们的处理方式是类似的。我们以多标签分类来说明，假设有10个标签，给定输入文本，可能是其中任意多个标签。这就意味着我们需要将10个标签的概率分布都表示出来。可以针对每个标签做个二分类，也就是说输出的大小是10×2的，每一行表示「是否是该标签」的概率分布。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "d7ed6710",
   "metadata": {},
   "outputs": [],
   "source": [
    "def norm(z):\n",
    "    axis = -1\n",
    "    exp = np.exp(z)\n",
    "    return exp / np.expand_dims(np.sum(exp, axis=axis), axis)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "id": "da315a25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 1, 2)"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "emd = np.random.normal(0, 1, (1, 32))\n",
    "W = np.random.random((10, 32, 2))\n",
    "y = norm(emd @ W)\n",
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "id": "d79cb3d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0.66293305, 0.33706695]],\n",
       "\n",
       "       [[0.76852603, 0.23147397]],\n",
       "\n",
       "       [[0.59404023, 0.40595977]],\n",
       "\n",
       "       [[0.04682992, 0.95317008]],\n",
       "\n",
       "       [[0.84782999, 0.15217001]],\n",
       "\n",
       "       [[0.01194495, 0.98805505]],\n",
       "\n",
       "       [[0.96779413, 0.03220587]],\n",
       "\n",
       "       [[0.04782398, 0.95217602]],\n",
       "\n",
       "       [[0.41894957, 0.58105043]],\n",
       "\n",
       "       [[0.43668264, 0.56331736]]])"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "febf1595",
   "metadata": {},
   "source": [
    "&emsp;&emsp;这里的输出每一行有两个值，分别表示标签「是/0」和「是/1」的概率，比如第一行，0的概率为0.66，1的概率为0.34。需要注意的是归一化时，我们要指定维度，否则就变成所有的值加起来为1了，这就不对了。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3aa5a208",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面是句子级别分类（Sequence Classification）的逻辑，我们必须再次说明，实际比上面要复杂得多，但基本思路是这样的。我们在LLM时代也并不需要自己去构建模型了，本章后面会讲到如何使用LLM的API进行各类任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a28f22c0",
   "metadata": {},
   "source": [
    "### 1.2 Token级别的分类"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a58b1457",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来看Token级别的分类，有了刚刚的基础，这个看起来就比较容易了。它最大的特点是，Embedding是针对每个Token的。也就是说，如果给定文本长度为10，假定维度依然是32，那Embedding的大小就为：(1, 10, 32)。比刚刚的(1, 32)多了个10。换句话说，这个文本的每一个Token都是一个32维的向量。再通俗一点来说，对于模型来说，无论你是1个Token、2个Token还是100个Token，都可以统一看待--都是固定维度的一个向量表示。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c6b7513",
   "metadata": {},
   "source": [
    "&emsp;&emsp;下面我们假设Label共5个：B-PER，I-PER，B-WORK，I-WORK，O。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "id": "25f60f17",
   "metadata": {},
   "outputs": [],
   "source": [
    "emd = np.random.normal(0, 1, (1, 10, 32))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "id": "2a85ef9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "W = np.random.random((32, 5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "id": "dfea44be",
   "metadata": {},
   "outputs": [],
   "source": [
    "z = emd @ W\n",
    "y = norm(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "id": "7c33459f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 10, 5)"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "id": "fbadb6a4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0.23850186, 0.04651826, 0.12495322, 0.28764271, 0.30238396],\n",
       "        [0.06401011, 0.3422055 , 0.54911626, 0.01179874, 0.03286939],\n",
       "        [0.18309536, 0.62132479, 0.09037235, 0.06016401, 0.04504349],\n",
       "        [0.01570559, 0.0271437 , 0.20159052, 0.12386611, 0.63169408],\n",
       "        [0.1308541 , 0.06810165, 0.61293236, 0.00692553, 0.18118637],\n",
       "        [0.08011671, 0.04648297, 0.00200392, 0.02913598, 0.84226041],\n",
       "        [0.05143706, 0.09635837, 0.00115594, 0.83118412, 0.01986451],\n",
       "        [0.03721064, 0.14529403, 0.03049475, 0.76177941, 0.02522117],\n",
       "        [0.24154874, 0.28648044, 0.11024747, 0.35380566, 0.0079177 ],\n",
       "        [0.10965428, 0.00432547, 0.08823724, 0.00407713, 0.79370588]]])"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50cb184c",
   "metadata": {},
   "source": [
    "&emsp;&emsp;注意看，每一行表示一个Token是某个Label的概率分布（也就是每一行加起来为1），比如第一行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "id": "1f18bb8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.00000001"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum((0.23850186, 0.04651826, 0.12495322, 0.28764271, 0.30238396))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3b453f6",
   "metadata": {},
   "source": [
    "&emsp;&emsp;表示啥呢？表示第一个Token的Label是O，那真实的Label和这个预测的之间就可能有误差，通过误差就可以更新参数，从而使得之后预测时能预测到正确的Label（也就是正确位置的概率最大）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3603d695",
   "metadata": {},
   "source": [
    "&emsp;&emsp;好了，关于NLU常见问题的基本原理我们就介绍到这里了，如果你对其中的很多细节感兴趣，那么可以关注DataWhale的NLP相关教程，从小项目开始一步一步构建自己的知识体系。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "460b3221",
   "metadata": {},
   "source": [
    "## 2. 相关API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d9e87e0",
   "metadata": {},
   "source": [
    "### 2.1 ChatGPT API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e32af1a6",
   "metadata": {},
   "source": [
    "&emsp;&emsp;这里我们介绍openai的ChatGPT接口，利用大模型的In-Context能力进行Zero-Shot或Few-Shot的推理。这里有四个概念需要先稍微解释一下：\n",
    "\n",
    "- GPT：全称是Generative Pretrained Transformer，生成式预训练Transformer。大家只要知道它是一个大模型的名字即可。\n",
    "- In-Context：简单来说就是一种上下文能力，也就是模型只要根据输入的文本就可以自动给出对应的结果，这种能力是大模型在学习了非常多的文本后获得的。可以看作是一种内在的理解能力。\n",
    "- Zero-Shot：直接给模型文本，让它给出你要的标签或输出。\n",
    "- Few-Shot：给模型一些类似的Case（输入+输出），再拼上一个新的没有输出的输入，让模型给出输出。\n",
    "\n",
    "&emsp;&emsp;如果对In-Context更多细节感兴趣的，可以阅读【相关文献1】。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5dae92f",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来，我们就可以用同一个接口，只要通过构造不同的输入就可以完成不同的任务。换句话说，通过使用GPT大模型的In-Context能力，我们只需要输入的时候告诉模型我们的任务就行。\n",
    "\n",
    "&emsp;&emsp;我们看看具体的用法，相比上一章的`Embedding`接口，它的接口参数要复杂多了，重要的参数包括：\n",
    "\n",
    "- model：指定的模型，`gpt-3.5-turbo`就是其中一个模型，大家可以根据自己的需要，参考官方[列表](https://platform.openai.com/docs/models/gpt-3)进行选择，一般需要综合价格和效果进行权衡。\n",
    "\n",
    "- messages：会话消息，支持多轮，多轮就是多条。每一条消息为一个字典，包含「role」和「content」两个字段。如：`[{\"role\": \"user\", \"content\": \"Hello!\"}]`\n",
    "\n",
    "- max_tokens：生成的最大Token数，默认为16。注意这里的Token数不一定是字数（但对中文来说几乎一致）。Prompt+生成的文本，所有的Token长度不能超过模型的上下文长度（一般是2048，新的是4096，具体可以参考上面的链接）。\n",
    "\n",
    "- temperature：温度，默认为1。采样温度，介于0和2之间。较高的值（如0.8）将使输出更加随机，而较低的值（如0.2）将使其更加集中和确定。通常建议调整这个参数或下面的top_p，但不能同时更改两者。\n",
    "\n",
    "- top_p：采样topN分布，默认为1。0.1意味着Next Token只选择前10%概率的。\n",
    "\n",
    "- stop：停止的Token或序列，默认为null，最多4个，如果遇到该Token或序列就停止继续生成。注意生成的结果中不包含stop。\n",
    "\n",
    "- presence_penalty：存在惩罚，默认为0，介于-2.0和2.0之间的数字。正值会根据新Token到目前为止是否出现在文本中来惩罚它们，从而增加模型讨论新主题的可能性。\n",
    "\n",
    "- frequency_penalty：频率惩罚，默认为0，介于-2.0和2.0之间的数字。正值会根据新Token到目前为止在文本中的现有频率来惩罚新Token，降低模型重复生成同一行的可能性。\n",
    "\n",
    "\n",
    "\n",
    "&emsp;&emsp;更多可以参考：[API Reference - OpenAI API](https://platform.openai.com/docs/api-reference/chat)，在大部分情况下，我们只需考虑上面这几个参数即可，甚至大部分时候只需要前两个参数，其他的用默认也行。不过熟悉上面的参数将帮助你更好地使用API。值得再次一提的是，接口支持多轮，而且多轮非常简单，只需要把历史会话加进去就可以了。\n",
    "\n",
    "智谱AI接口可以参考：[智谱AI开放平台](https://open.bigmodel.cn/dev/api#glm-3-turbo)。\n",
    "\n",
    "&emsp;&emsp;关于temperature, top_p, top_k, freqency_penalty等参数的原理可以参考：https://github.com/datawhalechina/hugging-llm/blob/main/resources/231223%E6%97%A9%E6%97%A9%E8%81%8A.pptx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "598ce60e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "client = OpenAI(api_key=\"YOUR OPENAI KEY\")\n",
    "\n",
    "from zhipuai import ZhipuAI\n",
    "zhipu_client = ZhipuAI(api_key=\"YOUR ZHIPU KEY\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bfe6aad1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask(content):\n",
    "    response = client.chat.completions.create(\n",
    "        model=\"gpt-3.5-turbo\", \n",
    "        messages=[{\"role\": \"user\", \"content\": content}]\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f461baa6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask_zhipu(content):\n",
    "    response = zhipu_client.chat.completions.create(\n",
    "        model=\"glm-3-turbo\", \n",
    "        messages=[{\"role\": \"user\", \"content\": content}]\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e82f6cad",
   "metadata": {},
   "source": [
    "&emsp;&emsp;先来个最简单的情感分类的例子，我们分别展示一下Zero-Shot和Few-Shot。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "33bf0f31",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Zero-Shot\n",
    "# 来自openai官方文档\n",
    "prompt=\"\"\"The following is a list of companies and the categories they fall into:\n",
    "\n",
    "Apple, Facebook, Fedex\n",
    "\n",
    "Apple\n",
    "Category:\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c6b4f6bf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Technology/Electronics\n",
      "\n",
      "Facebook\n",
      "Category:\n",
      "Social media/Technology\n",
      "\n",
      "Fedex\n",
      "Category:\n",
      "Logistics/Shipping\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "a9d9f1f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Few-Shot\n",
    "prompt = \"\"\"今天真开心。-->正向\n",
    "心情不太好。-->负向\n",
    "我们是快乐的年轻人。-->\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "72f121f1",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "正向\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96292463",
   "metadata": {},
   "source": [
    "也看一下智谱的答案："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "33111bff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'正向激励和负向激励是团队管理中两种不同的激励方法。正向激励通常通过奖励和表扬来强化人们的行为，以提高他们的积极性和动力。例如，当一个人感到开心和快乐时，这可以被视为一种正向激励，因为这会鼓励他们保持积极的态度和行为。\\n\\n相反，负向激励通常通过惩罚和批评来阻止或减少不良行为。当一个人感到心情不好时，这可能是因为他们经历了负向激励，如批评或失败的经历，这可能会导致他们产生消极的情绪和态度。\\n\\n最后，\"我们是快乐的年轻人\"这句话表达了积极向上的态度和乐观的情感，这可以被视为一种正向激励，因为它鼓励人们保持快乐和积极的生活态度。'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "137991c7",
   "metadata": {},
   "source": [
    "看起来说的有点多。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8491a8bf",
   "metadata": {},
   "source": [
    "&emsp;&emsp;再来个实体识别的例子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e8436957",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Zero-Shot 来自openai官方文档\n",
    "prompt = \"\"\"\n",
    "From the text below, extract the following entities in the following format:\n",
    "Companies: <comma-separated list of companies mentioned>\n",
    "People & titles: <comma-separated list of people mentioned (with their titles or roles appended in parentheses)>\n",
    "\n",
    "Text:\n",
    "In March 1981, United States v. AT&T came to trial under Assistant Attorney General William Baxter. AT&T chairman Charles L. Brown thought the company would be gutted. He realized that AT&T would lose and, in December 1981, resumed negotiations with the Justice Department. Reaching an agreement less than a month later, Brown agreed to divestiture—the best and only realistic alternative. AT&T's decision allowed it to retain its research and manufacturing arms. The decree, titled the Modification of Final Judgment, was an adjustment of the Consent Decree of 14 January 1956. Judge Harold H. Greene was given the authority over the modified decree....\n",
    "\n",
    "In 1982, the U.S. government announced that AT&T would cease to exist as a monopolistic entity. On 1 January 1984, it was split into seven smaller regional companies, Bell South, Bell Atlantic, NYNEX, American Information Technologies, Southwestern Bell, US West, and Pacific Telesis, to handle regional phone services in the U.S. AT&T retains control of its long distance services, but was no longer protected from competition.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a6e58c9c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Companies: AT&T, Bell South, Bell Atlantic, NYNEX, American Information Technologies, Southwestern Bell, US West, Pacific Telesis\n",
      "\n",
      "People & titles: \n",
      "- Assistant Attorney General William Baxter\n",
      "- AT&T chairman Charles L. Brown\n",
      "- Judge Harold H. Greene\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d935682",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面是官方给的一个Zero-Shot的例子，我们来造一个Few-Shot的例子，实体给设置的稍微特殊一些。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "f80e1a88",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Few-Shot\n",
    "prompt = \"\"\"\n",
    "根据下面的示例格式抽取给定Text中的实体，格式为：\n",
    "和弦: <实体用逗号分割>\n",
    "\n",
    "\n",
    "示例Text:\n",
    "增三和弦是大三度+大三度的增五度音，减三和弦是小三度+小三度的减五度音。\n",
    "和弦：增三和弦,减三和弦\n",
    "\n",
    "给定Text:\n",
    "三和弦是由3个按照三度音程关系排列起来的一组音。大三和弦是大三度+小三度的纯五度音，小三和弦是小三度+大三度的纯五度音。\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "b960fa11",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "和弦: 三和弦,大三和弦,小三和弦\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "c2014ca8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'和弦：大三和弦,小三和弦'"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76914494",
   "metadata": {},
   "source": [
    "&emsp;&emsp;Nice，不是么。大家可以尝试如果不给这个例子，它会输出什么。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04fd05b0",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们也可以向它提出明确的要求和指令："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "b145fe04",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt=\"\"\"please output the category of the following companies:\n",
    "Apple, Facebook, Fedex\n",
    "\n",
    "The output format should be:\n",
    "<company>\n",
    "Category:\n",
    "<category>\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "f9298c74",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Apple\n",
      "Category:\n",
      "Technology/Electronics\n",
      "\n",
      "Facebook\n",
      "Category:\n",
      "Technology/Social Media\n",
      "\n",
      "Fedex\n",
      "Category:\n",
      "Logistics/Shipping\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "583eab3c",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们来试一下情感分类的例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "f9e474d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"请给出下面句子的情感倾向，情感倾向包括三种：正向、中性、负向。\n",
    "句子：我们是快乐的年轻人。\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "039a9a4f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "正向\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "696310c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'这个句子的情感倾向是正向。'"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29eac4aa",
   "metadata": {},
   "source": [
    "这次好多了，大家不妨试着让其只输出情感，不要输出其他任何多余的字。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09433e74",
   "metadata": {},
   "source": [
    "&emsp;&emsp;再来做一下实体的例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "16609b04",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "请抽取给定Text中的实体，实体包括Company和People&Title，对于People，请同时给出它们的Title或role，跟在实体后面，用括号括起。\n",
    "\n",
    "Text:\n",
    "In March 1981, United States v. AT&T came to trial under Assistant Attorney General William Baxter. AT&T chairman Charles L. Brown thought the company would be gutted. He realized that AT&T would lose and, in December 1981, resumed negotiations with the Justice Department. Reaching an agreement less than a month later, Brown agreed to divestiture—the best and only realistic alternative. AT&T's decision allowed it to retain its research and manufacturing arms. The decree, titled the Modification of Final Judgment, was an adjustment of the Consent Decree of 14 January 1956. Judge Harold H. Greene was given the authority over the modified decree....\n",
    "\n",
    "In 1982, the U.S. government announced that AT&T would cease to exist as a monopolistic entity. On 1 January 1984, it was split into seven smaller regional companies, Bell South, Bell Atlantic, NYNEX, American Information Technologies, Southwestern Bell, US West, and Pacific Telesis, to handle regional phone services in the U.S. AT&T retains control of its long distance services, but was no longer protected from competition.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "9849c021",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "实体抽取结果如下：\n",
      "\n",
      "1. Company:\n",
      "- AT&T\n",
      "- Bell South\n",
      "- Bell Atlantic\n",
      "- NYNEX\n",
      "- American Information Technologies\n",
      "- Southwestern Bell\n",
      "- US West\n",
      "- Pacific Telesis\n",
      "\n",
      "2. People & Title/Role:\n",
      "- Assistant Attorney General William Baxter\n",
      "- AT&T chairman Charles L. Brown\n",
      "- Judge Harold H. Greene\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fff5d083",
   "metadata": {},
   "source": [
    "&emsp;&emsp;看起来还行。我们最后试一下上面的另一个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "edae5d41",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "根据下面的格式抽取给定Text中的和弦实体，实体必须包括“和弦”两个字。\n",
    "\n",
    "\n",
    "Desired format:\n",
    "和弦：<用逗号隔开>\n",
    "\n",
    "Text:\n",
    "三和弦是由3个按照三度音程关系排列起来的一组音。大三和弦是大三度+小三度的纯五度音，小三和弦是小三度+大三度的纯五度音。\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "5d98f935",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "和弦：三和弦, 大三和弦, 小三和弦\n"
     ]
    }
   ],
   "source": [
    "ans = ask(prompt)\n",
    "print(ans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "53b223ba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'和弦：三和弦，大三和弦，小三和弦'"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5ce0da3",
   "metadata": {},
   "source": [
    "模型能力差不多的时候，描述越清晰，得到的结果越相近。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff402a46",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面的例子中，我们还尝试用了中英文混合，它也完全没问题。\n",
    "\n",
    "&emsp;&emsp;大家不妨多多尝试，也可以参考【相关文献 2-10】中的写法，总的来说，它并没有什么标准答案，最多也是一种习惯或约定。大家可以自由尝试，不用有任何负担。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "810392a1",
   "metadata": {},
   "source": [
    "&emsp;&emsp;实际上，关于这个怎么向“ChatGPT”提问早就有了一个成熟的技术方案：Prompt工程，大家可以进一步阅读【相关文献2】。这里给出一些常见的建议：\n",
    "\n",
    "- 清晰，切忌复杂或歧义，如果有术语，应定义清楚。\n",
    "- 具体，描述语言应尽量具体，不要抽象活模棱两可。\n",
    "- 聚焦，问题避免太泛或开放。\n",
    "- 简洁，避免不必要的描述。\n",
    "- 相关，主要指主题相关，而且是整个对话期间，不要东一瓢西一瓤。\n",
    "\n",
    "&emsp;&emsp;新手容易忽略的地方：\n",
    "\n",
    "- 没有说明具体的输出目标。\n",
    "- 在一次对话中混合多个主题。\n",
    "- 让语言模型做数学题。\n",
    "- 没有给出想要什么的示例样本。\n",
    "- 反向提示。也就是一些反面例子。\n",
    "- 要求他一次只做一件事。可以将步骤捆绑在一起一次说清，不要拆的太碎。\n",
    "\n",
    "\n",
    "一般来说，当任务比较复杂时，我们可以给它一些示例（Few-Shot）；正常情况下，只要直接描述清楚要求即可。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "538158d4",
   "metadata": {},
   "source": [
    "## 3. NLU应用"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e5afdd",
   "metadata": {},
   "source": [
    "### 3.1 文档问答"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f642c65",
   "metadata": {},
   "source": [
    "&emsp;&emsp;文档问答和上一章的QA有点类似，不过要稍微复杂一点。它会先用QA的方法召回一个相关的文档，然后让模型在这个文档中找出问题的答案。一般的流程还是先召回相关文档，然后做阅读理解任务。阅读理解和实体提取任务有些类似，但它预测的不是具体某个标签，而是答案的Index，即start和end的位置。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b56fe55",
   "metadata": {},
   "source": [
    "&emsp;&emsp;还是举个例子。假设我们的问题是：“北京奥运会举办于哪一年？”\n",
    "\n",
    "&emsp;&emsp;召回的文档可能是含有北京奥运会举办的新闻，比如类似下面这样的：\n",
    "\n",
    "> 第29届夏季奥林匹克运动会（Beijing 2008; Games of the XXIX Olympiad），又称2008年北京奥运会，2008年8月8日晚上8时整在中国首都北京开幕。8月24日闭幕。\n",
    "\n",
    "&emsp;&emsp;标注就是「2008年」这个答案的索引。\n",
    "\n",
    "&emsp;&emsp;当然，一个文档里可能有不止一个问题，比如上面的文档，还可以问：“北京奥运会啥时候开幕？”，“北京奥运会什么时候闭幕”，“北京奥运会是第几届奥运会”等问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d466ff8",
   "metadata": {},
   "source": [
    "&emsp;&emsp;根据之前的NLP方法，这里实际做起来方案会比较多，也有一定的复杂度；不过总的来说还是分类任务。现在我们有了LLM，问题就变得简单了。依然是两步：\n",
    "\n",
    "- 召回：与上一章的QA类似，这次召回的是Doc，这一步其实就是相似Embedding选择最相似的。\n",
    "- 回答：将召回来的文档和问题以Prompt的方式提交给Completion/ChatCompletion接口，直接得到答案。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bf4197c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "client = OpenAI(api_key=\"YOUR OPENAI KEY\")\n",
    "\n",
    "from zhipuai import ZhipuAI\n",
    "zhipu_client = ZhipuAI(api_key=\"YOUR ZHIPU KEY\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7ef5f750",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask(content):\n",
    "    response = client.chat.completions.create(\n",
    "        model=\"gpt-3.5-turbo\", \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        max_tokens=300,\n",
    "        top_p=1.0,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans\n",
    "\n",
    "# 注意，智谱AI的接口参数和OpenAI的类似，但有些参数不支持\n",
    "# 具体查看 https://open.bigmodel.cn/dev/api\n",
    "def ask_zhipu(content):\n",
    "    response = zhipu_client.chat.completions.create(\n",
    "        model=\"glm-3-turbo\", \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        max_tokens=300,\n",
    "        top_p=0.9,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c46d07d8",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们分别用中英文各举一例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e3f6c10a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 来自官方文档\n",
    "prompt = \"\"\"Answer the question as truthfully as possible using the provided text, and if the answer is not contained within the text below, say \"I don't know\"\n",
    "\n",
    "Context:\n",
    "The men's high jump event at the 2020 Summer Olympics took place between 30 July and 1 August 2021 at the Olympic Stadium.\n",
    "33 athletes from 24 nations competed; the total possible number depended on how many nations would use universality places \n",
    "to enter athletes in addition to the 32 qualifying through mark or ranking (no universality places were used in 2021).\n",
    "Italian athlete Gianmarco Tamberi along with Qatari athlete Mutaz Essa Barshim emerged as joint winners of the event following\n",
    "a tie between both of them as they cleared 2.37m. Both Tamberi and Barshim agreed to share the gold medal in a rare instance\n",
    "where the athletes of different nations had agreed to share the same medal in the history of Olympics. \n",
    "Barshim in particular was heard to ask a competition official \"Can we have two golds?\" in response to being offered a \n",
    "'jump off'. Maksim Nedasekau of Belarus took bronze. The medals were the first ever in the men's high jump for Italy and \n",
    "Belarus, the first gold in the men's high jump for Italy and Qatar, and the third consecutive medal in the men's high jump\n",
    "for Qatar (all by Barshim). Barshim became only the second man to earn three medals in high jump, joining Patrik Sjöberg\n",
    "of Sweden (1984 to 1992).\n",
    "\n",
    "Q: Who won the 2020 Summer Olympics men's high jump?\n",
    "A:\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "ce704e8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Gianmarco Tamberi and Mutaz Essa Barshim.'"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba815c39",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面的Context就是我们召回的文档。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "871e9753",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"请根据以下Context回答问题，直接输出答案即可，不用附带任何上下文。\n",
    "\n",
    "Context:\n",
    "诺曼人（诺曼人：Nourmands；法语：Normands；拉丁语：Normanni）是在10世纪和11世纪将名字命名为法国诺曼底的人。他们是北欧人的后裔（丹麦人，挪威人和挪威人）的海盗和海盗，他们在首相罗洛（Rollo）的领导下向西弗朗西亚国王查理三世宣誓效忠。经过几代人的同化，并与法兰克和罗马高卢人本地居民融合，他们的后代将逐渐与以西卡罗来纳州为基础的加洛林人文化融合。诺曼人独特的文化和种族身份最初出现于10世纪上半叶，并在随后的几个世纪中持续发展。\n",
    "\n",
    "问题：\n",
    "诺曼底在哪个国家/地区？\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "064c1070",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'法国'"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "4c084e80",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'法国'"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99c79a06",
   "metadata": {},
   "source": [
    "&emsp;&emsp;看起来还行，我们接下来就把整个流程串起来。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c21827f1",
   "metadata": {},
   "source": [
    "&emsp;&emsp;首先是加载数据集，取自：[openai-cookbook/olympics-1-collect-data.ipynb at 1f6c2304b401e931928e74e978d9a0b8a40d1cf7 · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/1f6c2304b401e931928e74e978d9a0b8a40d1cf7/examples/fine-tuned_qa/olympics-1-collect-data.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8651069a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3964, 4)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(\"./dataset/olympics_sections_text.csv\")\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6dd6fbe3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>heading</th>\n",
       "      <th>content</th>\n",
       "      <th>tokens</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020 Summer Olympics</td>\n",
       "      <td>Summary</td>\n",
       "      <td>The 2020 Summer Olympics (Japanese: 2020年夏季オリン...</td>\n",
       "      <td>726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020 Summer Olympics</td>\n",
       "      <td>Host city selection</td>\n",
       "      <td>The International Olympic Committee (IOC) vote...</td>\n",
       "      <td>126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020 Summer Olympics</td>\n",
       "      <td>Impact of the COVID-19 pandemic</td>\n",
       "      <td>In January 2020, concerns were raised about th...</td>\n",
       "      <td>374</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020 Summer Olympics</td>\n",
       "      <td>Qualifying event cancellation and postponement</td>\n",
       "      <td>Concerns about the pandemic began to affect qu...</td>\n",
       "      <td>298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020 Summer Olympics</td>\n",
       "      <td>Effect on doping tests</td>\n",
       "      <td>Mandatory doping tests were being severely res...</td>\n",
       "      <td>163</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  title                                         heading  \\\n",
       "0  2020 Summer Olympics                                         Summary   \n",
       "1  2020 Summer Olympics                             Host city selection   \n",
       "2  2020 Summer Olympics                 Impact of the COVID-19 pandemic   \n",
       "3  2020 Summer Olympics  Qualifying event cancellation and postponement   \n",
       "4  2020 Summer Olympics                          Effect on doping tests   \n",
       "\n",
       "                                             content  tokens  \n",
       "0  The 2020 Summer Olympics (Japanese: 2020年夏季オリン...     726  \n",
       "1  The International Olympic Committee (IOC) vote...     126  \n",
       "2  In January 2020, concerns were raised about th...     374  \n",
       "3  Concerns about the pandemic began to affect qu...     298  \n",
       "4  Mandatory doping tests were being severely res...     163  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b51703fa",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们这次不用Redis，换一个工具：[Qdrant - Vector Search Engine](https://qdrant.tech/)，Qdrant相比Redis的单线程更容易扩展。但我们切记，要根据实际情况选择工具，很多时候过度优化是原罪，适合的就是最好的。我们真正需要做的是将业务逻辑抽象，做到尽量不依赖任何工具，换工具只需要换一个适配器就好。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67e7bda3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;依然使用Docker，启动很简单：\n",
    "\n",
    "```shell\n",
    "docker run -p 6333:6333 -v $(pwd)/qdrant_storage:/qdrant/storage qdrant/qdrant`\n",
    "```\n",
    "\n",
    "&emsp;&emsp;自然也少不了客户端的安装：\n",
    "\n",
    "```shell\n",
    "pip install qdrant-client\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c80d4072",
   "metadata": {},
   "source": [
    "&emsp;&emsp;不过首先还是生成Embedding，这一步我们用Batch方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "00b78eef",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_embedding(inputs):\n",
    "    embed_model = \"text-embedding-ada-002\"\n",
    "    res = client.embeddings.create(\n",
    "        input=inputs, model=embed_model\n",
    "    )\n",
    "    return res.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bf114227",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_embedding_zhipu(inputs):\n",
    "    if isinstance(inputs, str):\n",
    "        inputs = [inputs]\n",
    "    res = []\n",
    "    for s in inputs:\n",
    "        resp = zhipu_client.embeddings.create(\n",
    "            input=s, model=\"embedding-2\"\n",
    "        )\n",
    "        emd = resp.data[0]\n",
    "        res.append(emd)\n",
    "    return res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "id": "5af2b85b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3964"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texts = [v.content for v in df.itertuples()]\n",
    "len(texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "id": "32d7fd6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pnlp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "id": "d67bfeda",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "batch: 0 done\n",
      "batch: 1 done\n",
      "batch: 2 done\n",
      "batch: 3 done\n",
      "batch: 4 done\n",
      "batch: 5 done\n",
      "batch: 6 done\n",
      "batch: 7 done\n",
      "batch: 8 done\n",
      "batch: 9 done\n",
      "batch: 10 done\n",
      "batch: 11 done\n",
      "batch: 12 done\n",
      "batch: 13 done\n",
      "batch: 14 done\n",
      "batch: 15 done\n",
      "batch: 16 done\n",
      "batch: 17 done\n",
      "batch: 18 done\n",
      "batch: 19 done\n"
     ]
    }
   ],
   "source": [
    "emds = []\n",
    "for idx, batch in enumerate(pnlp.generate_batches_by_size(texts, 200)):\n",
    "    emd_data = get_embedding(batch)\n",
    "    for v in emd_data:\n",
    "        emds.append(v.embedding)\n",
    "    print(f\"batch: {idx} done\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "id": "3f81a61a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3964, 1536)"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(emds), len(emds[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f201101",
   "metadata": {},
   "source": [
    "由于量比较大，我们把emds存储了一下，大家可以直接加载使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "91e3e158",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4888ef32",
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.load(\"emds.npz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "779c208e",
   "metadata": {},
   "outputs": [],
   "source": [
    "emds = arr[\"arr\"].tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0af3aead",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3964, 1536)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(emds), len(emds[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43d03232",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来是创建索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "96276b2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from qdrant_client import QdrantClient\n",
    "qc_client = QdrantClient(host=\"localhost\", port=6333)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff9e241b",
   "metadata": {},
   "source": [
    "&emsp;&emsp;值得注意的是，qdrant还支持内存/文件库，也就是说，可以直接："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "id": "c5a7ffec",
   "metadata": {},
   "outputs": [],
   "source": [
    "# qc_client = QdrantClient(\":memory:\")\n",
    "# 或\n",
    "# qc_client = QdrantClient(path=\"path/to/db\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aef2c7b9",
   "metadata": {},
   "source": [
    "&emsp;&emsp;我们还是用server的方式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "86c1a55a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/xt/mpcnl_9151dbq43hptdlv2jr0000gn/T/ipykernel_73899/2165250407.py:3: DeprecationWarning: `recreate_collection` method is deprecated and will be removed in the future. Use `collection_exists` to check collection existence and `create_collection` instead.\n",
      "  qc_client.recreate_collection(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from qdrant_client.models import Distance, VectorParams\n",
    "\n",
    "qc_client.recreate_collection(\n",
    "    collection_name=\"doc_qa\",\n",
    "    vectors_config=VectorParams(size=len(emds[0]), distance=Distance.COSINE),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "fc944214",
   "metadata": {},
   "outputs": [],
   "source": [
    "# qc_client.delete_collection(\"doc_qa\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2feefbc",
   "metadata": {},
   "source": [
    "&emsp;&emsp;然后是把向量入库："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f1051a9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "payload=[\n",
    "    {\"content\": v.content, \"heading\": v.heading, \"title\": v.title, \"tokens\": v.tokens} for v in df.itertuples()\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "41d11d8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "qc_client.upload_collection(\n",
    "    collection_name=\"doc_qa\",\n",
    "    vectors=emds,\n",
    "    payload=payload\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbad1d89",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来进行查询："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "add5110b",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Who won the 2020 Summer Olympics men's high jump?\"\n",
    "query_vector = get_embedding(query)[0].embedding\n",
    "hits = qc_client.search(\n",
    "    collection_name=\"doc_qa\",\n",
    "    query_vector=query_vector,\n",
    "    limit=5\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a502ddee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[ScoredPoint(id='f40dc5e1-3291-443a-b802-91e1998f3c3f', version=3, score=0.90292776, payload={'content': 'The men\\'s high jump event at the 2020 Summer Olympics took place between 30 July and 1 August 2021 at the Olympic Stadium. 33 athletes from 24 nations competed; the total possible number depended on how many nations would use universality places to enter athletes in addition to the 32 qualifying through mark or ranking (no universality places were used in 2021). Italian athlete Gianmarco Tamberi along with Qatari athlete Mutaz Essa Barshim emerged as joint winners of the event following a tie between both of them as they cleared 2.37m. Both Tamberi and Barshim agreed to share the gold medal in a rare instance where the athletes of different nations had agreed to share the same medal in the history of Olympics. Barshim in particular was heard to ask a competition official \"Can we have two golds?\" in response to being offered a \\'jump off\\'. Maksim Nedasekau of Belarus took bronze. The medals were the first ever in the men\\'s high jump for Italy and Belarus, the first gold in the men\\'s high jump for Italy and Qatar, and the third consecutive medal in the men\\'s high jump for Qatar (all by Barshim). Barshim became only the second man to earn three medals in high jump, joining Patrik Sjöberg of Sweden (1984 to 1992).', 'heading': 'Summary', 'title': \"Athletics at the 2020 Summer Olympics – Men's high jump\", 'tokens': 275}, vector=None, shard_key=None),\n",
       " ScoredPoint(id='4c97dca4-2f99-4b78-a034-1a3bdfe66bab', version=4, score=0.88223696, payload={'content': \"The men's long jump event at the 2020 Summer Olympics took place between 31 July and 2 August 2021 at the Japan National Stadium. Approximately 35 athletes were expected to compete; the exact number was dependent on how many nations use universality places to enter athletes in addition to the 32 qualifying through time or ranking (1 universality place was used in 2016). 31 athletes from 20 nations competed. Miltiadis Tentoglou won the gold medal, Greece's first medal in the men's long jump. Cuban athletes Juan Miguel Echevarría and Maykel Massó earned silver and bronze, respectively, the nation's first medals in the event since 2008.\", 'heading': 'Summary', 'title': \"Athletics at the 2020 Summer Olympics – Men's long jump\", 'tokens': 136}, vector=None, shard_key=None),\n",
       " ScoredPoint(id='dbe9e9b6-e2b8-491d-a389-390ac0743077', version=4, score=0.8820208, payload={'content': \"The men's pole vault event at the 2020 Summer Olympics took place between 31 July and 3 August 2021 at the Japan National Stadium. 29 athletes from 18 nations competed. Armand Duplantis of Sweden won gold, with Christopher Nilsen of the United States earning silver and Thiago Braz of Brazil taking bronze. It was Sweden's first victory in the event and first medal of any color in the men's pole vault since 1952. Braz, who had won in 2016, became the ninth man to earn multiple medals in the pole vault.\", 'heading': 'Summary', 'title': \"Athletics at the 2020 Summer Olympics – Men's pole vault\", 'tokens': 112}, vector=None, shard_key=None),\n",
       " ScoredPoint(id='6cfd284d-adf3-4f82-8089-90cd76c1b8af', version=3, score=0.8760853, payload={'content': \"The men's triple jump event at the 2020 Summer Olympics took place between 3 and 5 August 2021 at the Japan National Stadium. Approximately 35 athletes were expected to compete; the exact number was dependent on how many nations use universality places to enter athletes in addition to the 32 qualifying through time or ranking (2 universality places were used in 2016). 32 athletes from 19 nations competed. Pedro Pichardo of Portugal won the gold medal, the nation's second victory in the men's triple jump (after Nelson Évora in 2008). China's Zhu Yaming took silver, while Hugues Fabrice Zango earned Burkina Faso's first Olympic medal in any event.\", 'heading': 'Summary', 'title': \"Athletics at the 2020 Summer Olympics – Men's triple jump\", 'tokens': 139}, vector=None, shard_key=None),\n",
       " ScoredPoint(id='5083441b-bf20-4efb-8338-7fb0a6cef789', version=3, score=0.86042714, payload={'content': \"The men's 110 metres hurdles event at the 2020 Summer Olympics took place between 3 and 5 August 2021 at the Olympic Stadium. Approximately forty athletes were expected to compete; the exact number was dependent on how many nations used universality places to enter athletes in addition to the 40 qualifying through time or ranking (1 universality place was used in 2016). 40 athletes from 29 nations competed. Hansle Parchment of Jamaica won the gold medal, the nation's second consecutive victory in the event. His countryman Ronald Levy took bronze. American Grant Holloway earned silver, placing the United States back on the podium in the event after the nation missed the medals for the first time in Rio 2016 (excluding the boycotted 1980 Games).\", 'heading': 'Summary', 'title': \"Athletics at the 2020 Summer Olympics – Men's 110 metres hurdles\", 'tokens': 149}, vector=None, shard_key=None)]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hits"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f02bce6",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来将这个过程包装在Prompt生成过程中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "id": "6daeec5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "MAX_SECTION_LEN = 500\n",
    "SEPARATOR = \"\\n* \"\n",
    "separator_len = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "id": "fe232d50",
   "metadata": {},
   "outputs": [],
   "source": [
    "def construct_prompt(question: str):\n",
    "    query_vector = get_embedding(question)[0].embedding\n",
    "    hits = qc_client.search(\n",
    "        collection_name=\"doc_qa\",\n",
    "        query_vector=query_vector,\n",
    "        limit=5\n",
    "    )\n",
    "    \n",
    "    choose = []\n",
    "    length = 0\n",
    "    indexes = []\n",
    "     \n",
    "    for hit in hits:\n",
    "        doc = hit.payload\n",
    "        length += doc[\"tokens\"] + separator_len\n",
    "        if length > MAX_SECTION_LEN:\n",
    "            break\n",
    "            \n",
    "        choose.append(SEPARATOR + doc[\"content\"].replace(\"\\n\", \" \"))\n",
    "        indexes.append(doc[\"title\"] + doc[\"heading\"])\n",
    "            \n",
    "    # Useful diagnostic information\n",
    "    print(f\"Selected {len(choose)} document sections:\")\n",
    "    print(\"\\n\".join(indexes))\n",
    "    \n",
    "    header = \"\"\"Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say \"I don't know.\"\\n\\nContext:\\n\"\"\"\n",
    "    \n",
    "    return header + \"\".join(choose) + \"\\n\\n Q: \" + question + \"\\n A:\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "id": "4c3e2cf1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected 2 document sections:\n",
      "Athletics at the 2020 Summer Olympics – Men's high jumpSummary\n",
      "Athletics at the 2020 Summer Olympics – Men's long jumpSummary\n",
      "===\n",
      " Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say \"I don't know.\"\n",
      "\n",
      "Context:\n",
      "\n",
      "* The men's high jump event at the 2020 Summer Olympics took place between 30 July and 1 August 2021 at the Olympic Stadium. 33 athletes from 24 nations competed; the total possible number depended on how many nations would use universality places to enter athletes in addition to the 32 qualifying through mark or ranking (no universality places were used in 2021). Italian athlete Gianmarco Tamberi along with Qatari athlete Mutaz Essa Barshim emerged as joint winners of the event following a tie between both of them as they cleared 2.37m. Both Tamberi and Barshim agreed to share the gold medal in a rare instance where the athletes of different nations had agreed to share the same medal in the history of Olympics. Barshim in particular was heard to ask a competition official \"Can we have two golds?\" in response to being offered a 'jump off'. Maksim Nedasekau of Belarus took bronze. The medals were the first ever in the men's high jump for Italy and Belarus, the first gold in the men's high jump for Italy and Qatar, and the third consecutive medal in the men's high jump for Qatar (all by Barshim). Barshim became only the second man to earn three medals in high jump, joining Patrik Sjöberg of Sweden (1984 to 1992).\n",
      "* The men's long jump event at the 2020 Summer Olympics took place between 31 July and 2 August 2021 at the Japan National Stadium. Approximately 35 athletes were expected to compete; the exact number was dependent on how many nations use universality places to enter athletes in addition to the 32 qualifying through time or ranking (1 universality place was used in 2016). 31 athletes from 20 nations competed. Miltiadis Tentoglou won the gold medal, Greece's first medal in the men's long jump. Cuban athletes Juan Miguel Echevarría and Maykel Massó earned silver and bronze, respectively, the nation's first medals in the event since 2008.\n",
      "\n",
      " Q: Who won the 2020 Summer Olympics men's high jump?\n",
      " A:\n"
     ]
    }
   ],
   "source": [
    "prompt = construct_prompt(\"Who won the 2020 Summer Olympics men's high jump?\")\n",
    "\n",
    "print(\"===\\n\", prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "id": "89645f61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Italian athlete Gianmarco Tamberi and Qatari athlete Mutaz Essa Barshim emerged as joint winners of the men's high jump event at the 2020 Summer Olympics.\""
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "id": "ea347480",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Gianmarco Tamberi of Italy and Mutaz Essa Barshim of Qatar won the 2020 Summer Olympics men's high jump event, sharing the gold medal.\""
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d72d5e5",
   "metadata": {},
   "source": [
    "&emsp;&emsp;再看几个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "c98b779f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected 1 document sections:\n",
      "Concerns and controversies at the 2020 Summer OlympicsSummary\n",
      "\n",
      "Q: Why was the 2020 Summer Olympics originally postponed?\n",
      "A: The 2020 Summer Olympics were originally postponed due to the COVID-19 pandemic.\n"
     ]
    }
   ],
   "source": [
    "query = \"Why was the 2020 Summer Olympics originally postponed?\"\n",
    "prompt = construct_prompt(query)\n",
    "answer = ask(prompt)\n",
    "\n",
    "print(f\"\\nQ: {query}\\nA: {answer}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "id": "9331800f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected 2 document sections:\n",
      "2020 Summer Olympics medal tableSummary\n",
      "List of 2020 Summer Olympics medal winnersSummary\n",
      "\n",
      "Q: In the 2020 Summer Olympics, how many gold medals did the country which won the most medals win?\n",
      "A: The country which won the most medals overall in the 2020 Summer Olympics was the United States. They won a total of 39 gold medals.\n"
     ]
    }
   ],
   "source": [
    "query = \"In the 2020 Summer Olympics, how many gold medals did the country which won the most medals win?\"\n",
    "prompt = construct_prompt(query)\n",
    "answer = ask_zhipu(prompt)\n",
    "\n",
    "print(f\"\\nQ: {query}\\nA: {answer}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "id": "eb3959f7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Selected 3 document sections:\n",
      "Sport climbing at the 2020 Summer Olympics – Men's combinedRoute-setting\n",
      "Ski mountaineering at the 2020 Winter Youth Olympics – Boys' individualSummary\n",
      "Ski mountaineering at the 2020 Winter Youth Olympics – Girls' individualSummary\n",
      "\n",
      "Q: What is the tallest mountain in the world?\n",
      "A: I don't know.\n"
     ]
    }
   ],
   "source": [
    "query = \"What is the tallest mountain in the world?\"\n",
    "prompt = construct_prompt(query)\n",
    "answer = ask(prompt)\n",
    "\n",
    "print(f\"\\nQ: {query}\\nA: {answer}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "68791933",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Q: What is the tallest mountain in the world?\n",
      "A: I don't know.\n"
     ]
    }
   ],
   "source": [
    "# ChatGPT\n",
    "answer = ask_zhipu(prompt)\n",
    "\n",
    "print(f\"\\nQ: {query}\\nA: {answer}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ad4f716",
   "metadata": {},
   "source": [
    "&emsp;&emsp;Very Nice, isn't that!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89bf9747",
   "metadata": {},
   "source": [
    "### 3.2 分类/实体微调"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "530fc7aa",
   "metadata": {},
   "source": [
    "&emsp;&emsp;在前面\"相关API\"一节，我们已经介绍了各种分类和实体提取的用法。这里会给大家介绍更具体、常见的任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90b57987",
   "metadata": {},
   "source": [
    "&emsp;&emsp;首先是主题分类，简单来说就是给定文本，判断属于哪一类主题。\n",
    "\n",
    "&emsp;&emsp;我们找一个新闻主题分类的数据集看看，数据集取自：[CLUEbenchmark/CLUE](https://github.com/CLUEbenchmark/CLUE)，共15个类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1fd28186",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10000"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pnlp\n",
    "lines = pnlp.read_file_to_list_dict(\"./dataset/tnews.json\")\n",
    "len(lines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "12e99f87",
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "ct = Counter([v[\"label_desc\"] for v in lines])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ab1103e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('news_tech', 1089),\n",
       " ('news_finance', 956),\n",
       " ('news_entertainment', 910),\n",
       " ('news_world', 905),\n",
       " ('news_car', 791),\n",
       " ('news_sports', 767),\n",
       " ('news_culture', 736),\n",
       " ('news_military', 716),\n",
       " ('news_travel', 693),\n",
       " ('news_game', 659),\n",
       " ('news_edu', 646),\n",
       " ('news_agriculture', 494),\n",
       " ('news_house', 378),\n",
       " ('news_story', 215),\n",
       " ('news_stock', 45)]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ct.most_common()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79154787",
   "metadata": {},
   "source": [
    "&emsp;&emsp;`stock`这个类别太少了，我们先给它去掉："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c37bb67c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9955"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines = [v for v in lines if v[\"label_desc\"] != \"news_stock\"]\n",
    "len(lines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9aa552bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_prompt(text):\n",
    "    prompt = f\"\"\"对给定文本进行分类，类别包括：科技、金融、娱乐、世界、汽车、运动、文化、军事、旅游、游戏、教育、农业、房产、社会。\n",
    "\n",
    "给定文本：\n",
    "{text}\n",
    "类别：\n",
    "\"\"\"\n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4facd64a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'label': '101',\n",
       " 'label_desc': 'news_culture',\n",
       " 'sentence': '上联：银笛吹开云天月，下联怎么对？',\n",
       " 'keywords': ''}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines[59]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4df0614f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "对给定文本进行分类，类别包括：科技、金融、娱乐、世界、汽车、运动、文化、军事、旅游、游戏、教育、农业、房产、社会。\n",
      "\n",
      "给定文本：\n",
      "上联：银笛吹开云天月，下联怎么对？\n",
      "类别：\n",
      "\n"
     ]
    }
   ],
   "source": [
    "prompt = get_prompt(lines[59][\"sentence\"])\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f9a7cff",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "client = OpenAI(api_key=\"YOUR OPENAI KEY\")\n",
    "\n",
    "from zhipuai import ZhipuAI\n",
    "zhipu_client = ZhipuAI(api_key=\"YOUR ZHIPU KEY\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "af632a1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask(content, model=\"gpt-3.5-turbo\"):\n",
    "    response = client.chat.completions.create(\n",
    "        model=model, \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        temperature=0,\n",
    "        max_tokens=300,\n",
    "        top_p=1.0,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans\n",
    "\n",
    "def ask_zhipu(content, model=\"glm-3-turbo\"):\n",
    "    response = zhipu_client.chat.completions.create(\n",
    "        model=model, \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        temperature=0.1,\n",
    "        max_tokens=300,\n",
    "        top_p=0.9,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "id": "884ccc2a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'文化'"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "id": "14fa7905",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'给定文本“上联：银笛吹开云天月，下联怎么对？”属于文化类别。这是因为文本涉及对联的创作，对联是中国传统文化中的一种文学形式，体现了中国的语言艺术和文学修养。'"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f464c09",
   "metadata": {},
   "source": [
    "&emsp;&emsp;再试几个其他类别的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "id": "79213ecc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'label': '110',\n",
       " 'label_desc': 'news_military',\n",
       " 'sentence': '以色列大规模空袭开始！伊朗多个军事目标遭遇打击，誓言对等反击',\n",
       " 'keywords': '伊朗,圣城军,叙利亚,以色列国防军,以色列'}"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "id": "19385d92",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "对给定文本进行分类，类别包括：科技、金融、娱乐、世界、汽车、运动、文化、军事、旅游、游戏、教育、农业、房产、社会。\n",
      "\n",
      "给定文本：\n",
      "以色列大规模空袭开始！伊朗多个军事目标遭遇打击，誓言对等反击\n",
      "类别：\n",
      "\n"
     ]
    }
   ],
   "source": [
    "prompt = get_prompt(lines[1][\"sentence\"])\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "id": "79f4e61b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'军事'"
      ]
     },
     "execution_count": 214,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "id": "88a6b426",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'label': '104',\n",
       " 'label_desc': 'news_finance',\n",
       " 'sentence': '出栏一头猪亏损300元，究竟谁能笑到最后！',\n",
       " 'keywords': '商品猪,养猪,猪价,仔猪,饲料'}"
      ]
     },
     "execution_count": 235,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "id": "bb28db73",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "对给定文本进行分类，类别包括：科技、金融、娱乐、世界、汽车、运动、文化、军事、旅游、游戏、教育、农业、房产、社会。\n",
      "\n",
      "给定文本：\n",
      "出栏一头猪亏损300元，究竟谁能笑到最后！\n",
      "类别：\n",
      "\n"
     ]
    }
   ],
   "source": [
    "prompt = get_prompt(lines[2][\"sentence\"])\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "id": "9e88297f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'给定文本“出栏一头猪亏损300元，究竟谁能笑到最后！”可以被归类为“农业”类别。这是因为文本内容涉及农业生产和养殖业的情况，亏损300元反映的是农业领域的经济现象，而“谁能笑到最后”可能指的是在当前情况下，养殖业者如何通过各种方式转亏为盈，保持竞争力。'"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ask_zhipu(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80bb9d4f",
   "metadata": {},
   "source": [
    "&emsp;&emsp;这个有点迷糊，不过「农业」这个类别感觉也没问题。当然，遇到有错误的情况，我们起码还有两种手段来解决：\n",
    "\n",
    "- Few-Shot，可以每次随机从数据集里抽几条出来作为Prompt的一部分。\n",
    "- Fine-Tuning，把我们自己的数据集按指定格式准备好，提交给API，让它帮我们微调一个属于我们自己的模型，它在我们自己的数据集上学习过。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8568b38f",
   "metadata": {},
   "source": [
    "&emsp;&emsp;Few-Shot最关键的是如何找到这个「Few」，换句话说，我们拿什么Case给模型当做参考样本。对于类别比较多的多分类（实际工作中，成百上千中Label是很常见的），Few-Shot即使每个Label一个例子，这上下文长度也不得了。不太现实。这时候其实Few-Shot有点不太方便了。当然，如果我们非要用也不是不行，还是最常用的策略：先召回几个相似句，然后把相似句的内容和类别作为Few-Shot的例子，让接口来预测给定句子的类别。\n",
    "\n",
    "&emsp;&emsp;不过，我们还可以使用微调（Fine-Tuning）方法在自己的数据集上对模型进行微调，简单来说就是让模型「熟悉」我们独特的数据，进而让其具备在类似数据集上识别出类别的能力。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35dfeea5",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来，就让我们看看具体怎么做，一般包括三个主要步骤：\n",
    "\n",
    "- 准备数据：按接口要求的格式把数据准备好，这里的数据就是我们自己的数据集，至少包含一段文本和一个类别。\n",
    "- 微调：使用微调接口将刚刚的数据传递过去，由服务器自动完成微调，微调完成后可以得到一个新的model_id。注意，这个model_id只属于你自己，不要将它公开给其他人。\n",
    "- 使用新的模型进行推理：嗯，这个很简单，把原来接口里的`model`参数内容换成我们的model_id即可。\n",
    "\n",
    "&emsp;&emsp;咱们接下来就来调调这个多分类模型，我们只取后100条作为训练集（为了快速和省钱……）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "1a91a171",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_lines = lines[-100:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "08cec044",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "f729c32e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 4)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.DataFrame(train_lines)\n",
    "train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "a12c3fc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>label_desc</th>\n",
       "      <th>sentence</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>109</td>\n",
       "      <td>news_tech</td>\n",
       "      <td>直播预告：2018创新中国春季峰会丨5.9-10 北京</td>\n",
       "      <td>浏览器,凤岐融媒体,菜单栏,人工智能,凤岐茶社</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>104</td>\n",
       "      <td>news_finance</td>\n",
       "      <td>5.8黄金反弹受阻下行，晚间原油白银恒指分析策略！</td>\n",
       "      <td>天然气,回落做多,行情分析布局,原油,布林带</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>115</td>\n",
       "      <td>news_agriculture</td>\n",
       "      <td>国际糖业半年报之巴西篇</td>\n",
       "      <td>食糖,巴西,南部地区,甘蔗,圣保罗州</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>103</td>\n",
       "      <td>news_sports</td>\n",
       "      <td>运动完之后能立马喝凉水吗？</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>112</td>\n",
       "      <td>news_travel</td>\n",
       "      <td>耐高温吨袋使用注意事项有哪些？</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label        label_desc                     sentence  \\\n",
       "0   109         news_tech  直播预告：2018创新中国春季峰会丨5.9-10 北京   \n",
       "1   104      news_finance    5.8黄金反弹受阻下行，晚间原油白银恒指分析策略！   \n",
       "2   115  news_agriculture                  国际糖业半年报之巴西篇   \n",
       "3   103       news_sports                运动完之后能立马喝凉水吗？   \n",
       "4   112       news_travel              耐高温吨袋使用注意事项有哪些？   \n",
       "\n",
       "                  keywords  \n",
       "0  浏览器,凤岐融媒体,菜单栏,人工智能,凤岐茶社  \n",
       "1   天然气,回落做多,行情分析布局,原油,布林带  \n",
       "2       食糖,巴西,南部地区,甘蔗,圣保罗州  \n",
       "3                           \n",
       "4                           "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "d8712acc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "label_desc\n",
       "news_world            12\n",
       "news_tech             11\n",
       "news_finance          11\n",
       "news_game             11\n",
       "news_sports           10\n",
       "news_travel            9\n",
       "news_edu               7\n",
       "news_entertainment     7\n",
       "news_agriculture       5\n",
       "news_military          5\n",
       "news_house             5\n",
       "news_car               3\n",
       "news_story             2\n",
       "news_culture           2\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.label_desc.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "deb66882",
   "metadata": {},
   "source": [
    "**Step1：准备数据**\n",
    "\n",
    "&emsp;&emsp;要保证有两列为：role和content，支持多轮。具体可以进一步参考：[Fine-tuning - OpenAI API](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "e93886ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>label_desc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>直播预告：2018创新中国春季峰会丨5.9-10 北京</td>\n",
       "      <td>news_tech</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.8黄金反弹受阻下行，晚间原油白银恒指分析策略！</td>\n",
       "      <td>news_finance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>国际糖业半年报之巴西篇</td>\n",
       "      <td>news_agriculture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>运动完之后能立马喝凉水吗？</td>\n",
       "      <td>news_sports</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>耐高温吨袋使用注意事项有哪些？</td>\n",
       "      <td>news_travel</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      sentence        label_desc\n",
       "0  直播预告：2018创新中国春季峰会丨5.9-10 北京         news_tech\n",
       "1    5.8黄金反弹受阻下行，晚间原油白银恒指分析策略！      news_finance\n",
       "2                  国际糖业半年报之巴西篇  news_agriculture\n",
       "3                运动完之后能立马喝凉水吗？       news_sports\n",
       "4              耐高温吨袋使用注意事项有哪些？       news_travel"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train = train[[\"sentence\", \"label_desc\"]]\n",
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "1c974c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "desc_map = {\n",
    "    \"news_tech\": \"科技\",\n",
    "    \"news_finance\": \"金融\",\n",
    "    \"news_entertainment\": \"娱乐\",\n",
    "    \"news_game\": \"游戏\",\n",
    "    \"news_travel\": \"旅游\",\n",
    "    \"news_sports\": \"运动\",\n",
    "    \"news_military\": \"军事\",\n",
    "    \"news_world\": \"世界\",\n",
    "    \"news_car\": \"汽车\",\n",
    "    \"news_culture\": \"文化\",\n",
    "    \"news_edu\": \"教育\",\n",
    "    \"news_agriculture\": \"农业\",\n",
    "    \"news_house\": \"房产\",\n",
    "    \"news_story\": \"社会\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "bfa8685b",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_lines = []\n",
    "for v in df_train.itertuples():\n",
    "    msg = {\n",
    "        \"messages\": [\n",
    "            {\"role\": \"user\", \"content\": get_prompt(v.sentence)},\n",
    "            {\"role\": \"assistant\", \"content\": desc_map[v.label_desc]},\n",
    "        ]\n",
    "    }\n",
    "    data_lines.append(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "4af83fe0",
   "metadata": {},
   "outputs": [],
   "source": [
    "out_fn = \"./dataset/tnews-finetuning.jsonl\"\n",
    "pnlp.write_list_dict_to_file(out_fn, data_lines)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb2bba5d",
   "metadata": {},
   "source": [
    "接下来可以检查数据并统计Token，可参考文档。如果你对自己的数据比较熟悉，不做这些也是可以的。其实就是对数据做一下基本统计，做到心中有数。\n",
    "\n",
    "- https://cookbook.openai.com/examples/chat_finetuning_data_prep\n",
    "\n",
    "> 以下代码均来自文档。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ea0f3c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "df90a52f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import tiktoken # for token counting\n",
    "import numpy as np\n",
    "from collections import defaultdict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "10f3fa6b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No errors found\n"
     ]
    }
   ],
   "source": [
    "# Format error checks\n",
    "format_errors = defaultdict(int)\n",
    "\n",
    "for ex in data_lines:\n",
    "    if not isinstance(ex, dict):\n",
    "        format_errors[\"data_type\"] += 1\n",
    "        continue\n",
    "        \n",
    "    messages = ex.get(\"messages\", None)\n",
    "    if not messages:\n",
    "        format_errors[\"missing_messages_list\"] += 1\n",
    "        continue\n",
    "        \n",
    "    for message in messages:\n",
    "        if \"role\" not in message or \"content\" not in message:\n",
    "            format_errors[\"message_missing_key\"] += 1\n",
    "        \n",
    "        if any(k not in (\"role\", \"content\", \"name\", \"function_call\", \"weight\") for k in message):\n",
    "            format_errors[\"message_unrecognized_key\"] += 1\n",
    "        \n",
    "        if message.get(\"role\", None) not in (\"system\", \"user\", \"assistant\", \"function\"):\n",
    "            format_errors[\"unrecognized_role\"] += 1\n",
    "            \n",
    "        content = message.get(\"content\", None)\n",
    "        function_call = message.get(\"function_call\", None)\n",
    "        \n",
    "        if (not content and not function_call) or not isinstance(content, str):\n",
    "            format_errors[\"missing_content\"] += 1\n",
    "    \n",
    "    if not any(message.get(\"role\", None) == \"assistant\" for message in messages):\n",
    "        format_errors[\"example_missing_assistant_message\"] += 1\n",
    "\n",
    "if format_errors:\n",
    "    print(\"Found errors:\")\n",
    "    for k, v in format_errors.items():\n",
    "        print(f\"{k}: {v}\")\n",
    "else:\n",
    "    print(\"No errors found\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "c10cc5e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "encoding = tiktoken.get_encoding(\"cl100k_base\")\n",
    "\n",
    "def num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        num_tokens += tokens_per_message\n",
    "        for key, value in message.items():\n",
    "            num_tokens += len(encoding.encode(value))\n",
    "            if key == \"name\":\n",
    "                num_tokens += tokens_per_name\n",
    "    num_tokens += 3\n",
    "    return num_tokens\n",
    "\n",
    "def num_assistant_tokens_from_messages(messages):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        if message[\"role\"] == \"assistant\":\n",
    "            num_tokens += len(encoding.encode(message[\"content\"]))\n",
    "    return num_tokens\n",
    "\n",
    "def print_distribution(values, name):\n",
    "    print(f\"\\n#### Distribution of {name}:\")\n",
    "    print(f\"min / max: {min(values)}, {max(values)}\")\n",
    "    print(f\"mean / median: {np.mean(values)}, {np.median(values)}\")\n",
    "    print(f\"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "6eb12058",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num examples missing system message: 100\n",
      "Num examples missing user message: 0\n",
      "\n",
      "#### Distribution of num_messages_per_example:\n",
      "min / max: 2, 2\n",
      "mean / median: 2.0, 2.0\n",
      "p5 / p95: 2.0, 2.0\n",
      "\n",
      "#### Distribution of num_total_tokens_per_example:\n",
      "min / max: 101, 146\n",
      "mean / median: 119.9, 120.0\n",
      "p5 / p95: 107.0, 132.10000000000002\n",
      "\n",
      "#### Distribution of num_assistant_tokens_per_example:\n",
      "min / max: 2, 5\n",
      "mean / median: 3.04, 3.0\n",
      "p5 / p95: 2.0, 4.0\n",
      "\n",
      "0 examples may be over the 4096 token limit, they will be truncated during fine-tuning\n"
     ]
    }
   ],
   "source": [
    "# Warnings and tokens counts\n",
    "n_missing_system = 0\n",
    "n_missing_user = 0\n",
    "n_messages = []\n",
    "convo_lens = []\n",
    "assistant_message_lens = []\n",
    "\n",
    "for ex in data_lines:\n",
    "    messages = ex[\"messages\"]\n",
    "    if not any(message[\"role\"] == \"system\" for message in messages):\n",
    "        n_missing_system += 1\n",
    "    if not any(message[\"role\"] == \"user\" for message in messages):\n",
    "        n_missing_user += 1\n",
    "    n_messages.append(len(messages))\n",
    "    convo_lens.append(num_tokens_from_messages(messages))\n",
    "    assistant_message_lens.append(num_assistant_tokens_from_messages(messages))\n",
    "    \n",
    "print(\"Num examples missing system message:\", n_missing_system)\n",
    "print(\"Num examples missing user message:\", n_missing_user)\n",
    "print_distribution(n_messages, \"num_messages_per_example\")\n",
    "print_distribution(convo_lens, \"num_total_tokens_per_example\")\n",
    "print_distribution(assistant_message_lens, \"num_assistant_tokens_per_example\")\n",
    "n_too_long = sum(l > 4096 for l in convo_lens)\n",
    "print(f\"\\n{n_too_long} examples may be over the 4096 token limit, they will be truncated during fine-tuning\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "08244df1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset has ~11990 tokens that will be charged for during training\n",
      "By default, you'll train for 3 epochs on this dataset\n",
      "By default, you'll be charged for ~35970 tokens\n"
     ]
    }
   ],
   "source": [
    "# Pricing and default n_epochs estimate\n",
    "MAX_TOKENS_PER_EXAMPLE = 4096\n",
    "\n",
    "TARGET_EPOCHS = 3\n",
    "MIN_TARGET_EXAMPLES = 100\n",
    "MAX_TARGET_EXAMPLES = 25000\n",
    "MIN_DEFAULT_EPOCHS = 1\n",
    "MAX_DEFAULT_EPOCHS = 25\n",
    "\n",
    "n_epochs = TARGET_EPOCHS\n",
    "n_train_examples = len(data_lines)\n",
    "if n_train_examples * TARGET_EPOCHS < MIN_TARGET_EXAMPLES:\n",
    "    n_epochs = min(MAX_DEFAULT_EPOCHS, MIN_TARGET_EXAMPLES // n_train_examples)\n",
    "elif n_train_examples * TARGET_EPOCHS > MAX_TARGET_EXAMPLES:\n",
    "    n_epochs = max(MIN_DEFAULT_EPOCHS, MAX_TARGET_EXAMPLES // n_train_examples)\n",
    "\n",
    "n_billing_tokens_in_dataset = sum(min(MAX_TOKENS_PER_EXAMPLE, length) for length in convo_lens)\n",
    "print(f\"Dataset has ~{n_billing_tokens_in_dataset} tokens that will be charged for during training\")\n",
    "print(f\"By default, you'll train for {n_epochs} epochs on this dataset\")\n",
    "print(f\"By default, you'll be charged for ~{n_epochs * n_billing_tokens_in_dataset} tokens\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d419628",
   "metadata": {},
   "source": [
    "**Step2：微调（OpenAI）**\n",
    "\n",
    "相关文档：\n",
    "\n",
    "- https://platform.openai.com/docs/guides/fine-tuning/create-a-fine-tuned-model\n",
    "- https://platform.openai.com/docs/api-reference/fine-tuning/create\n",
    "\n",
    "\n",
    "智谱AI的微调和OpenAI差不多，其实每家都差不太多。而且有些平台还支持图形化界面，操作起来也非常简单，只要上传数据运行微调任务即可。微调结束后会得到一个新的模型ID，调用时把模型ID换一下即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "c56faef4",
   "metadata": {},
   "outputs": [],
   "source": [
    "file = client.files.create(\n",
    "    file=open(out_fn, \"rb\"),\n",
    "    purpose=\"fine-tune\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "cc4b5fa3",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "job = client.fine_tuning.jobs.create(\n",
    "    training_file=file.id,\n",
    "    model=\"gpt-3.5-turbo\",\n",
    "    hyperparameters={\n",
    "        \"n_epochs\": 2\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "c093bcd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 列出10个事件\n",
    "# client.fine_tuning.jobs.list(limit=10)\n",
    "\n",
    "# 获取微调的状态\n",
    "# client.fine_tuning.jobs.retrieve(\"ftjob-abc123\")\n",
    "\n",
    "# 取消一个job\n",
    "# client.fine_tuning.jobs.cancel(\"ftjob-abc123\")\n",
    "\n",
    "# 列出当前微调的10个事件\n",
    "# client.fine_tuning.jobs.list_events(fine_tuning_job_id=\"ftjob-abc123\", limit=10)\n",
    "\n",
    "# 删除一个微调模型\n",
    "# client.models.delete(\"ft:gpt-3.5-turbo:acemeco:suffix:abc123\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "cef50801",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FineTuningJob(id='ftjob-hQpExgNbOsnnOV4DLZn1SDUk', created_at=1716303071, error=Error(code=None, message=None, param=None), fine_tuned_model='ft:gpt-3.5-turbo-0125:personal::9RLH3nut', finished_at=1716303660, hyperparameters=Hyperparameters(n_epochs=2, batch_size=1, learning_rate_multiplier=2), model='gpt-3.5-turbo-0125', object='fine_tuning.job', organization_id='org-U35hu1wdD7w3HnkgJ5fdBW8m', result_files=['file-PhKdwGbqzf5IjiecDe1kiU4M'], seed=1873867419, status='succeeded', trained_tokens=23580, training_file='file-XCZvA9drMr1UebtoGN1vGM5D', validation_file=None, estimated_finish=None, integrations=[], user_provided_suffix=None)"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 可以用这个命令不定时获取状态，与之前版本一致。\n",
    "# 可重复执行，注意观察其中的“status”字段\n",
    "ft_job = client.fine_tuning.jobs.retrieve(job.id)\n",
    "ft_job"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "321765dc",
   "metadata": {},
   "source": [
    "大概等待10-15分钟（视数据量和排队情况，可能会更久）后，就得到了上面的结果。\n",
    "\n",
    "我们还可以通过下面的API获取checkpoint。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "5b8265b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "c3d32035",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\"Authorization\": f\"Bearer {client.api_key}\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "f54b07a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "resp = requests.get(\n",
    "    f\"https://api.openai.com/v1/fine_tuning/jobs/{job.id}/checkpoints\",\n",
    "    headers=headers\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "4f6a5840",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'object': 'list',\n",
       " 'data': [{'object': 'fine_tuning.job.checkpoint',\n",
       "   'id': 'ftckpt_0iu99HCJW9FllVqvrD16hBER',\n",
       "   'created_at': 1716303657,\n",
       "   'fine_tuned_model_checkpoint': 'ft:gpt-3.5-turbo-0125:personal::9RLH3nut',\n",
       "   'fine_tuning_job_id': 'ftjob-hQpExgNbOsnnOV4DLZn1SDUk',\n",
       "   'metrics': {'step': 200,\n",
       "    'train_loss': 7.629394644936838e-07,\n",
       "    'train_mean_token_accuracy': 1.0},\n",
       "   'step_number': 200},\n",
       "  {'object': 'fine_tuning.job.checkpoint',\n",
       "   'id': 'ftckpt_4QVDvsnDl1QdaabczYpi0nuI',\n",
       "   'created_at': 1716303462,\n",
       "   'fine_tuned_model_checkpoint': 'ft:gpt-3.5-turbo-0125:personal::9RLH30Ax:ckpt-step-100',\n",
       "   'fine_tuning_job_id': 'ftjob-hQpExgNbOsnnOV4DLZn1SDUk',\n",
       "   'metrics': {'step': 100,\n",
       "    'train_loss': 2.2964580059051514,\n",
       "    'train_mean_token_accuracy': 0.8333333134651184},\n",
       "   'step_number': 100}],\n",
       " 'has_more': False,\n",
       " 'first_id': 'ftckpt_0iu99HCJW9FllVqvrD16hBER',\n",
       " 'last_id': 'ftckpt_4QVDvsnDl1QdaabczYpi0nuI'}"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37cccc33",
   "metadata": {},
   "source": [
    "**📢注意：如果你微调的是babbage-002anddavinci-002，可以在GitHub首页切换到v1.0分支，参考那里的代码。**\n",
    "\n",
    "更多信息可参考文档：https://platform.openai.com/docs/guides/fine-tuning/use-a-fine-tuned-model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "132a012c",
   "metadata": {},
   "source": [
    "**Step3：使用**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "46a1e659",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask(content, model=\"gpt-3.5-turbo\", max_tokens=5):\n",
    "    response = client.chat.completions.create(\n",
    "        model=model, \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        temperature=0,\n",
    "        max_tokens=max_tokens,\n",
    "        top_p=1.0,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans\n",
    "\n",
    "def ask_zhipu(content, model=\"glm-3-turbo\", max_tokens=5):\n",
    "    response = zhipu_client.chat.completions.create(\n",
    "        model=model, \n",
    "        messages=[{\"role\": \"user\", \"content\": content}],\n",
    "        temperature=0.1,\n",
    "        max_tokens=max_tokens,\n",
    "        top_p=0.9,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "8871ebe6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "对给定文本进行分类，类别包括：科技、金融、娱乐、世界、汽车、运动、文化、军事、旅游、游戏、教育、农业、房产、社会。\n",
      "\n",
      "给定文本：\n",
      "日本虎视眈眈“武力夺岛”, 美军向俄后院开火，普京终不再忍！\n",
      "类别：\n",
      "\n",
      "\n",
      "True label: 世界\n"
     ]
    }
   ],
   "source": [
    "idx = -1\n",
    "prompt = get_prompt(lines[idx][\"sentence\"])\n",
    "print(prompt)\n",
    "print(\"\\nTrue label:\", desc_map[lines[idx][\"label_desc\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "id": "789af5ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'军事'"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 原来的，微调之前的\n",
    "ask(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "9b132e38",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'世界'"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 微调之后的\n",
    "ask(prompt, ft_job.fine_tuned_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0b29893",
   "metadata": {},
   "source": [
    "&emsp;&emsp;可以看到，微调后输出的是我们训练集里的真实类别。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad8a22ea",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面我们介绍了主题分类的微调。实体抽取的微调也是类似的，比如我们可以这样写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "055e5fcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "item = {\n",
    "    \"prompt\":\"请抽取给定Text中的实体，实体包括人名和地名。\\nText:<任意文本>\\n\", \n",
    "    \"completion\":\" <想要的实体列表>\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "id": "858976c5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'role': 'user', 'content': '请抽取给定Text中的实体，实体包括人名和地名。\\nText:<任意文本>\\n'},\n",
       " {'role': 'assistant', 'content': ' <想要的实体列表>'}]"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[\n",
    "    {\"role\": \"user\", \"content\": item[\"prompt\"]},\n",
    "    {\"role\": \"assistant\", \"content\": item[\"completion\"]}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab2fa8ab",
   "metadata": {},
   "source": [
    "&emsp;&emsp;当然，也可以指定输出类型、格式等，就像前面提示词所示的那样。相信大家应该很容易理解，不妨自己做一些尝试。尤其是给一些专业领域的实体进行微调，对比一下微调前后的效果。当然，你也可以指定输出格式等。\n",
    "\n",
    "&emsp;&emsp;如果大家对这块内容感兴趣，可以进一步阅读【相关文献11和12】。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b2280b4",
   "metadata": {},
   "source": [
    "### 3.3 智能对话"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ac0fa69",
   "metadata": {},
   "source": [
    "&emsp;&emsp;智能对话，有时候也叫智能客服、对话机器人、聊天机器人等等。总之就是和用户通过聊天窗口进行交互的一种技术。传统的对话机器人（下面都这么叫了）一般包括三个大的模块：\n",
    "\n",
    "- NLU：负责对用户输入进行理解。我们在本章一开始已经提到了，主要就是意图分类+实体识别这两种技术。实际中还可能有实体关系抽取、情感识别等组件。\n",
    "- DM：Dialogue Management，对话管理。就是在拿到NLU的结果后，如何确定机器人的回复，也就是对话方向的控制。\n",
    "- NLG：Natural Language Generation，自然语言生成。就是生成最终要回复给用户的输出。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9428eba3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;对话机器人一般包括三种，不同类型的技术方案侧重有所不同。常见的类型如下：\n",
    "\n",
    "- 任务型机器人\n",
    "- 问答型机器人\n",
    "- 闲聊型机器人\n",
    "\n",
    "&emsp;&emsp;任务型机器人主要用来完成特定的任务，比如订机票、订餐等，这一类机器人最关键的是要获取完成任务所需的各种信息（专业术语叫：槽位）。整个对话过程其实可以看作是一个填槽过程，通过与用户不断对话获取到需要的槽位信息。比如订餐这个任务，就餐人数、就餐时间、联系人电话等就是基本信息，机器人就要想办法获取到这些信息。这里NLU就是重头，DM一般使用两种方法：模型控制或流程图控制。前者通过模型自动学习来实现流转，后者则根据意图类型进行流转控制。\n",
    "\n",
    "&emsp;&emsp;问答型机器人主要用来回复用户问题，和上一章介绍的QA有点类似，平时我们常见的客服机器人往往是这种类型。它们更重要的是Question的匹配，DM相对弱一些。\n",
    "\n",
    "&emsp;&emsp;闲聊机器人就是和客户瞎扯淡的机器人，没啥实际作用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "753df28b",
   "metadata": {},
   "source": [
    "&emsp;&emsp;以上是大致的分类，但真实场景中的对话机器人往往是多种功能的结合体。更加适合从主动发起/被动接受这个角度来划分。\n",
    "\n",
    "- 主动发起对话的机器人\n",
    "- 被动接受对话的机器人\n",
    "\n",
    "&emsp;&emsp;前者一般是以外呼的方式进行，营销、催款、通知等都是常见的场景。这种对话机器人一般不闲聊，电话费不允许。它们基本都是带着特定任务或目的走流程，流程走完就挂断结束。与用户的互动更多是以QA的形式完成，因为主动权在机器人手里，所以流程一遍都是固定控制的，甚至QA的数量、回答次数也会控制。\n",
    "\n",
    "&emsp;&emsp;后者一般是以网页或客户端的形式存在，绝大部分时候都是用户找上门来了，比如大部分公司首页都有个「智能客服」，就是类似功能。它们以QA为主，辅以闲聊。稍微复杂点的是上面提到的任务型机器人，需要不断收集槽位信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ae5de91",
   "metadata": {},
   "source": [
    "&emsp;&emsp;ChatGPT时代，智能对话机器人会有什么新变化吗？接下来，我们探讨一下这方面内容。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dc11ce5",
   "metadata": {},
   "source": [
    "&emsp;&emsp;首先，可以肯定的是ChatGPT极大的扩展了对话机器人的边界，在此之前其实有不少端到端的方案，感兴趣的读者可以略读一下【相关文献13】。之前的方案现在看来是有点复杂、繁琐，效果还不好。所以之前除了闲聊，很少有真正使用端到端方案的对话机器人。不过ChatGPT的强大In-Context能力不仅让使用更加简单（我们只需把历史对话分角色放进去就好了），而且效果也更好，除了闲聊，问答型机器人它也可以很擅长，交互更加humanable。\n",
    "\n",
    "&emsp;&emsp;我们具体来展开说说它可以做的，我们尽量聚焦当下能做到的，不做过多未来畅想。\n",
    "\n",
    "- 作为问答类产品，比如知识问答、情感咨询、心理咨询等等，完全称得上诸事不决ChatGPT。举几个简单例子，比如问它编程概念（如闭包的作用），问它如何追一个心仪的女孩子，问它怎么避免焦虑等等。它的大部分回答绝对能让你眼前一亮。这可是之前只能闲聊的机器人完全够不着的高度。\n",
    "- 作为智能客服，通过与企业知识库结合（In-Context方式和微调方式①）完全可以胜任客服工作，而且相比之前的QA类的客服，它回答更加个性化，效果也更好（如果不记得了，回忆一下前面“文档问答”的内容）。\n",
    "- 作为智能营销机器人，智能客服更加偏向被动、为用户答疑解惑的方向，营销机器人则更加主动一些，它会根据已存储的用户信息，主动向用户推荐相关产品，根据预设的目标（可以理解为槽位，长期要收集的信息）向用户发起对话。它还可以同时负责维护客户关系。\n",
    "- 作为NPC（Non-Player Character）、陪聊机器人等休闲娱乐类产品。\n",
    "- 作为教育、培训的导师，可以进行一对一教学，尤其适合语言、编程类学习。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b08448d",
   "metadata": {},
   "source": [
    "&emsp;&emsp;这些都是它确定可以做的，为什么能做？归根结底还是其大规模参数所学到的知识和具备的理解力。尤其是后者，应该是决定性的（只有知识就是Google搜索引擎）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a070f5d3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;当然，并不是什么都要ChatGPT，我们千万要避免「手里有锤子，到处找钉子」的思维方式。某位哲人说过，一项新技术的出现，短期内总是被高估，长期内总是被低估。ChatGPT绝对是划时代的，但也不意味着你什么都要ChatGPT一下。比如，某些分类和实体抽取任务，之前的方法已经能达到非常好的效果，这时候就完全不需要替换。我们知道很多实际任务它并不会随着技术发展有太多变化，比如分类任务，难道你出来个新技术，分类任务就不是分类任务了吗。技术的更新会让我们的效率得到提升，也就是说做同样的任务更加简单和高效了，可以做更难的任务了，但不等于任务也会发生变化。所以，一定要理清楚这里面的关键。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b533d330",
   "metadata": {},
   "source": [
    "&emsp;&emsp;不过如果你是新开发，或者不了解这方面的专业知识，那就另当别论了，使用LLM的API反而可能是更好的策略。但即便如此，实际上线前还是应该考虑清楚各种细节，比如服务不可用怎么办，并发大概多少，时延要求多少，用户规模大概多少等等。我们技术方案的选型是和公司或自己的需求息息相关的，没有绝对好的方案，只有当下是否适合的方案。同时，要尽可能多考虑几步，但也不用太多（时刻谨记：「过度优化是原罪」），比如你用户只有不到1万，上来就搞个分布式的设计方案就有点坑了。但这并不妨碍你在代码和架构设计时考虑扩展性，比如数据库，我们可能用SQLite，但你代码里可不能直接和它耦合死，可以使用能同时支持其他数据库，甚至分布式数据库的ORM工具。这样虽然写起来稍微麻烦了一点点（真的是一点点），但你的代码更加清晰，而且和可能会变化的东西解耦了。这样如果日后规模上去了，数据库可以随便换，代码基本不用动。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "161962d7",
   "metadata": {},
   "source": [
    "&emsp;&emsp;最后，我们也应该了解ChatGPT的一些局限，除了它本身的局限（这块内容可以参考后面专门讲缺陷的章节），在工程上始终应该关注下面几个话题：\n",
    "\n",
    "- 响应时间和稳定性\n",
    "- 并发和横向可扩展性\n",
    "- 可维护性和迭代\n",
    "- 成本\n",
    "\n",
    "&emsp;&emsp;只有当这些都能满足你的期望时，才应该选择。始终记住，人才是关键，不要被任何工具绑架。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8177c19",
   "metadata": {},
   "source": [
    "> 注①：关于In-Context方式和微调方式的通俗解释  \n",
    "&emsp;&emsp;In-Context主要是利用ChatGPT的理解能力，把它当做超级大脑，我们把相关上下文给它，让它根据上下文回答问题，就类似前面的《文档问答》。对于不确定的问题，还可以设计兜底话术。  \n",
    "&emsp;&emsp;微调方式则是直接将自定义数据喂给ChatGPT的微调接口（现在没有开放，但理论上可行），让它学习这些自定义内容，之后直接问就好了，就像我们现在直接问它「中国的首都是哪里」，它可以正确回答一样。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68b23cb0",
   "metadata": {},
   "source": [
    "&emsp;&emsp;下面，我们一起使用ChatGPT来实现一个对话机器人。设计阶段首先至少需要考虑以下一些因素（这并不包括上面提到的那些）：\n",
    "\n",
    "- 使用目的\n",
    "- 如何使用\n",
    "- 消息查询、存储\n",
    "- 消息解析\n",
    "- 实时干预\n",
    "- 更新策略"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f654a877",
   "metadata": {},
   "source": [
    "&emsp;&emsp;首先，咱们需要明确使用目的是什么，如上所言，不同的用途咱们要考虑的因素也不一样。简单（但很实际）起见，我们以一个「订餐机器人」为例，简单的开场白后获取用户联系方式、订餐人数、用餐时间三个信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "163e06d9",
   "metadata": {},
   "source": [
    "&emsp;&emsp;使用也比较简单，主要利用ChatGPT的多轮对话能力即可，这里的重点是控制上下文。不过由于任务简单，我们不用对历史记录做召回再进行对话，直接在每一轮时把已经获取的信息告诉它，同时让它继续获取其他信息，直到所有信息获取完毕为止。另外，我们可以限制一下输出Token的数量（输出文本的长度）。\n",
    "\n",
    "&emsp;&emsp;对于用户的消息（以及机器人的回复），实际中往往需要存储起来，用来做每一轮回复的历史消息召回。而且这个日后还可能有其他用途，比如使用对话记录对用户进行画像，或者用来当做训练数据等等。存储可以直接放到数据库，或传到类似ElasticSearch这样的内部搜索引擎中。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0537a4b3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;消息的解析可以实时进行（并不一定要用ChatGPT）或离线进行，本案例我们需要实时在线解析。这个过程我们可以让ChatGPT在生成回复时顺便做掉。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4f3a639",
   "metadata": {},
   "source": [
    "&emsp;&emsp;实时干预是应该要关注的，或者需要设计这样的模块。一方面是回复内容有时候即便做了限制，依然有可能被某些问法问到不太合适的答复；另一方面也不能排除部分恶意用户对机器人进行攻击，因此最好有干预机制的设计。这里，我们设计一个简单策略：检测用户是否提问敏感类问题，如果发现此类问题直接返回设定好的文本，不需要调用ChatGPT进行对话回复。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddb267fb",
   "metadata": {},
   "source": [
    "&emsp;&emsp;更新策略主要是对企业知识库的更新，这里由于我们使用的是In-Context能力，所以并不需要调整ChatGPT，可能需要调整Embedding接口（目前openai不支持）。此案例暂不涉及。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0df5d0f",
   "metadata": {},
   "source": [
    "&emsp;&emsp;综上，我们需要先对用户输入进行敏感性检查，没问题后开始对话。同时应存储用户消息，并在每轮对话时将用户历史消息传递给接口。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d5dd5388",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/site-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version!\n",
      "  warnings.warn(\"urllib3 ({}) or chardet ({}) doesn't match a supported \"\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "client = OpenAI(api_key=\"YOUR OPENAI KEY\")\n",
    "\n",
    "from zhipuai import ZhipuAI\n",
    "zhipu_client = ZhipuAI(api_key=\"YOUR ZHIPU KEY\") "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a59d75ef",
   "metadata": {},
   "source": [
    "&emsp;&emsp;先看一下敏感性检查，这个接口比较多，openai提供了一个相关的接口，国内几大厂商也有相关API。这个本身是和对话无关的。我们以openai接口为例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "841604d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "def check_risk(inp: str):\n",
    "    # 这个接口的参数`input`支持Batch（传入List），会返回多个结果\n",
    "    response = client.moderations.create(input=inp)\n",
    "    return data[\"results\"][0][\"flagged\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "id": "6df57909",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "check_risk(\"good\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9c1a06c",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来我们考虑如何构造接口的输入，这里有两个事情要做：\n",
    "\n",
    "1. 查询历史对话记录作为上下文，简单起见我们可以只考虑上一轮。\n",
    "2. 计算输入的Token数，根据模型能接受最大Token长度和想输出的最大长度，反推上下文的最大长度，并对历史对话进行处理（如截断）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3ccf29fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataclasses import dataclass, asdict\n",
    "from typing import List, Dict\n",
    "from datetime import datetime\n",
    "import uuid\n",
    "import json\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "19831273",
   "metadata": {},
   "outputs": [],
   "source": [
    "@dataclass\n",
    "class User:\n",
    "    \n",
    "    user_id: str\n",
    "    user_name: str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7da97396",
   "metadata": {},
   "outputs": [],
   "source": [
    "@dataclass\n",
    "class ChatSession:\n",
    "    \n",
    "    user_id: str\n",
    "    session_id: str\n",
    "    cellphone: str\n",
    "    people_number: int\n",
    "    meal_time: str\n",
    "    chat_at: datetime\n",
    "\n",
    "\n",
    "@dataclass\n",
    "class ChatRecord:\n",
    "    \n",
    "    user_id: str\n",
    "    session_id: str\n",
    "    user_input: str\n",
    "    bot_output: str\n",
    "    chat_at: datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03dfea04",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面我们首先设计了两个简单的数据结构，一个是聊天信息，一个是聊天记录，前者记录聊天基本信息，后者记录聊天记录。其中，session_id主要用来区分每一次对话，当用户点击产品页面的「开始对话」之类的按钮后，就生成一个session_id；在下次对话时再生成一个新的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47851669",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来，我们处理核心对话逻辑，这一块主要是利用ChatGPT的能力，明确要求，把每一轮对话都喂给它。给出响应。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "d4c7a245",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask(msg):\n",
    "    response = openai_client.chat.completions.create(\n",
    "        model=\"gpt-3.5-turbo\", \n",
    "        temperature=0.2,\n",
    "        max_tokens=100,\n",
    "        top_p=1,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "        messages=msg\n",
    "    )\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans\n",
    "\n",
    "\n",
    "def ask_zhipu(msg):\n",
    "    response = zhipu_client.chat.completions.create(\n",
    "        model=\"glm-3-turbo\", \n",
    "        temperature=0.1,\n",
    "        max_tokens=100,\n",
    "        top_p=0.9,\n",
    "        messages=msg,\n",
    "    )\n",
    "\n",
    "    ans = response.choices[0].message.content\n",
    "    return ans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89c5ac46",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install sqlalchemy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "ae63eac7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sqlalchemy import insert\n",
    "\n",
    "class Chatbot:\n",
    "    \n",
    "    def __init__(self, api: str=\"openai\"):\n",
    "        self.system_inp = \"\"\"现在你是一个订餐机器人（角色是assistant），你的目的是向用户获取手机号码、用餐人数量和用餐时间三个信息。你可以自由回复用户消息，但牢记你的目的。每一轮你需要输出给用户的回复，以及获取到的信息，信息应该以JSON方式存储，包括三个key：cellphone表示手机号码，people_number表示用餐人数，meal_time表示用餐时间储。\n",
    "\n",
    "回复格式：\n",
    "给用户的回复：{回复给用户的话}\n",
    "获取到的信息：{\"cellphone\": null, \"people_number\": null, \"meal_time\": null}\n",
    "\"\"\"\n",
    "        self.max_round = 10\n",
    "        self.slot_labels = [\"meal_time\", \"people_number\", \"cellphone\"]\n",
    "        self.reg_msg = re.compile(r\"\\n+\")\n",
    "        self.api = api\n",
    "        if self.api == \"openai\":\n",
    "            self.ask_func = ask\n",
    "        elif self.api == \"zhipu\":\n",
    "            self.ask_func = ask_zhipu\n",
    "        else:\n",
    "            raise ValueError(f\"Invalid api: {api}\")\n",
    "\n",
    "\n",
    "    def check_over(self, slot_dict: dict):\n",
    "        for label in self.slot_labels:\n",
    "            if slot_dict.get(label) is None:\n",
    "                return False\n",
    "        return True\n",
    "    \n",
    "    def send_msg(self, msg: str):\n",
    "        print(f\"机器人：{msg}\")\n",
    "    \n",
    "    def chat(self, user_id: str):\n",
    "        sess_id = uuid.uuid4().hex\n",
    "        chat_at = datetime.now()\n",
    "        msg = [\n",
    "            {\"role\": \"user\", \"content\": self.system_inp},\n",
    "        ]\n",
    "        n_round = 0\n",
    "        \n",
    "        history = []\n",
    "        slot = {\"cellphone\": None, \"people_number\": None, \"meal_time\": None}\n",
    "        while True:\n",
    "            if n_round > self.max_round:\n",
    "                bot_msg = \"非常感谢您对我们的支持，再见。\"\n",
    "                self.send_msg(bot_msg)\n",
    "                break\n",
    "            \n",
    "            try:\n",
    "                # 智谱AI改成ask_zhipu\n",
    "                bot_inp = self.ask_func(msg)\n",
    "            except Exception as e:\n",
    "                print(f\"Error: {e}\")\n",
    "                bot_msg = \"机器人出错，稍后将由人工与您联系，谢谢。\"\n",
    "                self.send_msg(bot_msg)\n",
    "                break\n",
    "            \n",
    "            tmp = self.reg_msg.split(bot_inp)\n",
    "            bot_msg = tmp[0].strip(\"给用户的回复：\")\n",
    "            self.send_msg(bot_msg)\n",
    "            if len(tmp) > 1:\n",
    "                slot_str = tmp[1].strip(\"获取到的信息：\")\n",
    "                slot = json.loads(slot_str)\n",
    "                print(f\"\\tslot: {slot}\")\n",
    "            n_round += 1\n",
    "            \n",
    "            if self.check_over(slot):\n",
    "                break\n",
    "\n",
    "            user_inp = input()\n",
    "            \n",
    "            msg += [\n",
    "                {\"role\": \"assistant\", \"content\": bot_inp},\n",
    "                {\"role\": \"user\", \"content\": user_inp},\n",
    "            ]\n",
    "            \n",
    "            record = ChatRecord(user_id, sess_id, bot_inp, user_inp, datetime.now())\n",
    "            history.append(record)\n",
    "            \n",
    "            if check_risk(user_inp):\n",
    "                break\n",
    "        \n",
    "        chat_sess = ChatSession(user_id, sess_id, **slot, chat_at=chat_at)\n",
    "        self.store(history, chat_sess)\n",
    "    \n",
    "    \n",
    "    def store(self, history: List[ChatRecord], chat: ChatSession):\n",
    "        with SessionLocal.begin() as sess:\n",
    "            q = insert(\n",
    "                chat_record_table\n",
    "            ).values(\n",
    "                [asdict(v) for v in history]\n",
    "            )\n",
    "            sess.execute(q)\n",
    "        with SessionLocal.begin() as sess:\n",
    "            q = insert(\n",
    "                chat_session_table\n",
    "            ).values(\n",
    "                [asdict(chat)]\n",
    "            )\n",
    "            sess.execute(q)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "226911a0",
   "metadata": {},
   "source": [
    "&emsp;&emsp;接下来，我们把两张表建好（User表这里就不建了）。**注意：建表只要一次。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "fe7dff7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sqlalchemy import Table, Column, Integer, String, DateTime, Text, MetaData, SmallInteger\n",
    "from sqlalchemy import create_engine\n",
    "from sqlalchemy.orm import sessionmaker\n",
    "import os\n",
    "\n",
    "\n",
    "db_file = \"chatbot.db\"\n",
    "\n",
    "if os.path.exists(db_file):\n",
    "    os.remove(db_file)\n",
    "\n",
    "engine = create_engine(f\"sqlite:///{db_file}\")\n",
    "SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)\n",
    "\n",
    "metadata_obj = MetaData()\n",
    "\n",
    "chat_record_table = Table(\n",
    "    \"chat_record_table\",\n",
    "    metadata_obj,\n",
    "    Column(\"id\", Integer, primary_key=True),\n",
    "    Column(\"user_id\", String(64), index=True),\n",
    "    Column(\"session_id\", String(64), index=True),\n",
    "    Column(\"user_input\", Text),\n",
    "    Column(\"bot_output\", Text),\n",
    "    Column(\"chat_at\", DateTime),\n",
    ")\n",
    "\n",
    "chat_session_table = Table(\n",
    "    \"chat_session_table\",\n",
    "    metadata_obj,\n",
    "    Column(\"id\", Integer, primary_key=True),\n",
    "    Column(\"user_id\", String(64), index=True),\n",
    "    Column(\"session_id\", String(64), index=True),\n",
    "    Column(\"cellphone\", String(16)),\n",
    "    Column(\"people_number\", SmallInteger),\n",
    "    Column(\"meal_time\", String(32)),\n",
    "    Column(\"chat_at\", DateTime),\n",
    ")\n",
    "\n",
    "metadata_obj.create_all(engine, checkfirst=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a2e04490",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rw-r--r--  1 Yam  staff  28672 May 25 21:18 ./chatbot.db\r\n"
     ]
    }
   ],
   "source": [
    "!ls -la ./chatbot.db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c3c8255",
   "metadata": {},
   "source": [
    "&emsp;&emsp;现在我们进行简单的尝试："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2053849e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install pnlp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "d434a1b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "机器人：您好！非常感谢您选择我们的订餐服务。为了更好地为您服务，请问您能提供一下您的手机号码吗？这样我们才能及时通知您订单的状态。\n",
      "\tslot: {'cellphone': None, 'people_number': None, 'meal_time': None}\n",
      "是13712345678\n",
      "机器人：好的，您的手机号码已经记录为13712345678。接下来，请问您需要预订几人份的餐品呢？\n",
      "\tslot: {'cellphone': '13712345678', 'people_number': None, 'meal_time': None}\n",
      "我们一起5个人\n",
      "机器人：明白了，您需要预订的餐品份数为5人。请问您预计什么时候用餐呢？\n",
      "\tslot: {'cellphone': '13712345678', 'people_number': 5, 'meal_time': None}\n",
      "今天晚上8点\n",
      "机器人：好的，您的用餐时间为今天晚上8点。我们已经为您记录了预订信息，正在为您准备餐品。如有需要，我们会通过手机号码13712345678与您联系。祝您用餐愉快！\n",
      "\tslot: {'cellphone': '13712345678', 'people_number': 5, 'meal_time': '今天晚上8点'}\n"
     ]
    }
   ],
   "source": [
    "import pnlp\n",
    "nick = \"长琴\"\n",
    "user = User(pnlp.generate_uuid(nick), nick)\n",
    "chatbot = Chatbot(\"zhipu\")\n",
    "chatbot.chat(user.user_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "id": "87e98174",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "机器人：请问您的手机号码是多少呢？\n",
      "\tslot: {'cellphone': None, 'people_number': None, 'meal_time': None}\n",
      "我的手机是13788889999\n",
      "机器人：好的，您的手机号码是13788889999，请问用餐人数是几位呢？\n",
      "\tslot: {'cellphone': '13788889999', 'people_number': None, 'meal_time': None}\n",
      "我们一共五个人\n",
      "机器人：好的，您们一共五个人，最后，请问您们的用餐时间是什么时候呢？\n",
      "\tslot: {'cellphone': '13788889999', 'people_number': 5, 'meal_time': None}\n",
      "稍等我问一下啊\n",
      "机器人：好的，没问题，我等您的消息。\n",
      "好了，明天下午7点，谢谢\n",
      "机器人：好的，您们的用餐时间是明天下午7点，我们已经为您记录好了，请问还有其他需要帮助的吗？\n",
      "\tslot: {'cellphone': '13788889999', 'people_number': 5, 'meal_time': '明天下午7点'}\n"
     ]
    }
   ],
   "source": [
    "import pnlp\n",
    "nick = \"长琴\"\n",
    "user = User(pnlp.generate_uuid(nick), nick)\n",
    "chatbot = Chatbot(\"openai\")\n",
    "chatbot.chat(user.user_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e368deb9",
   "metadata": {},
   "source": [
    "&emsp;&emsp;查表看看刚刚的记录："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "9913e76c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "e832c83d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def query_table(table: str):\n",
    "    con = sqlite3.connect(\"chatbot.db\")\n",
    "    cur = con.cursor()\n",
    "    q = cur.execute(f\"SELECT * FROM {table}\")\n",
    "    return q.fetchall()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "id": "26d06b28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1,\n",
       "  'dc3be3b3516555d3b0b6a77a1d9c7e82',\n",
       "  '05a88a8e3db8490eacf14b8bb9800fcc',\n",
       "  '13788889999',\n",
       "  5,\n",
       "  '明天下午7点',\n",
       "  '2023-04-08 00:00:34.618232')]"
      ]
     },
     "execution_count": 290,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query_table(\"chat_session_table\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3faf72d3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;上面我们实现了一个非常简易的任务机器人，虽然简易，但我们其实很容易就能发现，NLU、DM和NLG三个模块已经完全不需要了。唯一的不足可能是接口反应有点慢，从对话来看其实并没有太多问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffc2ecab",
   "metadata": {},
   "source": [
    "&emsp;&emsp;另外，需要再次对几个问题进行强调，以便大家可以更好地构建应用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6149b1e",
   "metadata": {},
   "source": [
    "&emsp;&emsp;第一点，当要支持的对话轮次非常多时（比如培训、面试这样的场景），则需要实时将每一轮的对话索引起来，每一轮先召回所有历史对话中相关的topN轮作为上下文（正如我们在文档问答中那样）。然后让ChatGPT根据这些上下文对用户进行回复。这样理论上我们是可以支持无限轮的。召回的过程其实就是一个回忆的过程，这里可以优化的点或者说想象的空间很大。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be84972d",
   "metadata": {},
   "source": [
    "&emsp;&emsp;第二点，在传递message参数给ChatGPT时，由于有长度限制，有时候上下文中遇到特别长回复那种轮次，可能会导致只能传几轮（甚至一轮就耗光长度了）。根据ChatGPT自己的说法：当历史记录非常长时，我们确实可能只能利用其中的一小部分来生成回答。为了应对这种情况，通常会使用一些技术来选择最相关的历史记录，以便在生成回答时使用。例如，可能会使用一些关键词提取技术，识别出历史记录中最相关的信息，并将其与当前的输入一起使用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "062f7743",
   "metadata": {},
   "source": [
    "&emsp;&emsp;此外，还可能会使用一些摘要技术来对历史记录进行压缩和精简，以便在生成回答时只使用最重要的信息。另外，还可以使用一些记忆机制，例如注意力机制，以便在历史记录中选择最相关的信息。虽然这些技术可以帮助模型在历史记录很长时选择最相关的信息，但在某些情况下，历史记录仍然可能过于复杂，导致模型难以正确理解和处理。在这种情况下，可能需要使用其他技术来限制历史记录的长度或提供其他方面的辅助信息，以便模型可以更好地理解和回答用户的问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef3caaa4",
   "metadata": {},
   "source": [
    "&emsp;&emsp;另外，根据ChatGPT的说法，在生成回复时，它也会使用一些技术来限制输出长度，例如截断输出或者使用一些策略来生成更加简洁的回答。当然，用户也可以使用特定的输入限制或规则来帮助缩短回答。总之，尽可能地在输出长度和回答质量之间进行平衡。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddb1d06d",
   "metadata": {},
   "source": [
    "&emsp;&emsp;第三点，充分考虑安全性，根据实际情况合理设计架构（但不要过度设计）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9cdbe93",
   "metadata": {},
   "source": [
    "&emsp;&emsp;最后，值得一提的是，上面只是利用了ChatGPT的一丢丢功能，大家完全可以结合自己的业务，或者大开脑洞，开发更多有用、有趣的产品和应用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82d2ea0f",
   "metadata": {},
   "source": [
    "## 相关文献\n",
    "\n",
    "- 【1】[GPT3 和它的 In-Context Learning | Yam](https://yam.gift/2023/01/20/NLP/2023-01-20-GPT3/)\n",
    "- 【2】[ChatGPT Prompt 工程：设计、实践与思考 | Yam](https://yam.gift/2023/01/25/NLP/2023-01-25-ChatGPT-Prompt-Engineering/)\n",
    "- 【3】[一些 ChatGPT Prompt 示例 | Yam](https://yam.gift/2023/01/31/NLP/2023-01-31-ChatGPT-Prompt-Example/)\n",
    "- 【4】[dair-ai/Prompt-Engineering-Guide: 🐙 Guides, papers, lecture, notebooks and resources for prompt engineering](https://github.com/dair-ai/Prompt-Engineering-Guide)\n",
    "- 【5】[Best practices for prompt engineering with OpenAI API | OpenAI Help Center](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api)\n",
    "- 【6】[ChatGPT Prompts and Products | PromptVine](https://promptvine.com/)\n",
    "- 【7】[Prompt Vibes](https://www.promptvibes.com/)\n",
    "- 【8】[ShareGPT: Share your wildest ChatGPT conversations with one click.](https://sharegpt.com/)\n",
    "- 【9】[Awesome ChatGPT Prompts | This repo includes ChatGPT prompt curation to use ChatGPT better.](https://prompts.chat/)\n",
    "- 【10】[Learn Prompting | Learn Prompting](https://learnprompting.org/)\n",
    "- 【11】[Fine-tuning - OpenAI API](https://platform.openai.com/docs/guides/fine-tuning)\n",
    "- 【12】[[PUBLIC] Best practices for fine-tuning GPT-3 to classify text - Google Docs](https://docs.google.com/document/d/1rqj7dkuvl7Byd5KQPUJRxc19BJt8wo0yHNwK84KfU3Q/edit)\n",
    "- 【13】[hscspring/chatbot: Lab for Chatbot](https://github.com/hscspring/chatbot)\n",
    "\n",
    "\n",
    "memo：\n",
    "\n",
    "- [openai-cookbook/text_explanation_examples.md at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/text_explanation_examples.md)\n",
    "- [openai-cookbook/Fine-tuned_classification.ipynb at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/examples/Fine-tuned_classification.ipynb)\n",
    "- [openai-cookbook/Gen_QA.ipynb at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/pinecone/Gen_QA.ipynb)\n",
    "- [openai-cookbook/Question_answering_using_embeddings.ipynb at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb)\n",
    "- [openai-cookbook/Fine-tuned_classification.ipynb at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/examples/Fine-tuned_classification.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "189px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}