feat: 被动攻击算法

_toc.yml

@@ -14,3 +14,4 @@ parts:
     - file: docs/回归/逻辑回归
     - file: docs/回归/随机梯度下降
     - file: docs/回归/感知器
+    - file: docs/回归/被动攻击算法

docs/回归/感知器.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "## 感知器\n",
+    "# 感知器\n",
     "\n",
     "感知器（Perceptron）是Frank Rosenblatt在1957年就职于康奈尔航空实验室（Cornell Aeronautical Laboratory）时所发明的一种人工神经网络。感知器是一种最简单的神经网络模型。\n",
     "\n",

docs/回归/被动攻击算法.ipynb

@@ -0,0 +1,160 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "5f743a36",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# 被动攻击算法\n",
+    "\n",
+    "被动攻击算法（Passive Aggressive Algorithm）由Crammer等人于2006年提出，是一种在线学习算法。思路是在每次训练样本的输入上进行预测，并根据预测结果与真实标签之间的差异来调整模型的参数。其主要思想是在遇到错误预测时采取被动的反应，通过适当的更新规则来修正模型，以尽可能减小损失。\n",
+    "\n",
+    "对于二分类问题，被动攻击算法的步骤如下：\n",
+    "\n",
+    "---------\n",
+    "* 初始化权重向量和偏置为零向量或随机值。\n",
+    "* 对于每个训练样本 $(x, y)$，其中 $x$ 是输入特征向量，$y$ 是目标标签，值是1或-1：\n",
+    "    * 预测输出：$\\hat{y} = w · x + b$\n",
+    "    * 计算损失：$loss = max(0, 1 - y · \\hat{y})$\n",
+    "        * 若预测结果与真实标签一致，即 $y · \\hat{y} >= 1$，表示预测正确，损失为0；否则，损失大于0，需要进行调整。\n",
+    "    * 如果损失大于0，执行以下更新步骤：\n",
+    "        * $η$是学习率，控制每次更新的步长大小。\n",
+    "        * 更新权重和偏置：$w = w + η * y * x，b = b + η * y$\n",
+    "        * 这里的学习率$η$通常较小，可以控制每次更新的幅度。\n",
+    "* 重复步骤2，直到达到指定的迭代次数或训练误差满足预设条件。\n",
+    "---------\n",
+    "\n",
+    "被动攻击算法可用于分类和回归。在被动攻击算法的回归变体中，我们使用不同的损失函数和参数更新规则：\n",
+    "\n",
+    "---------\n",
+    "* 初始化权重向量$w$和偏置$b$为零向量或随机值。\n",
+    "* 对于每个训练样本$(x, y)$，其中$x$是输入特征向量，$y$是目标值：\n",
+    "    * 预测输出：$ \\hat{y} = w \\cdot x + b$\n",
+    "    * 计算损失：$loss = \\max(0, |y - \\hat{y}| - \\varepsilon) $\n",
+    "    * 如果loss > 0，执行以下更新步骤：\n",
+    "        * $η$ 是学习率，控制每次更新的步长大小。\n",
+    "        * 如果$y > \\hat{y}$，表示预测值低于目标值，则更新公式为：\n",
+    "            * $w = w + η * (y - \\hat{y}) * x$\n",
+    "            * $b = b + η * (y - \\hat{y})$\n",
+    "        * 如果y < \\hat{y}，表示预测值高于目标值，则更新公式为：\n",
+    "            * $w = w - η * (\\hat{y} - y) * x$\n",
+    "            * $b = b - η * (\\hat{y} - y)$\n",
+    "---------\n",
+    "\n",
+    "\n",
+    "被动攻击算法适用于在线学习和增量学习场景，可以逐步地调整模型以适应新的训练样本。由于其简单性和高效性，被动攻击算法在处理大规模数据集或实时数据流方面具有优势，并在文本分类、情感分析等任务中得到广泛应用。"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43640d61",
+   "metadata": {},
+   "source": [
+    "在Scikit-learn库中，被动攻击算法（Passive Aggressive Algorithm）有两个主要的实现：PassiveAggressiveRegressor用于回归问题，PassiveAggressiveClassifier用于分类问题。下面是针对这两个问题的示例代码：\n",
+    "\n",
+    "回归问题的示例："
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "ec021c84",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "预测结果： [11.85000006 13.80000006]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from sklearn.linear_model import PassiveAggressiveRegressor\n",
+    "\n",
+    "# 创建模型对象\n",
+    "regressor = PassiveAggressiveRegressor()\n",
+    "\n",
+    "# 拟合数据\n",
+    "X_train = [[1], [2], [3], [4], [5]]  # 输入特征\n",
+    "y_train = [2, 4, 6, 8, 10]           # 目标值\n",
+    "regressor.fit(X_train, y_train)\n",
+    "\n",
+    "# 预测新样本\n",
+    "X_test = [[6], [7]]  # 新样本输入特征\n",
+    "y_pred = regressor.predict(X_test)\n",
+    "print(\"预测结果：\", y_pred)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c6f3db99",
+   "metadata": {},
+   "source": [
+    "分类问题的示例："
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "c0c52c6d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "准确率： 1.0\n"
+     ]
+    }
+   ],
+   "source": [
+    "from sklearn.linear_model import PassiveAggressiveClassifier\n",
+    "from sklearn.datasets import make_classification\n",
+    "from sklearn.model_selection import train_test_split\n",
+    "from sklearn.metrics import accuracy_score\n",
+    "\n",
+    "# 创建模拟数据集\n",
+    "X, y = make_classification(n_samples=100, n_features=10, random_state=42)\n",
+    "\n",
+    "# 划分训练集和测试集\n",
+    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
+    "\n",
+    "# 创建模型对象\n",
+    "classifier = PassiveAggressiveClassifier()\n",
+    "\n",
+    "# 拟合数据\n",
+    "classifier.fit(X_train, y_train)\n",
+    "\n",
+    "# 预测测试集\n",
+    "y_pred = classifier.predict(X_test)\n",
+    "\n",
+    "# 计算准确率\n",
+    "accuracy = accuracy_score(y_test, y_pred)\n",
+    "print(\"准确率：\", accuracy)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/回归/随机梯度下降.ipynb

@@ -390,8 +390,8 @@
     "\n",
     "本文大量参考[1]，原文写得非常好，可以看看原文。\n",
     "\n",
-    "[1] 各种梯度下降方法及其可视化 https://www.ruder.io/optimizing-gradient-descent/\n",
-    "[2] 1的中文版本 https://zhuanlan.zhihu.com/p/480322641"
+    "* [1] 各种梯度下降方法及其可视化 https://www.ruder.io/optimizing-gradient-descent/\n",
+    "* [2] 1的中文版本 https://zhuanlan.zhihu.com/p/480322641"
    ]
   }
  ],