diff --git a/_toc.yml b/_toc.yml index 3530c65..d03ed2a 100644 --- a/_toc.yml +++ b/_toc.yml @@ -14,3 +14,4 @@ parts: - file: docs/回归/逻辑回归 - file: docs/回归/随机梯度下降 - file: docs/回归/感知器 + - file: docs/回归/被动攻击算法 diff --git "a/docs/\345\233\236\345\275\222/\346\204\237\347\237\245\345\231\250.ipynb" "b/docs/\345\233\236\345\275\222/\346\204\237\347\237\245\345\231\250.ipynb" index c16cf73..eea1cb6 100644 --- "a/docs/\345\233\236\345\275\222/\346\204\237\347\237\245\345\231\250.ipynb" +++ "b/docs/\345\233\236\345\275\222/\346\204\237\347\237\245\345\231\250.ipynb" @@ -6,7 +6,7 @@ "metadata": {}, "source": [ "\n", - "## 感知器\n", + "# 感知器\n", "\n", "感知器（Perceptron）是Frank Rosenblatt在1957年就职于康奈尔航空实验室（Cornell Aeronautical Laboratory）时所发明的一种人工神经网络。感知器是一种最简单的神经网络模型。\n", "\n", diff --git "a/docs/\345\233\236\345\275\222/\350\242\253\345\212\250\346\224\273\345\207\273\347\256\227\346\263\225.ipynb" "b/docs/\345\233\236\345\275\222/\350\242\253\345\212\250\346\224\273\345\207\273\347\256\227\346\263\225.ipynb" new file mode 100644 index 0000000..5a96428 --- /dev/null +++ "b/docs/\345\233\236\345\275\222/\350\242\253\345\212\250\346\224\273\345\207\273\347\256\227\346\263\225.ipynb" @@ -0,0 +1,160 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5f743a36", + "metadata": {}, + "source": [ + "\n", + "# 被动攻击算法\n", + "\n", + "被动攻击算法（Passive Aggressive Algorithm）由Crammer等人于2006年提出，是一种在线学习算法。思路是在每次训练样本的输入上进行预测，并根据预测结果与真实标签之间的差异来调整模型的参数。其主要思想是在遇到错误预测时采取被动的反应，通过适当的更新规则来修正模型，以尽可能减小损失。\n", + "\n", + "对于二分类问题，被动攻击算法的步骤如下：\n", + "\n", + "---------\n", + "* 初始化权重向量和偏置为零向量或随机值。\n", + "* 对于每个训练样本 $(x, y)$，其中 $x$ 是输入特征向量，$y$ 是目标标签，值是1或-1：\n", + " * 预测输出：$\\hat{y} = w · x + b$\n", + " * 计算损失：$loss = max(0, 1 - y · \\hat{y})$\n", + " * 若预测结果与真实标签一致，即 $y · \\hat{y} >= 1$，表示预测正确，损失为0；否则，损失大于0，需要进行调整。\n", + " * 如果损失大于0，执行以下更新步骤：\n", + " * $η$是学习率，控制每次更新的步长大小。\n", + " * 更新权重和偏置：$w = w + η * y * x，b = b + η * y$\n", + " * 这里的学习率$η$通常较小，可以控制每次更新的幅度。\n", + "* 重复步骤2，直到达到指定的迭代次数或训练误差满足预设条件。\n", + "---------\n", + "\n", + "被动攻击算法可用于分类和回归。在被动攻击算法的回归变体中，我们使用不同的损失函数和参数更新规则：\n", + "\n", + "---------\n", + "* 初始化权重向量$w$和偏置$b$为零向量或随机值。\n", + "* 对于每个训练样本$(x, y)$，其中$x$是输入特征向量，$y$是目标值：\n", + " * 预测输出：$ \\hat{y} = w \\cdot x + b$\n", + " * 计算损失：$loss = \\max(0, |y - \\hat{y}| - \\varepsilon) $\n", + " * 如果loss > 0，执行以下更新步骤：\n", + " * $η$ 是学习率，控制每次更新的步长大小。\n", + " * 如果$y > \\hat{y}$，表示预测值低于目标值，则更新公式为：\n", + " * $w = w + η * (y - \\hat{y}) * x$\n", + " * $b = b + η * (y - \\hat{y})$\n", + " * 如果y < \\hat{y}，表示预测值高于目标值，则更新公式为：\n", + " * $w = w - η * (\\hat{y} - y) * x$\n", + " * $b = b - η * (\\hat{y} - y)$\n", + "---------\n", + "\n", + "\n", + "被动攻击算法适用于在线学习和增量学习场景，可以逐步地调整模型以适应新的训练样本。由于其简单性和高效性，被动攻击算法在处理大规模数据集或实时数据流方面具有优势，并在文本分类、情感分析等任务中得到广泛应用。" + ] + }, + { + "cell_type": "markdown", + "id": "43640d61", + "metadata": {}, + "source": [ + "在Scikit-learn库中，被动攻击算法（Passive Aggressive Algorithm）有两个主要的实现：PassiveAggressiveRegressor用于回归问题，PassiveAggressiveClassifier用于分类问题。下面是针对这两个问题的示例代码：\n", + "\n", + "回归问题的示例：" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "ec021c84", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "预测结果： [11.85000006 13.80000006]\n" + ] + } + ], + "source": [ + "from sklearn.linear_model import PassiveAggressiveRegressor\n", + "\n", + "# 创建模型对象\n", + "regressor = PassiveAggressiveRegressor()\n", + "\n", + "# 拟合数据\n", + "X_train = [[1], [2], [3], [4], [5]] # 输入特征\n", + "y_train = [2, 4, 6, 8, 10] # 目标值\n", + "regressor.fit(X_train, y_train)\n", + "\n", + "# 预测新样本\n", + "X_test = [[6], [7]] # 新样本输入特征\n", + "y_pred = regressor.predict(X_test)\n", + "print(\"预测结果：\", y_pred)" + ] + }, + { + "cell_type": "markdown", + "id": "c6f3db99", + "metadata": {}, + "source": [ + "分类问题的示例：" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "c0c52c6d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "准确率： 1.0\n" + ] + } + ], + "source": [ + "from sklearn.linear_model import PassiveAggressiveClassifier\n", + "from sklearn.datasets import make_classification\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.metrics import accuracy_score\n", + "\n", + "# 创建模拟数据集\n", + "X, y = make_classification(n_samples=100, n_features=10, random_state=42)\n", + "\n", + "# 划分训练集和测试集\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# 创建模型对象\n", + "classifier = PassiveAggressiveClassifier()\n", + "\n", + "# 拟合数据\n", + "classifier.fit(X_train, y_train)\n", + "\n", + "# 预测测试集\n", + "y_pred = classifier.predict(X_test)\n", + "\n", + "# 计算准确率\n", + "accuracy = accuracy_score(y_test, y_pred)\n", + "print(\"准确率：\", accuracy)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git "a/docs/\345\233\236\345\275\222/\351\232\217\346\234\272\346\242\257\345\272\246\344\270\213\351\231\215.ipynb" "b/docs/\345\233\236\345\275\222/\351\232\217\346\234\272\346\242\257\345\272\246\344\270\213\351\231\215.ipynb" index 1a4c2d0..109ca07 100644 --- "a/docs/\345\233\236\345\275\222/\351\232\217\346\234\272\346\242\257\345\272\246\344\270\213\351\231\215.ipynb" +++ "b/docs/\345\233\236\345\275\222/\351\232\217\346\234\272\346\242\257\345\272\246\344\270\213\351\231\215.ipynb" @@ -390,8 +390,8 @@ "\n", "本文大量参考[1]，原文写得非常好，可以看看原文。\n", "\n", - "[1] 各种梯度下降方法及其可视化 https://www.ruder.io/optimizing-gradient-descent/\n", - "[2] 1的中文版本 https://zhuanlan.zhihu.com/p/480322641" + "* [1] 各种梯度下降方法及其可视化 https://www.ruder.io/optimizing-gradient-descent/\n", + "* [2] 1的中文版本 https://zhuanlan.zhihu.com/p/480322641" ] } ],