feat: 弹性网络

tianxuzhang · Dec 3, 2023 · 97c21f5 · 97c21f5
1 parent b6a05a8
commit 97c21f5
Showing 1 changed file with 65 additions and 0 deletions.
diff --git a/docs/回归/正则化线性回归.ipynb b/docs/回归/正则化线性回归.ipynb
@@ -341,6 +341,71 @@
     "# 输出选择的最优alpha值\n",
     "print(reg.alpha_)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2cc25023",
+   "metadata": {},
+   "source": [
+    "## 弹性网络\n",
+    "\n",
+    "弹性网络（Elastic Net）是一种结合了L1范数和L2范数的线性回归正则化方法。它综合了Lasso回归和岭回归的优点，并旨在克服它们各自的限制。\n",
+    "\n",
+    "弹性网络的目标函数表达式如下：\n",
+    "\n",
+    "$$\\underset{w}{\\text{minimize }} \\frac{1}{2n_{samples}} ||X w - y||_2 ^ 2 + \\alpha \\rho ||w||_1 + \\frac{\\alpha(1-\\rho)}{2} ||w||_2 ^ 2$$\n",
+    "\n",
+    "其中，$X$ 是输入特征矩阵，$w$ 是模型的系数向量，$y$ 是观测值向量，$\\alpha$ 是正则化参数，$\\rho$ 是混合比例参数。\n",
+    "\n",
+    "* 第一项 $\\frac{1}{2n_{samples}} ||X w - y||_2 ^ 2$ 衡量了模型预测值与真实观测值之间的误差。这里使用了L2范数来计算误差的平方和，同时将其除以 $2n_{samples}$ 进行归一化，其中 $n_{samples}$ 是样本数量。\n",
+    "\n",
+    "* 第二项 $\\alpha \\rho ||w||_1$ 是L1范数的惩罚项，用于促使模型的系数稀疏化，即让部分系数趋向于零。\n",
+    "\n",
+    "* 第三项 $\\frac{\\alpha(1-\\rho)}{2} ||w||_2 ^ 2$ 是L2范数的惩罚项，用于控制模型的复杂度并避免过拟合。\n",
+    "\n",
+    "通过调整正则化参数 $\\alpha$ 和混合比例参数 $\\rho$ 的大小，可以控制模型的拟合程度、稀疏性和复杂度。\n",
+    "\n",
+    "弹性网络适用于在存在多个相关特征和噪声的情况下进行回归建模。它可以同时进行特征选择和模型拟合，并具有较好的泛化能力。\n",
+    "\n",
+    "在Scikit-learn中，可以使用ElasticNet类来实现弹性网络回归，并进行参数估计和模型拟合。"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "39860cd2",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "模型系数：\n",
+      "[ 0.         -0.         -0.          0.23961679 -0.07092795 -0.03552143\n",
+      "  0.05895168  0.          0.03982252 -0.05629927]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from sklearn.linear_model import ElasticNet\n",
+    "import numpy as np\n",
+    "\n",
+    "# 生成样本数据\n",
+    "np.random.seed(0)\n",
+    "n_samples, n_features = 100, 10\n",
+    "X = np.random.randn(n_samples, n_features)\n",
+    "y = np.random.randn(n_samples)\n",
+    "\n",
+    "# 创建ElasticNet对象并进行拟合\n",
+    "alpha = 0.1  # 正则化参数\n",
+    "l1_ratio = 0.5  # 混合比例参数\n",
+    "elastic_net = ElasticNet(alpha=alpha, l1_ratio=l1_ratio)\n",
+    "elastic_net.fit(X, y)\n",
+    "\n",
+    "# 输出模型系数\n",
+    "print(\"模型系数：\")\n",
+    "print(elastic_net.coef_)"
+   ]
   }
  ],
  "metadata": {