Skip to content

Commit

Permalink
deploy: 7d22c6b
Browse files Browse the repository at this point in the history
  • Loading branch information
@tianxuzhang
tianxuzhang committed Dec 9, 2023
1 parent 2ae63a0 commit 187c7e1
Show file tree
Hide file tree
Showing 29 changed files with 2,975 additions and 198 deletions.
53 changes: 53 additions & 0 deletions README.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,9 @@

<li class="toctree-l1"><a class="reference internal" href="docs/%E5%9B%9E%E5%BD%92/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95.html">最小二乘法</a></li>
<li class="toctree-l1"><a class="reference internal" href="docs/%E5%9B%9E%E5%BD%92/%E6%AD%A3%E5%88%99%E5%8C%96%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92.html">正则化线性回归</a></li>
<li class="toctree-l1"><a class="reference internal" href="docs/%E5%9B%9E%E5%BD%92/%E8%B4%9D%E5%8F%B6%E6%96%AF%E5%9B%9E%E5%BD%92.html">贝叶斯回归</a></li>


</ul>

</div>
Expand Down Expand Up @@ -409,6 +412,56 @@ <h2>回归<a class="headerlink" href="#id2" title="Permalink to this heading">#<
<li><p>算法：梯度提升、最近邻、随机森林、岭回归等等。</p></li>
</ul>
<p><img alt="" src="_images/sphx_glr_plot_adaboost_regression_thumb.png" /></p>
<p>本章内容包含：</p>
<hr class="docutils" />
<ul class="simple">
<li><p>线性回归</p>
<ul>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92.html">一元/多元线性回归</a></p></li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E8%B7%9D%E7%A6%BB%E5%92%8C%E8%8C%83%E6%95%B0.html">距离和范数</a></p></li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95.html">最小二乘法</a></p></li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E5%A4%9A%E9%A1%B9%E5%BC%8F%E5%9B%9E%E5%BD%92.html">多项式回归</a></p></li>
</ul>
</li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E6%AD%A3%E5%88%99%E5%8C%96%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92.html">正则化线性回归</a></p>
<ul>
<li><p>LASSO回归</p></li>
<li><p>多任务LASSO回归</p></li>
<li><p>岭回归</p></li>
<li><p>弹性网络</p></li>
<li><p>多任务弹性网络</p></li>
</ul>
</li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E8%BF%AD%E4%BB%A3%E6%B1%82%E8%A7%A3%E6%96%B9%E6%B3%95.html">迭代求解方法</a></p>
<ul>
<li><p>最小角回归（LARS）</p></li>
<li><p>LARS LASSO</p></li>
<li><p>正交匹配追踪法（OMP）</p></li>
</ul>
</li>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E8%B4%9D%E5%8F%B6%E6%96%AF%E6%96%B9%E6%B3%95.html">贝叶斯方法</a></p>
<ul>
<li><p>贝叶斯岭回归</p></li>
<li><p>主动相关决策理论 - ARD</p></li>
</ul>
</li>
<li><p>广义线性模型</p>
<ul>
<li><p><a class="reference external" href="https://tianxuzhang.github.io/introduction-to-machine-learning/docs/%E5%9B%9E%E5%BD%92/%E9%80%BB%E8%BE%91%E5%9B%9E%E5%BD%92.html">逻辑回归</a></p></li>
<li><p>多元逻辑回归</p></li>
<li><p>泊松回归</p></li>
<li><p>GLM模型</p></li>
</ul>
</li>
<li><p>其他</p>
<ul>
<li><p>随机梯度下降</p></li>
<li><p>感知器</p></li>
<li><p>被动攻击算法</p></li>
<li><p>稳健回归</p></li>
</ul>
</li>
</ul>
</section>
<section id="id3">
<h2>聚类<a class="headerlink" href="#id3" title="Permalink to this heading">#</a></h2>
Expand Down
Binary file renamed ...951c41abb4be8b5e5c5d50c717f5361647a64.png → ...49abdd636738bf245974ffbdb78a0ba18f2fb.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed ...30e98989d05081173550ac8bfdf257044f4f5.png → ...690f4c73a9e547351c1e93570bb14eae9960b.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed ...7aa13011afe0f8112adf0c9b0df3d8f395eea.png → ...49b53ece36c784b0d8eb89782f2be165090ee.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file renamed ...8fa74c463364f12612340bc9028e11e39e7c2.png → ...40f06e86c19d5f98a0f8dffc9a41ab406cbae.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed _images/f3059f195a0eff00b47daa668ae906ee2a125e11ad3a8b0054995d8129c3565f.png
View file
Binary file not shown.
Binary file added _images/f8a248761d27012dc4a57df4c9c44ecdc066deb6b6a15ee8a170804ae8924597.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
37 changes: 37 additions & 0 deletions _sources/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,43 @@

![](./images/ml-task/sphx_glr_plot_adaboost_regression_thumb.png)

本章内容包含：

--------

* 线性回归
* [一元/多元线性回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/线性回归.html)
* [距离和范数](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/距离和范数.html)
* [最小二乘法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/最小二乘法.html)
* [多项式回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/多项式回归.html)

* [正则化线性回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/正则化线性回归.html)
* LASSO回归
* 多任务LASSO回归
* 岭回归
* 弹性网络
* 多任务弹性网络

* [迭代求解方法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/迭代求解方法.html)
* 最小角回归（LARS）
* LARS LASSO
* 正交匹配追踪法（OMP）

* [贝叶斯方法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/贝叶斯方法.html)
* 贝叶斯岭回归
* 主动相关决策理论 - ARD

* 广义线性模型
* [逻辑回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/逻辑回归.html)
* 多元逻辑回归
* 泊松回归
* GLM模型

* 其他
* 随机梯度下降
* 感知器
* 被动攻击算法
* 稳健回归

## 聚类

Expand Down
35 changes: 32 additions & 3 deletions _sources/docs/回归/index.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,39 @@
"\n",
"\n",
"本章内容包含：\n",
"* 简单线性回归\n",
" * [线性回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/线性回归.html)\n",
"* 线性回归\n",
" * [一元/多元线性回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/线性回归.html)\n",
" * [距离和范数](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/距离和范数.html)\n",
" * [最小二乘法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/最小二乘法.html)"
" * [最小二乘法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/最小二乘法.html)\n",
" * [多项式回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/多项式回归.html)\n",
"\n",
"* [正则化线性回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/正则化线性回归.html)\n",
" * LASSO回归\n",
" * 多任务LASSO回归\n",
" * 岭回归\n",
" * 弹性网络\n",
" * 多任务弹性网络\n",
" \n",
"* [迭代求解方法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/迭代求解方法.html)\n",
" * 最小角回归（LARS）\n",
" * LARS LASSO\n",
" * 正交匹配追踪法（OMP）\n",
"\n",
"* [贝叶斯方法](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/贝叶斯方法.html)\n",
" * 贝叶斯岭回归\n",
" * 主动相关决策理论 - ARD\n",
"\n",
"* 广义线性模型\n",
" * [逻辑回归](https://tianxuzhang.github.io/introduction-to-machine-learning/docs/回归/逻辑回归.html)\n",
" * 多元逻辑回归\n",
" * 泊松回归\n",
" * GLM模型\n",
"\n",
"* 其他\n",
" * 随机梯度下降\n",
" * 感知器\n",
" * 被动攻击算法\n",
" * 稳健回归"
]
},
{
Expand Down
103 changes: 5 additions & 98 deletions _sources/docs/回归/正则化线性回归.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@
"\n",
"其中，$X$ 是输入特征矩阵，$Y$ 是多个目标变量的观测值矩阵，**$W$** 是模型的系数矩阵，$\\alpha$ 是正则化参数。\n",
"\n",
"* 第一项 $\\frac{1}{2n_{samples}} ||X W - Y||F ^ 2$ 衡量了模型预测值与真实观测值之间的误差。这里使用了Frobenius范数来计算误差的平方和，同时将其除以 (2n{samples}) 进行归一化，其中 (n_{samples}) 是样本数量。\n",
"* 第一项 $\\frac{1}{2n_{samples}} ||X W - Y||F ^ 2$ 衡量了模型预测值与真实观测值之间的误差。这里使用了Frobenius范数来计算误差的平方和，同时将其除以 $2n_{samples}$ 进行归一化，其中 $n_{samples}$ 是样本数量。\n",
"\n",
"* 第二项 $\\alpha ||W||_{21}$ 是指系数矩阵 $W$ 的 $L2,1$ 范数，也称为分组Lasso范数。$L2,1$ 范数将每个任务之间的系数向量视为一个分组，并对每个分组的L2范数进行惩罚。这鼓励模型选择相同或相似的特征子集来适应多个相关任务。\n",
"\n",
Expand Down Expand Up @@ -353,15 +353,15 @@
"\n",
"弹性网络的目标函数表达式如下：\n",
"\n",
"$$\\underset{w}{\\text{minimize }} \\frac{1}{2n_{samples}} ||X w - y||_2 ^ 2 + \\alpha \\rho ||w||_1 + \\frac{\\alpha(1-\\rho$}{2} ||w||_2 ^ 2$$\n",
"$ \\underset{w}{min\\,} { \\frac{1}{2n_{samples}} ||X w - y||_2 ^ 2 + \\alpha \\rho ||w||_1 +\\frac{\\alpha(1-\\rho)}{2} ||w||_2 ^ 2} $\n",
"\n",
"其中，$X$ 是输入特征矩阵，$w$ 是模型的系数向量，$y$ 是观测值向量，$\\alpha$ 是正则化参数，$\\rho$ 是混合比例参数。\n",
"\n",
"* 第一项 $\\frac{1}{2n_{samples}} ||X w - y||_2 ^ 2$ 衡量了模型预测值与真实观测值之间的误差。这里使用了L2范数来计算误差的平方和，同时将其除以 $2n_{samples}$ 进行归一化，其中 $n_{samples}$ 是样本数量。\n",
"\n",
"* 第二项 $\\alpha \\rho ||w||_1$ 是L1范数的惩罚项，用于促使模型的系数稀疏化，即让部分系数趋向于零。\n",
"\n",
"* 第三项 $\\frac{\\alpha(1-\\rho$}{2} ||w||_2 ^ 2$ 是L2范数的惩罚项，用于控制模型的复杂度并避免过拟合。\n",
"* 第三项 $ \\frac{\\alpha(1-\\rho)}{2} ||w||_2 ^ 2 $ 是L2范数的惩罚项，用于控制模型的复杂度并避免过拟合。\n",
"\n",
"通过调整正则化参数 $\\alpha$ 和混合比例参数 $\\rho$ 的大小，可以控制模型的拟合程度、稀疏性和复杂度。\n",
"\n",
Expand Down Expand Up @@ -424,7 +424,7 @@
"\n",
"* 第一项 $\\frac{1}{2n_{samples}} ||X W - Y||_{\\text{Fro}}^2$ 衡量了模型预测值与真实观测值之间的误差。这里使用了Frobenius范数来计算误差的平方和。\n",
"\n",
"* 第二项 $\\alpha \\rho ||W||_{2,1}) 是L2,1范数的惩罚项，用于促使模型的系数稀疏化，并鼓励共享特征选择。L2,1范数将每个任务之间的系数向量视为一个分组，并对每个分组的L2范数进行惩罚。\n",
"* 第二项 $\\alpha \\rho ||W||_{2,1}$ 是L2,1范数的惩罚项，用于促使模型的系数稀疏化，并鼓励共享特征选择。L2,1范数将每个任务之间的系数向量视为一个分组，并对每个分组的L2范数进行惩罚。\n",
"\n",
"* 第三项 $ \\frac{\\alpha(1-\\rho)}{2} ||W||_{\\text{Fro}}^2 $ 是L2范数的惩罚项，用于控制模型的复杂度并避免过拟合。\n",
"\n",
Expand Down Expand Up @@ -484,99 +484,6 @@
"print(\"模型系数：\")\n",
"print(multi_task_elastic_net.coef_)"
]
},
{
"cell_type": "markdown",
"id": "42f8c990",
"metadata": {},
"source": [
"## 最小角回归\n",
"最小角回归（Least Angle Regression，简称LARS）是一种用于线性回归和特征选择的迭代算法。它通过一系列步骤逐渐构建回归模型，并根据变量与目标变量之间的相关性来选择特征。\n",
"\n",
"LARS算法的主要思想是每次选择与目标变量具有最大相关性的特征，并沿着该方向移动。在每个步骤中，LARS将当前的解投影到残差向量上，然后确定一个新变量进入解集，并决定如何移动。\n",
"\n",
"LARS算法的步骤如下：\n",
"\n",
"* 初始化：将所有系数设为零，并计算初始残差。\n",
"* 选择特征：找到与当前残差具有最大相关性的特征，并将该特征添加到解集中。\n",
"* 移动步长：通过沿着这个特征的方向移动，使解向量与该特征的相关性等于其他已选择特征的相关性。\n",
"* 更新解和残差：更新解向量和残差向量，以反映新的解和新的步长。\n",
"* 重复步骤2-4，直到达到所需的特征数量或达到停止准则。\n",
"\n",
"LARS算法可以用于拟合线性回归模型并进行特征选择。它能够处理高维数据集，同时具有较低的计算复杂度。\n",
"\n",
"在Scikit-learn中，你可以使用Lars类来实现最小角回归。例如："
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "6ed5133b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"模型系数：\n",
"[ 0.02913523 -0.00313884 -0.03431681 0.29380121 -0.13901173 -0.08586778\n",
" 0.11000887 0.00837283 0.08400827 -0.11083981]\n"
]
}
],
"source": [
"from sklearn.linear_model import Lars\n",
"\n",
"# 创建Lars对象并进行拟合\n",
"lars = Lars()\n",
"lars.fit(X, y)\n",
"\n",
"# 输出模型系数\n",
"print(\"模型系数：\")\n",
"print(lars.coef_)"
]
},
{
"cell_type": "markdown",
"id": "36d1ba79",
"metadata": {},
"source": [
"### LARS Lasso\n",
"\n",
"LARS Lasso（Least Angle Regression Lasso）是一种结合了最小角回归（LARS）和Lasso回归的正则化方法。\n",
"\n",
"与传统的LASSO回归相比，LARS Lasso具有更高的计算效率，因为它使用了逐步向前的方式来确定特征的顺序，而不需要像坐标下降或拟梯度等方法那样进行迭代优化。\n",
"\n",
"在Scikit-learn中，你可以使用LassoLars类来实现LARS Lasso回归。以下是一个示例代码："
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "a99de35e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"模型系数：\n",
"[ 0. 0. 0. 0.21248063 -0.00992147 0.\n",
" 0.00512876 0. 0.00215299 -0.00265399]\n"
]
}
],
"source": [
"from sklearn.linear_model import LassoLars\n",
"\n",
"# 创建LassoLars对象并进行拟合\n",
"lasso_lars = LassoLars(alpha=0.1)\n",
"lasso_lars.fit(X, y)\n",
"\n",
"# 输出模型系数\n",
"print(\"模型系数：\")\n",
"print(lasso_lars.coef_)"
]
}
],
"metadata": {
Expand All @@ -595,7 +502,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.5"
"version": "3.9.12"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit 187c7e1

Please sign in to comment.