Softmax与分类模型.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"graffitiCellId":"id_z2fw17x","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"64AE0FB342E9428A8DA9C734B357BBB1","mdEditEnable":false},"source":"# softmax和分类模型\n内容包含：\n1. softmax回归的基本概念\n2. 如何获取Fashion-MNIST数据集和读取数据\n3. softmax回归模型的从零开始实现，实现一个对Fashion-MNIST训练集中的图像数据进行分类的模型\n4. 使用pytorch重新实现softmax回归模型\n"},{"attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAVAAAAB1CAYAAAAY/rnoAAAgAElEQVR4Ae2dCZgU1bn3T3dX9wyLMgqKEaK4ogR14hIXRHAFRRSixhgRJglJzBUBNe6ifLLojfoJgrniBrhlMUYQvC7RMKhg3CLGBRci4AKioojAzHRXd9/n11NnKIrq7lPr9GC9zzNT3V1VZ3mrzv+823lPTEQUccBfDtQKITpZilxo+b6tfe1n6dA3Qogllt+2pa81QoiDLB1aKYRYYfkt+hpxIOJAGQ4AmLfG4tq/hRD5Yn+xeGK9EGKmEGKEEIIB2FaJtp9OX4w+Fe1zPFn1NrwRQlgBtq31vYcQYowQ4tFiz9f0O9dwLfds8xTb5nsYdTAoDvSPxbXb8jn9gPadd03v2f+s1K61/UWqY43ovHetqOrYjJFfLlsi0hvWibXLlogPF/ylcc07L1bHEtqGfFa/WQgxVQixLqgGBlDudSIWv1jkc9vX9OiV3n/Qr1L0FaLvUJPRVz6veGGO+LD+4fSmtatSIhb/SORzAMucwoVt4x8geJ0Qok6rbp/ZtfZYsUe/M5Pb7dJDyD+68e1nK1r+Vi+pFx8ufDijN25KCiFmCSH+37YsmUYA2jZe5EpqZY+Ylrw/r2eO7nrA0Xqf0dO0LgaIqDQSgHnrr1PEG3++WdebGjaJfA6JtNJBZUgsoU2La8mda396War3mWNbJgiVPgMwr9x7TXrZ3x9MxbTkC3k9c2EbUPEBzvEdunTTD/3lRK3nwDqVrrZcw+SxaOoofeOXn2pCiCkGkLalybKlL6U+RABaijvROSsHamPxxHOddutZdfSY2wsSp/UC1e8SSF+bjYBSkFR+rnpvyNcVgGTfASPEIXXjC5KX2/pXLakXi24bnV238p1N+Vx2eIVOHDWxuPZcIpXar8+Y25NOgdPKm/eenFUA0my6aWk+px/TxjQOa3e2+h4B6FYsiX4owgFEkJl7n3huus/o6Smpohe5Vvnn5gF2QSarZ17K65nBFTbAsNnW9bt8pvAKJGaG1N9YJ95/ajY/MWmg5lYK1ca1qserO3XuMvDGx1NONItSHUACf+LKQelvPnqvKZ/LAqLbjIMtAtBSTz46JzkwBAfCISOuK0hh8ke/jthJ543um9UzTS/m9Uxfv8r1WM7MZLuO5w2+7fmEX0Bibs+bf50iXrz9In6qFBCticUTH9Xs3qv9abc9l/BrgpR9RuNYMGlY5uOXn2zM57IHbit20YTsYHSMOFCEA7WxhDZv35POSx01ClOW/9R+x13E9w8/Jf7BU7O+l9MznYUQT/pfi6MSxwohrjjlpqfiXXsd4ehG1YtluavfWMjkNFcI8ZnqvQFcV1DbO+3Ws+vp015I+g2etFdLVYvvH3FKYuWL87Wm9V+dIPK5PwohGgPoS6hFRgAaKrvbXGU1ca1q0c69Du80cNJj8SBbD4i279ItsXLRXBDrDSHEu0HWV6JsAG0mavseR/MxOMJz/82qZemvV7wzXOTzAEprOVlu1KranTLwhseT2+8SXPQRILr38efE3350Wpe8iB0octmHguNuOCUHOijC6UJUS4AcGBvXtJ0HTp6HJzVwws7Y+4wxQsTiGAhbJVYUb/vufU731eZZinHYk9vt0LUKj3ep6wI8R/zVmIE3zHcUTeG2PUi3AybPS+T1pkFCiObYL7eFVcB9EYBWwEOo0Cb0iCW0Sw775UQtCJWuWJ/xdGtV1e2EEKjRYVNdPqt3D8pUYdcZeHvYyEkpY4FBcOKfXeVCiJiWnEA4moxjLXKZrz9TF3USR+xrwa1QWASgrcD0NlLl+OpOXbQDzgwXxwCUg86+NAl4hy6FxuJTkYAJEg+TkLwJzG8FKbQ/8byH/WJCKBqGmafHXXW/xiIMohzMv7e1zxGAtrUnFlJ741rVibXnXF4dUnVbVEOgej6rdwxZxevPCqOwJwzZ8YPOvlRKofKnMI51AHeY0qfsFJPUbkcMyoT8jGX1vh1VAZQp2c5GwwoSlRkE923zmjf7plM+64VL/bWKTcy+udv8r7U5vWnX7xnLE8PuLVJo115H4qEN1ouzZceG1OzWsyFs6VM2wQRiofU5ltCO36vfWQB3q1C3Q05IxuKJH7dK5T5VqgKggBvL7XY3AE7qNwQAA6AYgq0gym9mwAU8zQBYb2k/97Mk5dgif5wrBcCW4qKvHjkwhPXtQcQ/qrZrz2N/Uh3m4IoltDO6HzYQ22urEMBtqPFhOVZqsffuHnCkQSlm9jh6iMjnstu1ZSlUxfbBAyXil9UDABmgSTYa0lctF0J8LYQoLKvwuKqCshYUYTjgHVF4HKjt1H0fR5IJgdIEhrPqBDrygluFFwAmSYcxuELpNWDCgHZCLACgzyRLob302YvDrfshJ6bWrXgnLEGhINA4fUYsAFixaG6hz/sOrBMHEDXhkkzSPn22ClUuSw33NhUJFEmTUYFEiaSIBApg8hsS4w8NVQugldKpm17sUUT6pI4o16Abjrq/p8akUiqV8vQ1Q0TnvQ4Sg29dUACS+WP7FzITKd1c+qKwJLLSrbCcZaKgj4DmGXe9LlIdOokXp3tzuAG+yQ6dsP2GQf216o5ZJxW9Nmu8YD3/SRMeFafeukC8/8TMQsYpJ2VYr+3Ubd+NFu3UeklFf1cBUGYHpE8JkLwlACV5EAFRAJbzBAE3ix/NXUbtR6Lkj+Sr5EWU3+2YgmRLWfIa69Fctt390W8+caCqUxezuaVsqaxnh6QDBqkGiYwUdmaS15l/K/bZKYAXK0fx9wJIk4pPlRZPHyuOHDWlRcpGel39xua80fSVRCkAjipRf2bjN4eoXu/1uh169GpwUgbSZ/8rZhWkbMCePstnjAby5iNTC32WWohK2e27fK+DynWVeo0KgKLX4NwBEAFBpFB+423hM3+ArFn1YOSwxheVnz8kSKRW+d1uquYegLmYHbRSebjNtUtv3LiLk05t+GxFSz5MeR+DSAISg+vpcUM9Syuy7KCOJpWybBUAB5OEmTp2bbY0AZ7kxSQgH6lNFVCc1G+u1+3nRLJK2ebLREB/zSYKnqskqYF876B+gmQpDsmL5uqwKn8vV7GBorrLP6ZT/vheKsu2dC7J1sJpAFJOxxJ0m0WXZgcT0z/2VCRPO6K+zVO83RXRb75wIBaLIe13dVuYlLqQRBlkDCjAwTz4ypVtHpzlrvXhfEG7ARRVJV/snmZ6/8lZLffST6RTjoAOAKoCjtSfqGq3JtvkSDA0N8PR54av11CRsskgvZGdSpqJ57Ny0Vxx4gQS0ItCkhl4x++qEwb35TKZJovm2lxBG/mvAqCyKwAcziOIFw5pEmnRjkqBK9dbZxymcsBUWvGRUPm+5VsaAagdr33/TW/c1OhkEBDuVJBA9sEcLsTiaWMKqh6fAZEBE+cUJDEnYVFSNQxpcDk2D+FAwYF0yIjxYvUb9QVVFtCEpDOKiYR+OFnZlMvqywqFBP9vxTefvK8MngVw/Pbrgpreea9a8drs8QUJWzqhOI+K/9YjU8VJE5Gf1Gj9muWYAtssqQAobwU2IgAUzvCyAXjFJEU3zKBsJFT+AFHAU6r51InEy7mIwuHAkvWrPjxaCKGUbIbBw6BBbYUGT6nfSuICTAiQVyWThOcY3FTrsF7nZNIAFJtV9QUFU8WpU+q3kLA5R1Z2J2CCVz+vZ3RruwL6XuArEqOqZnDG3UsEkvbqJQsEqQ157maSNnDeAwms5vN2nxvWrmaxRpsd2yoAKt96QIwRAuPpMNIn3+0kRXhFJm9JGIewoUrJ1KyOA5hStQeYqQ8VX14rw5sQb0IbTLLh39Hjks/efF4JPCV/GEzWASXPcQScVAcq1wM+sbj2Vj4XFp6IucsXPjKo58A6lTFR6FqxJMs4mJgAkLyd0KevPcPKHGc3Oalgy2sLoAXYSYl5y9Nbf+P5SZA0n8XOi0SOmYI/k/Zgvmyrzzxjg9osgKo4kWQnrUckUwAN6RFpVIKgvA57pfzDoYQTSX7HqSQJ0GzWfZqBk98BUQAa0CRXIvfLa+R90TE4DhTebNML7qkmJCurw6VcgZ+8+kw6n9MfKXedj+frVy1Z4FmdZKJAjf12zUox76JjBWCqQvBIb9zIRmwtqKJyn4dr1sW05CI/njGmGezcRB0ApqqaBloJk2RbFoycZKTHbolNkj9Ar83OGh5euu/MrbG49uY+J57bm7CVsImBNf+ignmdCTSs94z3ezkxjqoSmZ98AWjfnnP7pwT0+1lumbLGatUdfv+LJzYA3J5IOo9UVXcqe+DM7ulNaz/9H5O5zlMbWuNmJxIo0qZ06oT1UrcGT6I6CfLN6bewb48Tu6BfjHt15rU60lGI4EnTeb8XvvGnm0LPkg6P351/Vzqf1e/2i4eK5czKZprSSI1eCfXeCXhiI9609lNWu7VpzdKJBOqVx9H9bYwDSKG7HX5yzwGTH/Msoah23SR9ssHcRGMRBhO2nLylqYgoEL+pIIX6vYlcuUai/n7wzINBSZ+Y2FjIgtbIZ/74DDH+/5ls1/GSn/3l49DyviKt/qWuV2PD2tUz2rL0KRnYzMrof8SBrTmAm3VBWGotA+uxC49Of73i7RdNkR8AJgAgie0+ANMtXcDyrPfjrPadu51z1qy3fNt5tFSTsH3+7VeF8K+gNpcDMImvthJ8LPAwFk98vM9J53XsfzmbkAZPhrlio2GukBNj8BUHUIMTFT6A6qMiK5wDgNfsf0walmGgB031k4dn1n30LomF5VIWCZQMdkl7CSFeE0Ior6KRNyoexzasW/PFvDH9HK0TVyx7i8tQ3eeNOYZ6cLAGaWy+3VgNKOuX4Al/1+Vz2fMITyKOM2hCdcfJls/qo0xaRdDVBla+o1CVwFoRFVzJHJiTF2LAyhfm7tLzlF8k2BgsCEIq+eDp+xIinx8uhMD+KQmb5J+EECcbuzg+IYQ4XghxpRACGxrIDuh6JSS1AQWVMp/7Q+P6tYO+Xb08FZRDCWn7fy8dkN305eqlIp/DVPGYEKKnEGKNTzt0dhFCXC2EeFgIcbAQ4j0hxPbGEcnTLPmxgV/6k1eeOp5oiZrd9vPKS9v7mYSfvJzHWFiEEzxa27bC3x8jCdRffm6TpeX1zKnpDetWPFzXOx2EJFr/3z8vSCVGuBoAwuBGn5RqupREdzCcPYzw84UQJOP92LCV7uiC+dg8iU9mPSJqLiFErFf8cz6rD0MqWzD5vDRg5ycheRqmioZ8TsfW+7xRNzFPrxttof+0DWB3QjsJIW4y+ISUR/6KXYUQrHDC5ol0LztE/4nXJtYas8lschYgJfpNlPnYhUdnEql27D7q3WvldwNdlhc5kVwy7jt4W01MS85PaKkfHXf1A0k/JDOA6amrT9PXvP1iE4BlioHEdCAXUuAdB9hkCkU5+OUjQKRB0sKQeKcQ4vdCiNXyZIkjYAW4mAnwBFRkHbWxeOL5HXr8oN3gqQsTThYCmAs1f2YCQm3XGxuW5nN6X1NdACXStMx9S6w0n8nFqyKtkbsAqfzXQgjWl081+icXsFM+fYOfLMmm/9KZRKy1XEYNuF1HwDyp+vwgGR9qtEktMNaPikMoIwLQEJi8jVWBeDJi9yMHZ44afVtSJUmGXf8ZVP/+8826nm5ck8/qp1pClhjoSEWSAIFOQggkUAlu8pw8ArgAKUfaeKOlDHmdPFIHEtoZptwMdo6cHrG4Ni+RTO170DmXpdh0zg2QInW+eu+49Ad/fwCzA5MBQGLtCxI3i1LoL2OTCWGSzXWyDxy/Z/R7pBBigwGa7Hb5rfki47Pkq+SnvIRcvACrpLpYQpvefsddqvqMnqa5nSyJqFg0dRROQfpsx1tZX5s9RgDaZh9dqza8fyyuTcvn9N4s4WOAkcasHLAgfZHB5515Mxob132RzWf1mw3pygokdK4A1EYvGdxDLSBbjAGHCiHGCSFOQRUXQkww7H7W6ynvASHER4ZDijqkycB6LdLb2FhC+52Wqq7ab9BIbZ8BI5TiHsnezmofzAGxhEaoEqBdarUR/WYSeFkI8RODP0ihViLgnn6ikgOItwghpgshSFBcigBuwPYHxkVIqnZSIWCLNDqi6w+OamSLlR59Tt8qx4G1IiYK8qIunXdn45q3F2Mw/19DMv639dpt4XsEoNvCUwy/D6iUgADS2SUAKU3Y5cC+evX2nTXzsk3WhAOc6z5+j7i/6rhWtTqnNz1dRAIz9wTQAtRQ5ykfhyeguNR8UYnPvQyAOctw0ACk2Bcp5wYhxKWGuv+SEIIlo1YpzK5o2nRFLKENy2f1blp1h8xO+x2WJIBc5j7lJkBk/ar/6J/9+/nCunpjTT/OnOvtCrX8Rh2o8gAjkQaAPPZL+gHY085rhBDnCSHWGvZOVvOo5MBDgsdZta9h82TCACjtJjDZLNT88bGEdkI+q3eo2W2/hnY77tLOmvcAabNx3RdS2uRezAK8J/SFZ0hdpeqR9UXHiAPbNAeQkFgzzkCXxGcGPAOmXqtu/89EVbvPkh06vWoMHsAWaYbByHWqxD3cy8DHM49aeoLqzcZ1ewsh7jE89f8QQrwphFhvePVlUdIWKL8XO9J22gRxD5Ib/Kinr7GEtoG+G32GF1wPQEHc5ybQcjchBCCPk+sZIQTZVT4RQowWQjgJiQA0/2OEM/EZYJN9MZpY8kBfsJNyD4BYn2y//TL+YlryBeM3zklbqrkw3hfar8pn873R54gD2wQHAEkkOAaDl6Qb3OsERCXzkObuNQDEzU5m2FlRb3NCiFcsACrrKHWkzbTdCeiYywNYuN8piBLaRBgX7eb+J4UQ7c0FK3wmNSHSH+BOeJNTAvjsgvFVywFwaTtluHn2qvVE10UcqEgOIEWZwRPpxS3JgeRWGkHqI/gcMFWNY+YeUsWhRuPZxlOPNPsvw4lUzpQlwZO2uwVQVFju508FRDFB/MUAzg+FEL8SQgwy1HYkUiRTFSKuljhZwodw5jgl88Tp9F55vdRaZP/NqS7lNdEx4sA2yQEGENIL9kg5AJAo3JIsAy87Zbsh1HgAkHag3hcjNizDFkdYz28tF1E36+m/MuyqAI0dIJvBk7bbqaiWom2/mgGUcoqB6AFGXCoS5weGxGZul1TpsX8WotJta2v+kUgE6nIL+pSCCUU+M7eTHvXLMmR58COiiAPfCQ4AIjgzAC2ntjMrg+RA4uhlVwOC6XGqAMTYOa20jxDifcNeWFhsbr3A+L6dEOJyYwUQZQG0VcY5qbqaJw+3A98MoLI8M7DRxnkG0OAoIy622EIXJMlphnSKQ8wMsDQd2yiOMSaOnxbpt8rPSO7m5+W170xm2GExS0QUceA7w4FuxkAiVAZplIHlluSABIgpC2eLW+pshPsQxiMD7ynrNCMmEglVdYUSoHOhAcqrhBCXGCt4ADlUfSIHaLtbEEHi5f7FhtR7tiGBE3aFXZNzJBcG8IoBp5VPZxoOMfopVXoC6nHeIVkfbr3BwXcmD9oE2POcvPQdExDPm8gByrnCQTuiSyMOtHkO4MVmiaXXPbwZSAygvxurhQAR6aV2yySkMaRjvNO/NAAZ9Zf15apAZK6b1H3ESaI+oyb/wWizVOXdAih1AHSYCuABMbB41fkMuBDQX84Wa26n/IykjU2Utv7GkLhZ976nvMDlEQBl8sDh9qzRTi99l83ArEAUBEtOI4o4sM1zgJAXnDb/5UNPAUsG5WGG+jnQhzJlEQAmYITaSryoVwJ8f2Z4jgFkHE9+DHoSoSDh0lYkRbnTrZf2MomgHlMmEmMpu7CTejBv4IDi2aMp+AGgJDT5XAhxh5OGRNdGHGirHCDwGtug1c7mtT9IY34NoqOMQUk7NxkSrvKWvSU6gomAbFB4sJHEKBvbo5ttN8j0RCyrlDgBZb8mEDz0THLwFOmO4y4l+qV6ChMB7XUT+lSqDhLBoDF4MTGUKj86F3GgIjjQxxhADCS/iUFE0g83aqu5LawoYjA+aDh/WLX0qeFZ92oeuMwIf8KuCAGCzxlSGWFUds4r49KWA/GnqNgAEQHnJxlnsHvy54WQkgF0wBM+QKj0LJuEt2a7sHHa0eE+o+2OblK4mMn4bSMuVeHy6JKIA22TA6iYJC8OgpCQkMLcqoWol6yzRkLEZmmmnQ3HD44UpFM3BLDj5X/I5mayKAF+ABdB7oQeWYlwJ3gHcBJtUNglz3QR0if9d+uRRsKmDSzhBKTNxBJQgJWJhQQrbrQH+o9zjvuDICYSeOMlSiCIdkVlRhzwhQNy5YxbAFJpBCtj3Kjx+wshCC4n5ZsdeFE3XnUC0QmgP1elMZZryNPJAD/S8rv5K6FH5BIFCAlBwqPOmnUyvnMvzrIjzDeYPgNQhCsRZO6UiIpAgvtMCHFIiZvNXnqnKj2TBH0oxt8S1SqfYvLhGcqwMeUbowsjDlQyB1AN8eQCCkES4SxO1XhAQdo5SXFXjlj1AsDhBHJiLnjKkGLLlc95MhuhngM4/KGyA6blCDMGAA8gqhKACXAC0ir3uVXp4RemkCCJVWFoEFFYU5BcjsoOnQOoxKineOCDJMoH3IpJaea68TSztw/tutYhGOLpRtV9XHEdOfsu0S5yWJYiJhrCkphsAE7CkpBI8VwjXSPFliLWtJNRn9AeFUJVpx+o7k7Ww6PSs/JJqvQqEwnSManugqYorCloDkflh8oBVF8kHGI/wyCAplwwPZIWwezEOjrNxiT7gLpNLCvZmNjiohQBHNRVTLUksckvjDRzAC2rfsyqLl56khojKePQKRUcj/SFrbbcJnmslqIueAVwuyFiWfHSYzsupdKTMo8JgZCroCkKawqaw1H5oXKAbSGQclTUQz8aVk6Nx5MMmAGgXtsEcAKgACmxqHaEZIfzxE4qJMieYHXCpZCESdhM0o9iRNyolLBYVkqgfyFHqOkGVkqRJQp13o6okwz21Ee4kldSUekvNnjgxvnkpn1RWJMbrkX3VBwHGMyAB+urwyLWtCPt2KnxeIABjhkuMwnZ9QGARJUn4B6Hj5UYzNT5fdMJJNELDK88ajArn1RCmGQRLOPEFstEgGefJaNI+pJwpLEzplW1xsbLhnPkICCEyi8qp9ITR0rsa1gUhTWFxemonkA5wFp3BjlqVZgEeJglPoBDhiixGshvAqgmG8BtzRKPdxs7JgTIkXsUZwrOHjzmqLduiaWwxGtiIkEKRvomHItJBPXcHFjPUkxsq3ipS0m5btvCfaj0mBngtcwZwLNnkgiC76XaGoU1leJOdK7iOYB6jPPDbv+doBvP8k5UXIANWyKgQZiS2a4YRBuQQJFESbEGWBKTijQMkP3OADp4crcpWYcf7aAu9kainyQZBsRZb47kB7FCB7socbh+rwIyqmg5wGPstLQFcwkhXwBo2JMoDYrCmloeS/ShrXEAOxtbRWBzC5ukGo93nbAWJCIkszAIW+iXho0V1Z7PXxjAyl5DbpZtqrZbOqOYPLA7A96YTwB1HFNmNV+1TDfXmVV6wJStT1qDorCm1uB6VKdnDiCFYPdzE3DuuXLDvknKNNRYnFhhEwAOcAJgqOqs4mG74LAIrzqeeiQ/2oDDTKanC6sN1EPYFvW/Y1Lpw6yfuqTTzY/ELWG3ParvO8oBJD68061BAAWAgQSGGhkmYWslKz3gjaoOgCMB+5EhyUk/iHFF4gRAmcjYPgUgZy2626WeTuqX1xK2BIDKVV4/kidCPEZhTSEyO6rKOwdkwhCZ5MJ7ieolENMJeLFyRy6dRBoMmnCY4EQiJnKDsTUwzjPiP3GkAaRBrQG39g0bJ7Gw2DwBMBlYf6JhE6UtgGuprPrWMt1+p/8kdcZ8wv5RmBJwolmjA9yWr3pfFNakyqnoulbnAI6KsG1eDEgcR0hb043YSH7D68zvQRFqIUsUAU3Ac5KhqmK6AKhYgQTxHQmQtfRIh0ERK7GQutnmQiZAtgbWE971hCEZsgLJLtzLr/YheTKxSCJu1eqll+eCPEZhTUFyNyrbNw7IfI+kgAuLUJtJssHAtKbJw/5FSJPfRDq6W4068XqjttMOSS/apJYjiQpS4ctCCPKC+k1kZ0L6Js7TnAC5WGA9EiiSKEBPSj0/40LpG+8A6rsVoLGP4+QC6MNU6aOwJr/fuKg8XznALI/EQR7NsAgQYCBSLxmVrMTgZRD7pcbjCMIhhH0VFf0aG+/+wUadbBlsJfKJsrUHwe9+tYk6WFGEvZNgdbuoBxlYb121xL3YRO837kd7+LFPKjbOO6IP7NR1q0pv5VNQ36OwpqA4G5XrmQPYmXCceF0eqdqQXxsOGraeKBaixOAlOxNqrBci9IgQJGx4gALlFdvPicTIAKQdcNAGnBrEZ7IayOtWIdQhbaxXleggYI2kWSpXJgH9dxp9xGvOhm1ell4ihRPvWoqwh8JT7KPFnmGp+52ei8KanHIsuj4UDgAmrIZBrQ2aWAaJlItNkeD0coT0xSZrbgiPPiDAxED/qK9Ukg65ZQc7cJYigAmgxWZb7tpi5bCElPSASMNIjeUIeyfOpXKElE2CEdbTI9mzVr9YEpRiZeHIArDZzbQcocaj0vMXhkofhTWVeyLR+dA5MM5Y8y6X7wXVAACNwGw2EVNNzCxXAzlRmZHGWGYJSLPsEklJJQidLTsIW1JddcN6eFRv1sLbqdfF+IgNlvydSMOlEiCb72c1lJ1N0nyN+TMOMoLwyWUAD0gIUmryMN9LZikkS1WHWZgqvQxrcpN42tzH6HPEAV84AGjihQ46YP1kU4hSqdRp1k45UeOxByLdAmqo4QCcqvRFPdyDGuyEWO4ISLFBnNn5U6yMA40VXqyxdxIcT/vIyYkd0AnhHGOCZGEAf9h9zQ4zu7JY+8/mgU4pLJUecxOTSbQJndMnFF3vOweI9SORhYqE5qZyVtUQHoRK+P8dSmqyPtT4UuoryTVIJYdKTWo57Kt2zhhZnt1Rxp26SdRBJibq5a9UViYSIKNaEybmZpdQgAOp2o2dGjMN5qJLy7IAAB02SURBVAa2UJYZtuxW9yB10kbrvlJ2PLP7DTUexyAqfVC5C6KwJjvOR7+FzgFUXeyDxPcFQUi3OFyQcFXsacXaIDdcI3elmRigfzPAeZmR1Nit44QtOxaaC3f4GekTKRRwstv9EhUagEfCddtG7KZED5RLOF2q6Ujk7OvOfvGEjpHo2by+H8cYk52XpCU8dxxLlB/UuxWFNZV6ytG5UDiAukuwutuM5qUaiSRCMhIkESvwlbrP7hxSEYHu0htP+BMqJmoccaJ4nL30QW7ZYZcP1K49xX7DDoo9FBMC2ZUgwBLQBDxRcb0SCwsAJhVzQam6aCvp63j+TKI425hQy0n7pcq0ngtapY/Cmqwcj76HxgGkNwCo3D49bhqE7VGGtxQLF3JaLk4D1ufLVTjYEM/xCJyyDZgx8NJ7AWFZFkcpbbJ2HXUdldivQHekO8qTk4m5Xjef6fNPDKcWwA84+xmNEaRKH4U1uXni0T2+cOBpQ+X0pTCjELy8qG5INH5IW7JtBNSTYATABzhZsYRTxQ+SW3bgXPGTSECM1MlEUmorZDd1IiUi3Tvx/KvUwzODx6jwxOeq7CSqUi6gT4IawBmp10+Kwpr85GZUlhIHjjMGCqtu/CLUPwK4CXz3Kx6Qdi4w2gqAMgBxpPhJ0jHDIPeL8A4TooR5AaBj9RKrmPwilcB6N3VhHiB2lCQmSM6AKTuLsszUKzHhSZWeHUH9CryPwpq8PpnofsccIBbzr47vKn4DDiJslDhR7Dy7xe+0P4ODQO6rTmYmuawSm5fM0m5/p/NfkWhZCukXERSP1EmQPNItwfmsn2cdvWrsq0pbVAPrVcqS17xmcVAh+ZNUGiBlZRJRBF4pCJU+Cmvy+lSi+5U5IBP1+iER4SAhmxED7L89eJdl4wFKwpUoDzC22g1pO+qlkzhSWbbdUQbp+yUxE29J+26ymBhwgpHJiRAkv5JUOw2st+u/+Td2KIXvdlsXYy/HNINJglVhZ3u0F/ut0vMeMhGWCnUz9zX6HHHAFQd40VDRWBfulZA0ATkkT4LkvRAJi0mEwQBGwrQbxJSP2kd9fqnxSOGYBrwSMackAsEJM7xEYax3B2BZ/+7VYSUD69m/yQ+Cp4Rg8Y4UIxYq4BhjIsB7T8Z6t3ZY2k+OVXiGSq+6SqpY26KwpmKciX73jQOE1hCTubPHEpHYsHVi88T26YYYQGcYEg3Aia3NmjrNrlzUeNRXr4QUi0RVCvBU6iCciBR0pKJTsRUyWeBFJ4Wfm2B6c5sAPQDZjyz1OHlUty5mBRWOLJa9EjTPe+V2IQYxs7xLmJW8Bt5HYU3mtyP67CsHCCciCJvcl15orCGBoNK5kRqQvFDFyXQOcBKW5MTby73YGL2q8RMMfriVoOAhCZBJfgyI8FmVyLXJ+nSWZpr3m1e9X14nA+sBMy9EOfCUsDAnROISVpcxIRAGRi4BN84hnqUfXvoorMnJ04uudcQBgBMAdRuXyX2EtjDQWMnilADOYYZnGuDEyeJmWwoGKG3wosYDmqwLZ4mpWzrakDqxu7lZtYMWgPmAdrATqFsijMdrYP1QQ5VWTaJibSs2TXiJFI6zjHetxnpRme9+qfRRWFMZRkennXOA7D+o7gS4uyFWErGiiJAcp6CHTQ1bGaE8qJskqvCqqqHCe1HjceSgvsMXN4TaT6wrKq9qxiK7elB7sWEyIbhdBcUyTK+B9aTmYx96r0SSEuyahHCRL/VmFzw2q/RuVrBFYU1en2J0/1YcuN1IdFHKQbDVTcYPhCgBvjh3nMRKIuWRaR2nFWCFF9pNog67dhGMDei4URcpj5Act2FcSDhI0H7u1YTERpnsP4Qk5pS8BNZTH4Dn58IHTDusxsJMgZ30Dw4zT6HS877hMLRu86LCG7QT+Blla1LhVnRNSQ7g5MHT6VTCAQBZ0ofUONGB11gmq8AuCHA+5JOTw9xJqcZjD3VKcssOu4QfpcpCWmQPIoDbTb2lyuYcz4fkysRdYpN0Ql4C61klBdi4dQaWaifvAmBGlio897MdvAtM9tJLzzYsTuzt3BuFNZV6MtE5ZQ6Q5o0AaSeEBEAAOzYt1RAlAGa0oeYD2AyWUqndnLTH7lpUeDchPKirDC4nhKpPmBX2vSClGswjpBZkzT9xmU4IfvDMnEqwJF2mviAJQMPsgdOMSRUHpKoZx61KrxrWRDw0dTidUIPkV1R2wBzAQK/y0Ml2jnThZAUM15IxnrASlYS/SAbkmSQUBbvgPQFJM1aWItnYqfGSL3ZODLllB9tcqBJmB2y/xD3KrYZV73VzHcAJoAGkTuzNMrCeox3VGu8MRzMRDeHFmWYuq9xnHIlI2q8b7yWed5XQNbcqvTWsCbAcYdjhGRe2f/GE9rGhfUWgWu6JtoHzAEHhocfjCWxC5R46cYaSWA45X35ROF5qqPukuSuXyR2v/OUG2AJk2OHMeSUVqvN0CYMK88Ljxgtvz5dmnhGoDQ+vNYLFVVVkVkLhDGFdeLks7p46Y7mZ9qHKo9Kbn6flsi2+Inki4SGJApLXxeMak6AtX/g9ltBYYsr5IVuUFM4XVp0tNuonDysSYylCikVa5pmj0qs472RYE6vkcF7mq9q1Tx/af1B65FW35ifdt6Dwd9ezy1s+X3X7o/nBw8fkd+y6K+90PhaLs7iAd0eJnIr/SoVGF7niALPldTJ7zaH9B2Vqjzohucd+zQLEzt16iI3frhMb16NlC/HPZ+aINxb/Pf3RsndSsVh8fT6fI18mGYHYLhgPeinCpkgeSwDjQiHEXSUu5lpU9YuMQHDySCLBIIGGQUwoY+LxxCW5XHa7HXbaRR949m+03j9iVaYQHbavER22qxGff0q+YCGWv7tELFn8TObV+sdlhnrUd9Z0N19QuMr2Hw4wJgX4wh5BqJ5hEmMRwCC2EofV9QqVT5eRFjvstEvjD48eUH3ECUMK/ODePfavFcuXNu/PB39eenaueHnBvFwum40ntOQLWT3DUlS/cw2UazaLD8iEReIYzCQ40jDNAOx2hElJ7pxKWBzRHcWId+U5zAW77d0rPeyiSSn4oUprPlkhHps9Rcy7b6qIJ7RPclmdsVHSbBQBqCp3g7uuABAMms5du+nDxk7Ujv+xegYwHvo/Hp0l/vyHiblcPtcg8nleslIPHYBF2mGG5+1CvbIjJDBAE28t0ukMIQQeaVTNsGhIPKFN05LJnc8YeVnquKF1omt3tSX9G9avEw/fMVnUz70//c3XX6ZzWZ0QG/KANs9Am3uAmknGd0K+yLnJuvbWJEKuAAxicLEj4tW2Um1CS07L6pmj99j/h9kxN9yb2HN/q6ZuvWXz9zdfqhcPTh2nv/PaCzgOAVDCzspNMJsL8OcTiykQGJjcmOSYPAgTQ+K0EtoH6jkOQSY3u4iK2mSq6sGONTvuffLZ56d+Ogrlwx0xpmZMGCUnYSazootRIgB1x2O/7qqJx7XnUtVV+/9m3HRHwGltwOerVoqHbrtO/ONRfDmFHSoZFFYiRIS1zaxp5zPqipUAdLYDZvYFZFlHTwIRQl/CJNTwusHDx4hzLhwvOm5Ps9wRUsUDU67R041NS3M5/RgTiLK0ksGI7QtbnRPzh7vGqN2FXRqNgk3vsHFio5ZUl9CS03sedHjVuWMmaAcc3iyJy5NOjgDpjOsvSH/y4XtNuVyW8LWwpVGai3MJiZT3ERBH/ebZ48U3E+8i0SGYkQjTww6PDR7qH48nHvv+3r3a3/Dgcwkv74pRXuHw7N9mialXFoYRyb0RJqyTr2MPnrn86LM3DtTG44nnuu/Zs2rcjMdTqpJVuSp56Hdcf0FGz2ReyuoZss/z0JE0kLJ+a6iGqIdWlYmYT15OVh1xjpcUaQxPdJhUk9CS87Rk8vBLbn4w6UQFK9XID5cuETdf/NOGVSuW6blcFhBlQiDJNI4m7HNOIxdKVefHOZZ80j4CxTG14PxBYhvPpPKrq71sk7S5eUjqMyaMSi987EFsjKBFa20fTAQHQIoEjpaDxoDWg13YTKj0mFnYE4tJj/f29eOGjhBjb/S/6UwyE387ONuwcQMpELcSSiIJ1PxowvtcG09ozx/cd0DVxTc9kPRrxpTNByyuPLdvtqmh8Z1cTueFYyM2ElAwy5PEw0yE7GB3w8vNrM+mY6x33mq2Nd8U1GccITU77dzz2hmPp5yopSrtASwmnj9YX/qvxU35fA7VGDsu/MHrXomEhIwzBIkUs8zPxtwwUzgx8ah26o/Txos/Ti9oqq0JojSXSBBMKajqOPSYKbD1mrUlrkHd7xWLxasO6Xeydu2M+dLmrdpl5esYT2OHFAIkYNAWCyoiAFVmo28X1iRTVYt27bFvz2nz/o1aEgjx0C89+0g909SITYkwGRL/ohJKIjEEe8PjPGGW50XFRmh+UeW1YR1nVrfvMOzGh17Q/AZP2QFA9PyT9sl/u27txnw+T0A6q2cqndhB9KQLrr9TDDibxxUMmUAUtGj2PgVTlUqpcmInBA2HHhoRUqlJI4p90a7jdjves2Bl3G8hxNpAkzpPHoEWHwMG9IhC5ADq6XY7dN4bW02Q1QJAP71gHKo7qhkzuARPQo9YasdqEbLxoM6jLnJsTfAk61PduDvmBwae8JuBdsfTH8Sq23dol9CSOCYqndBWjuo3+NxAwRMmYGsm+gPTks9bkrjhMWo8dk6kTbQizE/EazLRM/mPT1W363Tb3DcCB08aj9SPmQCnprkzkQRq5kbwn3Gvz5wy53URlIRl7QJSxZ//Z9KGXFZnCR8edWLcvjZmc4DVamOyFhHG9xpiO8/+7dUdGcRhkEkt20KiCKNuJ3UktOTzPQ86/IgbH3qeyTBwQkK/8NTe+to1nxIbrB4OEnjLCnkQCKfjHd5JiFjmnFHXJsN6X+gevKnr2y2TbtxE6FXhRY0k0OAffEsN8YQ2gVksLPCk4sEjxop4Is7qIdR4PK3YOwk4xkFUCeBJM6d02rGLFuZg4Bn0O+3cdCwWx2xRqdSfUCW87WE1EAmdUDpjolWLGQuncdhDiT/m3X0ynohrvNthErw5Y+SlyXg8QeKUQlhIBKDhPYG6XFbvfs6ocCQs2S0e+pC6i6W5gHXdqEB2sYXylrCPheV2wy++gbX1odKwMRNT+XwOFbGSJK0WHsRi8dmo1F5ClVoKc/ABdZVAdONdcXBnKJc2xBNa70HnjorxbodNgDYLOgidom5VAOUlt4N7BmOpl49zKoFqlC/XNBc7hs8tH59OQkuORPr0K1zJSdOGX3KDqGrXgYQf6ssynFTg7dohtC0Iz3K5ZvEseCYVypcegPuwsRMD8y6X4s/QX16K7ZxlpZU27nogiLTG+wK/AO2ePzwSAaQwllQBlGUOVkYSdEXgK8BabBkE3ipErmLn5TMEaAkRYJmX3R/nypUhy6rEY01Wz/TxK6bRTQf7DDwTtUxlMnNTvOt7ElryTKNtrsvwcuMBzUtCKxEoChNLmOYeMx8P37wEstLemVblCzzqe/JPquPxBFEtShIo4AZYgrisVJDfuR8JFAYXk0SJJQRgVSJcKRuwtftrjRUS5vfJ6+fCbCXXb3stzM39FQoUrT6xVCpQtPbEgqRlqPGVprXU7nPAoY5swqxE40/SpP8aIgiQd0vkpzDUeCUABfyICQMIpUrOd9RuWoGUyR/n+M1K8tpyD4L7Kc/uD5BulcBua2dcfu+x74E/2uDEZkOyEDzoku6aNFYQi+aWSCxhUMsH+UMrHgttkQlTVNrBOuWrztssFMETeOOWeCa77r4vW2RUEl8EGkvXbnbDqXhPp1xR1wIMRBkAFHiO3VKfgWelEloyyByvbprWg8Q6Tujw44cUxhK8gEckoPFiV+Z+g/qrIDmtRVXnrQXcJIiShIK4QRlwy5OyS0ggJVfe8pYAVNkC05EpgvN42exI1mN3rtJ/q0lVt3O0pS3S6tQriD2rKyQLIcuQl+V7raUKqjwYJ3ZheS3AyaRw9+SxYsocb69GTZedq1atLJfASqUn/l7jVGNBy0DSgi+3XflzwZJPJ5O2XesTmrZXVrcuS7e7MpzfUtXtq51OLLwzaBo3jBoqNnzztZg6d+v3BYEFfqvwyzyWVGygACaOHaztbPnAd8COGFKOiEmAK7/bEQDKNQBsqVmeXgHWdjZQfmvLVOt01uRBnjZibOGh83Cvun3ruYffXZCz6dtFBQ5uKfU+FC2GSIY/Th8vrh5+rJh0f32LYw4J47HZU8WzzQlVit5vPbFdTedKsw+3iDjWtpb6jmOFiRagOPz407dY8smEA2+cSKQAcbqxgUxIFUMJTXOzI6rA/4DabjeO4A3Cigp4WhmhAqCAJOtjSYJKqh+kUB4woFqOAFYGLOAowbbYPZRJEosFRf5U6itWdqv/7nTWpMHMiKhixEeaHy5qLOAhcz2qdq66fUc88ZUEoDW77rEvG9g5Igb2xm+/EccN2RxTCzBcfV5/sXO33cXnnyzfwvxRrnCzRFHu2pDOu5pYaBvvDO+HOaYWEwe5ZOHN5AvKWdI295A8q5VG6cYG9qN3RIwhNJXeP+pX0ObMNyOEPPtoszZj/l31swqAIjkCmqjnABwDkN8A1FIE95FYpYGKMiC7JwjQUqaUPgFqVojI7/JoFNH2DjxEJwQgTDYkCbOkKV+G9h07FQaLkzIbN21A0nLWECcVOL92xarl7zsybVAF0sIe+x0kXnp2swROomlAA0mDWL03X5avW/lGGdfy/lUKqTfe1GIkqeZJNd9iC+U0PEGb4SgTcptuK/pRJqkuekErnMjqGd3JWCqMowuGCrSWMTcghU9pkcIpB6kU3hhOVqUemRxQ61QAFPWbQYeKTpIBXjRAlNRapQiJk6Sw5gEr1X/rDAvY8tJQF0fuAXj5Tl2MFD63VapHAlAlHvo1w48VI6+8VYy5cZZ46Zk5BamC+5GWrv7DnIJEgSTmgtQb4qJwh7fY2cxLFoETAF5Ovr9eYBYBNCDsXAAEhLTBoFAlPZ1hOwfze6p6a8Vcx6Cm36iogMWfbi9kViq0D4cJ59FanPAFAG3Xcfv/VEwnmxuyZP03XynvFvDWy/Vi5FW3FswZvCMjr5pS2L0AKX3yBUPFkScOLWTqF7FYC7A66O8SVQBlpPKmSo88L5u0S8rfZL2AISm4ICvoMXh5ywFEOfqRPuXLy2/cI+2pnCPpBeVVnj5hdFLl8NUXq5VVVaQIAryxaaG6A6JWaQBJwqzWl2uDadYsd2no53mZVYiJBVOItGMxGMzEeTz0SBMSTM3ni33+fNVya27UYpeG9ns8oW1k8KsStk9pD8Zh0vuwflsAApMtv1nfo1Llw083KnOpMn04t2Tpq8/LlXVli+M9ML8LjCmEECbh44eOEG++tKDwJ/J55fGERhhPaIUsXipe+GKNBOSkNIM0KkEQAES93/Lt3lwK1wGinOcopVKukGIDAMuo4o86WK8MUDeLGJvLaiuflqCqAhTSi1yq4UgM5jAL8wvAfageTp1SDLB4PLEhl8uqj8pSjfTnXL0BFB26dpdzZvGCmTDMtj0GgrRfMtix7wGq8rfiJW0+wzP56vPVLCOtJL6IXFZ/Ztlbr5HoWWmMmiVLM5+YOAm7gSfYR7H3qdLrLzzVkNUz7FtUSQQ2zATErOPCSSPN7w5lmd+rcuXAl1xWL2wroiKByvLMkiZAJ4EQgENVl+DHsRh4yrK4V9o9ZVmcYxTxO398BowpS4KtvL+tHec4lShKdRAJ1QlIUNYbi58lE/sjpcptjXMAxeKnHsG55Ymavag7FNQxEgMDqCqEHZWJpdIAFC3tlfr5SuBZrp+o9kQmELlgBtpS9zGxfPLhuyShAbAqidbF49pbZr+A18ah4amSlS9ROjtVznm/bs5h/U8dNG7GPM+DgoeIVKGqwgMmPztsB3rQ2tnG7bjIRDnzoVe+Vu6PXSFWE4VZgre7Xv525bnH6G+/+nylpW6jeZisvr7q9kc9SVoUxPuC6o4ar/rO4Gy598bfobGQOKPSqPDO3PXsciWNzs/G33LpeZnn5j30mZGERmklkp/1f5fLmoJEYR3obhiCGUB1IFD+vNlTRCwWJx1YpUkTNG8OEiBt9ELS7CGPKmXxLN5+tZBnU10EUSnYn2sQoRc+MOUauXGa61J5X+CL6jvDhPu3u3/fmMtl73FdabA3zsIG+cDUcaFG+GM6W/jYA8l8Ptey5acTFT5Ylmz7pdcntOSih2671rO66oRVSB+otMZDV9NrnVTg/dp1uVz2lr/dc1NaVe32XmVzCfdPubqRZ1KB6rvsYt1HH7ydkpEG8segj0xm69Z+gafb6gQOumrl8nNZfRRg5iSkSbnwIhfeNXG0brwvLRNuBKBFmBXEz1k9cw0ST5gD4s6JozOGx9CbiBcEQzaXOT6TTn/B8sOwiGfw7r8WV2f1zKiw6nRRD07U2fdPuSa0yYUJ99F7b9FzWZ39hypxwpVsRJuafeW5x2Rpc9DE9sZLX3+xyfq+RAAaNOe3LB9P79Q7rr8gHcbMSTKSVxbMS+ay+rAtm1F532gjjgFz1pygWgnveQZGdIeMHgmqOq/ljl3/9ZdrJ/32tMA1FzSACeefmk43Nr6r4Aj22i8/7h/b1NCwlDYHqb0w2fJnjKMt3pcIQP14jM7KGKtnMq8wcwb90E3b1FZUiE4RdtHGn989+SLhp4fVWhc8h/fZbJbtnWXkiPWySvq+Tk83nYL0gxQUJN162fDMJ/95N53L6X0rXPqUbFhHW2nzlcP6BSKJkj/A4PtFdj6ECEDlowjxmNUzp+qZ9PujTu0diCTKbGk8dJbEtthrQuyi26po62xWiNAHvwnJE54jtejppnP9Lj/A8pYg/Zieq69VNUuep2Vee+6JdC6XbSvgKXmADf2Ajz94Z+no0w7M+qnZMYYIATOiV2xNYMoR/bK10dEXDjTmstkHmxoaTl8w574dvr93r0T3Pdmi3Dshwd13C9u9FzL8twUJy9ppbFs7vPTs3CNIGHJw34HW866+I9VO+M2pmU0b1r+Sy+onthEJy9xX1Oo3Vr735qmLnn4kccygc+KpKu/bSAE4qMDvvf6iBM8tVFRzAyr487p8PvdHPZPu9eSfZuz3+aqVhUUDbvlDzOzE8wfrS19btCmfzw+wkzwlL6I4UMmJ1jsiao049vRh6Z+NnpBSWalk11RCcm697Dz9qy8+a8JD2cYkT7suFWL9dtvnB+nfjJueUo3rtBaEg+Gh28alF8x9gD1+kMgpty0T+8TPT1VVd2X3zNNGsMuvc0LqnHffVMKV0pmm9PttSG0v19m6eEKbmEyldhrwk1+nVHfBhR8vPTtXPPWXOxtxLhrvClEIJT1UEYCWexzhnB8ST2jTc1m9G2t1jx9aV0i9Va5q+dCf+es9BIMToC9XhJV86OXKraDzJJJhgun3g0P76iec+UuNPJcq8YxvvbywsGwRtTcWi3+cz+eGV3C4klOWE2Q/Np7QflfTeafEj0deXg1fVCZfJE6A4tF7btbTTY1MtnjbKzZcySljTNePjye0kYyp6nYdMn1OPisplz+zio+JVSb4WfTkw2nCxYx7HY2hCEBNHK+Aj3XJVNVlmXTT/jz02j4nJmXGJetDf33R03KmpNk8dGw0bcFZ5IbN/bVU1aU4U7h5v4OPavxhn5MK+is5K1mVJZNkLHvrtcyb//yHaGzYmEymqpZm0k2/3wak8WI8Y4IZH48nzsjlsh2777lfw8F9B7aTW06QVEWm9YM/ry/6e/qrNZ+mtFTVaj3ddKfxzlRyqFKxfjv5nQRFLA0fQjZ7PZPuncvqHSiATFMNG9Z/YkiZmI4cLzSJANTJowjvWgYGiVOGJLSkFk9oe2eaGrraPHQAk4e+rQ8CyXkkL/jCgOjRruP23Rs2rN+Lk8mqdmtyWX0Z+SINnsCXbUUSl/0vdSyABFnO2nfs1HHThm8OkRenqtu/lG7cxFa82DeR6NuinVN2p6KO/we8GQX264gI/AAAAABJRU5ErkJggg=="}},"cell_type":"markdown","metadata":{"graffitiCellId":"id_xq1py16","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"2031A689D83B4282ABA8CC2238D7FC9B","mdEditEnable":false},"source":"## softmax的基本概念\n\n- 分类问题  \n一个简单的图像分类问题，输入图像的高和宽均为2像素，色彩为灰度。  \n图像中的4像素分别记为$x_1, x_2, x_3, x_4$。  \n假设真实标签为狗、猫或者鸡，这些标签对应的离散值为$y_1, y_2, y_3$。  \n我们通常使用离散的数值来表示类别，例如$y_1=1, y_2=2, y_3=3$。\n\n- 权重矢量  \n$$\n \\begin{aligned} o_1 &= x_1 w_{11} + x_2 w_{21} + x_3 w_{31} + x_4 w_{41} + b_1 \\end{aligned} \n$$\n\n$$\n \\begin{aligned} o_2 &= x_1 w_{12} + x_2 w_{22} + x_3 w_{32} + x_4 w_{42} + b_2 \\end{aligned} \n$$\n\n$$\n \\begin{aligned} o_3 &= x_1 w_{13} + x_2 w_{23} + x_3 w_{33} + x_4 w_{43} + b_3 \\end{aligned} \n$$\n\n- 神经网络图  \n下图用神经网络图描绘了上面的计算。softmax回归同线性回归一样，也是一个单层神经网络。由于每个输出$o_1, o_2, o_3$的计算都要依赖于所有的输入$x_1, x_2, x_3, x_4$，softmax回归的输出层也是一个全连接层。\n\n![Image Name](https://cdn.kesci.com/upload/image/q5hmymezog.png)\n\n$$\n\\begin{aligned}softmax回归是一个单层神经网络\\end{aligned}\n$$\n\n既然分类问题需要得到离散的预测输出，一个简单的办法是将输出值$o_i$当作预测类别是$i$的置信度，并将值最大的输出所对应的类作为预测输出，即输出 $\\underset{i}{\\arg\\max} o_i$。例如，如果$o_1,o_2,o_3$分别为$0.1,10,0.1$，由于$o_2$最大，那么预测类别为2，其代表猫。\n\n- 输出问题  \n直接使用输出层的输出有两个问题：\n    1. 一方面，由于输出层的输出值的范围不确定，我们难以直观上判断这些值的意义。例如，刚才举的例子中的输出值10表示“很置信”图像类别为猫，因为该输出值是其他两类的输出值的100倍。但如果$o_1=o_3=10^3$，那么输出值10却又表示图像类别为猫的概率很低。\n    2. 另一方面，由于真实标签是离散值，这些离散值与不确定范围的输出值之间的误差难以衡量。\n\nsoftmax运算符（softmax operator）解决了以上两个问题。它通过下式将输出值变换成值为正且和为1的概率分布：\n\n$$\n \\hat{y}_1, \\hat{y}_2, \\hat{y}_3 = \\text{softmax}(o_1, o_2, o_3) \n$$\n\n其中\n\n$$\n \\hat{y}1 = \\frac{ \\exp(o_1)}{\\sum_{i=1}^3 \\exp(o_i)},\\quad \\hat{y}2 = \\frac{ \\exp(o_2)}{\\sum_{i=1}^3 \\exp(o_i)},\\quad \\hat{y}3 = \\frac{ \\exp(o_3)}{\\sum_{i=1}^3 \\exp(o_i)}. \n$$\n\n容易看出$\\hat{y}_1 + \\hat{y}_2 + \\hat{y}_3 = 1$且$0 \\leq \\hat{y}_1, \\hat{y}_2, \\hat{y}_3 \\leq 1$，因此$\\hat{y}_1, \\hat{y}_2, \\hat{y}_3$是一个合法的概率分布。这时候，如果$\\hat{y}_2=0.8$，不管$\\hat{y}_1$和$\\hat{y}_3$的值是多少，我们都知道图像类别为猫的概率是80%。此外，我们注意到\n\n$$\n \\underset{i}{\\arg\\max} o_i = \\underset{i}{\\arg\\max} \\hat{y}_i \n$$\n\n因此softmax运算不改变预测类别输出。\n\n- 计算效率\n    - 单样本矢量计算表达式  \n    为了提高计算效率，我们可以将单样本分类通过矢量计算来表达。在上面的图像分类问题中，假设softmax回归的权重和偏差参数分别为\n   \n$$\n \\boldsymbol{W} = \\begin{bmatrix} w_{11} & w_{12} & w_{13} \\\\ w_{21} & w_{22} & w_{23} \\\\ w_{31} & w_{32} & w_{33} \\\\ w_{41} & w_{42} & w_{43} \\end{bmatrix},\\quad \\boldsymbol{b} = \\begin{bmatrix} b_1 & b_2 & b_3 \\end{bmatrix}, \n$$\n\n设高和宽分别为2个像素的图像样本$i$的特征为\n   \n$$\n\\boldsymbol{x}^{(i)} = \\begin{bmatrix}x_1^{(i)} & x_2^{(i)} & x_3^{(i)} & x_4^{(i)}\\end{bmatrix},\n$$\n\n输出层的输出为\n\n$$\n\\boldsymbol{o}^{(i)} = \\begin{bmatrix}o_1^{(i)} & o_2^{(i)} & o_3^{(i)}\\end{bmatrix},\n$$\n\n预测为狗、猫或鸡的概率分布为\n    \n$$\n\\boldsymbol{\\hat{y}}^{(i)} = \\begin{bmatrix}\\hat{y}_1^{(i)} & \\hat{y}_2^{(i)} & \\hat{y}_3^{(i)}\\end{bmatrix}.\n$$\n\nsoftmax回归对样本$i$分类的矢量计算表达式为\n   \n$$\n \\begin{aligned} \\boldsymbol{o}^{(i)} &= \\boldsymbol{x}^{(i)} \\boldsymbol{W} + \\boldsymbol{b},\\\\ \\boldsymbol{\\hat{y}}^{(i)} &= \\text{softmax}(\\boldsymbol{o}^{(i)}). \\end{aligned} \n$$\n\n- 小批量矢量计算表达式  \n    为了进一步提升计算效率，我们通常对小批量数据做矢量计算。广义上讲，给定一个小批量样本，其批量大小为$n$，输入个数（特征数）为$d$，输出个数（类别数）为$q$。设批量特征为$\\boldsymbol{X} \\in \\mathbb{R}^{n \\times d}$。假设softmax回归的权重和偏差参数分别为$\\boldsymbol{W} \\in \\mathbb{R}^{d \\times q}$和$\\boldsymbol{b} \\in \\mathbb{R}^{1 \\times q}$。softmax回归的矢量计算表达式为\n\n$$\n \\begin{aligned} \\boldsymbol{O} &= \\boldsymbol{X} \\boldsymbol{W} + \\boldsymbol{b},\\\\ \\boldsymbol{\\hat{Y}} &= \\text{softmax}(\\boldsymbol{O}), \\end{aligned} \n$$\n\n其中的加法运算使用了广播机制，$\\boldsymbol{O}, \\boldsymbol{\\hat{Y}} \\in \\mathbb{R}^{n \\times q}$且这两个矩阵的第$i$行分别为样本$i$的输出$\\boldsymbol{o}^{(i)}$和概率分布$\\boldsymbol{\\hat{y}}^{(i)}$。\n"},{"metadata":{"id":"900A996DF3114CBE8545153F973B4640","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"mdEditEnable":false},"cell_type":"markdown","source":"## 交叉熵损失函数\n\n对于样本$i$，我们构造向量$\\boldsymbol{y}^{(i)}\\in \\mathbb{R}^{q}$ ，使其第$y^{(i)}$（样本$i$类别的离散数值）个元素为1，其余为0。这样我们的训练目标可以设为使预测概率分布$\\boldsymbol{\\hat y}^{(i)}$尽可能接近真实的标签概率分布$\\boldsymbol{y}^{(i)}$。\n\n- 平方损失估计  \n\n$$\n\\begin{aligned}Loss = |\\boldsymbol{\\hat y}^{(i)}-\\boldsymbol{y}^{(i)}|^2/2\\end{aligned}\n$$\n  \n\n然而，想要预测分类结果正确，我们其实并不需要预测概率完全等于标签概率。例如，在图像分类的例子里，如果$y^{(i)}=3$，那么我们只需要$\\hat{y}^{(i)}_3$比其他两个预测值$\\hat{y}^{(i)}_1$和$\\hat{y}^{(i)}_2$大就行了。即使$\\hat{y}^{(i)}_3$值为0.6，不管其他两个预测值为多少，类别预测均正确。而平方损失则过于严格，例如$\\hat y^{(i)}_1=\\hat y^{(i)}_2=0.2$比$\\hat y^{(i)}_1=0, \\hat y^{(i)}_2=0.4$的损失要小很多，虽然两者都有同样正确的分类预测结果。\n\n改善上述问题的一个方法是使用更适合衡量两个概率分布差异的测量函数。其中，交叉熵（cross entropy）是一个常用的衡量方法：\n\n\n$$\nH\\left(\\boldsymbol y^{(i)}, \\boldsymbol {\\hat y}^{(i)}\\right ) = -\\sum_{j=1}^q y_j^{(i)} \\log \\hat y_j^{(i)},\n$$\n\n\n其中带下标的$y_j^{(i)}$是向量$\\boldsymbol y^{(i)}$中非0即1的元素，需要注意将它与样本$i$类别的离散数值，即不带下标的$y^{(i)}$区分。在上式中，我们知道向量$\\boldsymbol y^{(i)}$中只有第$y^{(i)}$个元素$y^{(i)}{y^{(i)}}$为1，其余全为0，于是$H(\\boldsymbol y^{(i)}, \\boldsymbol {\\hat y}^{(i)}) = -\\log \\hat y_{y^{(i)}}^{(i)}$。也就是说，交叉熵只关心对正确类别的预测概率，因为只要其值足够大，就可以确保分类结果正确。当然，遇到一个样本有多个标签时，例如图像里含有不止一个物体时，我们并不能做这一步简化。但即便对于这种情况，交叉熵同样只关心对图像中出现的物体类别的预测概率。\n\n假设训练数据集的样本数为$n$，交叉熵损失函数定义为 \n$$\n\\ell(\\boldsymbol{\\Theta}) = \\frac{1}{n} \\sum_{i=1}^n H\\left(\\boldsymbol y^{(i)}, \\boldsymbol {\\hat y}^{(i)}\\right ),\n$$\n\n\n其中$\\boldsymbol{\\Theta}$代表模型参数。同样地，如果每个样本只有一个标签，那么交叉熵损失可以简写成$\\ell(\\boldsymbol{\\Theta}) = -(1/n) \\sum_{i=1}^n \\log \\hat y_{y^{(i)}}^{(i)}$。从另一个角度来看，我们知道最小化$\\ell(\\boldsymbol{\\Theta})$等价于最大化$\\exp(-n\\ell(\\boldsymbol{\\Theta}))=\\prod_{i=1}^n \\hat y_{y^{(i)}}^{(i)}$，即最小化交叉熵损失函数等价于最大化训练数据集所有标签类别的联合预测概率。\n"},{"metadata":{"id":"9221297B8C694D4F9B76FDFD29C02E5E","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"mdEditEnable":false},"cell_type":"markdown","source":"## 模型训练和预测\n在训练好softmax回归模型后，给定任一样本特征，就可以预测每个输出类别的概率。通常，我们把预测概率最大的类别作为输出类别。如果它与真实类别（标签）一致，说明这次预测是正确的。在3.6节的实验中，我们将使用准确率（accuracy）来评价模型的表现。它等于正确预测数量与总预测数量之比。"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_co5e0yp","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"3D1A30145B5A450E819DCA2FB094117B","mdEditEnable":false},"source":"# 获取Fashion-MNIST训练集和读取数据\n在介绍softmax回归的实现前我们先引入一个多类图像分类数据集。它将在后面的章节中被多次使用，以方便我们观察比较算法之间在模型精度和计算效率上的区别。图像分类数据集中最常用的是手写数字识别数据集MNIST[1]。但大部分模型在MNIST上的分类精度都超过了95%。为了更直观地观察算法之间的差异，我们将使用一个图像内容更加复杂的数据集Fashion-MNIST[2]。\n\n我这里我们会使用torchvision包，它是服务于PyTorch深度学习框架的，主要用来构建计算机视觉模型。torchvision主要由以下几部分构成：\n1. torchvision.datasets: 一些加载数据的函数及常用的数据集接口；\n2. torchvision.models: 包含常用的模型结构（含预训练模型），例如AlexNet、VGG、ResNet等；\n3. torchvision.transforms: 常用的图片变换，例如裁剪、旋转等；\n4. torchvision.utils: 其他的一些有用的方法。"},{"cell_type":"code","execution_count":13,"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","graffitiCellId":"id_my8ejol","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"9C804F2DABDA482BB4375CB0D36E011D","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"1.3.0\n0.4.1a0+d94043a\n","name":"stdout"}],"source":"# import needed package\n%matplotlib inline\nfrom IPython import display\nimport matplotlib.pyplot as plt\n\nimport torch\nimport torchvision\nimport torchvision.transforms as transforms\nimport time\n\nimport sys\nsys.path.append(\"/home/kesci/input\")\nimport d2lzh1981 as d2l\n\nprint(torch.__version__)\nprint(torchvision.__version__)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_75u5u2k","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"810E6357484047F08D83319B46F7438F","mdEditEnable":false},"source":"## get dataset"},{"cell_type":"code","execution_count":14,"metadata":{"graffitiCellId":"id_no4tgy0","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"6FF62DC73B1D46128E613FC7DFFB3C3C","collapsed":false,"scrolled":false},"outputs":[],"source":"mnist_train = torchvision.datasets.FashionMNIST(root='/home/kesci/input/FashionMNIST2065', train=True, download=True, transform=transforms.ToTensor())\nmnist_test = torchvision.datasets.FashionMNIST(root='/home/kesci/input/FashionMNIST2065', train=False, download=True, transform=transforms.ToTensor())"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_mekh927","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"DD5743BBD5614F49829E2AA08F422D77","mdEditEnable":false},"source":"class torchvision.datasets.FashionMNIST(root, train=True, transform=None, target_transform=None, download=False)\n- root（string）– 数据集的根目录，其中存放processed/training.pt和processed/test.pt文件。\n- train（bool, 可选）– 如果设置为True，从training.pt创建数据集，否则从test.pt创建。\n- download（bool, 可选）– 如果设置为True，从互联网下载数据并放到root文件夹下。如果root目录下已经存在数据，不会再次下载。\n- transform（可被调用 , 可选）– 一种函数或变换，输入PIL图片，返回变换之后的数据。如：transforms.RandomCrop。\n- target_transform（可被调用 , 可选）– 一种函数或变换，输入目标，进行变换。"},{"cell_type":"code","execution_count":15,"metadata":{"graffitiCellId":"id_d4dkkil","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"B4B6EB3C4CF74C1D8034A9A349870804","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"<class 'torchvision.datasets.mnist.FashionMNIST'>\n60000 10000\n","name":"stdout"}],"source":"# show result \nprint(type(mnist_train))\nprint(len(mnist_train), len(mnist_test))"},{"cell_type":"code","execution_count":16,"metadata":{"graffitiCellId":"id_yndoddn","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"E35B37B4A94A44AA89722155E8AB5788","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"torch.Size([1, 28, 28]) 9\n","name":"stdout"}],"source":"# 我们可以通过下标来访问任意一个样本\nfeature, label = mnist_train[0]\nprint(feature.shape, label)  # Channel x Height x Width\n"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_lkp5d0p","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"AFC6C721591743D789E443EE6637458E","mdEditEnable":false},"source":"如果不做变换输入的数据是图像，我们可以看一下图片的类型参数："},{"cell_type":"code","execution_count":17,"metadata":{"graffitiCellId":"id_2dk2d0m","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"C72CDF6B5BA64F36AAB028CFCB1242C9","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"<PIL.Image.Image image mode=L size=28x28 at 0x7F54A41612E8>\n","name":"stdout"}],"source":"mnist_PIL = torchvision.datasets.FashionMNIST(root='/home/kesci/input/FashionMNIST2065', train=True, download=True)\nPIL_feature, label = mnist_PIL[0]\nprint(PIL_feature)"},{"cell_type":"code","execution_count":18,"metadata":{"graffitiCellId":"id_pku7gp5","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"B7454582B1484EE4B227BF145B0ED3BA","collapsed":false,"scrolled":false},"outputs":[],"source":"# 本函数已保存在d2lzh包中方便以后使用\ndef get_fashion_mnist_labels(labels):\n    text_labels = ['t-shirt', 'trouser', 'pullover', 'dress', 'coat',\n                   'sandal', 'shirt', 'sneaker', 'bag', 'ankle boot']\n    return [text_labels[int(i)] for i in labels]"},{"cell_type":"code","execution_count":19,"metadata":{"graffitiCellId":"id_1x2gagm","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"D241CBCCD1CD47F28B9480080B93D4FD","collapsed":false,"scrolled":false},"outputs":[],"source":"def show_fashion_mnist(images, labels):\n    d2l.use_svg_display()\n    # 这里的_表示我们忽略（不使用）的变量\n    _, figs = plt.subplots(1, len(images), figsize=(12, 12))\n    for f, img, lbl in zip(figs, images, labels):\n        f.imshow(img.view((28, 28)).numpy())\n        f.set_title(lbl)\n        f.axes.get_xaxis().set_visible(False)\n        f.axes.get_yaxis().set_visible(False)\n    plt.show()"},{"cell_type":"code","execution_count":20,"metadata":{"graffitiCellId":"id_us9fuso","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"056F457B00454FFD81A3CB6AD966C508","collapsed":false,"scrolled":false},"outputs":[{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 864x864 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/056F457B00454FFD81A3CB6AD966C508/q5j7ehijw7.svg\">"},"transient":{}}],"source":"X, y = [], []\nfor i in range(10):\n    X.append(mnist_train[i][0]) # 将第i个feature加到X中\n    y.append(mnist_train[i][1]) # 将第i个label加到y中\nshow_fashion_mnist(X, get_fashion_mnist_labels(y))"},{"cell_type":"code","execution_count":21,"metadata":{"graffitiCellId":"id_uy1hu22","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"34AD2BCBAD9149428E1C29B6F5530BAC","collapsed":false,"scrolled":false},"outputs":[],"source":"# 读取数据\nbatch_size = 256\nnum_workers = 4\ntrain_iter = torch.utils.data.DataLoader(mnist_train, batch_size=batch_size, shuffle=True, num_workers=num_workers)\ntest_iter = torch.utils.data.DataLoader(mnist_test, batch_size=batch_size, shuffle=False, num_workers=num_workers)"},{"cell_type":"code","execution_count":22,"metadata":{"graffitiCellId":"id_2vdyxtx","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"2A439C938A5B43449F0160F5DACB4759","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"4.95 sec\n","name":"stdout"}],"source":"start = time.time()\nfor X, y in train_iter:\n    continue\nprint('%.2f sec' % (time.time() - start))"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_0nzlmif","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7886893A27B44B979B7E4B93F870B3CE","mdEditEnable":false},"source":"# softmax从零开始的实现"},{"cell_type":"code","execution_count":23,"metadata":{"graffitiCellId":"id_c0aor6l","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"C47E4519B73C46558F23670B6C6D2CF4","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"1.3.0\n0.4.1a0+d94043a\n","name":"stdout"}],"source":"import torch\nimport torchvision\nimport numpy as np\nimport sys\nsys.path.append(\"/home/kesci/input\")\nimport d2lzh1981 as d2l\n\nprint(torch.__version__)\nprint(torchvision.__version__)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_63x3cw1","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"24E441B54DB5429991AC9DEBB6014FEC","mdEditEnable":false},"source":"## 获取训练集数据和测试集数据"},{"cell_type":"code","execution_count":24,"metadata":{"graffitiCellId":"id_lkxl1n6","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"39EFD4C2466B4A649B15C7535800FAD6","collapsed":false,"scrolled":false},"outputs":[],"source":"batch_size = 256\ntrain_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, root='/home/kesci/input/FashionMNIST2065')"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_h6qpe01","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7DC915895B5C46278F4C395BD49815A0","mdEditEnable":false},"source":"## 模型参数初始化"},{"cell_type":"code","execution_count":25,"metadata":{"graffitiCellId":"id_xcafpiw","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"FD4BCCEF1A044DF99791FB17D8134B6A","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"784\n","name":"stdout"}],"source":"num_inputs = 784\nprint(28*28)\nnum_outputs = 10\n\nW = torch.tensor(np.random.normal(0, 0.01, (num_inputs, num_outputs)), dtype=torch.float)\nb = torch.zeros(num_outputs, dtype=torch.float)"},{"cell_type":"code","execution_count":26,"metadata":{"graffitiCellId":"id_1ow0q3n","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"D5372FA087BA4BAF87E940582444FB2D","collapsed":false,"scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True)"},"transient":{},"execution_count":26}],"source":"W.requires_grad_(requires_grad=True)\nb.requires_grad_(requires_grad=True)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_d5s0vs0","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"3BBF52F8DB29448B9D44F5D707596788","mdEditEnable":false},"source":"## 对多维Tensor按维度操作"},{"cell_type":"code","execution_count":27,"metadata":{"graffitiCellId":"id_e13vki6","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"6BFBFFFB54E44D05B1FC16BE841BB815","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"tensor([[5, 7, 9]])\ntensor([[ 6],\n        [15]])\ntensor([5, 7, 9])\ntensor([ 6, 15])\n","name":"stdout"}],"source":"X = torch.tensor([[1, 2, 3], [4, 5, 6]])\nprint(X.sum(dim=0, keepdim=True))  # dim为0，按照相同的列求和，并在结果中保留列特征\nprint(X.sum(dim=1, keepdim=True))  # dim为1，按照相同的行求和，并在结果中保留行特征\nprint(X.sum(dim=0, keepdim=False)) # dim为0，按照相同的列求和，不在结果中保留列特征\nprint(X.sum(dim=1, keepdim=False)) # dim为1，按照相同的行求和，不在结果中保留行特征"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_fn39y6u","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"F15E94B777BA4E9F88CB18BD16697A45","mdEditEnable":false},"source":"## 定义softmax操作\n\n$$\n \\hat{y}_j = \\frac{ \\exp(o_j)}{\\sum_{i=1}^3 \\exp(o_i)} \n$$\n\n"},{"cell_type":"code","execution_count":28,"metadata":{"graffitiCellId":"id_cr8iqvr","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7F882A63347F46318C5F334F4F55F123","collapsed":false,"scrolled":false},"outputs":[],"source":"def softmax(X):\n    X_exp = X.exp()\n    partition = X_exp.sum(dim=1, keepdim=True)\n    # print(\"X size is \", X_exp.size())\n    # print(\"partition size is \", partition, partition.size())\n    return X_exp / partition  # 这里应用了广播机制"},{"cell_type":"code","execution_count":29,"metadata":{"graffitiCellId":"id_vxj92pl","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"A77A982B8916433AAB521914CFD8467D","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"tensor([[0.2253, 0.1823, 0.1943, 0.2275, 0.1706],\n        [0.1588, 0.2409, 0.2310, 0.1670, 0.2024]]) \n tensor([1.0000, 1.0000])\n","name":"stdout"}],"source":"X = torch.rand((2, 5))\nX_prob = softmax(X)\nprint(X_prob, '\\n', X_prob.sum(dim=1))"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_bkbmqh1","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"F59CC29AE6F5407A84064647281E9ECA","mdEditEnable":false},"source":"## softmax回归模型\n \n$$\n \\begin{aligned} \\boldsymbol{o}^{(i)} &= \\boldsymbol{x}^{(i)} \\boldsymbol{W} + \\boldsymbol{b},\\\\ \\boldsymbol{\\hat{y}}^{(i)} &= \\text{softmax}(\\boldsymbol{o}^{(i)}). \\end{aligned} \n$$\n\n"},{"cell_type":"code","execution_count":30,"metadata":{"graffitiCellId":"id_76a2fpk","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"0B3F76DCDD23439D849A2AE842D7DBE5","collapsed":false,"scrolled":false},"outputs":[],"source":"def net(X):\n    return softmax(torch.mm(X.view((-1, num_inputs)), W) + b)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_9kc1jw8","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"5024E97053FF460A8CB5F2FCC75C0C27","mdEditEnable":false},"source":"## 定义损失函数\n\n$$\nH\\left(\\boldsymbol y^{(i)}, \\boldsymbol {\\hat y}^{(i)}\\right ) = -\\sum_{j=1}^q y_j^{(i)} \\log \\hat y_j^{(i)},\n$$\n  \n\n$$\n\\ell(\\boldsymbol{\\Theta}) = \\frac{1}{n} \\sum_{i=1}^n H\\left(\\boldsymbol y^{(i)}, \\boldsymbol {\\hat y}^{(i)}\\right ),\n$$\n  \n\n$$\n\\ell(\\boldsymbol{\\Theta}) = -(1/n) \\sum_{i=1}^n \\log \\hat y_{y^{(i)}}^{(i)}\n$$\n  \n\n"},{"cell_type":"code","execution_count":31,"metadata":{"graffitiCellId":"id_fl3cdvq","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"0244A13F3B4F43EA9B8D2FC5800238C7","collapsed":false,"scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"tensor([[0.1000],\n        [0.5000]])"},"transient":{},"execution_count":31}],"source":"y_hat = torch.tensor([[0.1, 0.3, 0.6], [0.3, 0.2, 0.5]])\ny = torch.LongTensor([0, 2])\ny_hat.gather(1, y.view(-1, 1))"},{"cell_type":"code","execution_count":32,"metadata":{"graffitiCellId":"id_4ymtpzn","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"D288CFCAE8DF464287ECA08AD5168F2E","collapsed":false,"scrolled":false},"outputs":[],"source":"def cross_entropy(y_hat, y):\n    return - torch.log(y_hat.gather(1, y.view(-1, 1)))"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_epbrym1","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"83DD3AF7D13F489D8683CB751F430CBF","mdEditEnable":false},"source":"## 定义准确率\n我们模型训练完了进行模型预测的时候，会用到我们这里定义的准确率。"},{"cell_type":"code","execution_count":33,"metadata":{"graffitiCellId":"id_8h5q5ic","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"B82A0C9B1F984CF089B9FA8747BE1A3B","collapsed":false,"scrolled":false},"outputs":[],"source":"def accuracy(y_hat, y):\n    return (y_hat.argmax(dim=1) == y).float().mean().item()"},{"cell_type":"code","execution_count":34,"metadata":{"graffitiCellId":"id_2szlvh2","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"9A6EF0B8618B400D8D252223B6B04E2D","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"0.5\n","name":"stdout"}],"source":"print(accuracy(y_hat, y))"},{"cell_type":"code","execution_count":35,"metadata":{"graffitiCellId":"id_gtpf1ob","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"963FEE28FD23446895193E95A4915F2A","collapsed":false,"scrolled":false},"outputs":[],"source":"# 本函数已保存在d2lzh_pytorch包中方便以后使用。该函数将被逐步改进：它的完整实现将在“图像增广”一节中描述\ndef evaluate_accuracy(data_iter, net):\n    acc_sum, n = 0.0, 0\n    for X, y in data_iter:\n        acc_sum += (net(X).argmax(dim=1) == y).float().sum().item()\n        n += y.shape[0]\n    return acc_sum / n"},{"cell_type":"code","execution_count":36,"metadata":{"graffitiCellId":"id_90h5xb7","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"BDDE1D7C593D47B78465BA7E4F991B82","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"0.1445\n","name":"stdout"}],"source":"print(evaluate_accuracy(test_iter, net))"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_1p8f5ji","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"A209780170A64CC29271427320284785","mdEditEnable":false},"source":"## 训练模型"},{"cell_type":"code","execution_count":37,"metadata":{"graffitiCellId":"id_smslnb6","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"EFDCDFDDE4B44FA3866BEABC48F44D57","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"epoch 1, loss 0.7851, train acc 0.750, test acc 0.791\nepoch 2, loss 0.5704, train acc 0.814, test acc 0.810\nepoch 3, loss 0.5258, train acc 0.825, test acc 0.819\nepoch 4, loss 0.5014, train acc 0.832, test acc 0.824\nepoch 5, loss 0.4865, train acc 0.836, test acc 0.827\n","name":"stdout"}],"source":"num_epochs, lr = 5, 0.1\n\n# 本函数已保存在d2lzh_pytorch包中方便以后使用\ndef train_ch3(net, train_iter, test_iter, loss, num_epochs, batch_size,\n              params=None, lr=None, optimizer=None):\n    for epoch in range(num_epochs):\n        train_l_sum, train_acc_sum, n = 0.0, 0.0, 0\n        for X, y in train_iter:\n            y_hat = net(X)\n            l = loss(y_hat, y).sum()\n            \n            # 梯度清零\n            if optimizer is not None:\n                optimizer.zero_grad()\n            elif params is not None and params[0].grad is not None:\n                for param in params:\n                    param.grad.data.zero_()\n            \n            l.backward()\n            if optimizer is None:\n                d2l.sgd(params, lr, batch_size)\n            else:\n                optimizer.step() \n            \n            \n            train_l_sum += l.item()\n            train_acc_sum += (y_hat.argmax(dim=1) == y).sum().item()\n            n += y.shape[0]\n        test_acc = evaluate_accuracy(test_iter, net)\n        print('epoch %d, loss %.4f, train acc %.3f, test acc %.3f'\n              % (epoch + 1, train_l_sum / n, train_acc_sum / n, test_acc))\n\ntrain_ch3(net, train_iter, test_iter, cross_entropy, num_epochs, batch_size, [W, b], lr)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_eptwt05","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"D7885C25CD7444BF8212D2102E99692C","mdEditEnable":false},"source":"## 模型预测\n现在我们的模型训练完了，可以进行一下预测，我们的这个模型训练的到底准确不准确。\n现在就可以演示如何对图像进行分类了。给定一系列图像（第三行图像输出），我们比较一下它们的真实标签（第一行文本输出）和模型预测结果（第二行文本输出）。"},{"cell_type":"code","execution_count":38,"metadata":{"graffitiCellId":"id_yt8sd9t","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"1DA8927186304BEBA2B3DCC4A9E027DD","collapsed":false,"scrolled":false},"outputs":[{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 864x864 with 9 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/1DA8927186304BEBA2B3DCC4A9E027DD/q5j7fq2jer.svg\">"},"transient":{}}],"source":"X, y = iter(test_iter).next()\n\ntrue_labels = d2l.get_fashion_mnist_labels(y.numpy())\npred_labels = d2l.get_fashion_mnist_labels(net(X).argmax(dim=1).numpy())\ntitles = [true + '\\n' + pred for true, pred in zip(true_labels, pred_labels)]\n\nd2l.show_fashion_mnist(X[0:9], titles[0:9])"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_623cj7f","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"740E43F6246C47078E68E5EB4399345A","mdEditEnable":false},"source":"# softmax的简洁实现"},{"cell_type":"code","execution_count":39,"metadata":{"graffitiCellId":"id_s0m2c8a","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"94B0411C93ED4E6C83B5545D9339E395","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"1.3.0\n","name":"stdout"}],"source":"# 加载各种包或者模块\nimport torch\nfrom torch import nn\nfrom torch.nn import init\nimport numpy as np\nimport sys\nsys.path.append(\"/home/kesci/input\")\nimport d2lzh1981 as d2l\n\nprint(torch.__version__)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_7ny2q9r","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"21E9BF3FC62E42FAA0D601526BC27572","mdEditEnable":false},"source":"## 初始化参数和获取数据"},{"cell_type":"code","execution_count":40,"metadata":{"graffitiCellId":"id_3ra54mv","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"08E33ED9A5684635874D11BFC0B488C6","collapsed":false,"scrolled":false},"outputs":[],"source":"batch_size = 256\ntrain_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, root='/home/kesci/input/FashionMNIST2065')"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_pvkd7mm","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7BCB961EF5464E1980619F1853FE2F84","mdEditEnable":false},"source":"## 定义网络模型"},{"cell_type":"code","execution_count":41,"metadata":{"graffitiCellId":"id_1s76ktf","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"01609B4BD2B444588B2C33751B48B9B0","collapsed":false,"scrolled":false},"outputs":[],"source":"num_inputs = 784\nnum_outputs = 10\n\nclass LinearNet(nn.Module):\n    def __init__(self, num_inputs, num_outputs):\n        super(LinearNet, self).__init__()\n        self.linear = nn.Linear(num_inputs, num_outputs)\n    def forward(self, x): # x 的形状: (batch, 1, 28, 28)\n        y = self.linear(x.view(x.shape[0], -1))\n        return y\n    \n# net = LinearNet(num_inputs, num_outputs)\n\nclass FlattenLayer(nn.Module):\n    def __init__(self):\n        super(FlattenLayer, self).__init__()\n    def forward(self, x): # x 的形状: (batch, *, *, ...)\n        return x.view(x.shape[0], -1)\n\nfrom collections import OrderedDict\nnet = nn.Sequential(\n        # FlattenLayer(),\n        # LinearNet(num_inputs, num_outputs) \n        OrderedDict([\n           ('flatten', FlattenLayer()),\n           ('linear', nn.Linear(num_inputs, num_outputs))]) # 或者写成我们自己定义的 LinearNet(num_inputs, num_outputs) 也可以\n        )"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_nssr12v","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"1C3A731775674DDFB88CA79FBCFABCFC","mdEditEnable":false},"source":"## 初始化模型参数"},{"cell_type":"code","execution_count":42,"metadata":{"graffitiCellId":"id_id7oqtn","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"444F87CD78CC4DCE9E76E56B9D096E48","collapsed":false,"scrolled":false},"outputs":[{"output_type":"execute_result","metadata":{},"data":{"text/plain":"Parameter containing:\ntensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True)"},"transient":{},"execution_count":42}],"source":"init.normal_(net.linear.weight, mean=0, std=0.01)\ninit.constant_(net.linear.bias, val=0)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_hn4au8n","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"EC61B11479EA443E812CCF03CC85D871","mdEditEnable":false},"source":"## 定义损失函数"},{"cell_type":"code","execution_count":43,"metadata":{"graffitiCellId":"id_knh2n50","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"EFA9FA88D866458F913194664286D8AB","collapsed":false,"scrolled":false},"outputs":[],"source":"loss = nn.CrossEntropyLoss() # 下面是他的函数原型\n# class torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean')"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_bdskdx7","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"FFD0D8B3BB1B4F6582610743373C576E","mdEditEnable":false},"source":"## 定义优化函数"},{"cell_type":"code","execution_count":44,"metadata":{"graffitiCellId":"id_6l357pq","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"7DF366DFD7D349B59B0CA9D3211EB140","collapsed":false,"scrolled":false},"outputs":[],"source":"optimizer = torch.optim.SGD(net.parameters(), lr=0.1) # 下面是函数原型\n# class torch.optim.SGD(params, lr=, momentum=0, dampening=0, weight_decay=0, nesterov=False)"},{"cell_type":"markdown","metadata":{"graffitiCellId":"id_kzma3d6","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"6A660AA6E71541EA85B4955AC0DCC66C","mdEditEnable":false},"source":"## 训练"},{"cell_type":"code","execution_count":45,"metadata":{"graffitiCellId":"id_0puuqtc","jupyter":{},"tags":[],"slideshow":{"slide_type":"slide"},"id":"A49C9427C0E14DB9914E8970D98C422B","collapsed":false,"scrolled":false},"outputs":[{"output_type":"stream","text":"epoch 1, loss 0.0031, train acc 0.751, test acc 0.795\nepoch 2, loss 0.0022, train acc 0.813, test acc 0.809\nepoch 3, loss 0.0021, train acc 0.825, test acc 0.806\nepoch 4, loss 0.0020, train acc 0.833, test acc 0.813\nepoch 5, loss 0.0019, train acc 0.837, test acc 0.822\n","name":"stdout"}],"source":"num_epochs = 5\nd2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, batch_size, None, None, optimizer)"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.3","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}