\n",
- "\n",
- "\n",
- "\n",
- "The different versions of the LIF model each have their own dynamics and use-cases. snnTorch currently supports four types of LIF neurons:\n",
- "* Lapicque's RC model: ``snntorch.Lapicque``\n",
- "* Non-physical 1st order model: ``snntorch.Leaky`` \n",
- "* Synaptic Conductance-based neuron model: ``snntorch.Synaptic``\n",
- "* Alpha neuron Model: ``snntorch.Alpha``\n",
- "\n",
- "Before learning how to use them, let's understand how to construct a simple LIF neuron model.\n",
- "\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Nea8oBorr_KZ"
- },
- "source": [
- "# 2. The Leaky Integrate-and-Fire Neuron Model"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "YKsrN5feQ2Dz"
- },
- "source": [
- "## 2.1 Spiking Neurons: Intuition"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "YGStnfjzsGKb"
- },
- "source": [
- "A neuron might be connected to 1,000 $-$ 10,000 other neurons. If one neuron spikes, all of these downhill neurons will feel it. But what determines whether a neuron spikes in the first place? The past century of experiments demonstrate that if a neuron experiences *sufficient* stimulus at its input, then we might expect it to become excited and fire its own spike. \n",
- "\n",
- "Where does this stimulus come from? It could be from:\n",
- "* the sensory periphery, \n",
- "* an invasive electrode artificially stimulating the neuron, or in most cases,\n",
- "* from other pre-synaptic neurons. \n",
- "\n",
- "
\n",
- "\n",
- "
\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "TaVTRojJ2Dl_"
- },
- "source": [
- "Given that these spikes are very short bursts of electrical activity, it is quite unlikely for all input spikes to arrive at the neuron body in precise unison. This indicates the presence of temporal dynamics that 'sustain' the input spikes, kind of like a delay.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "l44jI7A2ReB_"
- },
- "source": [
- "## 2.2 The Passive Membrane"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Od5tfXv6_wcq"
- },
- "source": [
- "Like all cells, a neuron is surrounded by a thin membrane. This membrane is a lipid bilayer that insulates the conductive saline solution within the neuron from the extracellular medium. Electrically, the two conductors separated by an insulator act as a capacitor. \n",
- "\n",
- "Another function of this membrane is to control what goes in and out of this cell (e.g., ions such as Na$^+$). The membrane is usually impermeable to ions which blocks them from entering and exiting the neuron body. But there are specific channels in the membrane that are triggered to open by injecting current into the neuron. This charge movement is electrically modelled by a resistor.\n",
- "\n",
- "
\n",
- "\n",
- "
\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "wT_S4b4HL2ir"
- },
- "source": [
- "Now say some arbitrary time-varying current $I_{\\rm in}(t)$ is injected into the neuron, be it via electrical stimulation or from other neurons. The total current in the circuit is conserved, so:\n",
- "\n",
- "$$I_{\\rm in}(t) = I_{R} + I_{C}$$\n",
- "\n",
- "From Ohm's Law, the membrane potential measured between the inside and outside of the neuron $U_{\\rm mem}$ is proportional to the current through the resistor:\n",
- "\n",
- "$$I_{R}(t) = \\frac{U_{\\rm mem}(t)}{R}$$\n",
- "\n",
- "The capacitance is a proportionality constant between the charge stored on the capacitor $Q$ and $U_{\\rm mem}(t)$:\n",
- "\n",
- "\n",
- "$$Q = CU_{\\rm mem}(t)$$\n",
- "\n",
- "The rate of change of charge gives the capacitive current:\n",
- "\n",
- "$$\\frac{dQ}{dt}=I_C(t) = C\\frac{dU_{\\rm mem}(t)}{dt}$$\n",
- "\n",
- "Therefore:\n",
- "\n",
- "$$I_{\\rm in}(t) = \\frac{U_{\\rm mem}(t)}{R} + C\\frac{dU_{\\rm mem}(t)}{dt}$$\n",
- "\n",
- "$$\\implies RC \\frac{dU_{\\rm mem}(t)}{dt} = -U_{\\rm mem}(t) + RI_{\\rm in}(t)$$\n",
- "\n",
- "The right hand side of the equation is of units **\\[Voltage]**. On the left hand side of the equation, the term $\\frac{dU_{\\rm mem}(t)}{dt}$ is of units **\\[Voltage/Time]**. To equate it to the left hand side (i.e., voltage), $RC$ must be of unit **\\[Time]**. We refer to $\\tau = RC$ as the time constant of the circuit:\n",
- "\n",
- "$$ \\tau \\frac{dU_{\\rm mem}(t)}{dt} = -U_{\\rm mem}(t) + RI_{\\rm in}(t)$$\n",
- "\n",
- "The passive membrane is therefore described by a linear differential equation.\n",
- "\n",
- "For a derivative of a function to be of the same form as the original function, i.e., $\\frac{dU_{\\rm mem}(t)}{dt} \\propto U_{\\rm mem}(t)$, this implies the solution is exponential with a time constant $\\tau$.\n",
- "\n",
- "Say the neuron starts at some value $U_{0}$ with no further input, i.e., $I_{\\rm in}(t)=0$. The solution of the linear differential equation is:\n",
- "\n",
- "$$U_{\\rm mem}(t) = U_0e^{-\\frac{t}{\\tau}}$$\n",
- "\n",
- "The general solution is shown below.\n",
- "\n",
- "
\n",
- "\n",
- "
"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "rTXS_aSmRs-3"
- },
- "source": [
- "## 2.3 Lapicque's LIF Neuron Model"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Jt2q6tZiWkgT"
- },
- "source": [
- "This similarity between nerve membranes and RC circuits was observed by [Louis Lapicque in 1907](https://core.ac.uk/download/pdf/21172797.pdf). He stimulated the nerve fiber of a frog with a brief electrical pulse, and found that membranes could be approximated as a capacitor with a leakage. We pay homage to his findings by naming the basic LIF neuron model in snnTorch after him. \n",
- "\n",
- "Most of the concepts in Lapicque's model carry forward to other LIF neuron models. Now let's simulate this neuron using snnTorch."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "WtveAGG0zE0n"
- },
- "source": [
- "### 2.3.1 Lapicque: Without Stimulus"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "um1s01gTzUC0"
- },
- "source": [
- "First, import the packages needed to run Lapicque's neuron model: snnTorch and PyTorch."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "sUyODsBtWkAG"
- },
- "source": [
- "import snntorch as snn\n",
- "import torch"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "6zpgNEbizd8s"
- },
- "source": [
- "The membrane potential has a time constant $\\tau = RC$ associated with it. This can be equivalently represented by a decay rate $\\beta$ that specifies the ratio of potential between subsequent time steps:\n",
- "\n",
- "$$\\beta = \\frac{U_0e^{-\\frac{1}{\\tau}}}{U_0e^{-\\frac{0}{\\tau}}} = \\frac{U_0e^{-\\frac{2}{\\tau}}}{U_0e^{-\\frac{1}{\\tau}}} = \\frac{U_0e^{-\\frac{3}{\\tau}}}{U_0e^{-\\frac{2}{\\tau}}}=~~...$$\n",
- "$$\\implies \\beta = e^{-\\frac{1}{\\tau}}$$\n",
- "\n",
- "Setting $\\tau = 5\\times 10^{-3} \\implies \\beta \\approx 0.819$:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "ohshwOCU6Vbm"
- },
- "source": [
- "# RC time constant\n",
- "tau_mem = 5e-3\n",
- "time_step = 1e-3 # one time step = 1ms\n",
- "\n",
- "# decay p/time step\n",
- "beta = float(torch.exp(torch.tensor(-time_step/tau_mem)))\n",
- "\n",
- "# Number of time steps to simulate\n",
- "num_steps = 200\n",
- "\n",
- "print(f\"Membrane decay rate ('beta'): {beta}\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "qi5EnND98cz3"
- },
- "source": [
- "Instantiating Lapicque's neuron only requires the following line of code:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "wI2Nahsg8d-t"
- },
- "source": [
- "# leaky integrate and fire neuron\n",
- "lif1 = snn.Lapicque(beta=beta)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "AUv8_QPCIsDT"
- },
- "source": [
- "The same thing can also be accomplished by specifying the RC values:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "Q0tk_CBoIv6d"
- },
- "source": [
- "R = 5\n",
- "C = 1e-3\n",
- "\n",
- "lif1 = snn.Lapicque(R=R, C=C, time_step=time_step)\n",
- "\n",
- "print(f\"Membrane decay rate ('beta'): {lif1.beta[0]}\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "kuytxXet8lc9"
- },
- "source": [
- "To use this neuron: \n",
- "\n",
- "**Inputs**\n",
- "* `spk_in`: each element of $I_{\\rm in}$, which are all `0` for now, is sequentially passed as an input\n",
- "* `mem`: the membrane potential at the present time $t$ is also passed as input. Initialize it arbitrarily as $U_0 = 0.9~V$.\n",
- "\n",
- "**Outputs**\n",
- "* `spk_out`: output spike $S_{\\rm out}[t+1]$ at the next time step ('1' if there is a spike; '0' if there is no spike)\n",
- "* `mem`: membrane potential $U_{\\rm mem}[t+1]$ at the next time step\n",
- "\n",
- "These all need to be of type `torch.Tensor`.\n"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "M_0O1Q3Y9KgM"
- },
- "source": [
- "# Initialize membrane, input, and output\n",
- "mem = torch.ones(1) * 0.9 # membrane potential of 0.9 at t=0\n",
- "cur_in = torch.zeros(num_steps) # input is 0 for all t \n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "_CEuNHsN8-67"
- },
- "source": [
- "These values are only for the initial time step $t=0$. We'd like to watch the evolution of `mem` over time. The list `mem_rec` is initialized to record these values at every time step."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "CUCxQBTQ9d_P"
- },
- "source": [
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec = [mem]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "i4TxVFaV9uf6"
- },
- "source": [
- "Now it's time to run a simulation! 200 time steps will be simulated, updating `mem` at each step and recording its value in `mem_rec`:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "j6GeMMiU8kc4"
- },
- "source": [
- "# pass updated value of mem and cur_in[step]=0 at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec.append(mem)\n",
- "\n",
- "# crunch the list of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "4lngTsykIc7k"
- },
- "source": [
- "Let's take a look at how the membrane potential and synaptic current evolved."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "SrfeQWT6I2JC"
- },
- "source": [
- "import matplotlib.pyplot as plt\n",
- "\n",
- "plt.title(\"Lapicque's Neuron Model Without Stimulus\")\n",
- "plt.plot(mem_rec, label=\"Membrane Potential\")\n",
- "plt.xlabel(\"Time step\")\n",
- "plt.ylabel(\"Membrane Potential\")\n",
- "plt.xlim([0, 50])\n",
- "plt.ylim([0, 1])\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "9k0PKng2JYcx"
- },
- "source": [
- "This matches the dynamics that were previously derived. We've shown ourselves that the membrane potential will decay over time in the absence of any input stimuli. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "7Z01CioKJpkr"
- },
- "source": [
- "### 2.3.2 Lapicque: Step Input"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "B6GZUEFgJ0l0"
- },
- "source": [
- "Now let's apply a step current $I_{\\rm in}(t)$ that switches on at $t=t_0$. Given the linear first-order differential equation:\n",
- "\n",
- "$$ \\tau \\frac{dU_{\\rm mem}}{dt} = -U_{\\rm mem} + RI_{\\rm in}(t),$$\n",
- "\n",
- "the general solution will be:\n",
- "\n",
- "$$U_{\\rm mem}=I_{\\rm in}(t)R + [U_0 - I_{\\rm in}(t)R]e^{-\\frac{t}{\\tau}}$$\n",
- "\n",
- "If the membrane potential is initialized to $U_{\\rm mem}(t=0) = 0 V$, then:\n",
- "\n",
- "$$U_{\\rm mem}(t)=I_{\\rm in}(t)R [1 - e^{-\\frac{t}{\\tau}}]$$\n",
- "\n",
- "Let's visualize what this looks like by triggering a current pulse of $I_{in}=100mA$ at $t_0 = 10ms$."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "kD7cLKSuLu9n"
- },
- "source": [
- "# Initialize input current pulse\n",
- "cur_in = torch.cat((torch.zeros(10), torch.ones(190)*0.1), 0) # input current turns on at t=10\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes\n",
- "\n",
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec = [mem]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "PW8ZDbKCNW8E"
- },
- "source": [
- "As before, 200 time steps will be simulated. But this time, the new values of `cur_in` will be passed:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "w9J2AsrZNXr8"
- },
- "source": [
- "# pass updated value of mem and cur_in[step] at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "_cC3zoy7OW_Y"
- },
- "source": [
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.2])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Step Input\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 0.6])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "uZi44ZEvNhkR"
- },
- "source": [
- "The membrane potential exponentially rises and then stabilizes at $U_{\\rm mem}(t\\rightarrow \\infty) =I_{\\rm in}R$:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "1l40Efnbk92o"
- },
- "source": [
- "print(f\"The calculated value of input pulse [A] x resistance [Ω] is: {cur_in[11]*lif1.R} V\")\n",
- "print(f\"The simulated value of steady-state membrane potential is: {mem_rec[200][0]} V\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "H0ql9cAzpN5D"
- },
- "source": [
- "Close enough!"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ZqEtGdKAaIbK"
- },
- "source": [
- "### 2.3.3 Lapicque: Pulse Input"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "HOgCeLPYaLKZ"
- },
- "source": [
- "Now what if the step input was clipped at $t=30ms$?"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "hjtupQNxaQ5D"
- },
- "source": [
- "# Initialize input current pulse.\n",
- "cur_in1 = torch.cat((torch.zeros(10), torch.ones(20)*(0.1), torch.zeros(170)), 0) # input turns on at t=10, off at t=30\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes\n",
- "\n",
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec1 = [mem]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "O8JFL9CGa0NW"
- },
- "source": [
- "# pass updated value of mem and cur_in[step] at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in1[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec1.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec1 = torch.stack(mem_rec1)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "UM8GGzWOa3GE"
- },
- "source": [
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in1, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.2])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Pulse\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec1)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=30, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "orJvjev6a9tR"
- },
- "source": [
- "It appears to rise just as it did for the step input, but now it decays with a time constant of $\\tau$ as in our first simulation. \n",
- "\n",
- "Let's deliver approximately the same amount of charge $Q = I \\times t$ to the circuit in half the time. This means our input current amplitude will need to be increased by a little, and the time window will be decreased."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "j_G_gWTzbo4J"
- },
- "source": [
- "# Increase amplitude of current pulse; half the time.\n",
- "cur_in2 = torch.cat((torch.zeros(10), torch.ones(10)*0.111, torch.zeros(180)), 0) # input turns on at t=10, off at t=20\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes\n",
- "\n",
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec2 = [mem]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "4YYr91yzbq-G"
- },
- "source": [
- "# pass updated value of mem and cur_in[step] at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in2[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec2.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec2 = torch.stack(mem_rec2)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "krBTZm92btXH"
- },
- "source": [
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in2, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.2])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Pulse: x1/2 pulse width\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec2)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=20, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "34azxvjWbqge"
- },
- "source": [
- "Let's do that again, but with an even faster input pulse and higher amplitude:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "650G94v9dgr3"
- },
- "source": [
- "# Increase amplitude of current pulse; quarter the time.\n",
- "cur_in3 = torch.cat((torch.zeros(10), torch.ones(5)*0.147, torch.zeros(185)), 0) # input turns on at t=10, off at t=15\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes\n",
- "\n",
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec3 = [mem]\n",
- "\n",
- "# pass updated value of mem and cur_in[step] at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in3[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec3.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec3 = torch.stack(mem_rec3)\n",
- "\n",
- "# Generate Plots\n",
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in3, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.2])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Pulse: x1/4 pulse width\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec3)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=15, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "fuQtNgw4sYhk"
- },
- "source": [
- "Let's compare all three experiments on the same plot:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "ZL7P3mEfsdAR"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in1)\n",
- "ax[0].plot(cur_in2)\n",
- "ax[0].plot(cur_in3)\n",
- "ax[0].set_ylim([0, 0.2])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Pulse: Varying inputs\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec1)\n",
- "ax[1].plot(mem_rec2)\n",
- "ax[1].plot(mem_rec3)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=15, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=20, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "ax[1].axvline(x=30, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "6WpZx0NStNdd"
- },
- "source": [
- "As the input current pulse amplitude increases, the rise time of the membrane potential speeds up. In the limit of the input current pulse width becoming infinitesimally small, $T_W \\rightarrow 0s$, the membrane potential will jump straight up in virtually zero rise time:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "h0EG17NpuzuN"
- },
- "source": [
- "# Current spike input\n",
- "cur_in4 = torch.cat((torch.zeros(10), torch.ones(1)*0.5, torch.zeros(189)), 0) # input only on for 1 time step\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to sequentially dump its output spikes\n",
- "\n",
- "# Initialize somewhere to store recordings of membrane potential\n",
- "mem_rec4 = [mem]\n",
- "\n",
- "# pass updated value of mem and cur_in[step] at every time step\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif1(cur_in4[step], mem)\n",
- "\n",
- " # Store recordings of membrane potential\n",
- " mem_rec4.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec4 = torch.stack(mem_rec4)\n",
- "\n",
- "# Generate Plots\n",
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in4, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.6])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Spike\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec4)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axvline(x=10, ymin=0, ymax=2.2, alpha = 0.25, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "DNY_EftJvREj"
- },
- "source": [
- "The current pulse width is now so short, it effectively looks like a spike. That is to say, charge is delivered in an infinitely short period of time, $I_{\\rm in}(t) = Q/t_0$ where $t_0 \\rightarrow 0$. More formally:\n",
- "\n",
- "$$I_{\\rm in}(t) = Q \\delta (t-t_0),$$\n",
- "\n",
- "where $\\delta (t-t_0)$ is the Dirac-Delta function. Physically, it is impossible to 'instantaneously' deposit charge. But integrating $I_{\\rm in}$ gives a result that makes physical sense, as we can obtain the charge delivered:\n",
- "\n",
- "$$1 = \\int^{t_0 + a}_{t_0 - a}\\delta(t-t_0)dt$$\n",
- "\n",
- "$$f(t_0) = \\int^{t_0 + a}_{t_0 - a}f(t)\\delta(t-t_0)dt$$\n",
- "\n",
- "Here, $f(t_0) = I_{\\rm in}(t_0=10) = 0.5A \\implies f(t) = Q = 0.5C$.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Danwy1q9xfvr"
- },
- "source": [
- "### 2.3.4 Lapicque: Firing"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "CUH8IYh9xlV-"
- },
- "source": [
- "So far, we have only seen how a neuron will react to spikes at the input. For a neuron to generate and emit its own spikes at the output, we need to combine the passive membrane model with a threshold.\n",
- "\n",
- "If the membrane potential exceeds this threshold, then a voltage spike will be generated, external to the passive membrane model. \n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "By default, `threshold=1` for all neuron models in snnTorch. So applying a step current input that is insufficient will result in the neuron to function only in the subthreshold regime. This time, we will create a list called `spk_rec` to record any output spikes if they occur. The current step will be set to $I_{\\rm in} = 0.15 A$. "
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "Fe61PZ33w49_"
- },
- "source": [
- "# Small step current input\n",
- "cur_in = torch.cat((torch.zeros(10), torch.ones(190)*0.15), 0)\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1)\n",
- "spk_out = torch.zeros(1) \n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "HPx-uuH90JTo"
- },
- "source": [
- "# Create a new neuron with a slow time constant\n",
- "lif2 = snn.Lapicque(R=5, C=10)\n",
- "\n",
- "print(f\"Membrane decay rate ('beta'): {lif2.beta[0]}\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "se0fnQ5EDHr9"
- },
- "source": [
- "Note how this new value of $\\beta$ is much larger than `lif1.beta`$\\approx 0.82$. \n",
- "\n",
- "For `lif2.beta`$=0.98$, the membrane potential is 98% of the value of that of the previous time step, and experiences a much slower decay rate."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "0yqODQazwg4s"
- },
- "source": [
- "# Simulation run across 200 time steps. \n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif2(cur_in[step], mem)\n",
- "\n",
- " # record outputs over time\n",
- " mem_rec.append(mem)\n",
- " spk_rec.append(spk_out)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "Z3Laa83DxPbJ"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(2, figsize=(8,6),sharex=True)\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.4])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Step Input\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 1.25])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=1.0, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "vrAD1fBextPM"
- },
- "source": [
- "The membrane potential fails to reach the threshold of 1.0. Instead, it reaches the steady-state value of $I_{\\rm in}R = 0.15A \\times 5Ω = 0.75V$:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "XtkMfF5Qxrvm"
- },
- "source": [
- "print(f\"The calculated steady state membrane potential is: {lif1.R*cur_in[199]}\")\n",
- "print(f\"The simulated steady state membrane potential is: {mem_rec[199][0]}\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "FN1Gmvuwy1ZA"
- },
- "source": [
- "> Note: these are non-biologically accurate values, and are chosen for simplicity.\n",
- "\n",
- "To reach the threshold, we need to ensure that $I_{\\rm in}R > U_{\\rm thr}$. So set $I_{\\rm in} = 0.21 A$:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "LU9dSGi6zEOP"
- },
- "source": [
- "# Larger current step\n",
- "cur_in = torch.cat((torch.zeros(10), torch.ones(190)*0.21), 0)\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1)\n",
- "spk_out = torch.zeros(1) \n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "td4oIyzNzLft"
- },
- "source": [
- "# Simulation run across 200 time steps.\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif2(cur_in[step], mem)\n",
- "\n",
- " # record outputs over time\n",
- " mem_rec.append(mem)\n",
- " spk_rec.append(spk_out)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "IeiswwDXGnay"
- },
- "source": [
- "To plot our results, let's import `snntorch.spikeplot`. From Tutorial 1, we learnt how to use it to create raster plots of spike responses."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "gxLfSFTIGvaU"
- },
- "source": [
- "from snntorch import spikeplot as splt"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "4BYXOPSizWH6"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [1, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.4])\n",
- "ax[0].set_xlim([0, 200])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Step Input\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 1.25])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=1.0, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[2], s=400, c=\"black\", marker=\"|\")\n",
- "ax[2].axvline(x=162, ymin=0, ymax=6.75, alpha = 0.15, linestyle=\"dashed\", c=\"black\", linewidth=2, zorder=0, clip_on=False)\n",
- "plt.ylabel(\"Output spikes\")\n",
- "plt.yticks([]) \n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "cm5tEp4YEoQm"
- },
- "source": [
- "The membrane potential exponentially rises and then hits the threshold, at which point it resets. We can roughly see this occurs between $155s < t_{\\rm spk} < 165s$:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "tWSGUUsgNr9D"
- },
- "source": [
- "print(spk_rec[155:165])"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "4xWW-ncoOWrJ"
- },
- "source": [
- "The absence of a spike is represented by $S_{\\rm out}=0$, and the occurrence of a spike is $S_{\\rm out}=1$. Here, the spike occurs at $S_{\\rm out}(t=162)=1$.\n",
- "\n",
- "If you are wondering why each of these entries is stored as a tensor, it is because soon we will simulate large scale neural networks. Each entry will contain the spike responses of many neurons, and tensors can be loaded into GPU memory to speed up the training process.\n",
- "\n",
- "If $I_{\\rm in}$ is increased, then the membrane potential approaches $U_{\\rm thr}$ faster:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "HY6G9HZ1POC7"
- },
- "source": [
- "# Even Larger current step\n",
- "cur_in = torch.cat((torch.zeros(10), torch.ones(190)*0.3), 0)\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1)\n",
- "spk_out = torch.zeros(1) \n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]\n",
- "\n",
- "# Simulation run across 200 time steps.\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif2(cur_in[step], mem)\n",
- "\n",
- " # record outputs over time\n",
- " mem_rec.append(mem)\n",
- " spk_rec.append(spk_out)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)\n",
- "\n",
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [1, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.4])\n",
- "ax[0].set_xlim([0, 200])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Periodic Firing\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 1.25])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=1.0, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[2], s=400, c=\"black\", marker=\"|\")\n",
- "plt.ylabel(\"Output spikes\")\n",
- "plt.yticks([]) \n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "GOjWXepnPleo"
- },
- "source": [
- "A similar increase in firing frequency can also be induced by decreasing the threshold. This requires initializing a new neuron model, but the rest of the code block is the exact same as above:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "HQ0VIBlJPrvr"
- },
- "source": [
- "# Half the threshold\n",
- "lif3 = snn.Lapicque(R=5, C=10, threshold=0.5)\n",
- "\n",
- "# Initialize membrane and output\n",
- "mem = torch.zeros(1)\n",
- "spk_out = torch.zeros(1) \n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]\n",
- "\n",
- "# Simulation run across 200 time steps.\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif3(cur_in[step], mem)\n",
- "\n",
- " # record outputs over time\n",
- " mem_rec.append(mem)\n",
- " spk_rec.append(spk_out)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)\n",
- "\n",
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [1, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "ax[0].plot(cur_in, c=\"tab:orange\")\n",
- "ax[0].set_ylim([0, 0.4])\n",
- "ax[0].set_xlim([0, 200])\n",
- "ax[0].set_ylabel(\"Input Current ($I_{in}$)\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Low Threshold\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 1.25])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=0.5, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[2], s=400, c=\"black\", marker=\"|\")\n",
- "plt.ylabel(\"Output spikes\")\n",
- "plt.yticks([]) \n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "rJwtafQ_Siav"
- },
- "source": [
- "That's what happens for a constant current injection. But in both deep neural networks and in the biological brain, most neurons will be connected to other neurons. They are more likely to receive spikes, rather than injections of constant current. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "sOr9akj9S2Ei"
- },
- "source": [
- "### 2.3.5 Lapicque: Spike Inputs"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "rw_FZDnMS-pc"
- },
- "source": [
- "Let's harness some of the skills we learnt in [Tutorial 1](https://colab.research.google.com/github/jeshraghian/snntorch/blob/tutorials/examples/tutorial_1_spikegen.ipynb), and use the `snntorch.spikegen` module to create some randomly generated input spikes."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "V7AXG4QqTCoZ"
- },
- "source": [
- "from snntorch import spikegen \n",
- "\n",
- "# Create a 1-D random spike train. Each element has a probability of 40% of firing.\n",
- "spk_in = spikegen.rate_conv(torch.ones((num_steps)) * 0.40)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "5CJxZg1TTNCJ"
- },
- "source": [
- "Run the following code block to see how many spikes have been generated."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "rbhLmpNSTPgK"
- },
- "source": [
- "# Tell me the number of spikes\n",
- "print(f\"There are {int(sum(spk_in))} total spikes out of {len(spk_in)} time steps.\")\n",
- "\n",
- "# Now show me the spikes\n",
- "from snntorch import spikeplot as splt\n",
- "\n",
- "fig = plt.figure(facecolor=\"w\", figsize=(8, 1))\n",
- "ax = fig.add_subplot(111)\n",
- "\n",
- "splt.raster(spk_in.reshape(num_steps, -1), ax, s=100, c=\"black\", marker=\"|\")\n",
- "\n",
- "plt.title(\"Input Spikes\")\n",
- "plt.xlabel(\"Time step\")\n",
- "plt.yticks([])\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "wo2_Voc3TvSc"
- },
- "source": [
- "# Refresh all our hidden and output variables\n",
- "mem = torch.ones(1)*0.5 # membrane potential of 0.5 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to dump its output spikes\n",
- "\n",
- "# Create a trace of the variables of interest\n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]\n",
- "\n",
- "# Run the simulation\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif3(spk_in[step], mem)\n",
- "\n",
- " # Store recordings of output and hidden states\n",
- " spk_rec.append(spk_out)\n",
- " mem_rec.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "uGYz-PJAT_1q"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [0.4, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "splt.raster(spk_in, ax[0], s=400, c=\"black\", marker=\"|\")\n",
- "ax[0].set_ylabel(\"Input Spikes\")\n",
- "ax[0].set_title(\"Lapicque's Neuron Model With Input Spikes\")\n",
- "plt.yticks([]) \n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec)\n",
- "ax[1].set_ylim([0, 1])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=0.5, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[2], s=400, c=\"black\", marker=\"|\")\n",
- "plt.ylabel(\"Output spikes\")\n",
- "plt.yticks([]) \n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "fGYnDIoTXBCo"
- },
- "source": [
- "### 2.3.6 Lapicque: Reset Mechanisms"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "lQsH8VOGXF2Q"
- },
- "source": [
- "The final detail of the Lapicque neuron we want to explore is the sharp drop of membrane potential every time the neuron emits an output spike. This sharp drops promotes a reduction of spike generation, which supplements part of the theory on how brains are so power efficient. Biologically, this is known as the 'refractory period' where the the neuron's firing ability is momentarily suppressed. Here, we use a reset mechanism to model the refractory period.\n",
- "\n",
- "There are two ways to implement the reset mechanism:\n",
- "\n",
- "1. *reset by subtraction* (default) $-$ subtract the threshold from the membrane potential each time a spike is generated;\n",
- "2. *reset to zero* $-$ force the membrane potential to zero each time a spike is generated.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "Let's instantiate another neuron model to demonstrate how to alternate between reset mechanisms. \n",
- "\n",
- "By default, snnTorch neuron models use `reset_mechanism = \"subtract\"`. This can be explicitly overridden by passing the argument `reset_mechanism = \"zero\"`."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "7bfidqhKXdri"
- },
- "source": [
- "# Reset mechanism: zero\n",
- "lif4 = snn.Lapicque(R=5, C=10, threshold=0.5, reset_mechanism=\"zero\")\n",
- "\n",
- "# Refresh all our hidden and output variables\n",
- "mem = torch.ones(1)*0.5 # membrane potential of 0.5 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to dump its output spikes\n",
- "\n",
- "# Create a trace of the variables of interest\n",
- "mem_rec0 = [mem]\n",
- "spk_rec0 = [spk_out]\n",
- "\n",
- "# Run the simulation\n",
- "for step in range(num_steps):\n",
- " spk_out, mem = lif4(spk_in[step], mem)\n",
- "\n",
- " # Store recordings of output and hidden states\n",
- " spk_rec0.append(spk_out)\n",
- " mem_rec0.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "mem_rec0 = torch.stack(mem_rec0)\n",
- "spk_rec0 = torch.stack(spk_rec0)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "ifU3betRYETs"
- },
- "source": [
- "# Generate Plots to Compare Reset Mechanisms\n",
- "fig, ax = plt.subplots(nrows=3, ncols=2, figsize=(10,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [0.4, 1, 0.4], 'wspace':0.05})\n",
- "\n",
- "# Reset by Subtraction: input spikes\n",
- "splt.raster(spk_in, ax[0][0], s=400, c=\"black\", marker=\"|\")\n",
- "ax[0][0].set_ylabel(\"Input Spikes\")\n",
- "ax[0][0].set_title(\"Reset by Subtraction\")\n",
- "ax[0][0].set_yticks([])\n",
- "\n",
- "# Reset by Subtraction: membrane potential \n",
- "ax[1][0].plot(mem_rec)\n",
- "ax[1][0].set_ylim([0, 0.7])\n",
- "ax[1][0].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1][0].axhline(y=0.5, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "\n",
- "# Reset by Subtraction: output spikes\n",
- "splt.raster(spk_rec, ax[2][0], s=400, c=\"black\", marker=\"|\")\n",
- "ax[2][0].set_yticks([])\n",
- "ax[2][0].set_xlabel(\"Time step\")\n",
- "ax[2][0].set_ylabel(\"Output Spikes\")\n",
- "\n",
- "# Reset to Zero: input spikes\n",
- "splt.raster(spk_in, ax[0][1], s=400, c=\"black\", marker=\"|\")\n",
- "ax[0][1].set_title(\"Reset to Zero\")\n",
- "ax[0][1].set_yticks([])\n",
- "\n",
- "# Reset to Zero: membrane potential\n",
- "ax[1][1].plot(mem_rec0)\n",
- "ax[1][1].set_ylim([0, 0.7])\n",
- "ax[1][1].axhline(y=0.5, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "ax[1][1].set_yticks([])\n",
- "ax[2][1].set_xlabel(\"Time step\")\n",
- "\n",
- "# Reset to Zero: output spikes\n",
- "splt.raster(spk_rec0, ax[2][1], s=400, c=\"black\", marker=\"|\")\n",
- "ax[2][1].set_yticks([])\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Dzs7qX2Kb8j_"
- },
- "source": [
- "Pay close attention to the evolution of the membrane potential, especially in the moments after it reaches the threshold. You may notice that for \"Reset to Zero\", the membrane potential is forced back to zero after each spike.\n",
- "\n",
- "So which one is better? Applying `\"subtract\"` (the default value in `reset_mechanism`) is less lossy, because it does not ignore how much the membrane exceeds the threshold by.\n",
- "\n",
- "On the other hand, applying a hard reset with `\"zero\"` promotes sparsity and potentially less power consumption when running on dedicated neuromorphic hardware. Both options are available for you to experiment with. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "3ukHPMnOieVV"
- },
- "source": [
- "## 2.4 Synaptic Conductance-based LIF Neuron Model"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "9b9GEzOfm_Iz"
- },
- "source": [
- "The passive membrane model allows discrete current spikes to be passed directly into the neuron. In reality, a spike will result in the gradual release of neurotransmitters from the pre-synaptic neuron to the post-synaptic neuron. This model accounts for the gradual temporal dynamics of input current, and is no longer strictly modelling a LIF neuron alone."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "YO6I0Wpmo3fx"
- },
- "source": [
- "### 2.4.1 Synaptic Current"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "KBuE963K_JmQ"
- },
- "source": [
- "If a pre-synaptic neuron fires, the voltage spike is transmitted down the axon of the neuron. It triggers the vesicles to release neurotransmitters into the synaptic cleft. These activate the post-synaptic receptors, which directly influence the effective current that flows into the post-synaptic neuron. \n",
- "\n",
- "Shown below are two types of excitatory receptors.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "The simplest model of synaptic current assumes an increasing current on a very fast time-scale (or instantaneous), followed by a relatively slow exponential decay. This is very similar to the membrane potential dynamics of Lapicque's model.\n",
- "\n",
- "The synaptic condutance-based neuron model combines the synaptic current dynamics with the passive membrane. It must be instantiated with two input arguments:\n",
- "* $\\alpha$: the decay rate of the synaptic current\n",
- "* $\\beta$: the decay rate of the membrane potential (as with Lapicque)"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "FjJQIlY9DGtR"
- },
- "source": [
- "# Decay rate of LIF states\n",
- "alpha = 0.9\n",
- "beta = 0.8\n",
- "\n",
- "# Initialize 2nd-order LIF neuron\n",
- "lif5 = snn.Synaptic(alpha=alpha, beta=beta)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ssWDmsKlDc34"
- },
- "source": [
- "Using this neuron is the exact same as Lapcique's neuron, but now with the addition of synaptic current `syn` as an input and output:\n",
- "\n",
- "**Inputs**\n",
- "* `spk_in`: each input voltage spike $S_{\\rm in}[t]$ is sequentially passed in\n",
- "* `syn`: synaptic current $I_{\\rm syn}[t]$ at the present time $t$\n",
- "* `mem`: membrane potential $U_{\\rm mem}[t]$ at the present time $t$\n",
- "\n",
- "**Outputs**\n",
- "* `spk_out`: output spike $S_{\\rm out}[t+1]$ at the next time step ('1' if there is a spike; '0' if there is no spike)\n",
- "* `syn`: synaptic current $I_{\\rm syn}[t+1]$ at the next time step\n",
- "* `mem`: membrane potential $U_{\\rm mem}[t+1]$ at the next time step\n",
- "\n",
- "These all need to be of type `torch.Tensor`."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "OCzmKeDRFeBb"
- },
- "source": [
- "Apply a periodic spiking input to see how current and membrane evolve with time:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "zWSjrxQGENPE"
- },
- "source": [
- "# Initialize hidden states and output\n",
- "syn = torch.zeros(1) # synaptic current of 0 at t=0\n",
- "mem = torch.zeros(1) # membrane potential of 0 at t=0\n",
- "spk_out = torch.zeros(1) # neuron needs somewhere to dump its output spikes\n",
- "\n",
- "# Periodic spiking input, spk_in = 0.2 V\n",
- "spk_period = torch.cat((torch.ones(1)*0.2, torch.zeros(9)), 0)\n",
- "spk_in = spk_period.repeat(20)\n",
- "\n",
- "syn_rec = [syn]\n",
- "mem_rec = [mem]\n",
- "spk_rec = [spk_out]\n",
- "\n",
- "for step in range(num_steps):\n",
- " spk_out, syn, mem = lif5(spk_in[step], syn, mem)\n",
- "\n",
- " # Store recordings of output and hidden states\n",
- " spk_rec.append(spk_out)\n",
- " syn_rec.append(syn)\n",
- " mem_rec.append(mem)\n",
- "\n",
- "# crunch -list- of tensors into one tensor\n",
- "spk_rec = torch.stack(spk_rec)\n",
- "syn_rec = torch.stack(syn_rec)\n",
- "mem_rec = torch.stack(mem_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "b1rVl69UG8xr"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(4, figsize=(8,7), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [0.4, 1, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "splt.raster(spk_in, ax[0], s=400, c=\"black\", marker=\"|\")\n",
- "ax[0].set_ylabel(\"Input Spikes\")\n",
- "ax[0].set_title(\"Synaptic Conductance-based Neuron Model With Input Spikes\")\n",
- "ax[0].set_yticks([]) \n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(syn_rec)\n",
- "ax[1].set_ylim([0, 0.5])\n",
- "ax[1].set_ylabel(\"Synaptic Current ($I_{syn}$)\")\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot membrane potential\n",
- "ax[2].plot(mem_rec)\n",
- "ax[2].set_ylim([0, 1.5])\n",
- "ax[2].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[2].axhline(y=1, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[3], s=400, c=\"black\", marker=\"|\")\n",
- "plt.ylabel(\"Output spikes\")\n",
- "ax[3].set_yticks([]) \n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "_w1uL7kbzfwW"
- },
- "source": [
- "If you're not interested in the mathematical detail, then feel free to skip this brief section. We represent the voltage spikes at the input with:\n",
- "\n",
- "$$S_{\\rm in} = \\sum_k \\delta(t-t_k),$$\n",
- "\n",
- "where each spike triggers a weighted jump in synaptic current at time $t_k$, and is followed by an exponential decay:\n",
- "\n",
- "$$I_{\\rm syn}(t) = \\sum_k W_{i,j} S_{in; i,j}(t) e^{-(t-t_k)/\\tau_{syn}}\\Theta(t-t_k)$$\n",
- "\n",
- "* $W_{i, j}$ is the weight between the the $i^{\\rm th}$ pre-synaptic neuron and the $j^{\\rm th}$ post-synaptic neuron\n",
- "\n",
- "* $t_k$ is the timing of each incident spike\n",
- "\n",
- "* $\\Theta(t)$ is the Heaviside step function, which clips the exponential term such that the contribution from each presynaptic spike commences at $t_k$\n",
- "\n",
- "* $\\tau_{syn}$ is the time constant of the synaptic current, independent of the membrane potential time constant\n",
- "\n",
- "The time constant $\\tau_{syn}$ can be equivalently represented by a decay rate $\\alpha$ that specifies the ratio of synaptic current between subsequent time steps:\n",
- "\n",
- "$$\\alpha = \\frac{e^{-\\frac{1}{\\tau_{syn}}}}{e^{-\\frac{0}{\\tau_{syn}}}} = \\frac{e^{-\\frac{2}{\\tau_{syn}}}}{e^{-\\frac{1}{\\tau_{syn}}}} = \\frac{e^{-\\frac{3}{\\tau_{syn}}}}{e^{-\\frac{2}{\\tau_{syn}}}}=~~...$$\n",
- "$$\\implies \\alpha = e^{-\\frac{1}{\\tau_{syn}}}$$\n",
- "\n",
- "\n",
- "When an input spike arrives at the neuron, the synaptic current will jump up $W_{i,j}S_{\\rm in}(t=t_k)$, where $S_{\\rm in}(t=t_k)=1$. \n",
- "\n",
- "That is to say: $\\Delta I_{\\rm syn}(t=t_k) = W_{i, j}$\n",
- "\n",
- "\n",
- "\n",
- "In summary, each spike contributes a shifted exponential decay to the synaptic current $I_{\\rm syn}$, which are all summed together. This current is then integrated by the passive membrane equation derived in the previous section, thus generating output spikes.\n",
- "\n",
- "If the math doesn't make sense, don't worry about it. A graphical intuition is usually sufficient to understand the essence of the synaptic conductance-based neuron model. \n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "This model has the same optional input arguments of `reset_mechanism` and `threshold` as described for Lapicque's neuron model."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "P-m632rfMXIb"
- },
- "source": [
- "# 3. A Feedforward Spiking Neural Network"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "iTV4g4JjMgef"
- },
- "source": [
- "So far, we have only considered how one neuron reacts to a single input stimulus. snnTorch makes it extremely straightforward to scale this up to a deep neural network. Here, we will create a 3-layer fully-connected neural network of dimensions 784-1000-10.\n",
- "\n",
- "Compared to our simulations so far, each neuron will now integrate over many more incoming input spikes. \n",
- "\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "We will use PyTorch to form the connections between neurons, and snnTorch is used to create the neurons."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "whg4LFGFNuH1"
- },
- "source": [
- "import torch \n",
- "import torch.nn as nn"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "pypN9pglN2C-"
- },
- "source": [
- "First, initialize all layers."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "LDE2ciwCOGa-"
- },
- "source": [
- "num_inputs = 784\n",
- "num_hidden = 1000\n",
- "num_outputs = 10\n",
- "\n",
- "# initialize layers\n",
- "fc1 = nn.Linear(num_inputs, num_hidden)\n",
- "lif1 = snn.Synaptic(alpha=alpha, beta=beta)\n",
- "fc2 = nn.Linear(num_hidden, num_outputs)\n",
- "lif2 = snn.Synaptic(alpha=alpha, beta=beta)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "3bh1Op3bOeLk"
- },
- "source": [
- "Next, initialize the hidden variables and outputs of each spiking neuron. \n",
- "As your networks increase in size, this will become a tedious process. So we can call a static method `init_synaptic()` to take care of this. All neurons in snnTorch have their own initialization methods that follow this same syntax, e.g., `init_lapicque()`. The shape of the hidden states are automatically initialized based on the input data dimensions during the first forward pass. "
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "p3X1_5puO3W7"
- },
- "source": [
- "# Initialize hidden states\n",
- "syn1, mem1 = lif1.init_synaptic()\n",
- "syn2, mem2 = lif2.init_synaptic()\n",
- "\n",
- "# Lists to record output traces\n",
- "mem2_rec = []\n",
- "spk1_rec = []\n",
- "spk2_rec = []"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "jotASwAQQmVJ"
- },
- "source": [
- "Create an input spike train to pass into the network. There are 200 time steps to simulate across 784 input neurons. We 'unsqueeze' the input along `dim=1` to denote this to be 'one batch' of data. So the dimensions of this input tensor must be 200 $\\times$ 1 $\\times$ 784:"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "ON4KKAjTQzf1"
- },
- "source": [
- "spk_in = spikegen.rate_conv(torch.rand((200, 784))*0.1).unsqueeze(1)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "CREME2dLPo-G"
- },
- "source": [
- "Now it's finally time to run a full simulation. \n",
- "An intuitive way to think about how PyTorch and snnTorch work together is that PyTorch routes the neurons together, and snnTorch loads the results into spiking neuron models. In terms of coding up a network, these spiking neurons can be treated like time-varying activation functions.\n",
- "\n",
- "Recall that the output of a spiking neuron is $S_{\\rm out}=1$ when a spike is triggered. This spike is then passed to the next layer. It is multiplied by the weight initialized by `nn.Linear` $S_{\\rm out; i}\\times W_{i, j}$, just as the output activation of a standard artificial neuron would be in a non-spiking neural network. The weighted spike is then passed as the input to the next layer of neurons for a given time step. If there is no spike, then nothing is passed to the post-synaptic neuron.\n",
- "\n",
- "The only difference from our simulations thus far is that we sequentially pass the output through additional layers of neurons. "
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "-5Zron7aNkxH"
- },
- "source": [
- "for step in range(num_steps):\n",
- " cur1 = fc1(spk_in[step])\n",
- " spk1, syn1, mem1 = lif1(cur1, syn1, mem1)\n",
- " cur2 = fc2(spk1)\n",
- " spk2, syn2, mem2 = lif2(cur2, syn2, mem2)\n",
- "\n",
- " mem2_rec.append(mem2)\n",
- " spk1_rec.append(spk1)\n",
- " spk2_rec.append(spk2)\n",
- "\n",
- "# convert output recordings to tensors\n",
- "mem2_rec = torch.stack(mem2_rec)\n",
- "spk1_rec = torch.stack(spk1_rec)\n",
- "spk2_rec = torch.stack(spk2_rec)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "xGpWl-S2YAyS"
- },
- "source": [
- "At this stage, the spikes don't have any real meaning. The inputs and weights are all randomly initialized, and no training has taken place. But let's take a look at the raster plots just to check that the spikes are propagating to the output layer."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "8FJuQGKfWEmR"
- },
- "source": [
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,7), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [1, 1, 0.4]})\n",
- "\n",
- "# Plot input spikes\n",
- "splt.raster(spk_in[:,0], ax[0], s=0.05, c=\"black\")\n",
- "ax[0].set_ylabel(\"Input Spikes\")\n",
- "ax[0].set_title(\"Fully Connected Spiking Neural Network\")\n",
- "\n",
- "# Plot hidden layer spikes\n",
- "splt.raster(spk1_rec.reshape(num_steps, -1), ax[1], s = 0.05, c=\"black\")\n",
- "ax[1].set_ylabel(\"Hidden Layer\")\n",
- "\n",
- "# Plot output spikes\n",
- "splt.raster(spk2_rec.reshape(num_steps, -1), ax[2], c=\"black\", marker=\"|\")\n",
- "ax[2].set_ylabel(\"Output Spikes\")\n",
- "ax[2].set_ylim([0, 10])\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "jfZwr307P5-u"
- },
- "source": [
- "We can also use `spikeplot.spike_count` to generate a spike counter of the output layer. \n",
- "Note: if you are running the notebook locally on your desktop, please uncomment the line below and modify the path to your ffmpeg.exe"
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "fAXeoiyGOm1H"
- },
- "source": [
- "from IPython.display import HTML\n",
- "\n",
- "fig, ax = plt.subplots(facecolor='w', figsize=(12, 7))\n",
- "labels=['0', '1', '2', '3', '4', '5', '6', '7', '8','9']\n",
- "spk2_rec = spk2_rec.squeeze(1).detach().cpu()\n",
- "\n",
- "# plt.rcParams['animation.ffmpeg_path'] = 'C:\\\\path\\\\to\\\\your\\\\ffmpeg.exe'\n",
- "\n",
- "# Plot spike count histogram\n",
- "anim = splt.spike_count(spk2_rec, fig, ax, labels=labels, animate=True)\n",
- "HTML(anim.to_html5_video())\n",
- "# anim.save(\"spike_bar.gif\")"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "xUXpWaPybVkd"
- },
- "source": [
- "We can also visualize the membrane potential traces with `spikeplot.traces`. We'll plot 9 out of 10 output neurons. Compare it to the animation and raster plot above to see if you can match the traces to the neuron. "
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "SpqvImMPxO36"
- },
- "source": [
- "splt.traces(mem2_rec.squeeze(1), spk=spk2_rec.squeeze(1))\n",
- "\n",
- "fig = plt.gcf() \n",
- "fig.set_size_inches(8, 6)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "z98Y9ucNac3e"
- },
- "source": [
- "# 4. Alpha Neuron Model (Hacked Spike Response Model)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "te-QotkqakDL"
- },
- "source": [
- "To finish up this tutorial, a recursive version of the Spike Response Model (SRM), or 'Alpha' neuron, is also available, called using `snntorch.Alpha`. The neuron models thus far have all been based on the passive membrane model, using ordinary differential equations to describe their dynamics.\n",
- "\n",
- "The SRM family of models, on the other hand, is interpreted in terms of a filter. Upon the arrival of an input spike, this spike is convolved with the filter to give the membrane potential response. The form of this filter can be exponential, as is the case with Lapicque's neuron, or they can be more complex such as a sum of exponentials. SRM models are appealing as they can arbitrarily add refractoriness, threshold adaptation, and any number of other features simply by embedding them into the filter. \n",
- "\n",
- "\n",
- "\n",
- " \n",
- "\n",
- "\n",
- "\n",
- " \n",
- "\n",
- "Formally, this process is represented by:\n",
- "\n",
- "$$U_{\\rm mem}(t) = \\sum_i W_{i, j}(\\epsilon * S_{\\rm in; i,j})(t)$$\n",
- "\n",
- "where the incoming spikes $S_{\\rm in; i,j}$ are convolved with a spike response kernel $\\epsilon( \\cdot )$. The spike response is scaled by a synaptic weight, $W_{i, j}$. In the figures above, the left kernel is an exponentially decaying function and would be the equivalent of Lapicque's neuron model. On the right, the kernel is an alpha function.\n",
- "\n",
- "In snnTorch, the spike response model is not directly implemented as a filter. Instead, it is recast into a recursive form such that only the previous time step of values are required to calculate the next set of values. This significantly reduces the memory overhead during learning.\n",
- "\n",
- "The filter adopted is unsurprisingly the alpha function on the right animation above, or equivalently a sum of two exponentials. This results in a membrane potential which peaks at some time delay $t_d$ after the input spike. This is often a desirable feature when training networks that rely on spike timing.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "\n",
- "As the membrane potential is now determined by the sum of two exponentials, each of these exponents has their own independent decay rate. $\\alpha$ defines the decay rate of the positive exponential, and $\\beta$ defines the decay rate of the negative exponential. "
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "3OiADDv7Z_z7"
- },
- "source": [
- "alpha = 0.8\n",
- "beta = 0.7\n",
- "\n",
- "# initialize neuron\n",
- "lif6 = snn.Alpha(alpha=alpha, beta=beta, threshold=0.5)"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "K8yVKbpyZV4J"
- },
- "source": [
- "Using this neuron is the same as the previous neurons, but the sum of two exponential functions requires the synaptic current `syn` to be split into a `syn_exc` and `syn_inh` component:\n",
- "\n",
- "**Inputs**\n",
- "* `spk_in`: each input voltage spike $S_{\\rm in}[t]$ is sequentially passed in\n",
- "* `syn_exc`: excitatory post-synaptic current $I_{\\rm syn_exc}[t]$ at the present time $t$\n",
- "* `syn_inh`: inhibitory post-synaptic current $I_{\\rm syn_inh}[t]$ at the present time $t$\n",
- "* `mem`: membrane potential $U_{\\rm mem}[t]$ at the present time $t$\n",
- "\n",
- "**Outputs**\n",
- "* `spk_out`: output spike $S_{\\rm out}[t+1]$ at the next time step ('1' if there is a spike; '0' if there is no spike)\n",
- "* `syn_exc`: excitatory post-synaptic $I_{\\rm syn_exc}[t+1]$ at the next time step $t$\n",
- "* `syn_inh`: inhibitory post-synaptic current $I_{\\rm syn_inh}[t+1]$ at the next time step $t$\n",
- "* `mem`: membrane potential $U_{\\rm mem}[t+1]$ at the next time step\n",
- "\n",
- "As with all other neuron models, these must be of type `torch.Tensor`."
- ]
- },
- {
- "cell_type": "code",
- "metadata": {
- "id": "Bf2fjxBZajeT"
- },
- "source": [
- "# input spike: initial spike, and then period spiking \n",
- "spk_in = (torch.cat((torch.zeros(10), torch.ones(1), torch.zeros(89), (torch.cat((torch.ones(1), torch.zeros(9)),0).repeat(10))), 0) * 0.85).unsqueeze(1)\n",
- "print(f\"spk_in contains {spk_in.size(1)} sample of data across {spk_in.size(0)} time steps.\")\n",
- "\n",
- "# initialize parameters - arg '1' is passed to indicate just one sample of data\n",
- "syn_exc, syn_inh, mem = lif6.init_alpha()\n",
- "mem_rec = []\n",
- "spk_rec = []\n",
- "\n",
- "# run simulation\n",
- "for step in range(num_steps):\n",
- " spk_out, syn_exc, syn_inh, mem = lif6(spk_in[step], syn_exc, syn_inh, mem)\n",
- "\n",
- " mem_rec.append(mem.squeeze(0))\n",
- " spk_rec.append(spk_out.squeeze(0))\n",
- "\n",
- "mem_rec = torch.stack(mem_rec)\n",
- "spk_rec = torch.stack(spk_rec)\n",
- "\n",
- "# Generate Plots\n",
- "fig, ax = plt.subplots(3, figsize=(8,6), sharex=True, \n",
- " gridspec_kw = {'height_ratios': [0.4, 1, 0.4]})\n",
- "\n",
- "# Plot input current\n",
- "splt.raster(spk_in, ax[0], s=400, c=\"black\", marker=\"|\")\n",
- "ax[0].set_ylabel(\"Input Spikes\")\n",
- "ax[0].set_title(\"Alpha Neuron Model With Input Spikes\")\n",
- "ax[0].set_yticks([]) \n",
- "\n",
- "# Plot membrane potential\n",
- "ax[1].plot(mem_rec.detach())\n",
- "ax[1].set_ylim([0, 0.6])\n",
- "ax[1].set_ylabel(\"Membrane Potential ($U_{mem}$)\")\n",
- "ax[1].axhline(y=0.5, alpha=0.25, linestyle=\"dashed\", c=\"black\", linewidth=2)\n",
- "plt.xlabel(\"Time step\")\n",
- "\n",
- "# Plot output spike using spikeplot\n",
- "splt.raster(spk_rec, ax[2], s=400, c=\"black\", marker=\"|\")\n",
- "ax[2].set_yticks([])\n",
- "ax[2].set_ylabel(\"Output Spikes\")\n",
- "\n",
- "plt.show()"
- ],
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "63V7WsZFa_Uo"
- },
- "source": [
- "As with the Lapicque and Synaptic models, the Alpha model also has options to modify the threshold and reset mechanism."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "PYWhh7idiS9t"
- },
- "source": [
- "# Conclusion"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "BGCY7g7R0Usy"
- },
- "source": [
- "Now you should understand the basics of several LIF neuron models, how to simulate them, and how to build your own feedforward spiking neural networks.\n",
- "\n",
- "For reference, the documentation [can be found here](https://snntorch.readthedocs.io/en/latest/snntorch.html).\n",
- "\n",
- "In the next tutorial, you will learn how to train these networks to classify spiking and static MNIST datasets."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "OTRCyd0Xa-QK"
- },
- "source": [
- "## Further Reading\n",
- "* [snnTorch documentation](https://snntorch.readthedocs.io/en/latest/snntorch.html) of the Lapicque, Leaky, Synaptic, and Alpha models\n",
- "* [*Neuronal Dynamics:\n",
- "From single neurons to networks and models of cognition*](https://neuronaldynamics.epfl.ch/index.html) by\n",
- "Wulfram Gerstner, Werner M. Kistler, Richard Naud and Liam Paninski.\n",
- "* [Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems](https://mitpress.mit.edu/books/theoretical-neuroscience) by Laurence F. Abbott and Peter Dayan"
- ]
- }
- ]
-}
\ No newline at end of file
diff --git a/examples/legacy/tutorial_3_FCN.ipynb b/examples/legacy/tutorial_3_FCN.ipynb
deleted file mode 100644
index bf4f77f0..00000000
--- a/examples/legacy/tutorial_3_FCN.ipynb
+++ /dev/null
@@ -1,1095 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "uSGZ6cdmpknm",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "\n",
- "\n",
- "# snnTorch - Deep Learning with ``snntorch``\n",
- "## Tutorial 3\n",
- "### By Jason K. Eshraghian (www.jasoneshraghian.com)\n",
- "\n",
- "\n",
- " \n",
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Ymi3sqJg28OQ"
- },
- "source": [
- "# Introduction\n",
- "In this tutorial, you will:\n",
- "* Learn how spiking neurons are implemented in a recurrent network\n",
- "* Understand backpropagation through time, and the associated challenges in SNNs such as target labeling, and the non-differentiability of spikes\n",
- "* Train a fully-connected network on the static MNIST dataset\n",
- "\n",
- "\n",
- "\n",
- ">Part of this tutorial was inspired by Friedemann Zenke's extensive work on SNNs. Check out his repo on surrogate gradients [here](https://github.com/fzenke/spytorch), and a favourite paper of mine: E. O. Neftci, H. Mostafa, F. Zenke, [Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks.](https://ieeexplore.ieee.org/document/8891809) IEEE Signal Processing Magazine 36, 51–63.\n",
- "\n",
- "As a quick recap, [Tutorial 1](https://colab.research.google.com/github/jeshraghian/snntorch/blob/tutorials/examples/tutorial_1_spikegen.ipynb) explained how to convert datasets into spikes using three encoding mechanisms:\n",
- "* Rate coding\n",
- "* Latency coding\n",
- "* Delta modulation\n",
- "\n",
- "[Tutorial 2](https://colab.research.google.com/github/jeshraghian/snntorch/blob/tutorials/examples/tutorial_2_neuronal_dynamics.ipynb) showed how to build neural networks using three different leaky integrate-and-fire (LIF) neuron models:\n",
- "* Lapicque's RC model\n",
- "* Synaptic Conductance-based model\n",
- "* Alpha neuron model\n",
- "\n",
- "At the end of the tutorial, a basic supervised learning algorithm will be implemented. We will use the original static MNIST dataset and train a multi-layer fully-connected spiking neural network using gradient descent to perform image classification. \n",
- "\n",
- "If running in Google Colab:\n",
- "* You may connect to GPU by checking `Runtime` > `Change runtime type` > `Hardware accelerator: GPU`\n",
- "* Next, install the latest PyPi distribution of snnTorch by clicking into the following cell and pressing `Shift+Enter`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "5tn_wUlopkon",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "!pip install snntorch"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "gt2xMbLY9dVE"
- },
- "source": [
- "# 1. A Recurrent Representation of SNNs"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "v7haBG7nA_TC"
- },
- "source": [
- "The following is a summary of the continuous time-domain representation LIF neurons, and applies the result to develop a recurrent representation that is more suitable for use in recurrent neural networks (RNNs). \n",
- "\n",
- "We derived the dynamics of the passive membrane using an RC circuit in the time-domain: \n",
- "\n",
- "$$ \\tau_{\\rm mem} \\frac{dU_{\\rm mem}(t)}{dt} = -U_{\\rm mem}(t) + RI_{\\rm syn}(t),$$\n",
- "\n",
- "where the general solution of this equation is:\n",
- "\n",
- "$$U_{\\rm mem}=I_{\\rm syn}(t)R + [U_0 - I_{\\rm syn}(t)R]e^{-t/\\tau_{\\rm mem}}$$\n",
- "\n",
- "In Lapicque's model, $I_{\\rm syn}(t)$ is also the input current, $I_{\\rm in}(t)$. \n",
- "\n",
- "In the Synaptic conductance-based model (which we will loosely refer to as the synaptic model), a more biologically plausible approach is taken that ensures $I_{\\rm syn}(t)$ follows an exponential decay as a function of the input:\n",
- "\n",
- "\n",
- "$$I_{\\rm syn}(t) = \\sum_k W_{i,j} S_{in; i,j}(t) e^{-(t-t_k)/\\tau_{syn}}\\Theta(t-t_k)$$\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "The synaptic model has two exponentially decaying terms: $I_{\\rm syn}(t)$ and $U_{\\rm mem}(t)$. The ratio between subsequent terms (i.e., decay rate) of $I_{\\rm syn}(t)$ is set to $\\alpha$, and that of $U_{\\rm mem}(t)$ is set to $\\beta$:\n",
- "\n",
- "$$ \\alpha = e^{-1/\\tau_{\\rm syn}}$$\n",
- "\n",
- "$$ \\beta = e^{-1/\\tau_{\\rm mem}}$$\n",
- "\n",
- "\n",
- "RNNs will process data sequentially, and so time must be discretised, and the neuron models must be converted into a recursive form. $\\alpha$ and $\\beta$ can be used to give a recursive representation of the Synaptic neuron model:\n",
- "\n",
- "$$I_{\\rm syn}[t+1]=\\underbrace{\\alpha I_{\\rm syn}[t]}_\\text{decay} + \\underbrace{WS_{\\rm in}[t+1]}_\\text{input}$$\n",
- "\n",
- "$$U[t+1] = \\underbrace{\\beta U[t]}_\\text{decay} + \\underbrace{I_{\\rm syn}[t+1]}_\\text{input} - \\underbrace{R[t+1]}_\\text{reset}$$\n",
- "\n",
- "**Spiking**\n",
- "\n",
- "If $U[t] > U_{\\rm thr}$, then an output spike is triggered: $S_{\\rm out}[t] = 1$. Otherwise, $S_{\\rm out}[t] = 0$. \n",
- "\n",
- "> Note: A variation of this is to set the output spike at the *next* time step to be triggered; i.e., $U[t] > U_{\\rm thr} \\implies S_{\\rm out}[t+1] = 1$. This is the approach taken in snnTorch, and will be explained in following sections.\n",
- "\n",
- "An alternative way to represent the relationship between $S_{\\rm out}$ and $U_{\\rm mem}$, which is also used to calculate the gradient in the backward pass, is:\n",
- "\n",
- "$$S_{\\rm out}[t] = \\Theta(U_{\\rm mem}[t] - U_{\\rm thr})$$ \n",
- "\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "**Reset**\n",
- "\n",
- "The reset term is activated only when the neuron triggers a spike. That is to say, if $S_{\\rm out}[t+1]=1$:\n",
- "\n",
- " * For `reset_mechanism=\"subtract\"`: $R[t+1]=U_{\\rm thr}$ \n",
- " * For `reset_mechanism=\"zero\"`: $R[t+1]=U[t+1]$\n",
- "\n",
- "> Note: In snnTorch, the reset will also take a one time step delay such that $R[t+1]$ is activated only when $S_{\\rm out}[t+1]=1$\n",
- "\n",
- "The other neurons follow a similar form, which is [detailed in the documentation](https://snntorch.readthedocs.io/en/latest/snntorch.html). The recursive neuron equations can be mapped into computation graphs, where the recurrent connections take place with a delay of a single time step, from the state at time $t$ to the state at time $t+1$. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "t256yMrzTU6M"
- },
- "source": [
- "An alternative way to represent recurrent models is to unfold the computational graph, in which each component is represented by a sequence of different variables, with one variable per time step. The unfolded form of the Synaptic model is shown below:\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "\n",
- "Up until now, the notation used for all variables have had an association with their electrical meanings. As we move from neuronal dynamics to deep learning, we will slightly modify the notation throughout the rest of the tutorial:\n",
- "\n",
- "* **Input spike:** $S_{\\rm in} \\rightarrow X$\n",
- "* **Input current (weighted spike):** $I_{\\rm in} \\rightarrow Y$\n",
- "* **Synaptic current:** $I_{\\rm syn} \\rightarrow I$\n",
- "* **Membrane potential:** $U_{\\rm mem} \\rightarrow U$\n",
- "* **Output spike:** $S_{\\rm out} \\rightarrow S$\n",
- "\n",
- "The benefit of an unrolled graph is that we now have an explicit description of how computations are performed. The process of unfolding illustrates the flow of information forward in time (from left to right) to compute outputs and losses, and backward in time to compute gradients. The more time steps that are simulated, the deeper the graph becomes. \n",
- "\n",
- "Conventional RNNs treat $\\alpha$ and $\\beta$ as learnable parameters. This is also possible for SNNs, but in snnTorch, they are treated as hyperparameters by default. This replaces the vanishing and exploding gradient problems with a parameter search."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "zqJdfllYbc16"
- },
- "source": [
- "# 2. Setting up the Static MNIST Dataset"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "SuOWJNEMe8l_"
- },
- "source": [
- "Much of the following code has already been explained in the first two tutorials. So we'll dive straight in. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "nbunSP5TbikZ"
- },
- "source": [
- "## 2.1 Import packages and setup the environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "bEFWu3nNpkoq",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "import snntorch as snn\n",
- "import torch\n",
- "import torch.nn as nn\n",
- "from torch.utils.data import DataLoader\n",
- "from torchvision import datasets, transforms\n",
- "import numpy as np\n",
- "import itertools\n",
- "import matplotlib.pyplot as plt"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "lI0GbgLgpkos",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Network Architecture\n",
- "num_inputs = 28*28\n",
- "num_hidden = 1000\n",
- "num_outputs = 10\n",
- "\n",
- "# Training Parameters\n",
- "batch_size=128\n",
- "data_path='/tmp/data/mnist'\n",
- "\n",
- "# Temporal Dynamics\n",
- "num_steps = 25\n",
- "alpha = 0.7\n",
- "beta = 0.8\n",
- "\n",
- "dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "nUS6YFXbpkos",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 2.2 Download MNIST Dataset"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "2fhRixcspkot",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Define a transform\n",
- "transform = transforms.Compose([\n",
- " transforms.Resize((28, 28)),\n",
- " transforms.Grayscale(),\n",
- " transforms.ToTensor(),\n",
- " transforms.Normalize((0,), (1,))])\n",
- "\n",
- "mnist_train = datasets.MNIST(data_path, train=True, download=True, transform=transform)\n",
- "mnist_test = datasets.MNIST(data_path, train=False, download=True, transform=transform)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "RAM_dP887uTq"
- },
- "source": [
- "If the above code blocks throws an error, e.g. the MNIST servers are down, then uncomment the following code instead."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "4jyJVqUNdXDo"
- },
- "outputs": [],
- "source": [
- "# # temporary dataloader if MNIST service is unavailable\n",
- "# !wget www.di.ens.fr/~lelarge/MNIST.tar.gz\n",
- "# !tar -zxvf MNIST.tar.gz\n",
- "\n",
- "# mnist_train = datasets.MNIST(root = './', train=True, download=True, transform=transform)\n",
- "# mnist_test = datasets.MNIST(root = './', train=False, download=True, transform=transform)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "aEtCbO6upkou",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Create DataLoaders\n",
- "train_loader = DataLoader(mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
- "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=True)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "GhFyzySNeT_e"
- },
- "source": [
- "# 3. Define the Network"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "RJkoAg-3pkow",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "The spiking neurons available in snnTorch are designed to be treated as activation units. The only difference is that these spiking neuron activations depend not only on their inputs, but also on their previous state (e.g., $I[t-1]$ and $U[t-1]$ for the Synaptic neuron). This can be implemented in a for-loop with ease.\n",
- "\n",
- "If you have a basic understanding of PyTorch, the following code block should look familiar. `nn.Linear` initializes the linear transformation layer, and instead of applying a sigmoid, ReLU or some other nonlinear activation, a spiking neuron is applied instead by calling `snn.Synaptic`:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "-uquHLLmpkox",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Define Network\n",
- "class Net(nn.Module):\n",
- " def __init__(self):\n",
- " super().__init__()\n",
- "\n",
- " # Initialize layers\n",
- " self.fc1 = nn.Linear(num_inputs, num_hidden)\n",
- " self.lif1 = snn.Synaptic(alpha=alpha, beta=beta)\n",
- " self.fc2 = nn.Linear(num_hidden, num_outputs)\n",
- " self.lif2 = snn.Synaptic(alpha=alpha, beta=beta)\n",
- "\n",
- " def forward(self, x):\n",
- "\n",
- " # Initialize hidden states and outputs at t=0\n",
- " syn1, mem1 = self.lif1.init_synaptic()\n",
- " syn2, mem2 = self.lif2.init_synaptic()\n",
- " \n",
- " # Record the final layer\n",
- " spk2_rec = []\n",
- " mem2_rec = []\n",
- "\n",
- " for step in range(num_steps):\n",
- " cur1 = self.fc1(x)\n",
- " spk1, syn1, mem1 = self.lif1(cur1, syn1, mem1)\n",
- " cur2 = self.fc2(spk1)\n",
- " spk2, syn2, mem2 = self.lif2(cur2, syn2, mem2)\n",
- "\n",
- " spk2_rec.append(spk2)\n",
- " mem2_rec.append(mem2)\n",
- "\n",
- " return torch.stack(spk2_rec, dim=0), torch.stack(mem2_rec, dim=0)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Y0fHcAKfrav6"
- },
- "source": [
- "The code in the `forward()` function will only be called once the input argument `x` is explicitly passed in:\n",
- "\n",
- "* `fc1` applies a linear transformation to the input: $W_{i, j}^{[1]}X_{i}^{[1]}[t] \\rightarrow Y_{j}^{[1]}[t]$, i.e., `cur1`\n",
- "* `lif1` integrates $Y^{[1]}_{j}[t]$ over time (with a decay), to generate $I_{j}^{[1]}[t]$ and $U_{j}^{[1]}[t]$. An output spike is triggered if $U_{j}^{[1]}[t] > U_{\\rm thr}$. Equivalently, `spk1=1` if `mem1` > `threshold=1.0`\n",
- "* `fc2` applies a linear transformation to `spk1`: $W_{j, k}^{[2]}S_{j}^{[1]}[t] \\rightarrow Y_{k}^{[2]}[t]$, i.e., `cur2`\n",
- "* `lif2` is another spiking neuron layer, and generates output spikes $S_{k}^{[2]}[t]$ which are returned in the variable `spk2`\n",
- "\n",
- "Here, $i$ denotes one of 784 input neurons, $j$ indexes one of the 1,000 neurons in the hidden layer, and $k$ points to one of 10 output neurons.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "3-yJXOSQqANb"
- },
- "source": [
- "The layers in `def __init__(self)` are automatically created upon instantiating `Net()`, as is done below:\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "EkTWSXj5fj2V"
- },
- "outputs": [],
- "source": [
- "# Load the network onto CUDA if available\n",
- "net = Net().to(device)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "6a7MdORCtIx4"
- },
- "source": [
- "# 4. Backpropagation for SNNs"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "ZlrNIMNnpkoy",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "A few questions arise when setting up a backprop-driven learning algorithm:\n",
- "\n",
- "1. **Targets**: What should the target of the output layer be?\n",
- "2. **Backprop through time**: How might the gradient flow back in time?\n",
- "3. **Spike non-differentiability**: If spikes are discrete, instantaneous bursts of information, doesn't that make them non-differentiable? If the output spike has no gradient with respect to the network parameters, wouldn't backprop be impossible?\n",
- "\n",
- "Let's tackle these one by one. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "NpJhgA6n8LPt"
- },
- "source": [
- "## 4.1 Target Labels"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "k0At_jfe8OLY"
- },
- "source": [
- "In [tutorial 1](https://colab.research.google.com/github/jeshraghian/snntorch/blob/tutorials/examples/tutorial_1_spikegen.ipynb), we learnt about rate and latency coding. Rate coding stores information in the frequency of spikes, and latency coding stores information in the timing of each spike. Previously, we used these encoding strategies to convert datasets into time-varying spikes. Here, they are used as encoding strategies for the output layer of our SNN. I.e., these codes will be used to teach the final layer of the network how to respond to certain inputs. \n",
- "\n",
- "The goal of the SNN is to predict a discrete variable with $n$ possible values, as is the case with MNIST where $n=10$. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "OnSx21wvAoee"
- },
- "source": [
- "### 4.1.1 Rate code"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "4kR56xe3Ari3"
- },
- "source": [
- "For rate encoding, the most naive implementation is to encourage the correct class to fire at every time step, and the incorrect classes to not fire at all. There are two ways to implement this, one of which is a lot more effective than the other:\n",
- "\n",
- "* Set the target of the output spike of the correct class $y_{\\rm spk} = 1$ for all $t$, or\n",
- "* Set the target of the membrane potential of the correct class $y_{\\rm mem} = U_{\\rm thr}$ for all $t$ \n",
- "\n",
- "Which is the better approach? \n",
- "\n",
- "**Spiking Targets**\n",
- "\n",
- "Consider the first option. The output spikes are discrete events, and rely on large perturbations of the membrane potential around the threshold to have any infleunce. If the output spiking behavior goes unchanged, the gradient of the output of the network with respect to its parameters would be $0$. This is problematic, because the training process would no longer have a guide for how to improve the weights. It would be an ineffective approach for gradient descent. \n",
- "\n",
- "**Membrane Potential Targets**\n",
- "\n",
- "Instead, it is better to promote spiking by applying the target to the membrane potential. As the membrane potential is a much stronger function of the parameters, (i.e., a small perturbation of the weights would directly perturb the membrane potential), this would ensure there is a strong gradient whenever the network obtains a wrong result. So we set $y_{\\rm mem} = U_{\\rm thr}$. By default, `threshold=1`. The outputs can then be applied to a softmax unit, which are then used to find the cross-entropy loss:\n",
- "\n",
- "$$CE = - \\sum^n_{i=1}y_{i,\\rm mem} {\\rm log}(p_i),$$\n",
- "\n",
- "where $y_{i, \\rm mem}$ is the target label at a given time step, $n$ is the number of classes, and $p_i$ is the softmax probability for the $i^{th}$ class. \n",
- "\n",
- "The accuracy of the network would then be measured by counting up how many times each neuron fired across all time steps. We could then use `torch.max()` to choose the neuron with the most spikes, or somewhat equivalently, the highest average firing rate. \n",
- "\n",
- "It is possible to increase the target of membrane potential beyond the threshold to excite the neuron further. While this may be desirable in some instances, it will likely trigger high-conductance pathways for the wrong class when training other samples."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "b-xv_G0_304H"
- },
- "source": [
- "*Our classifier will implement the simplest form of rate coding. It will encourage the correct class to fire 100% of time steps, and the incorrect class to fire 0% of the time. Although this is clearly not the most efficient method, it is the simplest.*"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "iknCYQEeCaHG"
- },
- "source": [
- "### 4.1.2 Latency code\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "92hSKSBogC0I"
- },
- "source": [
- "In latency encoding, the neuron that fires first is the predicted class. The target may be set to 1 for one of the first few time steps. Depending on the neuron model being used, it will take several time steps before the input can propagate to the output of the network. Therefore, it is inadvisable to set the target to `1` only for the first time step. \n",
- "\n",
- "Consider the case of a neuron receiving an input spike. Depending on the neuron model in use, the post-synaptic potential may experience a time delay $t_{\\rm psp}$ to reach the peak of its membrane potential, and subsequently emit an output spike. If this neuron is connected in a deep neural network, the minimum time before the final layer could generate output spikes *as a result of the input (and not biases)* would thus be $t_{\\rm min} = Lt_{\\rm psp}$, where $L$ is the number of layers in the network. \n",
- "\n",
- "For the Synaptic and Lapicque models, the membrane potential will immediately jump as a result of the input. But there is a time delay of one step before the output spike can be triggered as a result. Therefore, we set $t_{\\rm psp}=1$ time step. For the Alpha neuron model, it will take a longer time to reach the peak, and is a function of the decay rates, $\\alpha$ and $\\beta$. \n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "In absence of this post-synaptic potential delay, it becomes challenging to control the output layer in terms of spike timing. An input spike of a multi-layer SNN could effectively be transmitted straight to the output instantaneously, without considering the input data at any later time steps. A slight modification is made to the unrolled computational graph, which adds a delay of one time step between $U$ and $S$.\n",
- "\n",
- "
\n",
- "\n",
- "
"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "UYDvqr1oMHQp"
- },
- "source": [
- "\n",
- "As for the incorrect classes, it is acceptable to set their targets to 0. However, this could result in low conductance pathways that completely inhibit firing. It may be preferable to set their membrane potential target to something slightly higher, e.g., $U_{\\rm thr}/5$. The optimal point is a topic of further investigation. Note that all of the above can have a cross-entropy loss applied, just as with rate coding.\n",
- "\n",
- "A simple example across 4 time steps is provided in the image below, though the values and spiking periodicity should not be taken literally.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "An alternative approach is to treat the number of time steps as a continuous variable and use a mean square error loss to dictate when firing should occur:\n",
- "\n",
- "$$MSE = \\sum^n_{t=1}(t_{\\rm spk} - \\hat{t_{\\rm spk}}^2),$$\n",
- "\n",
- "where $t$ is the time step, and $n$ is the total number of steps. In such a case, a larger number of time steps are expected to improve performance as it will allow the flow of time to look more 'continuous'.\n",
- "\n",
- "Is there a preference between latency and rate codes? We briefly touched on this question in the context of data encoding, and the same arguments apply here. Latency codes are desirable because they only rely on a single spike to convey all necessary information. Rate coding spreads out information across many time steps, and there is much less information transfer within each spike. Therefore, latency codes are much more power efficient when running on neuromorphic hardware. On the other hand, the redundant spikes in rate codes makes them much more noise tolerant. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "9SBjiLAX-ZhX"
- },
- "source": [
- "## 4.2 Backpropragation Through Time"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "8UYEmLk4-e4G"
- },
- "source": [
- "Computing the gradient through an SNN is mostly the same as that of an RNN. The generalized backpropagation algorithm is applied to the unrolled computational graph. Working backward from the end of the sequence, the gradient flows from the loss to all descendents. Shown below are the various pathways of the gradient $\\nabla_W \\mathcal{L}$ from the parent ($\\mathcal{L}$: cross-entropy loss) to its leaf nodes ($W$). \n",
- "\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "The learnable parameter $W$ is shared across each time step. This means that multiple backprop paths exist between the loss and the same network parameter. To resolve this, all gradients $\\nabla_W \\mathcal{L}$ are simply summed together before applying a weight update.\n",
- "\n",
- "To find $\\nabla_W \\mathcal{L}$, the chain rule is applied to each pathway. \n",
- "\n",
- "**Shortest Pathway** \n",
- "\n",
- "Considering only the shortest pathway at $t=3$, where the superscript $^{<1>}$ indicates this is just one of many paths to be summed:\n",
- "\n",
- "$$\\nabla_W \\mathcal{L}^{<1>} = \\frac{\\partial{\\mathcal{L}}}{\\partial{p_i}} \\frac{\\partial{p_i}}{\\partial{U[3]}} \\frac{\\partial{U[3]}}{\\partial{Y[3]}} \\frac{\\partial{Y[3]}}{\\partial{W}}$$\n",
- "\n",
- "The first two terms can be analytically solved by taking the derivative of the cross-entropy loss and the softmax function. The third term must be decomposed into the following terms:\n",
- "\n",
- "$$ \\frac{\\partial{U[3]}}{\\partial{Y[3]}} = \\frac{\\partial{U[3]}}{\\partial{I[3]}} \\frac{\\partial{I[3]}}{\\partial{Y[3]}}$$\n",
- "\n",
- "Recall the recursive form of the Synaptic neuron model:\n",
- "\n",
- "\n",
- "$$I[t+1]=\\alpha I[t] + WX[t+1]$$\n",
- "\n",
- "$$U[t+1] = \\beta U[t] + I[t+1] - R[t+1]$$\n",
- "\n",
- "$WX=Y$ is directly added to $I$, which is directly added to $U$. Therefore, both partial derivative terms evaluate to 1:\n",
- "\n",
- "$$\\frac{\\partial{U[3]}}{\\partial{Y[3]}} = 1$$\n",
- "\n",
- "The final term $ \\frac{\\partial{Y[3]}}{\\partial{W}}$ evaluates to the input at that time step $X[3]$. \n",
- "\n",
- "**2nd Shortest Pathways**\n",
- "\n",
- "Consider the pathway that flows backwards one time step from $t=3$ to $t=2$ through $\\beta$:\n",
- "\n",
- "$$\\nabla_W \\mathcal{L}^{<2>} = \\frac{\\partial{\\mathcal{L}}}{\\partial{p_i}} \\frac{\\partial{p_i}}{\\partial{U[3]}} \\frac{\\partial{U[3]}}{\\partial{U[2]}} \n",
- "\\frac{\\partial{U[2]}}{\\partial{Y[2]}} \\frac{\\partial{Y[2]}}{\\partial{W}} $$\n",
- "\n",
- "Almost all terms are the same as the shortest pathway calculation, or at least evaluate to the same values. The only major difference is the third term, which signals the backwards flow through time: $U[3] \\rightarrow U[2]$. The derivative is simply $\\beta$. \n",
- "\n",
- "The parallel pathway flowing through $I[3] \\rightarrow I[2]$ follows the same method, but instead, $\\frac{\\partial{I[3]}}{\\partial{I[2]}} = \\alpha$. \n",
- "\n",
- "An interesting result arises: for each additional time step the graph flows through, the smaller that component of the gradient becomes. This is because each backwards path is recursively multiplied by either $\\alpha$ or $\\beta$, which gradually diminish the contribution of earlier states of the network to gradient.\n",
- "\n",
- "Luckily for you, all of this is automatically taken care of by PyTorch's autodifferentiation framework. Variations of backprop through time are also available within snnTorch, which will be demonstrated in future tutorials.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "c7nYrxNFLybB"
- },
- "source": [
- "## 4.3 Non-differentiability of Spikes"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "yXWeOeqZ5S_r"
- },
- "source": [
- "The above analysis only solved for parameter updates for the final layer. This was not an issue as we used membrane potential $U$ to calculate the loss, which is a continuous function. If we backpropagate to earlier layers, we need to take the derivative of spikes, i.e., a non-differentiable, non-continuous function.\n",
- "\n",
- "Let's open up the computational graph of the Synaptic neuron model to identify exactly where this problem occurs.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "Backpropagating through the shortest path gives:\n",
- "$$\\frac{\\partial{S[3]}}{\\partial{Y[2]}} = \\frac{\\partial{S[3]}}{\\partial{U[2]}} \\frac{\\partial{U[2]}}{\\partial{I[2]}}\\frac{\\partial{I[2]}}{\\partial{Y[2]}}$$\n",
- "\n",
- "The final two terms evaluate to 1 for the same reasons described above. But the first term is non-differentiable. Recall how $S=1$ only for $U>U_{\\rm thr}$, i.e., a shifted form of the Heaviside step function. The analytical derivative evaluates to 0 everywhere, except at $U_{\\rm thr}: \\frac{\\partial{S[t]}}{\\partial{U[t-1]}} \\rightarrow \\infty$. This is the result generated by PyTorch's default autodifferentiation framework, and will zero out the gradient thus immobilizing the network's ability to learn:\n",
- "\n",
- "$$W := W - \\eta \\nabla_W \\mathcal{L} $$\n",
- "\n",
- "where $\\nabla_W \\mathcal{L} \\rightarrow 0$. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "ktpVgJisQU03"
- },
- "source": [
- "How do we overcome this issue? Several approaches have been taken and yielded great results. Smooth approximations of the Heaviside function have been used, taking gradients of the continuous function instead. Friedemann Zenke's extensive work on surrogate gradients is among the most rigorous on this topic, and is [very well documented here](https://github.com/fzenke/spytorch). The option to use surrogate gradients is available in snnTorch as well, and can be called from the `snntorch.surrogate` library. More details are available [here](https://snntorch.readthedocs.io/en/latest/snntorch.surrogate.html).\n",
- "\n",
- "snnTorch takes a wholly different approach that is simple, yet effective. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "iBHz22CdgE0k"
- },
- "source": [
- "## 4.3.1 A Time-Evolution Approach to the Spiking Derivative"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "r8W_hVKkf4HM"
- },
- "source": [
- "What follows is a simple, intuitive description behind the approach taken. A rigorous mathematical treatment will be made available separately. \n",
- "\n",
- "The analytical derivative of $S$ with respect to $U$ neglects two features of spiking neurons:\n",
- "\n",
- "* the discrete time representation of SNNs \n",
- "* spike-induced reset and refractory periods of neurons\n",
- "\n",
- "**Discrete Time Representation**\n",
- "\n",
- "Given that SNNs (and more generally, RNNs) operate in discrete time, we can approximate the derivative to be the relative change across 1 time step:\n",
- "\n",
- "$$\\frac{\\partial S}{\\partial U} \\rightarrow \\frac{\\Delta S}{\\Delta U}$$\n",
- "\n",
- "Intuitively, the time derivative cannot be calculated by letting $\\Delta t \\rightarrow 0$, but rather, it must approach the smallest possible value $\\Delta t \\rightarrow 1$. It therefore follows that the derivative of a time-varying pair of functions must be treated similarly.\n",
- "\n",
- "**Spike-induced Reset**\n",
- "\n",
- "Next, the occurrence of a spike necessarily incurs a membrane potential reset. So when the spike mechanism switches off: $S: 1 \\rightarrow 0$, the membrane potential resets by subtraction of the threshold, which is set to one by default: $\\Delta U = U_{\\rm thr} \\rightarrow -1$:\n",
- "\n",
- "$$\\frac{\\Delta S}{\\Delta U} = \\frac{-1}{-1} = 1$$\n",
- "\n",
- "This situation is illustrated below:\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "If instead there is no spike, then $\\Delta S = 0$ for a finite change in $U$. Formally:\n",
- "\n",
- "\\begin{equation}\n",
- " \\frac{\\partial S}{\\partial U} \\approx \\Theta(U - U_{\\rm thr}) = \n",
- " \\begin{cases}\n",
- " 1 & \\text{if $S$ = $1$}\\\\\n",
- " 0 & \\text{if $S$ = $0$}\n",
- " \\end{cases} \n",
- "\\end{equation}\n",
- "\n",
- "This is simply the Heaviside step function shifted about the membrane threshold, $U_{\\rm thr} = \\theta$.\n",
- "\n",
- "
\n",
- "\n",
- "
\n",
- "\n",
- "What this suggests is that learning only takes place when neurons fire. This is generally not a concern, as a large enough network will have sufficient spiking to enable a gradient to flow through the computational graph. Armed with the knowledge that weight updates only take place when neurons fire, this approach echoes a rudimentary form of Hebbian learning.\n",
- "\n",
- "Importantly, the situation is more nuanced than what has been described above. But this should be sufficient to give you the big picture intuition. As a matter of interest, the Heaviside gradient takes a similar approach to how the gradient flows through a max-pooling unit, and also evaluates to the same derivative as a shifted ReLU activation. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "uDFwKmN9en4i"
- },
- "source": [
- "# 5. Training on Static MNIST"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "6D-fhT3Q7nXM"
- },
- "source": [
- "\n",
- "Time for training! Let's first define a couple of functions to print out test/train accuracy."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "-IxcnBAxpkoy",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "def print_batch_accuracy(data, targets, train=False):\n",
- " output, _ = net(data.view(batch_size, -1))\n",
- " _, idx = output.sum(dim=0).max(1)\n",
- " acc = np.mean((targets == idx).detach().cpu().numpy())\n",
- "\n",
- " if train:\n",
- " print(f\"Train Set Accuracy: {acc}\")\n",
- " else:\n",
- " print(f\"Test Set Accuracy: {acc}\")\n",
- "\n",
- "def train_printer():\n",
- " print(f\"Epoch {epoch}, Minibatch {minibatch_counter}\")\n",
- " print(f\"Train Set Loss: {loss_hist[counter]}\")\n",
- " print(f\"Test Set Loss: {test_loss_hist[counter]}\")\n",
- " print_batch_accuracy(data_it, targets_it, train=True)\n",
- " print_batch_accuracy(testdata_it, testtargets_it, train=False)\n",
- " print(\"\\n\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "OxfhunW6pkoz",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 5.1 Optimizer"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "T7ULsfh9bHr1"
- },
- "source": [
- "We will apply a softmax to the output of our network, and calculate the loss using the negative log-likelihood."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "iqdVyjCNtdlp"
- },
- "outputs": [],
- "source": [
- "optimizer = torch.optim.Adam(net.parameters(), lr=2e-4, betas=(0.9, 0.999))\n",
- "log_softmax_fn = nn.LogSoftmax(dim=-1)\n",
- "loss_fn = nn.NLLLoss()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "GiqAVKzVbfPn"
- },
- "source": [
- "## 5.2 Training Loop"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "yGDs_dF2e1Sx"
- },
- "source": [
- "We assume some working knowledge of PyTorch. The training loop is fairly standard, with the only exceptions being the following.\n",
- "\n",
- "**Inputs**\n",
- "\n",
- "The for-loop that iterates through each time step during the forward pass has already been nested within `net`. This means that the following line of code:\n",
- "\n",
- "`spk_rec, mem_rec = net(data_it.view(batch_size, -1))`\n",
- "\n",
- "passes the same sample at each step. That is why we refer to it as static MNIST.\n",
- "\n",
- "\n",
- "**Targets**\n",
- "\n",
- "The losses generated at each time steps are summed together in the for-loop that contains:\n",
- "\n",
- "`loss_val += loss_fn(log_p_y[step], targets_it)`\n",
- "\n",
- "Also note how `targets_it` is not indexed, because the same value is used as the target for each step. '1' is applied as the target for the correct class for all of time, and '0' is applied as the target for all other classes.\n",
- "\n",
- "Let's train this across 3 epochs to keep things quick."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "LMZMxEV8dcTC"
- },
- "outputs": [],
- "source": [
- "loss_hist = []\n",
- "test_loss_hist = []\n",
- "counter = 0\n",
- "\n",
- "# Outer training loop\n",
- "for epoch in range(3):\n",
- " minibatch_counter = 0\n",
- " train_batch = iter(train_loader)\n",
- "\n",
- " # Minibatch training loop\n",
- " for data_it, targets_it in train_batch:\n",
- " data_it = data_it.to(device)\n",
- " targets_it = targets_it.to(device)\n",
- "\n",
- " spk_rec, mem_rec = net(data_it.view(batch_size, -1))\n",
- " log_p_y = log_softmax_fn(mem_rec)\n",
- " loss_val = torch.zeros((1), dtype=dtype, device=device)\n",
- "\n",
- " # Sum loss over time steps: BPTT\n",
- " for step in range(num_steps):\n",
- " loss_val += loss_fn(log_p_y[step], targets_it)\n",
- "\n",
- " # Gradient calculation\n",
- " optimizer.zero_grad()\n",
- " loss_val.backward()\n",
- "\n",
- " # Weight Update\n",
- " optimizer.step()\n",
- "\n",
- " # Store loss history for future plotting\n",
- " loss_hist.append(loss_val.item())\n",
- "\n",
- " # Test set\n",
- " test_data = itertools.cycle(test_loader)\n",
- " testdata_it, testtargets_it = next(test_data)\n",
- " testdata_it = testdata_it.to(device)\n",
- " testtargets_it = testtargets_it.to(device)\n",
- "\n",
- " # Test set forward pass\n",
- " test_spk, test_mem = net(testdata_it.view(batch_size, -1))\n",
- "\n",
- " # Test set loss\n",
- " log_p_ytest = log_softmax_fn(test_mem)\n",
- " log_p_ytest = log_p_ytest.sum(dim=0)\n",
- " loss_val_test = loss_fn(log_p_ytest, testtargets_it)\n",
- " test_loss_hist.append(loss_val_test.item())\n",
- "\n",
- " # Print test/train loss/accuracy\n",
- " if counter % 50 == 0:\n",
- " train_printer()\n",
- " minibatch_counter += 1\n",
- " counter += 1\n",
- "\n",
- "loss_hist_true_grad = loss_hist\n",
- "test_loss_hist_true_grad = test_loss_hist"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Taf6WZLojHTz"
- },
- "source": [
- "If this was your first time training an SNN, then congratulations!"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "HxU7P7xFpko3",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "# 6. Results\n",
- "## 6.1 Plot Training/Test Loss"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "_Pk_EScnpkpj",
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Plot Loss\n",
- "fig = plt.figure(facecolor=\"w\", figsize=(10, 5))\n",
- "plt.plot(loss_hist)\n",
- "plt.plot(test_loss_hist)\n",
- "plt.legend([\"Train Loss\", \"Test Loss\"])\n",
- "plt.xlabel(\"Iteration\")\n",
- "plt.ylabel(\"Loss\")\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "g-Gd84OAl1rB"
- },
- "source": [
- "Taking a look at the training / test loss, the process is somewhat noisy. This could be a result of a variety of things: minibatch gradient descent is the obvious one, but the use of improper targets likely also contributes. By encouraging the correct class to fire at every time step, the loss function conflicts with the reset mechanism that tries to prevent this."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "Z3f0vBnBpkpk",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 6.2 Test Set Accuracy\n",
- "This function iterates over all minibatches to obtain a measure of accuracy over the full 10,000 samples in the test set."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "F5Rb4xHGndQh"
- },
- "outputs": [],
- "source": [
- "total = 0\n",
- "correct = 0\n",
- "\n",
- "# drop_last switched to False to keep all samples\n",
- "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=False)\n",
- "\n",
- "with torch.no_grad():\n",
- " net.eval()\n",
- " for data in test_loader:\n",
- " images, labels = data\n",
- " images = images.to(device)\n",
- " labels = labels.to(device)\n",
- "\n",
- " batch_size = images.size(0) # the final batch has a different size so must be updated\n",
- " outputs, _ = net(images.view(batch_size, -1))\n",
- "\n",
- " _, predicted = outputs.sum(dim=0).max(1)\n",
- " total += labels.size(0)\n",
- " correct += (predicted == labels).sum().item()\n",
- "\n",
- "print(f\"Total correctly classified test set images: {correct}/{total}\")\n",
- "print(f\"Test Set Accuracy: {100 * correct / total}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false,
- "id": "TBIXau4Zpkpl",
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "Voila! That's it for static MNIST. Feel free to tweak the network parameters, hyperparameters, decay rate, using a learning rate scheduler etc. to see if you can improve the network performance. "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "s0dAgWUt2o6E"
- },
- "source": [
- "# Conclusion\n",
- "Now you know how to construct and train a fully-connected network on a static dataset. The spiking neurons can actually be adapted to other layer types, including convolutions and skip connections. Armed with this knowledge, you should now be able to build many different types of SNNs.\n",
- "\n",
- "In the next tutorial, you will learn how to train a spiking convolutional network using a time-varying spiking dataset."
- ]
- }
- ],
- "metadata": {
- "accelerator": "GPU",
- "colab": {
- "name": "tutorial_2_FCN_truncatedfromscratch.ipynb",
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 2
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.6"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
diff --git a/examples/legacy/tutorial_4_CNN.ipynb b/examples/legacy/tutorial_4_CNN.ipynb
deleted file mode 100644
index 4495eede..00000000
--- a/examples/legacy/tutorial_4_CNN.ipynb
+++ /dev/null
@@ -1,981 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "# snnTorch - Tutorial 3\n",
- "### By Jason K. Eshraghian"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Gradient-based Learning in Convolutional Spiking Neural Networks\n",
- "In this tutorial, we'll use a convolutional neural network (CNN) to classify the MNIST dataset.\n",
- "We will use the backpropagation through time (BPTT) algorithm to do so. This tutorial is largely the same as tutorial 2, just with a different network architecture to show how to integrate convolutions with snnTorch.\n",
- "\n",
- "If running in Google Colab:\n",
- "* Ensure you are connected to GPU by checking Runtime > Change runtime type > Hardware accelerator: GPU\n",
- "* Next, install the Test PyPi distribution of snnTorch by clicking into the following cell and pressing `Shift+Enter`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Install the test PyPi Distribution of snntorch\n",
- "!pip install -i https://test.pypi.org/simple/ snntorch"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 1. Setting up the Static MNIST Dataset\n",
- "### 1.1. Import packages and setup environment"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "import snntorch as snn\n",
- "import torch\n",
- "import torch.nn as nn\n",
- "import torch.nn.functional as F\n",
- "from torch.utils.data import DataLoader\n",
- "from torchvision import datasets, transforms\n",
- "import numpy as np\n",
- "import itertools\n",
- "import matplotlib.pyplot as plt"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### 1.2 Define network and SNN parameters\n",
- "We will use a 2conv-2MaxPool-FCN architecture for a sequence of 25 time steps.\n",
- "\n",
- "* `alpha` is the decay rate of the synaptic current of a neuron\n",
- "* `beta` is the decay rate of the membrane potential of a neuron"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Network Architecture\n",
- "num_inputs = 28*28\n",
- "num_outputs = 10\n",
- "\n",
- "# Training Parameters\n",
- "batch_size=128\n",
- "data_path='/tmp/data/mnist'\n",
- "\n",
- "# Temporal Dynamics\n",
- "num_steps = 25\n",
- "time_step = 1e-3\n",
- "tau_mem = 6.5e-4\n",
- "tau_syn = 5.5e-4\n",
- "alpha = float(np.exp(-time_step/tau_syn))\n",
- "beta = float(np.exp(-time_step/tau_mem))\n",
- "\n",
- "dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 1.3 Download MNIST Dataset\n",
- "To see how to construct a validation set, refer to Tutorial 1."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Define a transform\n",
- "transform = transforms.Compose([\n",
- " transforms.Resize((28, 28)),\n",
- " transforms.Grayscale(),\n",
- " transforms.ToTensor(),\n",
- " transforms.Normalize((0,), (1,))])\n",
- "\n",
- "mnist_train = datasets.MNIST(data_path, train=True, download=True, transform=transform)\n",
- "mnist_test = datasets.MNIST(data_path, train=False, download=True, transform=transform)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 1.4 Create DataLoaders"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "train_loader = DataLoader(mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
- "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=True)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 2. Define Network\n",
- "snnTorch contains a series of neuron models and related functions to ease the training process.\n",
- "Neurons are treated as activations with recurrent connections, and integrate smoothly with PyTorch's pre-existing layer functions.\n",
- "* `snntorch.Stein` is a simple Leaky Integrate and Fire (LIF) neuron. Specifically, it uses Stein's model which assumes instantaneous rise times for synaptic current and membrane potential.\n",
- "* `snntorch.FastSigmoidSurrogate` defines separate forward and backward functions. The forward function is a Heaviside step function for spike generation. The backward function is the derivative of a fast sigmoid function, to ensure continuous differentiability.\n",
- "FSS is mostly derived from:\n",
- "\n",
- ">Neftci, E. O., Mostafa, H., and Zenke, F. (2019) Surrogate Gradient Learning in Spiking Neural Networks. https://arxiv.org/abs/1901/09948\n",
- "\n",
- "There are a few other surrogate gradient functions included.\n",
- "`snn.slope` is a variable that defines the slope of the backward surrogate.\n",
- "TO-DO: Include visualisation."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "spike_grad = snn.FastSigmoidSurrogate.apply\n",
- "snn.slope = 50"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "Now we can define our spiking neural network (SNN).\n",
- "If you have already worked through Tutorial 2, you may wish to skip ahead.\n",
- "\n",
- "Creating an instance of the `Stein` neuron requires two compulsory arguments and two optional arguments:\n",
- "1. $I_{syn}$ decay rate, $\\alpha$,\n",
- "2. $V_{mem}$ decay rate, $\\beta$,\n",
- "3. *optional*: the surrogate spiking function, `spike_grad` (*default*: the gradient of the Heaviside function), and\n",
- "4. *optional*: the threshold for spiking, (*default*: 1.0).\n",
- "\n",
- "snnTorch treats the LIF neuron as a recurrent activation. Therefore, it requires initialization of its internal states.\n",
- "For each layer, we initialize the synaptic current `syn1` and `syn2`, the membrane potential `mem1` and `mem2`, and the post-synaptic spikes `spk1` and `spk2` to zero.\n",
- "A class method `init_stein` will take care of this.\n",
- "\n",
- "For rate coding, the final layer of spikes and membrane potential are used to determine accuracy and loss, respectively.\n",
- "So their historical values are recorded in `spk3_rec` and `mem3_rec`.\n",
- "\n",
- "Keep in mind, the dataset we are using is just static MNIST. I.e., it is *not* time-varying.\n",
- "Therefore, we pass the same MNIST sample to the input at each time step.\n",
- "This is handled with `cur1 = F.max_pool2d(self.conv1(x), 2)`, where `x` is the same input over the whole for-loop."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "class Net(nn.Module):\n",
- " def __init__(self):\n",
- " super().__init__()\n",
- "\n",
- " # initialize layers\n",
- " self.conv1 = nn.Conv2d(in_channels=1, out_channels=12, kernel_size=5, stride=1, padding=1)\n",
- " self.lif1 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- " self.conv2 = nn.Conv2d(in_channels=12, out_channels=64, kernel_size=5, stride=1, padding=1)\n",
- " self.lif2 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- " self.fc2 = nn.Linear(64*5*5, 10)\n",
- " self.lif3 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- "\n",
- " def forward(self, x):\n",
- " # Initialize LIF state variables and spike output tensors\n",
- " spk1, syn1, mem1 = self.lif1.init_hidden(batch_size, 12, 13, 13)\n",
- " spk2, syn2, mem2 = self.lif1.init_hidden(batch_size, 64, 5, 5)\n",
- " spk3, syn3, mem3 = self.lif2.init_hidden(batch_size, 10)\n",
- "\n",
- " spk3_rec = []\n",
- " mem3_rec = []\n",
- "\n",
- " for step in range(num_steps):\n",
- " cur1 = F.max_pool2d(self.conv1(x), 2)\n",
- " spk1, syn1, mem1 = self.lif1(cur1, syn1, mem1)\n",
- " cur2 = F.max_pool2d(self.conv2(spk1), 2)\n",
- " spk2, syn2, mem2 = self.lif2(cur2, syn2, mem2)\n",
- " cur3 = self.fc2(spk2.view(batch_size, -1))\n",
- " spk3, syn3, mem3 = self.lif3(cur3, syn3, mem3)\n",
- "\n",
- " spk3_rec.append(spk3)\n",
- " mem3_rec.append(mem3)\n",
- "\n",
- " return torch.stack(spk3_rec, dim=0), torch.stack(mem3_rec, dim=0)\n",
- "\n",
- "net = Net().to(device)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 3. Training\n",
- "Time for training! Let's first define a couple of functions to print out test/train accuracy."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "def print_batch_accuracy(data, targets, train=False):\n",
- " output, _ = net(data.view(batch_size, 1, 28, 28))\n",
- " _, idx = output.sum(dim=0).max(1)\n",
- " acc = np.mean((targets == idx).detach().cpu().numpy())\n",
- "\n",
- " if train:\n",
- " print(f\"Train Set Accuracy: {acc}\")\n",
- " else:\n",
- " print(f\"Test Set Accuracy: {acc}\")\n",
- "\n",
- "def train_printer():\n",
- " print(f\"Epoch {epoch}, Minibatch {minibatch_counter}\")\n",
- " print(f\"Train Set Loss: {loss_hist[counter]}\")\n",
- " print(f\"Test Set Loss: {test_loss_hist[counter]}\")\n",
- " print_batch_accuracy(data_it, targets_it, train=True)\n",
- " print_batch_accuracy(testdata_it, testtargets_it, train=False)\n",
- " print(\"\\n\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 3.1 Optimizer & Loss\n",
- "* *Output Activation*: We'll apply the softmax function to the membrane potentials of the output layer, rather than the spikes.\n",
- "* *Loss*: This will then be used to calculate the negative log-likelihood loss.\n",
- "By encouraging the membrane of the correct neuron class to reach the threshold, we expect that neuron will fire more frequently.\n",
- "The loss could be applied to the spike count as well, but the membrane is continuous whereas spike count is discrete.\n",
- "* *Optimizer*: The Adam optimizer is used for weight updates.\n",
- "* *Accuracy*: Accuracy is measured by counting the spikes of the output neurons. The neuron that fires the most frequently will be our predicted class."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "optimizer = torch.optim.Adam(net.parameters(), lr=2e-4, betas=(0.9, 0.999))\n",
- "log_softmax_fn = nn.LogSoftmax(dim=-1)\n",
- "loss_fn = nn.NLLLoss()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 3.2 Training Loop\n",
- "Now just sit back, relax, and wait for convergence."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "loss_hist = []\n",
- "test_loss_hist = []\n",
- "counter = 0\n",
- "\n",
- "# Outer training loop\n",
- "for epoch in range(5):\n",
- "\n",
- " minibatch_counter = 0\n",
- " train_batch = iter(train_loader)\n",
- "\n",
- " # Minibatch training loop\n",
- " for data_it, targets_it in train_batch:\n",
- " data_it = data_it.to(device)\n",
- " targets_it = targets_it.to(device)\n",
- "\n",
- " output, mem_rec = net(data_it.view(batch_size, 1, 28, 28)) # [28x28] or [1x28x28]?\n",
- " log_p_y = log_softmax_fn(mem_rec)\n",
- " loss_val = torch.zeros((1), dtype=dtype, device=device)\n",
- "\n",
- " # Sum loss over time steps to perform BPTT\n",
- " for step in range(num_steps):\n",
- " loss_val += loss_fn(log_p_y[step], targets_it)\n",
- "\n",
- " # Gradient calculation\n",
- " optimizer.zero_grad()\n",
- " loss_val.backward(retain_graph=True)\n",
- "\n",
- " # Weight Update\n",
- " nn.utils.clip_grad_norm_(net.parameters(), 1)\n",
- " optimizer.step()\n",
- "\n",
- " # Store loss history for future plotting\n",
- " loss_hist.append(loss_val.item())\n",
- "\n",
- " # Test set\n",
- " test_data = itertools.cycle(test_loader)\n",
- " testdata_it, testtargets_it = next(test_data)\n",
- " testdata_it = testdata_it.to(device)\n",
- " testtargets_it = testtargets_it.to(device)\n",
- "\n",
- " # Test set forward pass\n",
- " test_output, test_mem_rec = net(testdata_it.view(batch_size, 1, 28, 28))\n",
- "\n",
- " # Test set loss\n",
- " log_p_ytest = log_softmax_fn(test_mem_rec)\n",
- " log_p_ytest = log_p_ytest.sum(dim=0)\n",
- " loss_val_test = loss_fn(log_p_ytest, testtargets_it)\n",
- " test_loss_hist.append(loss_val_test.item())\n",
- "\n",
- " # Print test/train loss/accuracy\n",
- " if counter % 50 == 0:\n",
- " train_printer()\n",
- " minibatch_counter += 1\n",
- " counter += 1\n",
- "\n",
- "loss_hist_true_grad = loss_hist\n",
- "test_loss_hist_true_grad = test_loss_hist"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 4. Results\n",
- "### 4.1 Plot Training/Test Loss"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Plot Loss\n",
- "fig = plt.figure(facecolor=\"w\", figsize=(10, 5))\n",
- "plt.plot(loss_hist)\n",
- "plt.plot(test_loss_hist)\n",
- "plt.legend([\"Test Loss\", \"Train Loss\"])\n",
- "plt.xlabel(\"Epoch\")\n",
- "plt.ylabel(\"Loss\")\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 4.2 Test Set Accuracy\n",
- "This function just iterates over all minibatches to obtain a measure of accuracy over the full 10,000 samples in the test set."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "total = 0\n",
- "correct = 0\n",
- "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=False)\n",
- "\n",
- "with torch.no_grad():\n",
- " net.eval()\n",
- " for data in test_loader:\n",
- " images, labels = data\n",
- " images = images.to(device)\n",
- " labels = labels.to(device)\n",
- "\n",
- " # If current batch matches batch_size, just do the usual thing\n",
- " if images.size()[0] == batch_size:\n",
- " outputs, _ = net(images.view(batch_size, 1, 28, 28))\n",
- "\n",
- " # If current batch does not match batch_size (e.g., is the final minibatch),\n",
- " # modify batch_size in a temp variable and restore it at the end of the else block\n",
- " else:\n",
- " temp_bs = batch_size\n",
- " batch_size = images.size()[0]\n",
- " outputs, _ = net(images.view(images.size()[0], 1, 28, 28))\n",
- " batch_size = temp_bs\n",
- "\n",
- " _, predicted = outputs.sum(dim=0).max(1)\n",
- " total += labels.size(0)\n",
- " correct += (predicted == labels).sum().item()\n",
- "\n",
- "print(f\"Total correctly classified test set images: {correct}/{total}\")\n",
- "print(f\"Test Set Accuracy: {100 * correct / total}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "That's it for static MNIST!"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 5. Spiking MNIST\n",
- "As before, there isn't anything all that impressive about training a network on the static MNIST dataset.\n",
- "So let's apply rate-coding to convert it into a time-varying stream of spikes."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "from snntorch import spikegen\n",
- "\n",
- "# MNIST to spiking-MNIST\n",
- "spike_data, spike_targets = spikegen.rate(data_it, targets_it, num_outputs=num_outputs, num_steps=num_steps,\n",
- " gain=1, offset=0, convert_targets=False, temporal_targets=False)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 5.1 Visualiser\n",
- "Just so you're damn sure it's a spiking input."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "!pip install celluloid # matplotlib animations made easy"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Note: if you are running the notebook locally on your desktop, please uncomment the line below and modify the path to your ffmpeg.exe"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "from celluloid import Camera\n",
- "from IPython.display import HTML\n",
- "\n",
- "# Animator\n",
- "spike_data_sample = spike_data[:, 0, 0].cpu()\n",
- "\n",
- "fig, ax = plt.subplots()\n",
- "camera = Camera(fig)\n",
- "plt.axis('off')\n",
- "\n",
- "# plt.rcParams['animation.ffmpeg_path'] = 'C:\\\\path\\\\to\\\\your\\\\ffmpeg.exe'\n",
- "\n",
- "for step in range(num_steps):\n",
- " im = ax.imshow(spike_data_sample[step, :, :], cmap='plasma')\n",
- " camera.snap()\n",
- "\n",
- "# interval=40 specifies 40ms delay between frames\n",
- "a = camera.animate(interval=40)\n",
- "HTML(a.to_html5_video())"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "print(spike_targets[0])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 6. Define Network\n",
- "The convolutional network is the same as before. The one difference is that the for-loop iterates through the first dimension of the input:\n",
- "`cur1 = F.max_pool2d(self.conv1(x[step]), 2)`"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "spike_grad = snn.FastSigmoidSurrogate.apply\n",
- "snn.slope = 50\n",
- "\n",
- "# Define a different network\n",
- "class Net(nn.Module):\n",
- " def __init__(self):\n",
- " super().__init__()\n",
- "\n",
- " # initialize layers\n",
- " self.conv1 = nn.Conv2d(in_channels=1, out_channels=12, kernel_size=5, stride=1, padding=1)\n",
- " self.lif1 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- " self.conv2 = nn.Conv2d(in_channels=12, out_channels=64, kernel_size=5, stride=1, padding=1)\n",
- " self.lif1 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- " self.fc2 = nn.Linear(64*5*5, 10)\n",
- " self.lif1 = snn.Stein(alpha=alpha, beta=beta, spike_grad=spike_grad)\n",
- "\n",
- " # self.conv1 = nn.Conv2d(in_channels=1, out_channels=3, kernel_size=3, stride=1, padding=0)\n",
- " # self.lif1 = LIF(spike_fn=spike_fn, alpha=alpha, beta=beta)\n",
- " # self.fc1 = nn.Linear(26*26*3, 10)\n",
- " # self.lif2 = LIF(spike_fn=spike_fn, alpha=alpha, beta=beta)\n",
- "\n",
- " def forward(self, x):\n",
- " # Initialize LIF state variables and spike output tensors\n",
- " spk1, syn1, mem1 = self.lif1.init_stein(batch_size, 12, 13, 13)\n",
- " spk2, syn2, mem2 = self.lif1.init_stein(batch_size, 64, 5, 5)\n",
- " spk3, syn3, mem3 = self.lif2.init_stein(batch_size, 10)\n",
- "\n",
- " spk3_rec = []\n",
- " mem3_rec = []\n",
- "\n",
- " for step in range(num_steps):\n",
- " cur1 = F.max_pool2d(self.conv1(x[step]), 2) # add max-pooling to membrane or spikes?\n",
- " spk1, syn1, mem1 = self.lif1(cur1, syn1, mem1)\n",
- " cur2 = F.max_pool2d(self.conv2(spk1), 2)\n",
- " spk2, syn2, mem2 = self.lif2(cur2, syn2, mem2)\n",
- " cur3 = self.fc2(spk2.view(batch_size, -1))\n",
- " spk3, syn3, mem3 = self.lif3(cur3, syn3, mem3)\n",
- "\n",
- " spk3_rec.append(spk3)\n",
- " mem3_rec.append(mem3)\n",
- "\n",
- " return torch.stack(spk3_rec, dim=0), torch.stack(mem3_rec, dim=0)\n",
- "\n",
- "net = Net().to(device)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "## 7. Training\n",
- "We make a slight modification to our print-out functions to handle the new first dimension of the input:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "def print_batch_accuracy(data, targets, train=False):\n",
- " output, _ = net(data.view(num_steps, batch_size, 1, 28, 28))\n",
- " _, idx = output.sum(dim=0).max(1)\n",
- " acc = np.mean((targets == idx).detach().cpu().numpy())\n",
- "\n",
- " if train:\n",
- " print(f\"Train Set Accuracy: {acc}\")\n",
- " else:\n",
- " print(f\"Test Set Accuracy: {acc}\")\n",
- "\n",
- "def train_printer():\n",
- " print(f\"Epoch {epoch}, Minibatch {minibatch_counter}\")\n",
- " print(f\"Train Set Loss: {loss_hist[counter]}\")\n",
- " print(f\"Test Set Loss: {test_loss_hist[counter]}\")\n",
- " print_batch_accuracy(spike_data, spike_targets, train=True)\n",
- " print_batch_accuracy(test_spike_data, test_spike_targets, train=False)\n",
- " print(\"\\n\")\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 7.1 Optimizer & Loss\n",
- "We'll keep our optimizer and loss the exact same as the static MNIST case."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "optimizer = torch.optim.Adam(net.parameters(), lr=2e-4, betas=(0.9, 0.999))\n",
- "log_softmax_fn = nn.LogSoftmax(dim=-1)\n",
- "loss_fn = nn.NLLLoss()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "pycharm": {
- "name": "#%% md\n"
- }
- },
- "source": [
- "### 7.2 Training Loop\n",
- "The training loop is identical to the static MNIST case, but we pass each minibatch through `spikegen.rate` before running it through the feedforward network."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "loss_hist = []\n",
- "test_loss_hist = []\n",
- "counter = 0\n",
- "\n",
- "# Outer training loop\n",
- "for epoch in range(5):\n",
- " minibatch_counter = 0\n",
- " data = iter(train_loader)\n",
- "\n",
- " # Minibatch training loop\n",
- " for data_it, targets_it in data:\n",
- " data_it = data_it.to(device)\n",
- " targets_it = targets_it.to(device)\n",
- "\n",
- " # Spike generator\n",
- " spike_data, spike_targets = spikegen.rate(data_it, targets_it, num_outputs=num_outputs, num_steps=num_steps,\n",
- " gain=1, offset=0, convert_targets=False, temporal_targets=False)\n",
- "\n",
- " # Forward pass\n",
- " output, mem_rec = net(spike_data.view(num_steps, batch_size, 1, 28, 28))\n",
- " log_p_y = log_softmax_fn(mem_rec)\n",
- " loss_val = torch.zeros(1, dtype=dtype, device=device)\n",
- "\n",
- " # Sum loss over time steps to perform BPTT\n",
- " for step in range(num_steps):\n",
- " loss_val += loss_fn(log_p_y[step], targets_it)\n",
- "\n",
- " # Gradient Calculation\n",
- " optimizer.zero_grad()\n",
- " loss_val.backward(retain_graph=True)\n",
- " nn.utils.clip_grad_norm_(net.parameters(), 1)\n",
- "\n",
- " # Weight Update\n",
- " optimizer.step()\n",
- "\n",
- " # Store Loss history\n",
- " loss_hist.append(loss_val.item())\n",
- "\n",
- " # Test set\n",
- " test_data = itertools.cycle(test_loader)\n",
- " testdata_it, testtargets_it = next(test_data)\n",
- " testdata_it = testdata_it.to(device)\n",
- " testtargets_it = testtargets_it.to(device)\n",
- "\n",
- " # Test set spike conversion\n",
- " test_spike_data, test_spike_targets = spikegen.rate(testdata_it, testtargets_it, num_outputs=num_outputs,\n",
- " num_steps=num_steps, gain=1, offset=0, convert_targets=False,\n",
- " temporal_targets=False)\n",
- "\n",
- " # Test set forward pass\n",
- " test_output, test_mem_rec = net(test_spike_data.view(num_steps, batch_size, 1, 28, 28))\n",
- "\n",
- " # Test set loss\n",
- " log_p_ytest = log_softmax_fn(test_mem_rec)\n",
- " log_p_ytest = log_p_ytest.sum(dim=0)\n",
- " loss_val_test = loss_fn(log_p_ytest, test_spike_targets)\n",
- " test_loss_hist.append(loss_val_test.item())\n",
- "\n",
- " # Print test/train loss/accuracy\n",
- " if counter % 50 == 0:\n",
- " train_printer()\n",
- " minibatch_counter += 1\n",
- " counter += 1\n",
- "\n",
- "loss_hist_true_grad = loss_hist\n",
- "test_loss_hist_true_grad = test_loss_hist"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## 8. Spiking MNIST Results\n",
- "### 8.1 Plot Training/Test Loss"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "# Plot Loss\n",
- "fig = plt.figure(facecolor=\"w\", figsize=(10, 5))\n",
- "plt.plot(loss_hist)\n",
- "plt.plot(test_loss_hist)\n",
- "plt.legend([\"Test Loss\", \"Train Loss\"])\n",
- "plt.xlabel(\"Epoch\")\n",
- "plt.ylabel(\"Loss\")\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### 8.2 Test Set Accuracy"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "pycharm": {
- "name": "#%%\n"
- }
- },
- "outputs": [],
- "source": [
- "total = 0\n",
- "correct = 0\n",
- "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=False)\n",
- "\n",
- "with torch.no_grad():\n",
- " net.eval()\n",
- " for data in test_loader:\n",
- " images, labels = data\n",
- " images = images.to(device)\n",
- " labels = labels.to(device)\n",
- "\n",
- " # If current batch matches batch_size, just do the usual thing\n",
- " if images.size()[0] == batch_size:\n",
- " spike_test, spike_targets = spikegen.rate(images, labels, num_outputs=num_outputs, num_steps=num_steps,\n",
- " gain=1, offset=0, convert_targets=False, temporal_targets=False)\n",
- "\n",
- " outputs, _ = net(spike_test.view(num_steps, batch_size, 1, 28, 28))\n",
- "\n",
- " # If current batch does not match batch_size (e.g., is the final minibatch),\n",
- " # modify batch_size in a temp variable and restore it at the end of the else block\n",
- " else:\n",
- " temp_bs = batch_size\n",
- " batch_size = images.size()[0]\n",
- " spike_test, spike_targets = spikegen.rate(images, labels, num_outputs=num_outputs, num_steps=num_steps,\n",
- " gain=1, offset=0, convert_targets=False, temporal_targets=False)\n",
- " outputs, _ = net(spike_test.view(num_steps, images.size()[0], 1, 28, 28))\n",
- " batch_size = temp_bs\n",
- "\n",
- " _, predicted = outputs.sum(dim=0).max(1)\n",
- " total += spike_targets.size(0)\n",
- " correct += (predicted == spike_targets).sum().item()\n",
- "\n",
- "print(f\"Total correctly classified test set images: {correct}/{total}\")\n",
- "print(f\"Test Set Accuracy: {100 * correct / total}%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "To-do:\n",
- "* Add figures to explain in better detail\n",
- "* See if SRM0 model can reach acceptable acc within reasonable num_steps\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.8"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 1
-}
diff --git a/examples/legacy/tutorial_7_tonic.ipynb b/examples/legacy/tutorial_7_tonic.ipynb
deleted file mode 100644
index f3815c11..00000000
--- a/examples/legacy/tutorial_7_tonic.ipynb
+++ /dev/null
@@ -1,2342 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "colab_type": "text",
- "id": "view-in-github"
- },
- "source": [
- ""
- ]
- },
- {
- "cell_type": "markdown",
- "id": "47d5313e-c29d-4581-a9c7-a45122337069",
- "metadata": {
- "id": "47d5313e-c29d-4581-a9c7-a45122337069"
- },
- "source": [
- "[](https://github.com/jeshraghian/snntorch/) \n",
- "[](https://github.com/neuromorphs/tonic/)\n",
- "\n",
- "\n",
- "# Neuromorphic Datasets with Tonic + snnTorch\n",
- "## Tutorial 7\n",
- "### By Gregor Lenz (https://lenzgregor.com) and Jason K. Eshraghian (www.jasoneshraghian.com)\n",
- "\n",
- "\n",
- " \n",
- "\n",
- "\n",
- "[](https://github.com/jeshraghian/snntorch/) [](https://github.com/jeshraghian/snntorch/)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "oll2NNFeG1NG",
- "metadata": {
- "id": "oll2NNFeG1NG"
- },
- "source": [
- "The snnTorch tutorial series is based on the following paper. If you find these resources or code useful in your work, please consider citing the following source:\n",
- "\n",
- "> [Jason K. Eshraghian, Max Ward, Emre Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D. Lu. \"Training Spiking Neural Networks Using Lessons From Deep Learning\". arXiv preprint arXiv:2109.12894, September 2021.](https://arxiv.org/abs/2109.12894) "
- ]
- },
- {
- "cell_type": "markdown",
- "id": "ClgsZMOfBVby",
- "metadata": {
- "id": "ClgsZMOfBVby"
- },
- "source": [
- "# Introduction\n",
- "In this tutorial, you will:\n",
- "* Learn how to load neuromorphic datasets using [Tonic](https://github.com/neuromorphs/tonic)\n",
- "* Make use of caching to speed up dataloading\n",
- "* Train a CSNN with the [Neuromorphic-MNIST](https://tonic.readthedocs.io/en/latest/datasets.html#n-mnist) Dataset\n",
- "\n",
- "If running in Google Colab:\n",
- "* You may connect to GPU by checking `Runtime` > `Change runtime type` > `Hardware accelerator: GPU`\n",
- "* Next, install the latest PyPi distribution of snnTorch and Tonic by clicking into the following cell and pressing `Shift+Enter`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "hDnIEHOKB8LD",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "hDnIEHOKB8LD",
- "outputId": "3d7a9981-9f06-4188-eba7-77b48ccb045d"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[K |████████████████████████████████| 92 kB 3.1 MB/s \n",
- "\u001b[K |████████████████████████████████| 8.9 MB 26.5 MB/s \n",
- "\u001b[K |████████████████████████████████| 395 kB 45.8 MB/s \n",
- "\u001b[?25h Building wheel for importRosbag (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- " Building wheel for loris (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- "\u001b[K |████████████████████████████████| 59 kB 3.1 MB/s \n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "!pip install tonic --quiet \n",
- "!pip install snntorch --quiet"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e93694d9-0f0a-46a0-b17f-c04ac9b73a63",
- "metadata": {
- "id": "e93694d9-0f0a-46a0-b17f-c04ac9b73a63"
- },
- "source": [
- "# 1. Using Tonic to Load Neuromorphic Datasets\n",
- "Loading datasets from neuromorphic sensors is made super simple thanks to [Tonic](https://github.com/neuromorphs/tonic), which works much like PyTorch vision.\n",
- "\n",
- "Let's start by loading the neuromorphic version of the MNIST dataset, called [N-MNIST](https://tonic.readthedocs.io/en/latest/reference/datasets.html#n-mnist). We can have a look at some raw events to get a feel for what we're working with."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "7d286ef9-5fe6-4578-a686-91559a1f81d2",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 137,
- "referenced_widgets": [
- "9f46301d16924494ada03e7496af98ee",
- "397436bca741424c886e7f67805eebc3",
- "e965cc586d5f4d5aaf8c435ebd4bf35c",
- "be2f2e3ca1b34106874e6b5c8cd2051e",
- "b3a18813d8684b3ab7b9d944d8e74e4f",
- "37deac4746684f189a32858514a780a4",
- "83b65be63ec841288496bd9ad33dbbab",
- "c87ffeb8f6f54e1b871d129cd783de9f",
- "2789fbacce8f4cc8a240daa6c6d45d15",
- "bfe2d8fcea404d22bbc61a69ae60c802",
- "64ab952b23bd4406af5cc6d4f72b257e"
- ]
- },
- "id": "7d286ef9-5fe6-4578-a686-91559a1f81d2",
- "outputId": "f9438c1a-440f-4815-b4e6-101ba31e5cca"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Downloading https://uca6892c4e08b631fef762aee80b.dl.dropboxusercontent.com/zip_download_get/A8yubn6iJIfzLC4NXQBrS1I7ATTQ14Bx1aKLTntxskbCo02s7rFtnwUSW2bHzvIbJVxXlmWSWs7kTHeWQty4Q8hQHssWjWM0UpkC79TENd9K1A?dl=1 to ./data/NMNIST/nmnist-archive.zip\n"
- ]
- },
- {
- "data": {
- "application/vnd.jupyter.widget-view+json": {
- "model_id": "9f46301d16924494ada03e7496af98ee",
- "version_major": 2,
- "version_minor": 0
- },
- "text/plain": [
- " 0%| | 0/1181572961 [00:00"
- ]
- },
- "metadata": {
- "needs_background": "light"
- },
- "output_type": "display_data"
- }
- ],
- "source": [
- "tonic.utils.plot_event_grid(events)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f6bcc031-d11a-4471-b3aa-335eec76d7ad",
- "metadata": {
- "id": "f6bcc031-d11a-4471-b3aa-335eec76d7ad"
- },
- "source": [
- "## 1.1 Transformations\n",
- "\n",
- "However, neural nets don't take lists of events as input. The raw data must be converted into a suitable representation, such as a tensor. We can choose a set of transforms to apply to our data before feeding it to our network. The neuromorphic camera sensor has a temporal resolution of microseconds, which when converted into a dense representation, ends up as a very large tensor. That is why we bin events into a smaller number of frames using the [ToFrame transformation](https://tonic.readthedocs.io/en/latest/reference/transformations.html#frames), which reduces temporal precision but also allows us to work with it in a dense format.\n",
- "\n",
- "* `time_window=1000` integrates events into 1000$~\\mu$s bins\n",
- "\n",
- "* Denoise removes isolated, one-off events. If no event occurs within a neighbourhood of 1 pixel across `filter_time` microseconds, the event is filtered. Smaller `filter_time` will filter more events."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "30f249be-8a65-4c1c-a21c-d561e904b4bf",
- "metadata": {
- "id": "30f249be-8a65-4c1c-a21c-d561e904b4bf"
- },
- "outputs": [],
- "source": [
- "import tonic.transforms as transforms\n",
- "\n",
- "sensor_size = tonic.datasets.NMNIST.sensor_size\n",
- "\n",
- "# Denoise removes isolated, one-off events\n",
- "# time_window\n",
- "frame_transform = transforms.Compose([transforms.Denoise(filter_time=10000), \n",
- " transforms.ToFrame(sensor_size=sensor_size, \n",
- " time_window=1000)\n",
- " ])\n",
- "\n",
- "trainset = tonic.datasets.NMNIST(save_to='./data', transform=frame_transform, train=True)\n",
- "testset = tonic.datasets.NMNIST(save_to='./data', transform=frame_transform, train=False)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "70be0b77-9405-44f6-af49-dfc4d49566c6",
- "metadata": {
- "id": "70be0b77-9405-44f6-af49-dfc4d49566c6"
- },
- "source": [
- "## 1.2 Fast Dataloading via Caching\n",
- "\n",
- "The original data is stored in a format that is slow to read. To speed up dataloading, we can make use of disk caching. That means that once files are loaded from the original file, they are written to disk in an efficient format in our cache directory. Let's compare some file reading speeds to read 100 examples."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "3a6bf2a2-ff9f-4cdc-8cb3-02a1a9d71a11",
- "metadata": {
- "id": "3a6bf2a2-ff9f-4cdc-8cb3-02a1a9d71a11"
- },
- "outputs": [],
- "source": [
- "def load_sample_simple():\n",
- " for i in range(100):\n",
- " events, target = trainset[i]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1a9d3b28-b303-4a17-be78-b9918911a7cd",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "1a9d3b28-b303-4a17-be78-b9918911a7cd",
- "outputId": "47fb99f1-0685-460a-bfd3-5aca4bc069ed"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "1 loop, best of 5: 2.76 s per loop\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- ""
- ]
- },
- "execution_count": 6,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "%timeit -o load_sample_simple()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b957b32f-d76b-42c0-8c1c-b6b63e84e2ef",
- "metadata": {
- "id": "b957b32f-d76b-42c0-8c1c-b6b63e84e2ef"
- },
- "source": [
- "We can decrease the time it takes to read 100 samples by using a PyTorch DataLoader in addition to disk caching."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "f47e0798-5259-491a-9d3a-59b13b1b0983",
- "metadata": {
- "id": "f47e0798-5259-491a-9d3a-59b13b1b0983"
- },
- "outputs": [],
- "source": [
- "from torch.utils.data import DataLoader\n",
- "from tonic import CachedDataset\n",
- "\n",
- "cached_trainset = CachedDataset(trainset, cache_path='./cache/nmnist/train')\n",
- "cached_dataloader = DataLoader(cached_trainset)\n",
- "\n",
- "def load_sample_cached():\n",
- " for i, (events, target) in enumerate(iter(cached_dataloader)):\n",
- " if i > 99: break"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "17a0219b-4d15-4f0b-b5be-8c728b5e24a9",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "17a0219b-4d15-4f0b-b5be-8c728b5e24a9",
- "outputId": "fe8526e1-ffef-4190-a07c-0f62e1ba32fd"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "1 loop, best of 20: 1.8 s per loop\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- ""
- ]
- },
- "execution_count": 8,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "%timeit -o -r 20 load_sample_cached()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "3831428d-0511-4fde-84d9-11d08fa45df7",
- "metadata": {
- "id": "3831428d-0511-4fde-84d9-11d08fa45df7"
- },
- "source": [
- "## 1.3 Even Faster DataLoading via Batching\n",
- "\n",
- "Now that we've reduced our loading time, we also want to use batching to make efficient use of the GPU. \n",
- "\n",
- "Because event recordings have different lengths, we are going to provide a collation function `tonic.collation.PadTensors()` that will pad out shorter recordings to ensure all samples in a batch have the same dimensions. "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5b35c7cd-d292-47cd-9203-7f31aa7f7207",
- "metadata": {
- "id": "5b35c7cd-d292-47cd-9203-7f31aa7f7207"
- },
- "outputs": [],
- "source": [
- "batch_size = 100\n",
- "trainloader = DataLoader(cached_trainset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors())"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "14b9af4f-141e-4301-8451-445957ec8707",
- "metadata": {
- "id": "14b9af4f-141e-4301-8451-445957ec8707"
- },
- "outputs": [],
- "source": [
- "def load_sample_batched():\n",
- " events, target = next(iter(cached_dataloader))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "3dc4b27a-63ac-4edc-94e9-589d548c4769",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "3dc4b27a-63ac-4edc-94e9-589d548c4769",
- "outputId": "e3a16676-9c5d-4c9e-e692-96bb4f8f5099"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "100 loops, best of 10: 17.4 ms per loop\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- ""
- ]
- },
- "execution_count": 11,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "%timeit -o -r 10 load_sample_batched()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "a82a7afd-c011-4cd6-ba04-1e7cd438bc1f",
- "metadata": {
- "id": "a82a7afd-c011-4cd6-ba04-1e7cd438bc1f"
- },
- "source": [
- "By using disk caching and a PyTorch dataloader with multithreading and batching support, we have reduced loading times to less than a tenth per sample in comparison to naively iterating over the dataset!"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "2ded1bd9-e2f1-479e-899c-c6c2652e6fc9",
- "metadata": {
- "id": "2ded1bd9-e2f1-479e-899c-c6c2652e6fc9"
- },
- "source": [
- "# 2. Training our network using frames created from events"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9be82d75-69ef-4c1b-ad85-4eca84c73ccf",
- "metadata": {
- "id": "9be82d75-69ef-4c1b-ad85-4eca84c73ccf"
- },
- "source": [
- "Now let's actually train a network on the N-MNIST classification task. We start by defining our caching wrappers and dataloaders. While doing that, we're also going to apply some augmentations to the training data. The samples we receive from the cached dataset are frames, so we can make use of PyTorch Vision to apply whatever random transform we would like."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "ace6cd0b-7b56-4422-b3bd-23bac65db9bd",
- "metadata": {
- "id": "ace6cd0b-7b56-4422-b3bd-23bac65db9bd"
- },
- "outputs": [],
- "source": [
- "import torch\n",
- "import torchvision\n",
- "\n",
- "transform = tonic.transforms.Compose([torch.from_numpy,\n",
- " torchvision.transforms.RandomRotation([-10,10])])\n",
- "\n",
- "cached_trainset = CachedDataset(trainset, transform=transform, cache_path='./cache/nmnist/train')\n",
- "\n",
- "# no augmentations for the testset\n",
- "cached_testset = CachedDataset(testset, cache_path='./cache/nmnist/test')\n",
- "\n",
- "batch_size = 128\n",
- "trainloader = DataLoader(cached_trainset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors(), shuffle=True)\n",
- "testloader = DataLoader(cached_testset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors())"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "528fe384-a365-4b53-bbfc-ed4dd261d222",
- "metadata": {
- "id": "528fe384-a365-4b53-bbfc-ed4dd261d222"
- },
- "source": [
- "A mini-batch now has the dimensions (time steps, batch size, channels, height, width). The number of time steps will be set to that of the longest recording in the mini-batch, and all other samples will be padded with zeros to match it."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c9e37337-ad4a-43d5-b429-81a18de5148e",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "c9e37337-ad4a-43d5-b429-81a18de5148e",
- "outputId": "88a6c0b0-aaaa-43ee-a62b-0eabcc90ebf0"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "torch.Size([311, 128, 2, 34, 34])\n"
- ]
- }
- ],
- "source": [
- "event_tensor, target = next(iter(trainloader))\n",
- "print(event_tensor.shape)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "61ae5d4b-2bb3-4191-9f96-04b3c6ba4c41",
- "metadata": {
- "id": "61ae5d4b-2bb3-4191-9f96-04b3c6ba4c41"
- },
- "source": [
- "## 2.1 Defining our network\n",
- "We will use snnTorch + PyTorch to construct a CSNN, just as in the previous tutorial. The convolutional network architecture to be used is: 12C5-MP2-32C5-MP2-800FC10\n",
- "\n",
- "- 12C5 is a 5$\\times$5 convolutional kernel with 12 filters\n",
- "- MP2 is a 2$\\times$2 max-pooling function\n",
- "- 800FC10 is a fully-connected layer that maps 800 neurons to 10 outputs"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "HpKDIkRKUIAB",
- "metadata": {
- "id": "HpKDIkRKUIAB"
- },
- "outputs": [],
- "source": [
- "import snntorch as snn\n",
- "from snntorch import surrogate\n",
- "from snntorch import functional as SF\n",
- "from snntorch import utils\n",
- "from snntorch import spikeplot as splt\n",
- "\n",
- "import torch\n",
- "import torch.nn as nn"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "107cb645-0227-4290-9e1b-25d6ae7eac87",
- "metadata": {
- "id": "107cb645-0227-4290-9e1b-25d6ae7eac87"
- },
- "outputs": [],
- "source": [
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")\n",
- "\n",
- "# neuron and simulation parameters\n",
- "spike_grad = surrogate.fast_sigmoid(slope=75)\n",
- "beta = 0.5\n",
- "\n",
- "# Initialize Network\n",
- "net = nn.Sequential(nn.Conv2d(2, 12, 5),\n",
- " nn.MaxPool2d(2),\n",
- " snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True),\n",
- " nn.Conv2d(12, 32, 5),\n",
- " nn.MaxPool2d(2),\n",
- " snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True),\n",
- " nn.Flatten(),\n",
- " nn.Linear(32*5*5, 10),\n",
- " snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True, output=True)\n",
- " ).to(device)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "zPFvlqOGi_uW",
- "metadata": {
- "id": "zPFvlqOGi_uW"
- },
- "outputs": [],
- "source": [
- "# this time, we won't return membrane as we don't need it \n",
- "\n",
- "def forward_pass(net, data): \n",
- " spk_rec = []\n",
- " utils.reset(net) # resets hidden states for all LIF neurons in net\n",
- "\n",
- " for step in range(data.size(0)): # data.size(0) = number of time steps\n",
- " spk_out, mem_out = net(data[step])\n",
- " spk_rec.append(spk_out)\n",
- " \n",
- " return torch.stack(spk_rec)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "23569dfc-e4a7-490f-8a68-c9ade5e03028",
- "metadata": {
- "id": "23569dfc-e4a7-490f-8a68-c9ade5e03028"
- },
- "source": [
- "## 2.2 Training\n",
- "\n",
- "In the previous tutorial, Cross Entropy Loss was applied to the total spike count to maximize the number of spikes from the correct class.\n",
- "\n",
- "Another option from the `snn.functional` module is to specify the target number of spikes from correct and incorrect classes. The approach below uses the *Mean Square Error Spike Count Loss*, which aims to elicit spikes from the correct class 80\\% of the time, and 20\\% of the time from incorrect classes. Encouraging incorrect neurons to fire could be motivated to avoid dead neurons."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "VocYbtD7Vwp7",
- "metadata": {
- "id": "VocYbtD7Vwp7"
- },
- "outputs": [],
- "source": [
- "optimizer = torch.optim.Adam(net.parameters(), lr=2e-2, betas=(0.9, 0.999))\n",
- "loss_fn = SF.mse_count_loss(correct_rate=0.8, incorrect_rate=0.2)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7xkKLsqnmzcw",
- "metadata": {
- "id": "7xkKLsqnmzcw"
- },
- "source": [
- "Training neuromorphic data is expensive as it requires sequentially iterating through many time steps (approximately 300 time steps in the N-MNIST dataset). The following simulation will take some time, so we will just stick to training across 50 iterations (which is roughly 1/10th of a full epoch). Feel free to change `num_iters` if you have more time to kill. As we are printing results at each iteration, the results will be quite noisy and will also take some time before we start to see any sort of improvement.\n",
- "\n",
- "In our own experiments, it took about 20 iterations before we saw any improvement, and after 50 iterations, managed to crack ~60% accuracy. \n",
- "\n",
- "> Warning: the following simulation will take a while. Go make yourself a coffee, or ten. "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "R4GbPSdTUcUR",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "R4GbPSdTUcUR",
- "outputId": "76a3bfd8-7038-4e4d-d597-f864b92740a0"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Epoch 0, Iteration 0 \n",
- "Train Loss: 30.96\n",
- "Accuracy: 5.47%\n",
- "\n",
- "Epoch 0, Iteration 1 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 8.59%\n",
- "\n",
- "Epoch 0, Iteration 2 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 9.38%\n",
- "\n",
- "Epoch 0, Iteration 3 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 14.06%\n",
- "\n",
- "Epoch 0, Iteration 4 \n",
- "Train Loss: 31.00\n",
- "Accuracy: 6.25%\n",
- "\n",
- "Epoch 0, Iteration 5 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 7.03%\n",
- "\n",
- "Epoch 0, Iteration 6 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 6.25%\n",
- "\n",
- "Epoch 0, Iteration 7 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 10.16%\n",
- "\n",
- "Epoch 0, Iteration 8 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 14.84%\n",
- "\n",
- "Epoch 0, Iteration 9 \n",
- "Train Loss: 31.00\n",
- "Accuracy: 7.03%\n",
- "\n",
- "Epoch 0, Iteration 10 \n",
- "Train Loss: 30.90\n",
- "Accuracy: 10.16%\n",
- "\n",
- "Epoch 0, Iteration 11 \n",
- "Train Loss: 30.58\n",
- "Accuracy: 12.50%\n",
- "\n",
- "Epoch 0, Iteration 12 \n",
- "Train Loss: 29.67\n",
- "Accuracy: 5.47%\n",
- "\n",
- "Epoch 0, Iteration 13 \n",
- "Train Loss: 17.50\n",
- "Accuracy: 6.25%\n",
- "\n",
- "Epoch 0, Iteration 14 \n",
- "Train Loss: 13.11\n",
- "Accuracy: 12.50%\n",
- "\n",
- "Epoch 0, Iteration 15 \n",
- "Train Loss: 18.77\n",
- "Accuracy: 14.06%\n",
- "\n",
- "Epoch 0, Iteration 16 \n",
- "Train Loss: 19.44\n",
- "Accuracy: 16.41%\n",
- "\n",
- "Epoch 0, Iteration 17 \n",
- "Train Loss: 17.33\n",
- "Accuracy: 15.62%\n",
- "\n",
- "Epoch 0, Iteration 18 \n",
- "Train Loss: 13.39\n",
- "Accuracy: 10.16%\n",
- "\n",
- "Epoch 0, Iteration 19 \n",
- "Train Loss: 12.56\n",
- "Accuracy: 12.50%\n",
- "\n",
- "Epoch 0, Iteration 20 \n",
- "Train Loss: 14.20\n",
- "Accuracy: 18.75%\n",
- "\n",
- "Epoch 0, Iteration 21 \n",
- "Train Loss: 14.74\n",
- "Accuracy: 20.31%\n",
- "\n",
- "Epoch 0, Iteration 22 \n",
- "Train Loss: 11.32\n",
- "Accuracy: 24.22%\n",
- "\n",
- "Epoch 0, Iteration 23 \n",
- "Train Loss: 11.99\n",
- "Accuracy: 27.34%\n",
- "\n",
- "Epoch 0, Iteration 24 \n",
- "Train Loss: 12.59\n",
- "Accuracy: 39.06%\n",
- "\n",
- "Epoch 0, Iteration 25 \n",
- "Train Loss: 12.47\n",
- "Accuracy: 42.97%\n",
- "\n",
- "Epoch 0, Iteration 26 \n",
- "Train Loss: 10.73\n",
- "Accuracy: 42.19%\n",
- "\n",
- "Epoch 0, Iteration 27 \n",
- "Train Loss: 9.97\n",
- "Accuracy: 36.72%\n",
- "\n",
- "Epoch 0, Iteration 28 \n",
- "Train Loss: 11.09\n",
- "Accuracy: 32.03%\n",
- "\n",
- "Epoch 0, Iteration 29 \n",
- "Train Loss: 11.04\n",
- "Accuracy: 45.31%\n",
- "\n",
- "Epoch 0, Iteration 30 \n",
- "Train Loss: 10.03\n",
- "Accuracy: 50.78%\n",
- "\n",
- "Epoch 0, Iteration 31 \n",
- "Train Loss: 9.99\n",
- "Accuracy: 46.88%\n",
- "\n",
- "Epoch 0, Iteration 32 \n",
- "Train Loss: 9.43\n",
- "Accuracy: 51.56%\n",
- "\n",
- "Epoch 0, Iteration 33 \n",
- "Train Loss: 9.78\n",
- "Accuracy: 48.44%\n",
- "\n",
- "Epoch 0, Iteration 34 \n",
- "Train Loss: 9.56\n",
- "Accuracy: 53.91%\n",
- "\n",
- "Epoch 0, Iteration 35 \n",
- "Train Loss: 9.20\n",
- "Accuracy: 54.69%\n",
- "\n",
- "Epoch 0, Iteration 36 \n",
- "Train Loss: 9.02\n",
- "Accuracy: 50.78%\n",
- "\n",
- "Epoch 0, Iteration 37 \n",
- "Train Loss: 9.06\n",
- "Accuracy: 51.56%\n",
- "\n",
- "Epoch 0, Iteration 38 \n",
- "Train Loss: 8.93\n",
- "Accuracy: 57.81%\n",
- "\n",
- "Epoch 0, Iteration 39 \n",
- "Train Loss: 8.71\n",
- "Accuracy: 60.94%\n",
- "\n",
- "Epoch 0, Iteration 40 \n",
- "Train Loss: 8.68\n",
- "Accuracy: 53.12%\n",
- "\n",
- "Epoch 0, Iteration 41 \n",
- "Train Loss: 8.58\n",
- "Accuracy: 56.25%\n",
- "\n",
- "Epoch 0, Iteration 42 \n",
- "Train Loss: 8.17\n",
- "Accuracy: 64.84%\n",
- "\n",
- "Epoch 0, Iteration 43 \n",
- "Train Loss: 8.36\n",
- "Accuracy: 53.12%\n",
- "\n",
- "Epoch 0, Iteration 44 \n",
- "Train Loss: 7.77\n",
- "Accuracy: 58.59%\n",
- "\n",
- "Epoch 0, Iteration 45 \n",
- "Train Loss: 8.26\n",
- "Accuracy: 60.94%\n",
- "\n",
- "Epoch 0, Iteration 46 \n",
- "Train Loss: 7.71\n",
- "Accuracy: 68.75%\n",
- "\n",
- "Epoch 0, Iteration 47 \n",
- "Train Loss: 7.06\n",
- "Accuracy: 78.12%\n",
- "\n",
- "Epoch 0, Iteration 48 \n",
- "Train Loss: 7.97\n",
- "Accuracy: 61.72%\n",
- "\n",
- "Epoch 0, Iteration 49 \n",
- "Train Loss: 7.73\n",
- "Accuracy: 58.59%\n",
- "\n",
- "Epoch 0, Iteration 50 \n",
- "Train Loss: 7.49\n",
- "Accuracy: 65.62%\n",
- "\n"
- ]
- }
- ],
- "source": [
- "num_iters = 50\n",
- "\n",
- "loss_hist = []\n",
- "acc_hist = []\n",
- "\n",
- "# training loop\n",
- "for epoch in range(num_epochs):\n",
- " for i, (data, targets) in enumerate(iter(trainloader)):\n",
- " data = data.to(device)\n",
- " targets = targets.to(device)\n",
- "\n",
- " net.train()\n",
- " spk_rec = forward_pass(net, data)\n",
- " loss_val = loss_fn(spk_rec, targets)\n",
- "\n",
- " # Gradient calculation + weight update\n",
- " optimizer.zero_grad()\n",
- " loss_val.backward()\n",
- " optimizer.step()\n",
- "\n",
- " # Store loss history for future plotting\n",
- " loss_hist.append(loss_val.item())\n",
- " \n",
- " print(f\"Epoch {epoch}, Iteration {i} \\nTrain Loss: {loss_val.item():.2f}\")\n",
- "\n",
- " acc = SF.accuracy_rate(spk_rec, targets) \n",
- " acc_hist.append(acc)\n",
- " print(f\"Accuracy: {acc * 100:.2f}%\\n\")\n",
- "\n",
- " if i == num_iters:\n",
- " break"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "YVjUzNcX0wld",
- "metadata": {
- "id": "YVjUzNcX0wld"
- },
- "source": [
- "# 3. Results\n",
- "## 3.1 Plot Test Accuracy"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "yp2aTX2_1zFG",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 295
- },
- "id": "yp2aTX2_1zFG",
- "outputId": "9b85de65-dbae-4818-bfc9-776625498e1e"
- },
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd1RUd/o/8PcwQ+9laEOTDqOACoIaFTWKIYqaqEFNcVPIZo1p3035bTaamKbpRZOIyUZNFIy6q9iIDaxRRFEjI1KkDEMH6QwDM5/fH8SJhDaUC8g8r3M4h3vv5977XA/OM/dTeYwxBkIIIVpLZ6gDIIQQMrQoERBCiJajREAIIVqOEgEhhGg5SgSEEKLlKBEQQoiWo0RA7lkPPPAAtm7dOtRhEHLPo0RABpWJiYn6R0dHB4aGhurt7du39+pahw8fxhNPPNGnOM6cOYNJkybB3NwcVlZWmDx5Mi5evKjRuTweD9nZ2T2WS05OBo/Hw/r16/sUIyGDhRIBGVT19fXqHxcXF+zfv1+9vXz5cnW51tZWzmKora3F3LlzsWrVKlRVVUEmk2HNmjXQ19cf0Pts3boVVlZW2LZt24BetyeMMahUqkG9J7m3USIgw0JycjKcnJywfv162Nvb429/+xtu376NuXPnQigUwtLSEnPnzkVhYaH6nPDwcHz//fcAgC1btuC+++7DP//5T1haWmLUqFE4fPhwp/fKzMwEACxduhR8Ph+GhoaYPXs2AgIC1GX+85//wM/PD5aWloiIiEB+fj4AYOrUqQCAwMBAmJiYYOfOnZ3eo6GhAbt378bGjRuRlZWF1NTUdsc3b94MPz8/mJqawt/fH5cvXwYASKVSPPTQQxAKhbC2tsbzzz8PAHj77bfx6KOPqs/Py8sDj8dTJ8zw8HC8+eabmDx5MoyMjHDr1i38+OOP6nu4u7tj06ZN7WLYt28fgoKCYGZmBg8PDyQmJmLXrl0YP358u3KfffYZ5s+f3+lzkhGCETJEXF1d2dGjRxljjCUlJTE+n89ee+01JpfLWWNjI6uoqGC7d+9mDQ0NrLa2li1atIjNnz9fff60adPY5s2bGWOM/fjjj0wgELDY2FjW2trKvvnmG+bg4MBUKlWH+9bU1DArKyv2+OOPs0OHDrGqqqp2x/fu3cs8PDyYRCJhLS0t7N1332UTJ05UHwfAsrKyun22bdu2MXt7e9ba2srmzp3Lnn/+efWxX375hTk6OrKUlBSmUqlYVlYWy8vLY62trSwgIIC99NJLrL6+njU1NbHTp08zxhhbs2YNW758ufoaubm5DABraWlR/1s4Ozuz69evs5aWFqZQKNiBAwdYdnY2U6lULDk5mRkaGrJLly4xxhi7cOECMzMzY0eOHGFKpZIVFhayGzduMLlcziwtLZlEIlHfKygoiO3evbvb5yX3NkoEZMj8NRHo6uqypqamLsunpaUxCwsL9fZfE4GHh4f6WENDAwPAiouLO72WRCJhTzzxBBOJRIzP57N58+axkpISxhhjc+bMYd9//726rFKpZIaGhiwvL48xplkimDlzJnvxxRcZY4zt2LGD2djYMIVCwRhjbPbs2eyLL77ocM65c+eYjY2N+sP9bpokgrfeeqvbmObPn6++b0xMDHvppZc6Lff3v/+d/etf/2KMMXb9+nVmYWHB5HJ5t9cm9zaqGiLDhlAohIGBgXq7sbERzz77LFxdXWFmZoapU6eiuroaSqWy0/Pt7e3VvxsZGQFoa5PojJ+fH7Zs2YLCwkJcv34dRUVFeOmllwAA+fn5ePHFF2FhYQELCwtYWVmBMQaZTKbRc0ilUiQlJanbPObPnw+5XI6DBw+qj3t4eHR6nqurKwQCgUb3+StnZ+d224cPH0ZYWBisrKxgYWGBQ4cOoaKiotsYAOCJJ57Ajh07wBjDTz/9hCVLlgx4+wkZXigRkGGDx+O12/70009x8+ZNXLhwAbW1tTh16hSAtsbQgeTr64sVK1bg+vXrANo+UDdt2oTq6mr1T1NTEyZNmqTR9X766SeoVCrMmzcP9vb2cHd3h1wuV3d1dXZ2Rk5OTofznJ2dUVBQ0GlDubGxMRobG9XbJSUlHcrc/e/X3NyMhx9+GP/85z9RWlqK6upqREZGqv/tuooBAMLCwqCnp4fTp09jx44deOyxxzR6bnLvokRAhq26ujoYGhrCwsICVVVVeOeddwbkuhkZGfj000/VDc9SqRRxcXEICwsDAPz973/Hhx9+iPT0dABATU0Ndu3apT7fzs4Ot27d6vL6W7duxZo1a3DlyhX1z549e3Do0CFUVlbi6aefxieffIJLly6BMYbs7Gzk5+djwoQJcHBwwBtvvIGGhgbI5XKcPXsWABAUFIRTp06hoKAANTU1+PDDD7t9RoVCgebmZgiFQggEAhw+fBhHjhxRH3/qqafw448/4vjx41CpVJDJZMjIyFAff/zxx/H8889DV1cX9913Xy//hcm9hhIBGbZeeuklNDU1wcbGBmFhYZgzZ86AXNfU1BQXLlxAaGgojI2NERYWhtGjR+PTTz8FACxcuBCvv/46oqOjYWZmhtGjR7frgfT222/jiSeegIWFBX755Zd21z5//jzy8/OxcuVK2Nvbq3+ioqLg6emJuLg4LF68GG+++SaWLVsGU1NTLFiwAFVVVeDz+di/fz+ys7Ph4uICJycnda+kWbNm4ZFHHkFAQADGjx+PuXPn9viMX331FZYsWQJLS0vs2LEDUVFR6uMTJkzAjz/+iJdffhnm5uaYNm2aumcUADz22GO4fv16u55KZOTisYF+zyaE3POamppga2uLy5cvw8vLa6jDIRyjNwJCSAfffvstQkJCKAloib51TyCEjFhubm5gjGHv3r1DHQoZJFQ1RAghWo6qhgghRMvdc1VDNjY2cHNzG+owCCHknpKXl6ceUPhX91wicHNz6zCBFyGEkO4FBwd3eYzTqqHExET4+PjA09MT69at63C8oKAA06dPx9ixYxEQEIBDhw5xGQ4hhJBOcJYIlEolVq5cicOHD0MikSAuLg4SiaRdmffeew9LlixBWloa4uPj8Y9//IOrcAghhHSBs0SQkpICT09PuLu7Q09PD9HR0di3b1+7MjweD7W1tQDahvE7OjpyFQ4hhJAucNZGIJPJ2s2G6OTkhAsXLrQr8/bbb2P27Nn4+uuv0dDQgGPHjnV6rdjYWMTGxgIAysvLuQqZEEK00pB2H42Li8OKFStQWFiIQ4cO4bHHHut0ib2YmBikpqYiNTUVQqFwCCIlhJCRi7NEIBKJIJVK1duFhYUQiUTtyvzwww9YsmQJAGDixImQy+Vddm8ihBDCDc4SQUhICLKyspCbmwuFQoH4+Ph2sx8CgIuLC44fPw4AuHHjBuRyOX3jJ4SQQcZZIhAIBNiwYQMiIiLg5+eHJUuWQCwWY/Xq1UhISADQtvDI5s2bERgYiKVLl2LLli0dFichhJB7VUNzK35JlQ74YkoD7Z6bayg4OJgGlBFC7gk/nMnFuwckOPTCFPg7mg1pLN19dtJcQ4QQwpGU3EoAQEFVwxBH0j1KBIQQwgHGGFLzbgMA8isbeyg9tCgREEIIB3LKG1DZoAAA5FdRIiCEEK2TklsFALA21kMBvREQQoj2uZhXBRsTPUzytEE+tREQQoj2ScmtQrCrFdysjVBULUeLsuOsCcMFJQJCCBlgRdVNkFU3YcIoK7hYGUGpYpDdbhrqsLpEiYAQQgbYxby29oEJo6zgam0MYHg3GFMiIISQAZaSWwUTfQH8HMzgam0EACioHL7tBJQICCFkgF3Mq8I4V0vwdXiwNdWHga7OsB5LQImAEEIG0O0GBTJL6zHBzRJA2wJcLlZGVDVECCHaIjW/bTRxiJuVep+LlTGklAgIIUQ7XMyrgh5fB4HOFup9rtZGKKhqHLazkFIiIISQAZSSW4UAJ3MY6PLV+1ysjNCoUKK8vnkII+saJQJCCBkgjYpWXJfVIGSUVbv9LuqeQ8OzeogSASGEDJC0gmq0qhgmuLVPBK5WbYlguPYcokRACCEDJCW3CjweMM7Vst1+J0sj6PCG76AySgSEEDJALuZVwdfeDOaGuu326wl04GBuOGwHlXGaCBITE+Hj4wNPT0+sW7euw/GXX34ZQUFBCAoKgre3NywsLDq5CiGEDH8tShXSCqoR+pf2gTtcrYfvWAIBVxdWKpVYuXIljh49CicnJ4SEhCAqKgr+/v7qMp9//rn696+//hppaWlchUMIIZy6LqtBU4uy3fiBu7laG+FIeukgR6UZzt4IUlJS4OnpCXd3d+jp6SE6Ohr79u3rsnxcXByWLl3KVTiEEMKpOxPNhYyy7PS4i5UxKhsUqG9uHcywNMJZIpDJZHB2dlZvOzk5QSaTdVo2Pz8fubm5mDFjRqfHY2NjERwcjODgYJSXl3MSLyGE9EdK7m24WRvB1tSg0+N3Jp/LH4btBMOisTg+Ph6LFi0Cn8/v9HhMTAxSU1ORmpoKoVA4yNERQkj3VCqG1PyqLquFgLZBZcDwHEvAWSIQiUSQSqXq7cLCQohEok7LxsfHU7UQIeSelV1ej+rGlg4Dye6mHlQ2DBuMOUsEISEhyMrKQm5uLhQKBeLj4xEVFdWhXEZGBm7fvo2JEydyFQohhHDqwh8L1f91INndzAx0YWmkOyx7DnGWCAQCATZs2ICIiAj4+flhyZIlEIvFWL16NRISEtTl4uPjER0dDR6Px1UohBDCqZM3yyGyMFS3A3TFxdp4WFYNcdZ9FAAiIyMRGRnZbt/atWvbbb/99ttchkAIIZxqblXiXE4FFo4V9fiF1tXKCGnS24MUmeaGRWMxIYTcq1LzbqNRocR0H9sey7paG6GoWo4WpWoQItMcJQJCCOmHpIwy6PF1MMnTuseyLlZGUKoYZLebBiEyzVEiIISQfkjOLEeouxWM9HquaXe1NgYw/Cafo0RACCF9JK1qRHZZPaZ5aza+yVW9LsHwGlRGiYAQQvooObNtpoNwDdoHAMDWVB/6Ap1hty4BJQJCCOmjkzfL4GxlCA+hsUbleTweXKx6PwupSsXw4aEbuC6r6UuYPaJEQAghfdDcqsTZ7EqEe9v2ahyUq7VRr8cSXC2sxqZTt5BZWtfbMDVCiYAQMuLdKq9H4e2BrY5Jya1CU4sS4T69m//MxcoYBVWNYIxpfM6v6aUQ6PAw09eut2FqhBIBIWTEe2ZbKt7Y8/uAXjP5Zjn0BDqY6NFzt9G7uVoboalFifL6Zo3KM8bwa3oJJnpYw9xIt+cT+oASASFkRCuqbkJOeQOuFVb36lt4T5JvliF0lGbdRu+mnnxOw+qhrLJ65FY0YLbYvtcxaooSASFkRDuXUwkAqJW3onCABnJJqxqRU96gcW+hu7la3VmXQLNEkHi9BDweEOHPTbUQQImAEDLCncuuwJ223PSi2gG5ZvLNMgDA9F62DwCAk6URdHiaDyr7Nb0EY50tYGvW+YI3A4ESASFkxGKM4VxOJWb62oKvw4OkaGC6XybfLIeLlRFG2WjWbfRuegIdOJgbajSoTFrViPSiWswZzV21EECJgBAygt2qaEBJrRzTfW3hITQekDcCeYsS53IqEe4j7PP0+a7Wmo0l+DW9BAAQwWH7AECJgBAygp3LrgAATPawgdjRfEASwZ1uo5rMNtoVFyvNxhIcSS+Fr72peo4irlAiIISMWOdyKtULxogdzVBSK0elht02u3Kn22iYe++6jd7NxdoIlQ0K1De3dlmmvK4ZF/OrOH8bACgREEJGKJWK4bdblZjoYQ0ejwd/BzMA/W8wTs4sQ5i7NQz1+H2+hqtV2zf8G8Vdx3LsRikY475aCKBEQAgZoSTFtahubMHkP9YJ8HfsfyIoqGzErfKGPvUWutskD2vYmOjhrb3XIW9Rdlom8XoJXKyM4Odg2q97aYLTRJCYmAgfHx94enpi3bp1nZb55Zdf4O/vD7FYjGXLlnEZDiHkHtKkUOLVXVdxrbC6T+efy2lrH5jkYQMAsDDSg8jCEOn96Dl08PdiAJrPNtoVS2M9fLwoEBkldVifmNHheK28BedyKhAhthuU9dw5W7NYqVRi5cqVOHr0KJycnBASEoKoqCj4+/ury2RlZeHDDz/E2bNnYWlpibKyMq7CIYTcY94/JMGuS4UAgI8XW/T6/LPZlfAQGsPurv73YkczSLqpjunOr+kl+OTITUzxsulTt9G/mu5rixWT3PDj2TxM9Ra2a3xOyihDi5Jx3m30Ds7eCFJSUuDp6Ql3d3fo6ekhOjoa+/bta1dm8+bNWLlyJSwtLQEAtrb9y7KEkJHhqKQUP58vgKEuH8mZ5VCpejc1hKJVhZTcKkz2tGm3X+xojtyKBjR000jbmZOZ5Vi1Iw1jROb49tHxvTq3O2884Atfe1O8uusqyuv+bMT+Nb0EQlN9jHW2HLB7dYezRCCTyeDs7KzednJygkwma1cmMzMTmZmZmDx5MsLCwpCYmMhVOISQe0RprRyv7b4KfwczvDXXH+V1zb3+Fn+1sBpNLUp1tdAdYkczMAZklGh+vQu3KvHsT6nwtDXB1r9NgIn+wFWkGOjy8WX0WNTJW/Hq7qtgjEHeokTyzXLM9reDjg731ULAEDcWt7a2IisrC8nJyYiLi8MzzzyD6uqO9YGxsbEIDg5GcHAwysvLhyBSQshgUKkY/u+Xq2hqUeKrpWMx64/5dU5m9u7//dk/ppUIc7dqt18s6l2D8RVpNZ7cchFOlkb46akJnMz+6WNvijcf9EPyzXJsOZeH01kVaFQoB6W30B2cJQKRSASpVKreLiwshEgkalfGyckJUVFR0NXVxahRo+Dt7Y2srKwO14qJiUFqaipSU1MhFPavtZ4QMnz9cCYXZ7Ir8NZcf3jamkBoqo/RIjMkZfSu/fBcdiVGO5rDwkiv3X57MwNYGukiXdZzIpAU1eLxHy7A2kQfPz8VCmsT/V7F0BuPhblipq8tPjycgdhTOTA1EPRrnEJvcZYIQkJCkJWVhdzcXCgUCsTHxyMqKqpdmQULFiA5ORkAUFFRgczMTLi7u3MVEiFkGLsuq8FHv2Zgtr8dlk1wUe+f7mOLywW3UdPYotF1GhWtSJPexiTPjh+kPB6vbYRxcfc9h3LK6/HYDxdgrC/A9qdDYW/O3YRvd+L6aFEAzA11cTHvNu73s4OeYPAqbDi7k0AgwIYNGxAREQE/Pz8sWbIEYrEYq1evRkJCAgAgIiIC1tbW8Pf3x/Tp0/Hxxx/D2nrwsiAhZHhoUijxYnwarIz1sP7hgHZdJsN9hFAx4HS2ZtVDF/Nuo0XJOrQP3CF2NENmST1alKour7FmXzpUjGH706Fw/mPaaK5Zm+jj08WBEOjwEBXkOCj3vIOz7qMAEBkZicjIyHb71q5dq/6dx+Phs88+w2effcZlGISQYe7dgxLcqmjAz0+FwtK4fXVOkLMlzA11kZRRjrkBPX9AnsuugC6fhxC3znvc+DuaQaFUIau0Xj3I7G65FQ04k12B/5vlDXehSd8eqI+megtxZc3sAW2Q1gSNLCaEDClJUS12XCjAM1PcO3T3BAC+Dg9TvYU4qWE30rM5FRjrYtnlymFiR/O2+3bRE2nHhXwIdHh4JMS50+NcG+wkAFAiIIQMsf+lFUKgw8Nz0zy6LBPuLURFfc/dSKsbFUgvqsWkbtYRHmVjDENdfqcjjOUtSuy6VIjZYjtOF4IZbigREEKGjFLFkHC1COE+th2qhO421butt+CdlcG6cv5WJRhDp28Wd/B1ePB1MO20C+nBa8WobmzBo6GuGj7ByECJgBAyZM7fqkRpbTMWjO2+7l9oqo8AJ3Mk3ey+wfhsdiWM9PgIdOp+SgqxoxluFNV2qGr6+UI+3IXGmNjNG8VIRImAEDJk/pcmg4m+APf79bwwe7i3EGkFt1HdqOj0eJNCiRMZZQhxs+qx66XY0Rx1za2Q3v5zcZj0ohqkFVRjeajroEz0NpxQIiCEDAl5ixKJ10vwwGh7GOj2PLf/NB/btm6kWRWdHn//kASy6iY8M6XnsUjiTqak3n6hAPoCHTw8TtTVaSMWJQJCyJA4dqMU9c2tWDBWsw/eIGcLWBjpIqmTdoI7k9Q9M2UU7vPqun3gDm87U/B1eOoG4zp5C/amyTAv0LHDaGRtQImAEDIk9qbJYGemr/FUCnwdHqZ6CXHqL91I70xSJ3Y0wz8jfDS6loEuH162Juo3gr1XitCoUOLRMO1qJL6DEgEhpNdyKxrw5bEsvLLzCpoUna+w1Z2qBgWSb5ZjfpAI/F7MsBnuI0RFvUL9AX73JHVfRo+FvkDz5SP9Hc0gKaoFYwzbz+djtMgMgU7mvX6WkWDwRy4QQu5JJTVyHLhWhISrRbhWWAMeD2AMcLU2xov3e/XqWgd/L0arimFBUO/q4+90I026WYYxTubqSeo+WDgGnra9GwXs72CG/16W4df0EmSU1GHdQ2O0rpH4DkoEhJBuZZfV4d97r+NCbhUYA8aIzPHvB/0wN8AR7x6Q4LuTOVgS4gQHc0ONr7kvTQZvO5Ner8drY6KPQCdzJN8swwxfW3z0awYixHZYOqH3o4DvjDB+98ANmOoLBn1+n+GEqoYIId364FAGJEW1eHGmF0783zTsX3Ufnp7iDntzA7zxgC+UjGH94Y7r7naloLIRqfm3sWCsqE/fwKf52OKKtBrP77gMK2M9rHsooE/XuTPPkKy6CQ+NE3U5JYU2oERACOmStKoRSTfL8MQkN7x0f8dJ2JytjPDMlFHYe6UIlwtua3TNfVfaViqc38tqoTvuzEaaX9WIz5cEdTsiuTvmhrpwtmp7i1mupY3Ed1AiIIR0Kf5iAXgAou9aH+Cv/hHuCVtTfazdL+lxUjjGGPZekWHCKCuILDSvSrpboJMFvO1M8MIML0zqZioJTczys8ccsT287XpXRTXSaO+7ECGkW4pWFXZelGKGr223H9rG+gK8NscX/9x1FfuuyrBwrFOXZa/LapFT3oCnNRj01RW+Dg9HXp7W5/Pvtnqe/4Bc515HbwSEkE79ml6CinqFRtUmD40VIcDJHOsP30SjorXLcnuvyKDH10HkaIeBDJX0EyUCQrRMk0Kp0bKPP5/Ph7OVIaZ59bxOuI4OD6vn+qOkVo7vTt7qtEyLUoWEq0WY7ivkZBF40neUCAjRMjE/pWLmZ8kor2vuskx2WR0u5FZh2QRX6Gg44CvYzQpzAxyw6WQOZNVNANoGfKXmVeGtvdcR9sFxlNc1Y9H4oVnwhXSN2ggIucc1tyrB5/Eg4Pf8ve5S/m31pG2v7r6KH1eEdNr18ufzBdDl87A4uOv6/s78v0g/HJWUYvXe6/CyM8X+q0WQVTdBX6CD+/3t8NBYEWZqMNMoGVycvhEkJibCx8cHnp6eWLduXYfjW7ZsgVAoRFBQEIKCgvD9999zGQ4hIw5jDAs3nsOquDSNyn+bnA1LI1288YAvkm+WY8u5vA5lGhWt2HO5EA+MdoCNiX6v4hFZGOLZqe44nlGGzadvwdvOBJ8/EohLb83CxmXjKAkMU5y9ESiVSqxcuRJHjx6Fk5MTQkJCEBUVBX//9q30jzzyCDZs2MBVGISMaL/dqoSkuBaS4lqczirHlG7q828U1+LYjTK8Mssbz051x8XcKnx4KANh7tbwc/hzEff9V4tQJ2/t8wRsz8/wglhkjhA3K1j1sY8/GVycvRGkpKTA09MT7u7u0NPTQ3R0NPbt28fV7QjRStvPF8DcUBcuVkZ494AErUpVl2W/Sc6BsR4fT0x0A4/Hw0eLAmBupIsX4tIgb/lz4rjtFwrgbWeCEDfLPsWkJ9BBhNieksA9hLNEIJPJ4Oz8Z6OQk5MTZDJZh3J79uxBQEAAFi1aBKlU2um1YmNjERwcjODgYJSXd79UHSHaoqxWjl/TS7B4vBP+FemHzNJ6xKUUdFo2t6IBB68V4dGJruoeO9Ym+vh0cSCyyurxwaEbAIBrhdW4Vlijlat0abMh7TU0b9485OXl4dq1a5g1axaeeOKJTsvFxMQgNTUVqampEAp77spGiDbYeVGKVhXD8jBXRIjtEOZuhc+OZnbaNXTTyRzo8nXw9H3tB3JN9Rbi6ftGYdtv+TgmKcXP5/NhqMvHQi1cpUub9ZgI9u/fD5Wq69fNrohEonbf8AsLCyEStf/jsra2hr5+W2PU008/jUuXLvX6PoRoI6WKIS6lAPd52mCUjTF4PB5WzxWjpqkFXx7Pale2qLoJey4X4pEQZwhNOzb+vjrHB/4OZnhtzzUkXC3CgrGOMDOgfv7apMdEsHPnTnh5eeG1115DRobmMwyGhIQgKysLubm5UCgUiI+PR1RUVLsyxcXF6t8TEhLg5+fXi9AJ0V5JGWUoqpFjeeifcwD5O5rhkRAXbPstD9ll9er9m0/fAmNAzNTOp3XQF/Dx1dIgNCpaIW9RYXmodk/Apo16TAQ///wz0tLS4OHhgRUrVmDixImIjY1FXV1dt+cJBAJs2LABERER8PPzw5IlSyAWi7F69WokJCQAAL766iuIxWIEBgbiq6++wpYtWwbkoQgZ6X6+kA9bU33c79++O+b/zfaGoS4f7x+UAAAq65sRl1KA+UEiOFkadXk9T1tTfBk9Fv8I98BokXau0qXNeIyx7qcL/ENlZSV++uknfPHFF/Dz80N2djZeeOEFrFq1iusY2wkODkZqauqg3pOQ4URa1YipHydh1QwvvDLLu8Px2FM5+OBQBrb8LQSpebexMTkbR1+e1usVvMjI0t1nZ49vBAkJCVi4cCHCw8PR0tKClJQUHD58GFevXsWnn3464MESQrq3I6VtauiuVuVaMWkU3KyNsPaABFt/y8MDo+0pCZBu9TigbM+ePXj55ZcxderUdvuNjIzwww8/cBYYIaSj5lYlfrkoxUw/uy6XhtQT6OBfkX6I+amt88U/wj0HM0RyD+oxEbz99ttwcPhzytimpiaUlpbCzc0NM2fO5DQ4Qkh7iddLUNmg6HHU7yx/O8z2t4OeQIfq/EmPeqwaWrx4MXR0/izG5/OxePFiToMihHRu+/kCuFgZYUoPK3PxeDzEPh6MDcvGDVJk5F7WYyJobW2Fnt6fQ8X19AjJKYMAACAASURBVPSgUCg4DYoQ0tHNkjqk5FVhWaiLxlNDE6KJHhOBUChUd/cEgH379sHGpn/rhBJCem/HhXzo8XWweHzvpoYmpCc9thF89913WL58OZ5//nkwxuDs7Ixt27YNRmyEkD8wxpBwtQgRo+1h3cupoQnpSY+JwMPDA+fPn0d9fdtIRRMT6oZGyGArrW3G7caWPs8ISkh3NFqP4ODBg0hPT4dcLlfvW716NWdBEULayyxtG8nvZWs6xJGQkajHNoK///3v2LlzJ77++mswxrBr1y7k5+cPRmyEkD/cSQTedvRGTgZej4ng3Llz2LZtGywtLbFmzRr89ttvyMzMHIzYCCF/yCqth7WxHrUPEE70mAgMDAwAtI0kLioqgq6ubrtZQwkh3Mssq4MXvQ0QjvSYCObNm4fq6mq8+uqrGDduHNzc3LBs2bLBiI2Qe0ZmaR3mbzyL9w9K8HthDTScy1EjjDFkldbD247aBwg3um0sVqlUmDlzJiwsLPDwww9j7ty5kMvlMDenIeuE3MEYw5p96bhZUgtJUQ02n86Fu40x5gU6IirIER7C/n2TL6qRo765FV6UCAhHun0j0NHRwcqVK9Xb+vr6lAQI+YsjklL8dqsS/4r0w8U378e6h8bAzswAX53IwsxPTyJqwxnIqpv6fP07DcU+lAgIR3qsGpo5cyb27NkzoK+6hIwUza1KfHDoBrxsTbBsggssjPQQPcEFcTFhOP//ZuLfD/oho7gOm0/d6vM9sqjHEOFYj4lg06ZNWLx4MfT19WFmZgZTU1OYmZkNRmyEDHtbzuYhv7IRb831h4Df/r+TnZkBnp7ijgfG2GPPpUI0Klr7dI/M0noITfVhYaTXc2FC+qDHRFBXVweVSgWFQoHa2lrU1dWhtrZ2MGIjZFgrr2vG1yeyMdPXFlO9hV2WezTMFXXNrdh/tahP98kqraO3AcKpHhPBqVOnOv3RRGJiInx8fODp6Yl169Z1WW7Pnj3g8Xi0BCW5p3x29CbkLUq8+aBft+WCXS3hY2eKn88X9PoeKhVDVlk9jSgmnOpxiomPP/5Y/btcLkdKSgrGjx+PEydOdHueUqnEypUrcfToUTg5OSEkJARRUVHw9/dvV66urg5ffvklQkND+/gIhAy+9KIaxF+U4snJo+DeQ68gHo+H5WEuWL0vHVel1Qh0ttD4PrLqJjQqlNR1lHCqxzeC/fv3q3+OHj2K69evw9Ky54mvUlJS4OnpCXd3d+jp6SE6Ohr79u3rUO6tt97C66+/rh64RshwxxjD2v0SWBrp4YWZXhqds3CsCEZ6fPx8vnfTs9DUEmQw9JgI/srJyQk3btzosZxMJoOz85+Lazs5OUEmk7Urc/nyZUilUjz44IPdXis2NhbBwcEIDg5GeXl5b0MmZED9ml6CC7lVeGWWN8wNdTU6x9RAF/ODRNh/rQg1jS0a3yuztG3WXxpDQLjUY9XQqlWrwOO1rYakUqlw5coVjBvX/+XvVCoVXnnlFWzZsqXHsjExMYiJiQEABAcH9/vehPSVvEWJ9w/dgI+dKaJDnHs+4S7LQ10Ql1KAPZcL8eR9ozQ6J6u0DvZmBhonHEL6osdEcPcHr0AgwNKlSzF58uQeLywSiSCVStXbhYWFEIlE6u26ujpcv34d4eHhAICSkhJERUUhISGBPuzJsHX4ejGkVU3Y9uSEDt1FezJaZI4gZwtsv5CPv012U3/B6g7NMUQGQ4+JYNGiRTAwMACfzwfQ1gjc2NgIIyOjbs8LCQlBVlYWcnNzIRKJEB8fjx07dqiPm5ubo6KiQr0dHh6OTz75hJIAGdauSmtgqMvH5B4Wj+/Ko2Gu+Oeuqzh/qwoTPay7LatSMWSX1WN5qGuf7kWIpjQaWdzU9Ofw+KamJtx///09XlggEGDDhg2IiIiAn58flixZArFYjNWrV7dbA5mQe4mkqBZ+Dqbg93Hx+LkBDjA31MXPF3puNJbeboS8RUUNxYRzPb4RyOXydstTmpiYoLGxUaOLR0ZGIjIyst2+tWvXdlo2OTlZo2sSMlRUKgZJcS0WjhX1XLgLBrp8LB7vhC3n8lBWJ4etade95W6W/LEqGTUUE471+EZgbGyMy5cvq7cvXboEQ0NDToMiZDiS3m5EfXMrxI79m2JlWagLWlUMv1yUdlsuq+yPHkO29EZAuNXjG8EXX3yBxYsXw9HREYwxlJSUYOfOnYMRGyHDSnpR29QqYsf+zcDrLjTBZE9rxKVI8Vy4Z5fVTJmldXA0N4CpAfUYItzqMRGEhIQgIyMDN2/eBAD4+PhAV5f+MIn2SS+qgUCHNyC9eB4NdcVz2y/jREYZZvnbdVoms7Qe3vZULUS412PV0MaNG9HQ0IDRo0dj9OjRqK+vxzfffDMYsREyrKQX1cLT1gQGuvx+X+t+fzuILAzx3cmcTqd4V6oYcsppVTIyOHpMBJs3b4aFxZ9zo1haWmLz5s2cBkXIcJReVAv/frYP3KHL18Gz09xxKf82LuRWdTieX9kARauK2gfIoOgxESiVynbfWJRKJRQKBadBETLclNXJUV7X3O/2gbstCXaGjYk+NiZldzh2Z2oJeiMgg6HHRDBnzhw88sgjOH78OI4fP46lS5figQceGIzYCBk2/mwoHrhFmQx0+Xh6yiiczqrAtcLqdsfuTDbnSW8EZBD0mAjWr1+PGTNm4LvvvsN3332HMWPGtBtgRog2kPyRCAaqauiO5aEuMDMQ4JuknHb7M0vr4GRpCGP9HvtzENJvPSYCHR0dhIaGws3NDSkpKThx4gT8/LpfiIOQkSa9qAYuVkYwG+CunKYGulgxyQ2J6SXqtYkBIKuUGorJ4OkyEWRmZuKdd96Br68vVq1aBRcXFwBAUlISnn/++UELkJDhIL2odkCrhe62YvIoGOry8e3JtreCFqUKtyrqabI5Mmi6TAS+vr44ceIEDhw4gDNnzmDVqlXqiecI0SZ18hbkVzbC34GbRGBlrIdloS7Yd6UI0qpG5Fc2oEXJ4ENvBGSQdJkI/vvf/8LBwQHTp0/HM888g+PHj3fa35mQke5GcVuVjVjETSIAgGemuEOHB2w6lUM9hsig6zIRLFiwAPHx8cjIyMD06dPxxRdfoKysDM899xyOHDkymDESMqTSi2oA9H9qie7Ymxtg0Xgn/JJaiDPZFeDxAI8e1kImZKBoNOncsmXLsH//fhQWFmLs2LFYv379YMRGyLCQXlQLGxM92Jrqc3qfZ6d6oFWpQlxKAVysjGCoR1WxZHD0aoklS0tLxMTE4Pjx41zFQ8iw0zai2FyjFcX6w83GGPMCHcEY4GVL1UJk8PR68XpCtElzqxJZpXWc9Rj6q+fCPQAAfg6UCMjgodEqhHQjq7QerSo2aInA194MO2PC4EOzjpJBRImAkG7caSjmqutoZ0Ldu1/LmJCBxmnVUGJiInx8fODp6Yl169Z1OH5nyoqgoCDcd999kEgkXIZDSK+lF9XCWI8PN2vjoQ6FEM5wlgiUSiVWrlyJw4cPQyKRIC4ursMH/bJly/D777/jypUreO211/DKK69wFQ4hfZJeVAs/BzPo9HGxekLuBZwlgpSUFHh6esLd3R16enqIjo7Gvn372pUxM/vzdbuhoYHzXhmE9IZKxXCjmLupJQgZLjhrI5DJZHB2dlZvOzk54cKFCx3Kbdy4EZ999hkUCgVOnDjR6bViY2MRGxsLACgvL+cmYEL+Iq+yAY0KJacDyQgZDoa8++jKlSuRk5OD9evX47333uu0TExMDFJTU5GamgqhUDjIERJtlc7R1NOEDDecJQKRSASpVKreLiwshEgk6rJ8dHQ09u7dy1U4hPRaelEtdPk8mvOHjHicJYKQkBBkZWUhNzcXCoUC8fHxiIqKalcmKytL/fvBgwfh5eXFVTiE9Fp6UQ28bE2hJxjyF2dCOMVZG4FAIMCGDRsQEREBpVKJJ598EmKxGKtXr0ZwcDCioqKwYcMGHDt2DLq6urC0tMTWrVu5CoeQXmGMQVJUi+m+tkMdCiGc43RAWWRkJCIjI9vtW7t2rfr3L7/8ksvbE9JnpbXNqGxQUI8hohXonZeQTgzG1NOEDBeUCAj5C8YYdlwogJ5Ah3oMEa1AiYCQv/j5fD6OZ5Th9Tm+MNGn6bjIyEeJgJC7ZJbW4b2DNzDNW4i/TXIb6nAIGRSUCAj5g7xFiRfi0mCiL8AniwNpfiGiNei9l5A/rE/MQEZJHf6zIhhCjpelJGQ4oTcCQgAk3yzDj2fzsGKSG2b42g11OIQMKkoEROtV1Dfjn7uuwcfOFG884DvU4RAy6KhqiGg1xhhe3XUVtfIWbH86FAa6/KEOiZBBR28ERKttv1CApJvleDPSj9YJJlqLEgHRWtWNCnxy5CYmeVjj8YmuQx0OIUOGEgHRWl8cy0JtUwtWz/On1fGIVqNEQLRSdlkdfjqfj6UTXOBrT9NIEO1GiYBopXcP3ICRHh+vzPIe6lAIGXKUCIjWScoow8nMcrw40wvWJjRwjBBKBESrtChVePegBO42xnh8ottQh0PIsECJgGiVn37Lx63yBrz5oB8tQUnIH+h/AtEatxsU+OJYJqZ42WAGLUFJiBqniSAxMRE+Pj7w9PTEunXrOhz/7LPP4O/vj4CAAMycORP5+flchkO03OfHMlHf3Iq35lJ3UULuxlkiUCqVWLlyJQ4fPgyJRIK4uDhIJJJ2ZcaOHYvU1FRcu3YNixYtwmuvvcZVOETLZZbWYfuFAiwPdYW3HY0gJuRunCWClJQUeHp6wt3dHXp6eoiOjsa+ffvalZk+fTqMjIwAAGFhYSgsLOQqHKLFzmZX4In/pMBYj4+XqbsoIR1wlghkMhmcnZ3V205OTpDJZF2W/+GHH/DAAw90eiw2NhbBwcEIDg5GeXn5gMdKRiZ5ixLvHpBg+fcXYKjHx/anw2BlrDfUYREy7AyL2Ud//vlnpKam4uTJk50ej4mJQUxMDAAgODh4MEMj9yhJUS1e2pmGzNJ6PBbmin9F+sFQj2YWJaQznCUCkUgEqVSq3i4sLIRIJOpQ7tixY3j//fdx8uRJ6OvT4B7SP0oVw+bTt/DpkZuwMNLDj38LwXQf6iFESHc4SwQhISHIyspCbm4uRCIR4uPjsWPHjnZl0tLS8OyzzyIxMRG2tvSflfTfW/uuY8eFAswR2+ODh8ZQVRAhGuAsEQgEAmzYsAERERFQKpV48sknIRaLsXr1agQHByMqKgqvvvoq6uvrsXjxYgCAi4sLEhISuAqJjHB18hbsuVSIxeOd8NGiAOoiSoiGOG0jiIyMRGRkZLt9a9euVf9+7NgxLm9PtEzi9RI0t6qwNNSFkgAhvUAji8mIsfeKDK7WRhjrbDHUoRByT6FEQEaE0lo5zuVUYn6QiN4GCOklSgRkREi4UgTGgAVBjkMdCiH3HEoEZNAoWlX4+ngW9qbJ0NDcOqDX/l+aDIHOFnAXmgzodQnRBsNiQBkZ+VqVKry88woO/l4MADDQ1cH9fnaYHyTCVG8b6Av6Ptgrs7QOkuJarJnnP1DhEqJVKBEQzqlUDK/v+R0Hfy/GvyJ9EeRsiYSrMhy8VowD14phZiDAA6Md8PIsb9ibG/T6+nvTZODr8DA3gKqFCOkLSgSEU4wxrElIx57LhXj5fm/ETPUAAEwYZYU188Q4k12B/VeKsPeKDDVNLfjusfG9ur5KxbDvShGmeNlAaEoj0wnpC0oE9xBpVSN0dHgQWRgOdSgaYYxh3eEM/HQ+H89Oc8cLMz3bHdfl62C6jy2m+9hCaKqP78/koqRG3qu3gtT825BVN+HVCJ+BDp8QrUGNxfeIVqUKy74/j39svzzUoWjsq+PZ2HTqFh4Lc8Ubc3y77da5LNQFShVD/MWCXt3jf2kyGOnxMVts199wCdFalAj6QFbdBMbYoN5z/7UiSKuacFVajfK65kG9d19sPnULnx/LxKLxTngnStxj335Xa2NM8bJBfIoUrUqVRvdoblXi0O/FiBDbw0iPXm4J6StKBL10LqcCk9edwPEbZYN2T5WK4ZukHFj/MYHaqczhuyZDfXMrXt99De8fuoEHAxyw/uEA6OhoNsDr0TBXlNTKcTxDs3/b5JvlqGlqwXwaO0BIv1Ai6KUNJ7IBAEckJYN2zyOSUmSV1WP1PH8ITfWRdHPwklBvXMqvQuSXp7HrkhT/CPfAF48Ega9hEgCAmb62cDA3wM/nNVu7em+aDDYmerjP06avIRNCQImgVy4X3Ma5nEoY6fFxMrN8UKqHGGP4JjkbrtZGeHCMA6Z5C3E6q0Lj6pPBoGhV4eNfM7D4u9/AwLDz2Yl4bY4vdPm9+/MS8HUQHeKC01kVyKto6LZsTVMLjmeUYV6gIwS9vA8hpD36H9QL3yTlwMJIF6/P8UVpbTNuFNdxfs8z2RW4VliD56Z5QPBHL5uaphZcLazm/N6ayC6rw0PfnsXGpBw8PM4Jh16YghA3qz5f75EQZ/B1eIhL6b7ROPF6MRStKiwI6rjYESGkdygRaCijpBbHbpTib5NG4YHR9gAwKFU0G05kw97MAAvHtX3g3edlA74OD0kZfW8n2HAiC4u+PYeWfr5V5JTXY+7XZyC73YTvHh2HjxcHwtRAt1/XtDc3wCw/O/ySKoW8RdlpmYbmVmw5lw93G2MEOJn3636EEEoEGvsmKQfGenw8MckVtmYGEDua4eRNbhttU/OqcCG3Cs9MdVdPwWBuqItxLhZIzuxbEvomORufHMlEav5tXLhV1a/4frkoRauS4cALUzBntEO/rnW3R8NccbuxBYevF3c4Jm9R4pltqcgsrcP/i/SjmUYJGQCUCDSQV9GAA9eK8GiYKyyM2nruhPsIcangNmqaWji77zfJObA00sXSCc7t9of72OK6rBZldfJeXW/L2Vx8lHgTDwY4wFCXj8T0jh+0mlL+MaJ3mrdwwAe4TfKwxigbY2w/3756SNGqwj+2X8ZvtyrxyeIAzPKnsQOEDARKBBrYdCoHAr4OnpoySr0v3McWShXDmawKTu6ZXlSDExlleHLyqA595MN9hADQqzeSXy5K8fZ+CWb72+GLR4IQ7iPEkfRSqFR9a/C+kFuJklo5Fowd+Dp6HR0elk1wQWr+bWSU1AL4c9K6ExlleH/BGCwc6zTg9yVEW3GaCBITE+Hj4wNPT0+sW7euw/FTp05h3LhxEAgE2L17N5eh9FlJjRy7LxXikWBn2Jr+OfXBWGcLmBkIkMxRO8G3yTkw0Rfg8YluHY75O5jB1lQfyRqOJ0i4WoTX/3sNU72F+HrZWOjydRAhtkdZXTPSpH1rdN6bJoOJvgD3+3HzrXzReCfoCXTw8/l8qFQMr+25hoO/F+PfD/phWagLJ/ckRFtxlgiUSiVWrlyJw4cPQyKRIC4uDhKJpF0ZFxcXbNmyBcuWLeMqjH6LPXULKgbETHVvt1/A18EUbyEn3Uhvldfj4O/FeGyiK8yNOja+8ng8hPsIcTqzvMdupEfSS/DyzisIcbPCpkfHq9sapvvaQqDDw5H03o+HkLcocfj3EswZbQ9Dvb5PH90dS2M9zA1wwP8uy/D//vs7/ntZhv+b5Y2np7j3fDIhpFc4SwQpKSnw9PSEu7s79PT0EB0djX379rUr4+bmhoCAAOjoDM8aqsr6ZsSlFGB+kCOcrYw6HA/3FqKsrhmS4toBve+3yTnQ4+vgycmjuiwT7mOLWnkrrnTzjf5cdgWe35GG0SJz/GdFSLsPbXNDXUzytEFiekmvE9mJjDLUNbdy3nXz0TBXNCiU2Jkqxd+neeD5GZ49n0QI6TXOPoFlMhmcnf9s5HRycoJMJuPqdpzYci4P8lYl/hHu0enxaX/U1ScPYO+hmyV12HO5EEsnuHQ7rfJkzz+6kXZRNVVWJ8equDS4Whth699CYKLfcS6eCLEd8isbcbO0d+Mh/pcmg62pPiZ6WPfqvN4a62yBeYGO+Ee4B16f40M9hAjhyPD8Kv4XsbGxCA4ORnBwMMrLB2eenYySWmw5l4cIf3t42pp2WsbW1ACjRWYD1k7AGMO7ByQwNdDFizO9ui1rbqiL8a6WnSYhlYrh1V3XUN/cio3Lx6l7Ov3VLH878HhA4nXNq4eqGxVIvlmG+UGOvZo+oi94PB6+XjoWr/UwcykhpH84SwQikQhSqVS9XVhYCJGob1UJMTExSE1NRWpqKoRC4UCF2IG0qhEbk7Ix54tTmPPFaShaVVg1s/vqiHBvW1wuqEZNY/+7kR6/UYYz2RV46X4vWBp3/uHd7t4+QqQX1aKstn030i3n8nAysxxvPugHb7vOkxjQlsjGu1ji1/RSjWM8+HsxWpQM82lELyEjBmeJICQkBFlZWcjNzYVCoUB8fDyioqK4ul2fNbcqsfVcHh765iymfJSEj3+9CWN9AdbOF+PsGzMgdux+5Gq4jxBKFcPp7P69qShaVXj/0A14CI3xaJirRueEe9sCQLveQ5KiWqw7nIGZvrZ4TIPrzBltjxvFtSiobNTonnvTZPCyNYHY0Uyj8oSQ4Y+zRCAQCLBhwwZERETAz88PS5YsgVgsxurVq5GQkAAAuHjxIpycnLBr1y48++yzEIvFXIXTpdV707EmIR2NCiVen+OLM69Px57nJuHxiW6wMel56cMgZwuYG+r2u51g2295yK1owL/n+ms8WZufgynszPTV4wnkLUq8GJ8GcyNdfLQoQKPqlAhx23QZv2rQe0ha1YiLebexYKyIqmoIGUE4Xc0jMjISkZGR7fatXbtW/XtISAgKCwu5DKFbh38vxs5UKZ4L98Drc3z7dA0BXwdTvGxwMrMcKhXTeO79u1XWN+PL41kI9xFiuo+txufxeDyEe9vi0PVitCpVeP/gDWSV1WPbkxNgrUESAwBnKyP4O5jh1/QSPDO1+66ZCVeLAIDm/ydkhLknGou5UFTdhDf++zsCnczxyizvfl0r3McW5f3oRvrZ0Uw0KpT494N+fbi3EHXyVnz86038dD4fT983ClO9e9eOEiG2x6WC291OWcEYw//SZJjgZgUny45daQkh9y6tTARKFcPLO6+gRanCl9Fjez1v/l9N877TjbTz3kPNrcou++rfKK5FXEoBHgtz7bJ3Uncme9lAoMPDplO34O9ghlfn9H4R94jRdmAMOCrputE4vagW2WX1nEwpQQgZWlqZCL47mYMLuVV4J0oMNxvjfl9PaKqPMSLzdu0EDc2t2Jsmw5NbLkK8+leEf5KMT4/cRNZdffbvdBc1M9TFS/d33120K2YGugh2s4SBrg6+WhqkHjncGz52pnCzNuq2G+neNBl0+TxEjrHvU5yEkOFL61b8viqtxudHM/FggAMWjR+4icvCfYTYmJSNfVdkOCopxbEbpZC3qOBgboBHw1yRXVaPjUnZ+PpENnztTTE/SAQzQwHO5VTinShxl339NbHuoQDUNLX06Y0CaGtriBDb44czuahpaoG5YftpLZQqhoSrRZjuY9uvOAkhw5NWJYL65la8GJ8GOzMDfLBgzID2fAn3scXXJ7LxYvwVWBrpYtF4J0QFihDsaqluQC6rk+PQtWLsu1qE9YkZAAAvWxMs7+ckagPxVhMx2h6bTt1CUkaZuvqnSaHE8YxS/PeyDGV1zVhI1UKEjEhalQjeTkhHQVUj4mMmdjqZW3+Mc7HAO1FiuFgb4T5Pm07bHWxNDbBi8iismDwKBZWNOCIpwX1eNsNizd0gJwvYmurjwLVimBvqIuFqEY6kl6BBoYTQVB/PhXvQ/P+EjFBakwgOXCvC7kuFWDXDExNG9X1N3a7weDw8MclN4/Iu1kbDaiZNHZ226qGfzufj2I1SmBkIMC/QEVGBjgh1t+Z8OglCyNDRmkRgbqiL2f52eKGHOXy02VP3jQIDwzRvW0z1tulTwzMh5N6jNYlgipcQU7y4m6doJHCzMcZ7C8YMdRiEkEE29JXThBBChhQlAkII0XKUCAghRMtRIiCEEC1HiYAQQrQcJQJCCNFylAgIIUTLUSIghBAtx2NdTZQ/TNnY2MDNza1P55aXl0Mo1K5BZfTM2oGeWTv055nz8vJQUVHR6bF7LhH0R3BwMFJTU4c6jEFFz6wd6Jm1A1fPTFVDhBCi5SgREEKIluO//fbbbw91EINp/PjxQx3CoKNn1g70zNqBi2fWqjYCQgghHVHVECGEaDlKBIQQouW0JhEkJibCx8cHnp6eWLdu3VCHw4knn3wStra2GD16tHpfVVUVZs2aBS8vL8yaNQu3b98ewggHllQqxfTp0+Hv7w+xWIwvv/wSwMh+ZrlcjgkTJiAwMBBisRhr1qwBAOTm5iI0NBSenp545JFHoFAohjjSgadUKjF27FjMnTsXwMh/Zjc3N4wZMwZBQUEIDg4GwN3ftlYkAqVSiZUrV+Lw4cOQSCSIi4uDRCIZ6rAG3IoVK5CYmNhu37p16zBz5kxkZWVh5syZIyoJCgQCfPrpp5BIJDh//jw2btwIiUQyop9ZX18fJ06cwNWrV3HlyhUkJibi/PnzeP311/Hyyy8jOzsblpaW+OGHH4Y61AH35Zdfws/PT72tDc+clJSEK1euqMcOcPa3zbTAuXPn2OzZs9XbH3zwAfvggw+GMCLu5ObmMrFYrN729vZmRUVFjDHGioqKmLe391CFxrmoqCh25MgRrXnmhoYGNnbsWHb+/HlmbW3NWlpaGGMd/95HAqlUymbMmMGOHz/OHnzwQaZSqUb8M7u6urLy8vJ2+7j629aKNwKZTAZnZ2f1tpOTE2Qy2RBGNHhKS0vh4OAAALC3t0dpaekQR8SNvLw8pKWlITQ0dMQ/s1KpRFBQEGxtbTFr1ix4eHjAwsICAkHbEuQj8e/7pZdewkcffQQdnbaPrMrKyhH/zDweD7Nnz8b48eMRGxsLgLv/z1qzeD1p+8Pi8XhDHcaAq6+vx8MPP4wvvvgCZmZm7Y6NxGfm8/m4cuUKACohWwAABNFJREFUqqursXDhQmRkZAx1SJw6cOAAbG1tMX78eCQnJw91OIPmzJkzEIlEKCsrw6xZs+Dr69vu+ED+bWtFIhCJRJBKpertwsJCiESiIYxo8NjZ2aG4uBgODg4oLi6Gra3tUIc0oFpaWvDwww9j+fLleOihhwCM/Ge+w8LCAtOnT8dvv/2G6upqtLa2QiAQjLi/77NnzyIhIQGHDh2CXC5HbW0tXnzxxRH9zADUz2Nra4uFCxciJSWFs79tragaCgkJQVZWFnJzc6FQKBAfH4+oqKihDmtQREVFYevWrQCArVu3Yv78+UMc0cBhjOGpp56Cn58fXnnlFfX+kfzM5eXlqK6uBgA0NTXh6NGj8PPzw/Tp07F7924AI++ZP/zwQxQWFiIvLw/x8fGYMWMGtm/fPqKfuaGhAXV1derfjxw5gtGjR3P3tz0gLQ33gIMHDzIvLy/m7u7O3nvvvaEOhxPR0dHM3t6eCQQCJhKJ2Pfff88qKirYjBkzmKenJ5s5cyarrKwc6jAHzOnTpxkANmbMGBYYGMgCAwPZwYMHR/QzX716lQUFBbExY8YwsVjM3nnnHcYYYzk5OSwkJIR5eHiwRYsWMblcPsSRciMpKYk9+OCDjLGR/cw5OTksICCABQQEMH9/f/VnFld/2zTFBCGEaDmtqBoihBDSNUoEhBCi5SgREEKIlqNEQAghWo4SASGEaDlKBERrmZiYAGibnmLHjh0Deu0PPvig3fakSZMG9PqEDCRKBETr9SURtLa2dnv8r4ng3LlzvY6LkMFCiYBovTfeeAOnT59GUFAQPv/8cyiVSrz66qsICQlBQEAANm3aBABITk7GlClTEBUVBX9/fwDAggULMH78eIjFYvXEYG+88QaampoQFBSE5cuXA/jz7YMxhldffRWjR4/GmDFjsHPnTvW1w8PDsWjRIvj6+mL58uWgIT5k0AzIsDRC7kHGxsaMsfajVRljbNOmTezdd99ljDEml8vZ+PHj2a1bt1hSUhIzMjJit27dUpe9M7KzsbGRicViVlFR0e7af73X7t272f33389aW1tZSUkJc3Z2ZkVFRSwpKYmZmZkxqVTKlEolCwsLY6dPn+bu4Qm5C70REPIXR44cwbZt2xAUFITQ0FBUVlYiKysLADBhwgSMGjVKXfarr75CYGAgwsLCIJVK1eW6cubMGSxduhR8Ph92dnaYNm0aLl68qL62k5MTdHR0EBQUhLy8PM6ekZC7acXso4T0BmMMX3/9NSIiItrtT05OhrGxcbvtY8eO4bfffoORkRHCw8Mhl8v7fF99fX3173w+v8d2CEIGCr0REK1namqqnukRACIiIvDtt9+ipaUFAJCZmYmGhoYO59XU1MDS0hJGRkbIyMjA+fPn1cd0dXXV599typQp2LlzJ5RKJcrLy3Hq1ClMmDCBg6ciRHP0RkC0XkBAAPh8PgIDA7FixQq8+OKLyMvLw7hx48AYg1AoxN69ezuc9//bu2MaAEIgCqIfFzggNJBQIYEWE4hBAQbQgi06NFxu55WLgEm2WFprWmspxqgQgmqt722MoZSSSinae795713nHOWc5ZzTnFPe+99/LoNv4/ooABjHaggAjCMEAGAcIQAA4wgBABhHCADAOEIAAMYRAgAw7gKS+UQuni37kQAAAABJRU5ErkJggg==",
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "import matplotlib.pyplot as plt\n",
- "\n",
- "# Plot Loss\n",
- "fig = plt.figure(facecolor=\"w\")\n",
- "plt.plot(acc_hist)\n",
- "plt.title(\"Train Set Accuracy\")\n",
- "plt.xlabel(\"Iterationspk_rec, mem_rec = forward_pass(net, num_steps, data)\")\n",
- "plt.ylabel(\"Accuracy\")\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "1gb1wCQb2bMd",
- "metadata": {
- "id": "1gb1wCQb2bMd"
- },
- "source": [
- "## 3.2 Spike Counter\n",
- "\n",
- "Run a forward pass on a batch of data to obtain spike recordings."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "qLAfvj9D2AYd",
- "metadata": {
- "id": "qLAfvj9D2AYd"
- },
- "outputs": [],
- "source": [
- "spk_rec = forward_pass(net, data)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "VQnj40YC2hUV",
- "metadata": {
- "id": "VQnj40YC2hUV"
- },
- "source": [
- "Changing `idx` allows you to index into various samples from the simulated minibatch. Use `splt.spike_count` to explore the spiking behaviour of a few different samples. Generating the following animation will take some time.\n",
- "\n",
- "> Note: if you are running the notebook locally on your desktop, please uncomment the line below and modify the path to your ffmpeg.exe\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "oTKhuyk22M57",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 991
- },
- "id": "oTKhuyk22M57",
- "outputId": "f6e0b621-0084-4fa2-d8a9-30644c0b7e65"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "The target label is: 3\n"
- ]
- },
- {
- "data": {
- "text/html": [
- ""
- ],
- "text/plain": [
- ""
- ]
- },
- "execution_count": 88,
- "metadata": {},
- "output_type": "execute_result"
- },
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAHBCAYAAACSZYZnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxVBf3/8feFgWGQ1Q0RRRTJXTFM09RySUihXMsNzQ0tTTRL03JJy9Qs0cqvotlCVv6+hSVmaImiuYY7aa7si4rAgKwy3N8f1CQRnhnTe4evz+fj0aO55557z2c8j9Pj1fHMOaVyuVwOAACwWq2qPQAAALR0ohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqsEdH8pa9eUO0RAAD4AFsjonlu/bxqjwAAwAfYGhHNAABQTaIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAAChQU+kNTpg0JZcP+58898JL6dq5c4Z+4bjstcdulR4DAACarKJnmpcta8hZX78ku+/6kYy57Vf5+ldOy/nf/l4mTZlWyTEAAKBZKhrNEydPyeuzZueoww5M69at85EP75Adtt06d9w1ppJjAABAs1T9muZyuZyXJ0yq9hgAALBaFb2muVfPjbJ21875+a9/m6MOOzDjnng6jz81PjvtuN0q644cNTq3jhqdJJlTX1/JMQEAYCWlcrlcruQGX3x5Qr57zfV5ecKkbLXF5unauXPatG2TC84eutrPDB5yRkYMH1bBKQEA4F8qfveMPr03zfCrL2t8ffypX8kB/fep9BgAANBkFb+m+cWXJ2TJkqVZvHhxRvx6ZGa9MSeDBuxb6TEAAKDJKn6m+Y677snv/nBnli1ryI7bb5MfXXlJ2rZtU+kxAACgySp+TfO74ZpmAACqqeq3nAMAgJZONAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFCgptIbnD7j1Vw27No887e/p22bNtn74x/LWacNSU1N60qPAgAATVLxM82XDbs2a3fpktG/HZGbb/xBHn9qfH7z+z9UegwAAGiyikfz9BmvZt+9dk9tbdusu07X7LZzv7w8cXKlxwAAgCareDQfcehncteY+7J48eK89vqsPPDIuOy284crPQYAADRZxa9p/vAO2+bW20fn4/t/Ng3Ll2dg/33yid13XWW9kaNG59ZRo5Mkc+rrKz0mAAA0KpXL5XKlNrZ8+fJ8+ogTctDAARn8uYOzcNGiXHzF1dlk4x4Zesrxq/3c4CFnZMTwYZUaEwAAVlLRyzPmzZufma++ns8dNDBt27ZJl86d8ukB++aBh8dVcgwAAGiWikZzly6d06N7t/zm93dk2bKGzJ//Zm6/8+706b1pJccAAIBmqfgfAl5x8dfz4KOP5ZMHHpkDjxqSmpqafPnUEys9BgAANFnF/xBwiz6bZfjVl1V6swAA8K55jDYAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFaiq5sT0GHLrS6yVLl+bQz+yfs4eeUskxAACgWSoazfeP/k3jzwsXLkr/gwdn30/sXskRAACg2ap2ecaY+x7M2l07Z8ftt6nWCAAA0CRVi+bb77w7+++3d0qlUrVGAACAJqno5Rn/NGPma3n8qfE5/+zTV7vOyFGjc+uo0UmSOfX1lRoNAABWUZVo/sNdY9J3u63To/sGq13n4EEDcvCgAUmSwUPOqNRoAACwiqpcnnHHXWNyQP+9q7FpAABotopH81Pjn8trs95w1wwAANYYFY/m2++8O3vtsVvWat++0psGAIB3peLXNH/9rNMqvUkAAPiveIw2AAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUqEo033n32Bx6zCnZfcAh+cyRJ+aJp8dXYwwAAGiSmkpv8OFxT+QHw3+a71xwTrbZ6kOZ9cbsSo8AAADNUvFoHv6Tm3PiMUdku222TJKsv966lR4BAACapaKXZzQ0NOTZ51/K3Ln1OfDIk7L/ocfm8mH/k8VLllRyDAAAaJaKnmmePWduli1blrvHPpAbf3B5alq3zpe/8a38eMQtOfXEY1Zad+So0bl11OgkyZz6+kqOCQAAK6nomeba2tokyecOHpR111k7Xbp0zlGHHZgHHh63yroHDxqQEcOHZcTwYenauXMlxwQAgJVUNJo7deyQbuutm5T+taxUKq3+AwAA0AJU/JZzgz61b/7fyNsze87czJv/Zn75v7/LHrt+pNJjAABAk1X87hknHnN45tbPy8FHn5zatm2y71575PijP1fpMQAAoMlK5XK5XO0higweckZGDB9W7TEAAPiA8hhtAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACohmAAAoIJoBAKCAaAYAgAKiGQAACtQ0Z+UJk6ZkwqQp6d5t/Wy1xeb5+wsv5Yc3/Cxz5s7Lbrv0yxeOPzqtWr1zhw8Z+rWMf/b5tG7dOkmy3nrrZOSI69/9bwAAAO+zZkXz8J/+MnePfSBf+dKQfGjzTfPl8y7JrNlzUi6X8+LLE1LXrl2OP/qzhd9z9tBTcuDA/u96aAAAqKRmXZ7x3PMvJUl22WnHPPf8S3n9jdlZZ+0u2XrLPimXy7nz7rHvy5AAAFBNzYrmN2bPSZJ077Z+Xnh5QpLkuKM+m2HfuTBJMvO115v0PT+84WfZ59NH5vjTvppxTzzdnBEAAKDimnV5RqvWKxr7zQUL8uLLE1IqlbJZr55pX1eXJCkvLxd+x+knH5dNe22cNjVtcteY+/Ll8y7JL2+8Jhv16L7SeiNHjc6to0YnSebU1zdnTAAAeE8160xzj+4bJElOOO2rue2Pf04pSZ/em+a112clSdbu2qXwO7bdeous1b592rZtk4ED9skO226VvzwybpX1Dh40ICOGD8uI4cPStXPn5owJAADvqWZF80ED+6dcLmfq9JlZunRpdt9153Tu1DF//cclFltv2afZA5RKpaRcfIYaAACqpVmXZxx24AHp3Kljnhr/XLp3Wy+HHnhAkqRzp4456dgjsnO/vu/4+fnz38z4557Ph3fYLq1bt86f7rkvjz89Pmd9aci7/w0AAOB9ViqXK3ead87c+gw956JMnDw1rVq1Sq+eG+WUE47OR3fa8R0/N3jIGRkxfFiFpgQAgJUVnmm+/c67m/WFA/vvs9r3unbpnJ9ff1Wzvg8AAKqtMJq/edmwFdcdN0Ep7xzNAACwJmrSNc1NvoKjiXENAABrksJovu6qSysxBwAAtFiF0dyv73aVmAMAAFqsZt1yLkmWLVuW2/7454x74unMn/9mfvDdi/PE0+NTLidb9umd9u3r3o85AQCgapoVzQsXLsopXz4vf3/h5ZTL5cY/EPzFLb/L/Q89mrNOOymfO3jQ+zIoAABUS7OeCHj9T2/Oc8+/tMofBh40aMWTAsf+5eH3dDgAAGgJmhXNY8Y+mFKplCsuPnel5X233TpJMnHy1PduMgAAaCGaFc2zZs9Okuz+0Y+stLympnWSZG79vPdoLAAAaDmaFc2dOnZMkkybPnOl5WPue2jF+506vkdjAQBAy9GsaP7n7efO+sa3G5d99YJLc/EVV6dUKuUjO27/3k4HAAAtQLOiecjnj0xdu9pMnjqt8c4ZY//ycBoaGlLXrjYnHHP4+zIkAABUU7OiuVfPjXLDNZdnpx23S6lUWnHbuaw4A3391ZelV8+N3qcxAQCgepr9cJMPbb5Z/uf7l2bxkiWZP//NdOzYIe1qa9+P2QAAoEVodjQnyZPPPJunxj+b12e9kfXWXSd9t9smO2y71Xs9GwAAtAjNiub589/MeRdfkUcee3KV9z6604759vlfTceOHd6z4QAAoCVo1jXNV1x9XR4e90TK5fIq/3l43BO54prr3685AQCgapp1pvm+Bx9JqVTKTn23y/GDP5f11l0nr896IzeNuCV/feLp3PeAx2gDAPB/T7OiuU2bNlm0eEkuveDsdOnSOUmyycY9slmvnul/8ODU+oNAAAD+D2rW5RkH9N87STJr9pyVls+eM3fF+/vt/R6NBQAALUfhmebb77y78edNe26ctbt2yelnX5jPHLBfuq23bl59fVZuu+NPWXedrtlk4x7v67AAAFANpXK5XH6nFT6y16DGp/+9k3K5nFalUh4Zc9t7Ntw/DR5yRkYMH/aefy8AADRFk65pLujqf633X40CAAAtU2E0X3fVpZWYAwAAWqzCaO7Xd7tKzAEAAC3Wu3qM9vMvvpLJU6dlydKlq7w3sP8+//VQAADQkjQrmufWz8uZ534zf/v7i//x/VJEMwAA//c0K5qvvfHnGf/cC6tfoQl32QAAgDVNsx5u8uCjj6VUKmXI549MkpRKpVx16QXZYdutsnGP7rnqOxe8L0MCAEA1NSua3/jHkwCPPOwzjct23/Uj+fb5Z2fKtBm59y8PN/m7Jk+dlt0+eVDO/9aVzRkBAAAqrlnR3LZt2yRJbdva1Nau+Hny1Glp1WrFZRl33/uXJn/X5cOuy9Zb9mnO5gEAoCqadU3zOmt3zdRpMzK3fl569tgwL02YlJPPODetWrVe8WU1Tfu6O+8em44d1sr222yZqdNmNH9qAACooGadad6yT++Uy+U8+/wLGfDJT6RcLmfWG3Py2uuzkiT77b1H4Xe8uWBhrv/JzTnziye+u4kBAKDCmnWm+ezTT86Qzx+Ztbt2yZ677ZJWrVplzNgH8tZby9J7001y2IEDC7/juptG5NP775du66/7juuNHDU6t44anSSZU1/fnDEBAOA9VSqXy+X/9kseevSxnH7ORWlVKuWRMbetdr3nX3wl53/7ytx8w9Vp06ZNrv/JzZk6bUYu+cZX3vH7Bw85IyOGD/tvxwQAgHflXT0RcHWK6vuxJ5/J9JmvZuBnj0uSLFy0OMuXL88rJw3NzTdc/V6OAgAA75n3NJqLHDyof/bbe8/G17+4ZWSmz3w153751EqOAQAAzVLRaG7Xrl3atWvX+Lqurl1q27ZN1y6dKzkGAAA0S2E0P/7U+MIveemVSe9q4ycfd9S7+hwAAFRSYTSffMa5KZVKlZgFAABapCZdnvEe3GADAADWWIXRPLD/PpWYAwAAWqzCaL7wa2dUYg4AAGixmvUYbQAA+CASzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAFaiq9wfO/dWUeffypLF68OOus3TXHHH5IDhzYv9JjAABAk1U8mj9/1GE5/+yhadu2TSZOmpKTzzg3W/Tpna222LzSowAAQJNU/PKM3ptukrZt26x4USolpVKmTp9R6TEAAKDJKn6mOUkuu+rajBp9d5YsWZIt+vTOx3bZqRpjAABAk5TK5XK5GhtuaGjIM3/7e8Y9+Uw+f+ShqalZud9HjhqdW0eNTpLMqa/P7bf8pBpjAgBA9aL5ny793g+zWa+eOfyQT692ncFDzsiI4cMqOBUAAPxL1W8519Cw3DXNAAC0aBWN5tlz5ubOu8dm4cJFaWhoyEOPPpY7x4zNRz7ct5JjAABAs1T0DwFLpVJ+e9sf853vX5tyeXk26LZ+zjrtpHz8Y7tUcgwAAGiWikZz1y6dM/zqyyq5SQAA+K9V/ZpmAABo6UQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABUQzAAAUEM0AAFBANAMAQAHRDAAABWoqubGlS9/KZcOuzaOPPZl5897MRhtukFOHHJuP7bJTJccAAIBmqeiZ5oaGhnRbb90MH3ZZ7v3DLfnCCYNz7kWXZ/qMVys5BgAANEtFzzTX1bXLyccd1fh6j912zobdu+W5F17Kht27VXIUAABosopG8797Y/acTJ4yLb179VzlvZGjRufWUaOTJHPq6ys9GgAANCqVy+VyNTa8bNmynH72henRo3u+ftZp77ju4CFnZMTwYRWaDAAAVlaVu2csX74853/7e6lpU5Nzhp5SjREAAKDJKn55RrlcziVXXJPZc+bm6ssvSk1NVa8QAQCAQhUv1u98/0eZMGlKrv3et9KutrbSmwcAgGaraDTPmPlaRo4anbZt2qT/wYMbl5931qn51Cf3quQoAADQZBWN5u4brJ9x995eyU0CAMB/zWO0AQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKCCaAQCggGgGAIACohkAAAqIZgAAKFDxaL5l5KgMHnJGdv3kgbnoO1dVevMAANBsNZXe4HrrrpMTBn8uD/318SxZsrTSmwcAgGareDTvveduSZJnn38pr70+q9KbBwCAZnNNMwAAFKj4meamGjlqdG4dNTpJMqe+vsrTAADwQdZio/ngQQNy8KABSZLBQ86o8jQAAHyQuTwDAAAKVPxM87JlDWloaMjy5Q1pWL48S5YsTevWrVNT07rSowAAQJNUPJp/POLXueFnv2p8/cc/3ZOTjj0iJx93VKVHAQCAJql4NJ983FECGQCANYprmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAAChQ8Wiunzc/X/nGt7L7gEMy8HPHZfSf7630CAAA0Cw1ld7g5cP+J23atMldI3+RF156JUPP/Wb69N40vTfdpNKjAABAk1T0TPOiRYsz5r4Hc8rxR6d9+7r03X6b7LnbLrnjrnsqOQYAADRLRaN50tRpad26dTbZuEfjsg/13jSvTJxUyTEAAKBZKnp5xqJFi9Khfd1Kyzp0aJ8FCxetsu7IUaNz66jRSZI59fUVmQ8AAP6TikZzXV1d3vy3QF6wYGHW+reQTpKDBw3IwYMGJEkGDzmjIvMBAMB/UtHLMzbZqEcaGhoyeeq0xmUvvDwhm/XyR4AAALRcFY3murp22WuPXXPdTTdn0aLFefKZZzP2gUey/357VXIMAABolorfp/lrZ34xS5YszScPOipfv+S7OffML7rdHAAALVrF79PcuVPHfO/b36j0ZgEA4F3zGG0AACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKiGYAACggmgEAoIBoBgCAAqIZAAAKrBHR3KVzp2qPAADAB1ipXC6Xqz0EAAC0ZGvEmWYAAKgm0QwAAAVEMwAAFBDNAABQQDQDAEAB0QwAAAVEMwAAFBDNAABQQDQDAEAB0QwAAAVEMwAAFBDNAABQQDQDAECBmmoP0BSf/fwXU9u2bbXHoInm1Nena+fO1R6DJrCv1iz215rF/lpz2Fdrlvd7f3Xp3Ck/+O7FqyxfI6K5tm3bjBg+rNpj0ESDh5xhf60h7Ks1i/21ZrG/1hz21ZqlWvvL5RkAAFBANAMAQIHWF1100UXVHqIpttpi82qPQDPYX2sO+2rNYn+tWeyvNYd9tWapxv4qlcvlcsW3CgAAaxCXZwAAQAHRDAAABVr0Lefq583PJVdcnYfHPZEunTvltJOOzYB9P1HtsfiHIUO/lvHPPp/WrVsnSdZbb52MHHF9kmT0n+/ND2/4WebWz8su/XbMBecMTedOHas57gfKLSNH5fbRd+elCRPTf++P56Jzz2x879HHnszlw67LzNdez7ZbfSgXfe3MdN9g/STJ0qVv5TtX/Shjxj6QdrW1GXzEITn6swdV69f4wFjd/po+49V8+ogTUteuXeO6xx55SE485ogk9lc1LF36Vi4bdm0efezJzJv3ZjbacIOcOuTYfGyXnZI4vlqad9pfjq+W5/xvXZlHH38qixcvzjprd80xhx+SAwf2T9JCjq1yC3buNy8vf+2iy8oLFiwsP/HU+PKe+x9WfumVidUei3846fRzyreOGr3K8pdemVjeY8Ch5ceefKa8YMHC8nkXX1H+2kWXVWHCD667xz5Qvue+B8uXfu+H5Qsv/X7j8jlz5pb33P+w8p/uub+8ePGS8rBrf1w+9pQvN77/g+t/Uj7htK+W6+fNL78ycXJ5vwOPKj/w8Lhq/AofKKvbX9Omzyz3+/gB5bfeWvYfP2d/Vd7ChYvK1930i/K06TPLDQ0N5fseeKS8x4BDy9Omz3R8tUDvtL8cXy3PS69MLC9ZsrRcLpfLE/7xz/zZv7/YYo6tFnt5xqJFizPmvgdzyvFHp337uvTdfpvsudsuueOue6o9GgVG//ne7LHbzvnwDtumffu6fOH4o3PP/Q9lwcKF1R7tA2PvPXfLJ/bYNZ07dVpp+Zj7H0rvXj2z7yd2T21t2wz5/JF58eUJmThpSpLk9jvH5MRjDk+njh2y6SYb58CB/TNq9J+r8St8oKxufxWxvyqvrq5dTj7uqGzYvVtatWqVPXbbORt275bnXnjJ8dUCvdP+KmJ/VV7vTTdJ27ZtVrwolZJSKVOnz2gxx1aLjeZJU6eldevW2WTjHo3LPtR707wycVIVp+Lf/fCGn2WfTx+Z40/7asY98XSS5OWJk9On96aN62zUo3va1NRk8pTp1RqTf3hl4qSV9k1dXbv02HCDvDxxcubNfzOz3pi90vt9em+aVyZOrsaovM2gw4/L/ocem29eNixz59Ynif3VQrwxe04mT5mW3r16Or7WAG/fX//k+GpZLrvq2nys/yE59JhTsu46a+dju+zUYo6tFhvNixYtSof2dSst69ChfRYsXFSlifh3p598XH7/qxvzx9/8LAcPHJAvn3dJpk6bsWLfrdV+pXVX7Dtnmqtt4aLF6bDWWist69BhrSxcuCgLF604tt7+/j/fozq6dO6Un193VUb9+icZMXxYFixcmG98+8oksb9agGXLluX8b12ZAwbsk16bbOz4auH+fX85vlqmr535xdx3x//Ljddcnr322DVt27ZpMcdWi43murq6vPlvv/CCBQuz1r+FNNWz7dZbZK327dO2bZsMHLBPdth2q/zlkXGpq6tbJZAXLFiUtdq3X803USnt69r9h32zMO3b16V93Ypj6+3v//M9qqN9+7psvWWf1NS0zjprd83ZQ7+Qh//6RBYsXGh/Vdny5ctz/re/l5o2NTln6ClJHF8t2X/cX46vFqt169bpu/02ee31WfnN7+9oMcdWi43mTTbqkYaGhkyeOq1x2QsvT8hmvTap4lS8k1KplJTL6d2rZ158eULj8qnTZ2bpW2+l58YbVnE6kmSzXpvkhbftm0WLFmfq9Jnp3atnOnXskHXXWXul9198eUI2e9u/xqS6SqUV/11eXra/qqhcLueSK67J7Dlzc8XF56WmZsWNqBxfLdPq9te/c3y1PA0NyzN1+owWc2y12Giuq2uXvfbYNdfddHMWLVqcJ595NmMfeCT777dXtUcjyfz5b+ahRx/LkiVLs2xZQ/74p3vy+NPjs+vO/TJg30/k/gcfzRNPj8+iRYtz/U2/yF577OpMcwUtW9aQJUuWZvnyhjQsX964n/baY9e8PGFS7h77QJYsWZobfv6r9NmsV3ptsnGS5ID99s5NI27JvPlvZuKkKbn19jszaMC+Vf5t/u9b3f4a/+zzmTh5apYvX5659fNy5Q+Gp1/f7dKhw4p/DWl/Vcd3vv+jTJg0JVddekHa1dY2Lnd8tUyr21+Or5Zl9py5ufPusVm4cFEaGhry0KOP5c4xY/ORD/dtMcdWi36Mdv28+bn48qvzyGNPpHOnTvnSEPdpbinmzK3P0HMuysTJU9OqVav06rlRTjnh6Hx0px2TrLiDxg+G/yz18+Zl5359c+E5Z7hPcwVd/5Obc8PPfrXSspOOPSInH3dUHhn3ZK64+rrMfPW1bPOPe11u2L1bkpXvdVlbW5tj3Je0Ila3vzbpuVGuveHnmT13btZq3z677NQ3p598fNZdp2sS+6saZsx8LYMOPz5t27RpvEd9kpx31qn51Cf3cny1MO+0v0qtWjm+WpA5c+tzzoXfyQsvTUi5vDwbdFs/hx8yKAcNHJAkLeLYatHRDAAALUGLvTwDAABaCtEMAAAFRDMAABQQzQAAUEA0AwBAAdEMAAAF/vNjcQD4jwZ97vjMePW1wvWuu+rSzAzqSsIAAAaQSURBVJj5Wr55+bDG1zvtuP37PV6hZcuW5X9/94fc9sc/Zdr0V1MqJV26dM5mvXqm/z4fz4B9Pt647vU/uTlJssXmm+UTe+xarZEBWgTRDPABcvEVV+eOu+5ZadmChYsybfrMtKutXSma//nAlYH99xHNwAeeaAZohlG33NT487gnns4pZ56XZEVYXnTumausP+hTLeexu7PemN0YzAcN7J9TTzwmtbW1mTp9Rh545LEsWbKkyhMCtFyiGeB9MuqPf17l8oy3h/Y5Z3whf3/h5fzpnvvTsWOHnHLcUdl/v71y/U9+md/e9sfU1LTO/p/cK6eedGxqav71CODxzz6fH//iljw9/rksWLgoG26wfvbfb698/sjDUlOz+v9Znz7z1caf++24fbp06Zwk6dN70/TpvWnje2+fMUluv/Pu3H7n3UmSC885I4M+tW+WL1+e//e7P+S2O/6USVOmpVWplK222DwnDD48u+zUt/GzQ4Z+LY8/NT7du62fi849M1dde2NemTA53Tfoli+ccHT2/cTu/80/YoCKEc0AVXLdTb9I/bz5SZKFixbl4iuuzpj7H8z9Dz7auM6IW0amx4Yb5NDP7J8keejRx3LmeZdk2bJljetMnjo91910c/723Au56jsXrnZ7662zTuPPl1x+dcaMfSD9+m6Xfn23y+ab9WrW7BdddtUql3k8/tT4PPH0+bn0grPzyb32WOm9OfX1Of3sC7Nk6dIkyaQpU3PexVdk3bW7pu/22zRr2wDV4O4ZAFXSrrY2t/5ieK669IIkSblczgMPj8sPrvhmbvvVj9O+ri5JMmbsA42fuXzYdVm2bFm233arjPr1TXngzpH58qknJUnuf+ivefCRx1a7ve4brJ/ddumXJFmydGnG3PdgvnvN9Tn8+NNy+PGn5Zm//T1JVpwRv/f2xs8N7L9Pxt17e8bde3sGfWrfPPH0+MZg/uKJx+S+O/43o387Iv36bpdyuZzv/+iGLF++fKVtL168JIcdeEDuvf2WfPeS81IqlbJ8+fJc/9Ob/9t/jAAVIZoBqmTggH2y8UYbZud+/7qcYYdtt8quO/fLht27ZfPNNkmSzHx9VpJk0pRpmTp9RpLk6fHPZdDhx+dj/Q/O9390Q+Pnxz359Dtu84qLz8vgww/O2l27rLT8pVcm5ivf+FbeXLCwcO63h/m1N/48e+5/WAYcMjiPPflMkuT1WbMzacq0lT7Tpk1NTjnh6HTosFb22mO39N1u63/8Hn9PuVwu3CZAtYlmgCrp3m39JEltbdvGZRusv17jz/+8PvmtpW8lSebMrS/8znn/uNxjddrV1mboKcfnj7/5WX523fcz5PNHpnOnjkmSN+bMzfhn/164jabMUf9vc3Tu1CntamsbX6+37opLRZYsXbrKugAtkWuaAaqkdevWTVr2T106d2r8+YhDP5OzTjtplXXe6azt0qVvpVWrUmpqatK6detss+WHss2WH0qnTh1z5TXXJ1k1dovm+OWN1+RDm2+2ygylUmmlZfXz5mXJkqWN/wfh9VlvJElq27ZtjHaAlsyZZoA1xCYb90iP7t2SJL//w1154JFxWbJkaebMrc+f7/1Lhgz9WmbMXP2DV6bPmJlDBp+cEb8emYmTpuStt97K67PeyIOPjPvXNnpu1Phzp44dkiRTps3I4rfdjm7Xnfs1/vy9H96QKVOn56233srEyVPz05v/N1+/5LurbPutt5blup/8Im8uWJh77n8wTz7zbJJk+223XCWwAVoiZ5oB1hClUilfHXpKzvr6t7Jw0aIMPeeiZn/HtBmv5urrbsrV1920yns7f3iHbNmnd+Prrbfsk4f/+kSeGv9sdu9/SJLk1l8MT7++26X/Ph/PnXePzWNPPpODjh6y0vd8eIdtV/nu9nV1+c3v7siIX49sXNaqVauc/Pmjmv07AFSDM80Aa5DdP/qR3HjN5dnzY7ukc6dOadOmJt3WXy+77dIvX//KaVlv3bVX+9kNuq2Xs4eekj0/tkt6bLhB6tq1S5s2Nem50YY55ohDcuW3vrHS+l/50snp13e7rNW+bpXvuuTrZ+Urp5+cLfr0Tm3btmlfV5dePTfKQYMG5IsnHrPK+p07dcwPr7wkW2/ZJ23btEnPjXvk0gvOdrs5YI1RKvuzZQDeJ29/uMnbn6YIsKZxphkAAAqIZgAAKODyDAAAKOBMMwAAFBDNAABQQDQDAEAB0QwAAAVEMwAAFBDNAABQ4P8DdJ7/tZssjnsAAAAASUVORK5CYII=",
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "from IPython.display import HTML\n",
- "\n",
- "idx = 0\n",
- "\n",
- "fig, ax = plt.subplots(facecolor='w', figsize=(12, 7))\n",
- "labels=['0', '1', '2', '3', '4', '5', '6', '7', '8','9']\n",
- "print(f\"The target label is: {targets[idx]}\")\n",
- "\n",
- "# plt.rcParams['animation.ffmpeg_path'] = 'C:\\\\path\\\\to\\\\your\\\\ffmpeg.exe'\n",
- "\n",
- "# Plot spike count histogram\n",
- "anim = splt.spike_count(spk_rec[:, idx].detach().cpu(), fig, ax, labels=labels, \n",
- " animate=True, interpolate=1)\n",
- "\n",
- "HTML(anim.to_html5_video())\n",
- "# anim.save(\"spike_bar.mp4\")"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "-iSGTq0Q3Lcm",
- "metadata": {
- "id": "-iSGTq0Q3Lcm"
- },
- "source": [
- "# Conclusion\n",
- "If you made it this far, then congratulations - you have the patience of a monk. You should now also understand how to load neuromorphic datasets using Tonic and then train a network using snnTorch. [In the next tutorial](https://snntorch.readthedocs.io/en/latest/tutorials/index.html), we will learn more advanced techniques, such as introducing long-term temporal dynamics into our SNNs.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "h-K_DUnsMKnv",
- "metadata": {
- "id": "h-K_DUnsMKnv"
- },
- "source": [
- "# Additional Resources\n",
- "* [Check out the snnTorch GitHub project here.](https://github.com/jeshraghian/snntorch)\n",
- "* [The Tonic GitHub project can be found here.](https://github.com/neuromorphs/tonic)\n",
- "* The N-MNIST Dataset was originally published in the following paper: [Orchard, G.; Cohen, G.; Jayawant, A.; and Thakor, N. “Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades\", Frontiers in Neuroscience, vol.9, no.437, Oct. 2015.](https://www.frontiersin.org/articles/10.3389/fnins.2015.00437/full) \n",
- "* For further information about how N-MNIST was created, please refer to [Garrick Orchard's website here.](https://www.garrickorchard.com/datasets/n-mnist)"
- ]
- }
- ],
- "metadata": {
- "accelerator": "GPU",
- "colab": {
- "include_colab_link": true,
- "name": "Copy of tutorial_5_neuromorphic_datasets.ipynb",
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.11"
- },
- "widgets": {
- "application/vnd.jupyter.widget-state+json": {
- "2789fbacce8f4cc8a240daa6c6d45d15": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "37deac4746684f189a32858514a780a4": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "397436bca741424c886e7f67805eebc3": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "64ab952b23bd4406af5cc6d4f72b257e": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "83b65be63ec841288496bd9ad33dbbab": {
- "model_module": "@jupyter-widgets/base",
- "model_module_version": "1.2.0",
- "model_name": "LayoutModel",
- "state": {
- "_model_module": "@jupyter-widgets/base",
- "_model_module_version": "1.2.0",
- "_model_name": "LayoutModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "LayoutView",
- "align_content": null,
- "align_items": null,
- "align_self": null,
- "border": null,
- "bottom": null,
- "display": null,
- "flex": null,
- "flex_flow": null,
- "grid_area": null,
- "grid_auto_columns": null,
- "grid_auto_flow": null,
- "grid_auto_rows": null,
- "grid_column": null,
- "grid_gap": null,
- "grid_row": null,
- "grid_template_areas": null,
- "grid_template_columns": null,
- "grid_template_rows": null,
- "height": null,
- "justify_content": null,
- "justify_items": null,
- "left": null,
- "margin": null,
- "max_height": null,
- "max_width": null,
- "min_height": null,
- "min_width": null,
- "object_fit": null,
- "object_position": null,
- "order": null,
- "overflow": null,
- "overflow_x": null,
- "overflow_y": null,
- "padding": null,
- "right": null,
- "top": null,
- "visibility": null,
- "width": null
- }
- },
- "9f46301d16924494ada03e7496af98ee": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HBoxModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HBoxModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HBoxView",
- "box_style": "",
- "children": [
- "IPY_MODEL_e965cc586d5f4d5aaf8c435ebd4bf35c",
- "IPY_MODEL_be2f2e3ca1b34106874e6b5c8cd2051e",
- "IPY_MODEL_b3a18813d8684b3ab7b9d944d8e74e4f"
- ],
- "layout": "IPY_MODEL_397436bca741424c886e7f67805eebc3"
- }
- },
- "b3a18813d8684b3ab7b9d944d8e74e4f": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_64ab952b23bd4406af5cc6d4f72b257e",
- "placeholder": "",
- "style": "IPY_MODEL_bfe2d8fcea404d22bbc61a69ae60c802",
- "value": " 1181573120/? [00:20<00:00, 60231750.25it/s]"
- }
- },
- "be2f2e3ca1b34106874e6b5c8cd2051e": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "FloatProgressModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "FloatProgressModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "ProgressView",
- "bar_style": "success",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_2789fbacce8f4cc8a240daa6c6d45d15",
- "max": 1181572961,
- "min": 0,
- "orientation": "horizontal",
- "style": "IPY_MODEL_c87ffeb8f6f54e1b871d129cd783de9f",
- "value": 1181572961
- }
- },
- "bfe2d8fcea404d22bbc61a69ae60c802": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "DescriptionStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "DescriptionStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "description_width": ""
- }
- },
- "c87ffeb8f6f54e1b871d129cd783de9f": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "ProgressStyleModel",
- "state": {
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "ProgressStyleModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/base",
- "_view_module_version": "1.2.0",
- "_view_name": "StyleView",
- "bar_color": null,
- "description_width": ""
- }
- },
- "e965cc586d5f4d5aaf8c435ebd4bf35c": {
- "model_module": "@jupyter-widgets/controls",
- "model_module_version": "1.5.0",
- "model_name": "HTMLModel",
- "state": {
- "_dom_classes": [],
- "_model_module": "@jupyter-widgets/controls",
- "_model_module_version": "1.5.0",
- "_model_name": "HTMLModel",
- "_view_count": null,
- "_view_module": "@jupyter-widgets/controls",
- "_view_module_version": "1.5.0",
- "_view_name": "HTMLView",
- "description": "",
- "description_tooltip": null,
- "layout": "IPY_MODEL_83b65be63ec841288496bd9ad33dbbab",
- "placeholder": "",
- "style": "IPY_MODEL_37deac4746684f189a32858514a780a4",
- "value": ""
- }
- }
- }
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/examples/quickstart.ipynb b/examples/quickstart.ipynb
index 2abaa53b..262004a1 100644
--- a/examples/quickstart.ipynb
+++ b/examples/quickstart.ipynb
@@ -88,8 +88,8 @@
"outputs": [],
"source": [
"batch_size = 128\n",
- "data_path='/tmp/data/mnist'\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
+ "data_path='/data/mnist'\n",
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
]
},
{
diff --git a/examples/tutorial_5_FCN.ipynb b/examples/tutorial_5_FCN.ipynb
index a430b29b..6a38c78b 100644
--- a/examples/tutorial_5_FCN.ipynb
+++ b/examples/tutorial_5_FCN.ipynb
@@ -401,7 +401,7 @@
"data_path='/tmp/data/mnist'\n",
"\n",
"dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
]
},
{
diff --git a/examples/tutorial_6_CNN.ipynb b/examples/tutorial_6_CNN.ipynb
index d6f8f903..891e7f98 100644
--- a/examples/tutorial_6_CNN.ipynb
+++ b/examples/tutorial_6_CNN.ipynb
@@ -3,8 +3,8 @@
{
"cell_type": "markdown",
"metadata": {
- "colab_type": "text",
- "id": "view-in-github"
+ "id": "view-in-github",
+ "colab_type": "text"
},
"source": [
""
@@ -261,10 +261,10 @@
"source": [
"# dataloader arguments\n",
"batch_size = 128\n",
- "data_path='/tmp/data/mnist'\n",
+ "data_path='/data/mnist'\n",
"\n",
"dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
]
},
{
@@ -630,11 +630,6 @@
},
{
"cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "y_VQ9es-gSO3"
- },
- "outputs": [],
"source": [
"optimizer = torch.optim.Adam(net.parameters(), lr=1e-2, betas=(0.9, 0.999))\n",
"num_epochs = 1\n",
@@ -676,7 +671,12 @@
" test_acc_hist.append(test_acc.item())\n",
"\n",
" counter += 1"
- ]
+ ],
+ "metadata": {
+ "id": "y_VQ9es-gSO3"
+ },
+ "execution_count": null,
+ "outputs": []
},
{
"cell_type": "markdown",
@@ -816,9 +816,9 @@
"metadata": {
"accelerator": "GPU",
"colab": {
- "include_colab_link": true,
"name": "tutorial_6_CNN.ipynb",
- "provenance": []
+ "provenance": [],
+ "include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
diff --git a/examples/tutorial_7_neuromorphic_datasets.ipynb b/examples/tutorial_7_neuromorphic_datasets.ipynb
index 8a7f894a..0d0c38fe 100644
--- a/examples/tutorial_7_neuromorphic_datasets.ipynb
+++ b/examples/tutorial_7_neuromorphic_datasets.ipynb
@@ -3,8 +3,8 @@
{
"cell_type": "markdown",
"metadata": {
- "colab_type": "text",
- "id": "view-in-github"
+ "id": "view-in-github",
+ "colab_type": "text"
},
"source": [
""
@@ -386,7 +386,8 @@
},
"outputs": [],
"source": [
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")\n",
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
+ "\n",
"# neuron and simulation parameters\n",
"spike_grad = surrogate.atan()\n",
"beta = 0.5\n",
@@ -640,11 +641,10 @@
"metadata": {
"accelerator": "GPU",
"colab": {
- "include_colab_link": true,
"name": "Copy of tutorial_5_neuromorphic_datasets.ipynb",
- "provenance": []
+ "provenance": [],
+ "include_colab_link": true
},
- "gpuClass": "standard",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
@@ -661,7 +661,8 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.11"
- }
+ },
+ "gpuClass": "standard"
},
"nbformat": 4,
"nbformat_minor": 5
diff --git a/examples/tutorial_regression_1.ipynb b/examples/tutorial_regression_1.ipynb
index cd19fd7e..960256a3 100644
--- a/examples/tutorial_regression_1.ipynb
+++ b/examples/tutorial_regression_1.ipynb
@@ -387,9 +387,8 @@
"outputs": [],
"source": [
"hidden = 128\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")\n",
- "\n",
- "model = Net(timesteps=num_steps, hidden=hidden).to(device)"
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
+ "model = Net(timesteps=num_steps, hidden=hidden).to(device)\n"
]
},
{
diff --git a/examples/tutorial_regression_2.ipynb b/examples/tutorial_regression_2.ipynb
index d0f1be46..fa84e58d 100644
--- a/examples/tutorial_regression_2.ipynb
+++ b/examples/tutorial_regression_2.ipynb
@@ -407,8 +407,7 @@
"outputs": [],
"source": [
"hidden = 128\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")\n",
- "\n",
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
"model = Net(timesteps=num_steps, hidden=hidden, beta=0.9).to(device)"
]
},
diff --git a/examples/tutorial_sae.ipynb b/examples/tutorial_sae.ipynb
index e7e4f9d6..b8ed467d 100755
--- a/examples/tutorial_sae.ipynb
+++ b/examples/tutorial_sae.ipynb
@@ -199,10 +199,10 @@
"source": [
"# dataloader arguments\n",
"batch_size = 250\n",
- "data_path='/tmp/data/mnist'\n",
+ "data_path='/data/mnist'\n",
"\n",
"dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")"
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n"
]
},
{
@@ -762,7 +762,7 @@
"\n",
"#setup GPU\n",
"dtype = torch.float\n",
- "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device("mps") if torch.backends.mps.is_available() else torch.device(\"cpu\")\n",
+ "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n",
"\n",
"# neuron and simulation parameters\n",
"spike_grad = surrogate.atan(alpha=2.0)# alternate surrogate gradient fast_sigmoid(slope=25) \n",
![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
\n",
- "
\n",
- "\n",
- "# snnTorch - Gradient-based Learning in Spiking Neural Networks\n",
- "## Tutorial 2\n",
- "### By Jason K. Eshraghian\n",
- "\n",
- "\n",
- " 
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
\n",
- "
](https://github.com/neuromorphs/tonic/)\n",
- "\n",
- "\n",
- "# Neuromorphic Datasets with Tonic + snnTorch\n",
- "## Tutorial 7\n",
- "### By Gregor Lenz (https://lenzgregor.com) and Jason K. Eshraghian (www.jasoneshraghian.com)\n",
- "\n",
- "\n",
- " 
](https://github.com/jeshraghian/snntorch/) [
](https://github.com/jeshraghian/snntorch/)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "oll2NNFeG1NG",
- "metadata": {
- "id": "oll2NNFeG1NG"
- },
- "source": [
- "The snnTorch tutorial series is based on the following paper. If you find these resources or code useful in your work, please consider citing the following source:\n",
- "\n",
- "> [Jason K. Eshraghian, Max Ward, Emre Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D. Lu. \"Training Spiking Neural Networks Using Lessons From Deep Learning\". arXiv preprint arXiv:2109.12894, September 2021.](https://arxiv.org/abs/2109.12894) "
- ]
- },
- {
- "cell_type": "markdown",
- "id": "ClgsZMOfBVby",
- "metadata": {
- "id": "ClgsZMOfBVby"
- },
- "source": [
- "# Introduction\n",
- "In this tutorial, you will:\n",
- "* Learn how to load neuromorphic datasets using [Tonic](https://github.com/neuromorphs/tonic)\n",
- "* Make use of caching to speed up dataloading\n",
- "* Train a CSNN with the [Neuromorphic-MNIST](https://tonic.readthedocs.io/en/latest/datasets.html#n-mnist) Dataset\n",
- "\n",
- "If running in Google Colab:\n",
- "* You may connect to GPU by checking `Runtime` > `Change runtime type` > `Hardware accelerator: GPU`\n",
- "* Next, install the latest PyPi distribution of snnTorch and Tonic by clicking into the following cell and pressing `Shift+Enter`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "hDnIEHOKB8LD",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "hDnIEHOKB8LD",
- "outputId": "3d7a9981-9f06-4188-eba7-77b48ccb045d"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\u001b[K |████████████████████████████████| 92 kB 3.1 MB/s \n",
- "\u001b[K |████████████████████████████████| 8.9 MB 26.5 MB/s \n",
- "\u001b[K |████████████████████████████████| 395 kB 45.8 MB/s \n",
- "\u001b[?25h Building wheel for importRosbag (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- " Building wheel for loris (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
- "\u001b[K |████████████████████████████████| 59 kB 3.1 MB/s \n",
- "\u001b[?25h"
- ]
- }
- ],
- "source": [
- "!pip install tonic --quiet \n",
- "!pip install snntorch --quiet"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e93694d9-0f0a-46a0-b17f-c04ac9b73a63",
- "metadata": {
- "id": "e93694d9-0f0a-46a0-b17f-c04ac9b73a63"
- },
- "source": [
- "# 1. Using Tonic to Load Neuromorphic Datasets\n",
- "Loading datasets from neuromorphic sensors is made super simple thanks to [Tonic](https://github.com/neuromorphs/tonic), which works much like PyTorch vision.\n",
- "\n",
- "Let's start by loading the neuromorphic version of the MNIST dataset, called [N-MNIST](https://tonic.readthedocs.io/en/latest/reference/datasets.html#n-mnist). We can have a look at some raw events to get a feel for what we're working with."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "7d286ef9-5fe6-4578-a686-91559a1f81d2",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 137,
- "referenced_widgets": [
- "9f46301d16924494ada03e7496af98ee",
- "397436bca741424c886e7f67805eebc3",
- "e965cc586d5f4d5aaf8c435ebd4bf35c",
- "be2f2e3ca1b34106874e6b5c8cd2051e",
- "b3a18813d8684b3ab7b9d944d8e74e4f",
- "37deac4746684f189a32858514a780a4",
- "83b65be63ec841288496bd9ad33dbbab",
- "c87ffeb8f6f54e1b871d129cd783de9f",
- "2789fbacce8f4cc8a240daa6c6d45d15",
- "bfe2d8fcea404d22bbc61a69ae60c802",
- "64ab952b23bd4406af5cc6d4f72b257e"
- ]
- },
- "id": "7d286ef9-5fe6-4578-a686-91559a1f81d2",
- "outputId": "f9438c1a-440f-4815-b4e6-101ba31e5cca"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Downloading https://uca6892c4e08b631fef762aee80b.dl.dropboxusercontent.com/zip_download_get/A8yubn6iJIfzLC4NXQBrS1I7ATTQ14Bx1aKLTntxskbCo02s7rFtnwUSW2bHzvIbJVxXlmWSWs7kTHeWQty4Q8hQHssWjWM0UpkC79TENd9K1A?dl=1 to ./data/NMNIST/nmnist-archive.zip\n"
- ]
- },
- {
- "data": {
- "application/vnd.jupyter.widget-view+json": {
- "model_id": "9f46301d16924494ada03e7496af98ee",
- "version_major": 2,
- "version_minor": 0
- },
- "text/plain": [
- " 0%| | 0/1181572961 [00:00"
- ]
- },
- "metadata": {
- "needs_background": "light"
- },
- "output_type": "display_data"
- }
- ],
- "source": [
- "tonic.utils.plot_event_grid(events)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f6bcc031-d11a-4471-b3aa-335eec76d7ad",
- "metadata": {
- "id": "f6bcc031-d11a-4471-b3aa-335eec76d7ad"
- },
- "source": [
- "## 1.1 Transformations\n",
- "\n",
- "However, neural nets don't take lists of events as input. The raw data must be converted into a suitable representation, such as a tensor. We can choose a set of transforms to apply to our data before feeding it to our network. The neuromorphic camera sensor has a temporal resolution of microseconds, which when converted into a dense representation, ends up as a very large tensor. That is why we bin events into a smaller number of frames using the [ToFrame transformation](https://tonic.readthedocs.io/en/latest/reference/transformations.html#frames), which reduces temporal precision but also allows us to work with it in a dense format.\n",
- "\n",
- "* `time_window=1000` integrates events into 1000$~\\mu$s bins\n",
- "\n",
- "* Denoise removes isolated, one-off events. If no event occurs within a neighbourhood of 1 pixel across `filter_time` microseconds, the event is filtered. Smaller `filter_time` will filter more events."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "30f249be-8a65-4c1c-a21c-d561e904b4bf",
- "metadata": {
- "id": "30f249be-8a65-4c1c-a21c-d561e904b4bf"
- },
- "outputs": [],
- "source": [
- "import tonic.transforms as transforms\n",
- "\n",
- "sensor_size = tonic.datasets.NMNIST.sensor_size\n",
- "\n",
- "# Denoise removes isolated, one-off events\n",
- "# time_window\n",
- "frame_transform = transforms.Compose([transforms.Denoise(filter_time=10000), \n",
- " transforms.ToFrame(sensor_size=sensor_size, \n",
- " time_window=1000)\n",
- " ])\n",
- "\n",
- "trainset = tonic.datasets.NMNIST(save_to='./data', transform=frame_transform, train=True)\n",
- "testset = tonic.datasets.NMNIST(save_to='./data', transform=frame_transform, train=False)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "70be0b77-9405-44f6-af49-dfc4d49566c6",
- "metadata": {
- "id": "70be0b77-9405-44f6-af49-dfc4d49566c6"
- },
- "source": [
- "## 1.2 Fast Dataloading via Caching\n",
- "\n",
- "The original data is stored in a format that is slow to read. To speed up dataloading, we can make use of disk caching. That means that once files are loaded from the original file, they are written to disk in an efficient format in our cache directory. Let's compare some file reading speeds to read 100 examples."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "3a6bf2a2-ff9f-4cdc-8cb3-02a1a9d71a11",
- "metadata": {
- "id": "3a6bf2a2-ff9f-4cdc-8cb3-02a1a9d71a11"
- },
- "outputs": [],
- "source": [
- "def load_sample_simple():\n",
- " for i in range(100):\n",
- " events, target = trainset[i]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1a9d3b28-b303-4a17-be78-b9918911a7cd",
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "1a9d3b28-b303-4a17-be78-b9918911a7cd",
- "outputId": "47fb99f1-0685-460a-bfd3-5aca4bc069ed"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "1 loop, best of 5: 2.76 s per loop\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "