---
# System prepended metadata

title: Comparing ALIF model formulas with code
tags: [code, NDDL]

---

###### tags: `NDDL`, `code`
# Comparing ALIF model formulas with code
This note serves to compare the implementation of the ALIF model with the description of the model in the paper 'Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks' 

***ALIF model***

![](https://i.imgur.com/t5jS3ZU.png)

***ALIF code***

    def mem_update_adp(inputs, mem, spike, tau_adp, tau_m, b, dt=1, isAdapt=1):
        """
        Parameters:
        - mem       : membrane potential
        - spike     : 0 or 1, whether there was a spike
        - tau_adp   : time-constant for decay of the threshold
        - tau_m     : time-constant for decay of the membrane potential
        - b         : parameter for adaptive change of the firing threshold theta
        - dt        : [NOT USED]
        - isAdapt   : 0 or 1, whether neuron uses adaptive threshold
        """
        # Alpha expresses the single time-step decay of the membrane potential
        # with time-constant tau_m
        alpha = tau_m

        # Ro expresses the decay single-timestep decay of the threshold 
        # with time-constant tau_adp
        ro = tau_adp

        # Beta is a constant that controls the size of adaptation of the threshold;
        # We set beta to 1.8 for adaptive neurons as default
        if isAdapt:
            beta = 1.8
        else:
            beta = 0.0

        # B is a dynamical threshold comprised of a fixed minimal threshold b_j0 
        # and an adaptive contribution beta x b
        b = ro * b + (1 - ro) * spike
        B = b_j0 + beta * b

        # Update membrane potential
        d_mem = -mem + inputs
        mem = mem + d_mem * alpha

        # Use approximation firing function to generate a spike, or not
        inputs_ = mem - B 
        spike = ActFun_adp.apply(inputs_)
        
        # Reset mem to zero after spike
        mem = (1 - spike) * mem

        return mem, spike, B, b

***Comparing model and code***

| | Model | Code |
| --- | --- | --- |
| 1. | $\eta_t = \rho \eta_{t-1} + (1-\rho) S_{t-1}$ | `b = ro * b + (1-ro) * spike` |
| 2. | $\theta = \theta_0 + \beta \eta_t$ | `B = b_j0 + beta * b` | 
| 3. | $u_t = \alpha u_{t-1} + (1-\alpha) x_t - \theta S_{t-1}$| `d_mem = -mem + inputs` |
| | | `mem = alpha * d_mem + mem` |
| 4. | $S_t = f_s(u_t, \theta)$ | `inputs_ = mem - B` |
| | | `spike = ActFun_adp.apply(inputs_)`|
| 5. | If $S_t = 1$, set $u_t=0$ | `mem = (1 - spike) * mem` |

***Observations***
:::info
The role of `alpha` is inverted to $\alpha$. 
The effect is equivalent if you set `alpha` $= 1-\alpha$.
:::

:::danger
It seems that the equations for $u_t$ and `mem` in (3) are equivalent when $S_{t-1} = 0$ [if you take `alpha` = 1 - $\alpha$], but ***not*** when $S_{t-1} = 1$. 
:::
In that case `mem` becomes `alpha * inputs + (1-alpha) * mem`, which is equal to `alpha * inputs` [because `mem` is 0 if there was a spike in previous timestep]. This lacks the term for deduction of the threshold ($\theta$ in the formulas, `B` in the code).
For the calculation of the next spike in the code, this is compensated because the threshold `B` is subtracted from the membrane potential `mem`. 
But the value for `mem` that is passed to the next iteration is higher in the code than in the formula. In other words: in the ALIF model the input value $x_t$ *in the timestep just after a spike* is dampened by threshold $\theta$, while this is not the case in the code.
The code can be made equivalent by using:

    mem = alpha * d_mem + mem - (B if spike == 1 else 0)

but I'm not sure whether that is what is supposed to happen!

I have not tested yet if this has impact on the behavior of the SRNN.

