###### tags: `NDDL`, `code`
# Comparing ALIF model formulas with code
This note serves to compare the implementation of the ALIF model with the description of the model in the paper 'Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks'

***ALIF model***


***ALIF code***

def mem_update_adp(inputs, mem, spike, tau_adp, tau_m, b, dt=1, isAdapt=1):
"""
Parameters:
- mem : membrane potential
- spike : 0 or 1, whether there was a spike
- tau_adp : time-constant for decay of the threshold
- tau_m : time-constant for decay of the membrane potential
- b : parameter for adaptive change of the firing threshold theta
- dt : [NOT USED]
- isAdapt : 0 or 1, whether neuron uses adaptive threshold
"""
# Alpha expresses the single time-step decay of the membrane potential
# with time-constant tau_m
alpha = tau_m

# Ro expresses the decay single-timestep decay of the threshold
# with time-constant tau_adp
ro = tau_adp

# Beta is a constant that controls the size of adaptation of the threshold;
# We set beta to 1.8 for adaptive neurons as default
if isAdapt:
beta = 1.8
else:
beta = 0.0

# B is a dynamical threshold comprised of a fixed minimal threshold b_j0
# and an adaptive contribution beta x b
b = ro * b + (1 - ro) * spike
B = b_j0 + beta * b

# Update membrane potential
d_mem = -mem + inputs
mem = mem + d_mem * alpha

# Use approximation firing function to generate a spike, or not
inputs_ = mem - B
spike = ActFun_adp.apply(inputs_)

# Reset mem to zero after spike
mem = (1 - spike) * mem

return mem, spike, B, b

***Comparing model and code***

| | Model | Code |
| --- | --- | --- |
| 1. | $\eta_t = \rho \eta_{t-1} + (1-\rho) S_{t-1}$ | `b = ro * b + (1-ro) * spike` |
| 2. | $\theta = \theta_0 + \beta \eta_t$ | `B = b_j0 + beta * b` |
| 3. | $u_t = \alpha u_{t-1} + (1-\alpha) x_t - \theta S_{t-1}$| `d_mem = -mem + inputs` |
| | | `mem = alpha * d_mem + mem` |
| 4. | $S_t = f_s(u_t, \theta)$ | `inputs_ = mem - B` |
| | | `spike = ActFun_adp.apply(inputs_)`|
| 5. | If $S_t = 1$, set $u_t=0$ | `mem = (1 - spike) * mem` |

***Observations***
:::info
The role of `alpha` is inverted to $\alpha$.
The effect is equivalent if you set `alpha` $= 1-\alpha$.
:::

:::danger
It seems that the equations for $u_t$ and `mem` in (3) are equivalent when $S_{t-1} = 0$ [if you take `alpha` = 1 - $\alpha$], but ***not*** when $S_{t-1} = 1$.
:::
In that case `mem` becomes `alpha * inputs + (1-alpha) * mem`, which is equal to `alpha * inputs` [because `mem` is 0 if there was a spike in previous timestep]. This lacks the term for deduction of the threshold ($\theta$ in the formulas, `B` in the code).
For the calculation of the next spike in the code, this is compensated because the threshold `B` is subtracted from the membrane potential `mem`.
But the value for `mem` that is passed to the next iteration is higher in the code than in the formula. In other words: in the ALIF model the input value $x_t$ *in the timestep just after a spike* is dampened by threshold $\theta$, while this is not the case in the code.
The code can be made equivalent by using:

mem = alpha * d_mem + mem - (B if spike == 1 else 0)

but I'm not sure whether that is what is supposed to happen!

I have not tested yet if this has impact on the behavior of the SRNN.