Paper
https://arxiv.org/abs/2306.16922
Equations
$$\begin{aligned}
\boldsymbol{\kappa}_m(t) &= \exp\left(-\frac{\Delta t}{\tau_m}\right), \\\
\boldsymbol{\kappa}_s(t) &= \exp\left(-\frac{\Delta t}{\tau_s}\right), \\\
\mathbf{s}(t) &= \boldsymbol{\kappa}_s(t) \circ \mathbf{s}(t-1)
+ \mathbf{w}_s \circ \mathbf{x}(t), \\\
\Delta \mathbf{m}(t) &= \tanh\left(
\mathrm{MLP}_{\mathbf{w}_p}\!\left[
\mathbf{s}(t),\;
\boldsymbol{\kappa}_m(t) \circ \mathbf{m}(t-1)
\right]
\right), \\\
\mathbf{m}(t) &= \boldsymbol{\kappa}_m(t) \circ \mathbf{m}(t-1)
+ \lambda \left(1 - \boldsymbol{\kappa}_m(t)\right)
\circ \Delta \mathbf{m}(t), \\\
\mathbf{y}(t) &= \mathbf{w}_y \cdot \mathbf{m}(t).
\end{aligned}$$
Official implementation
https://github.com/AaronSpieler/elmneuron (Jax)
