Continuous-time reservoir computing

[MartinuzziFrancesco]

In order to build a first pass of a continuous reservoir computer I think we need to address the following points:

1. Conceptually, the reservoir should be built around any AbstractSciMLProblem, since the long-term goal is to support continuous-time substrates more generally. In practice though, the first version should probably focus on ODEProblem, because that already gives a clean path to a working continuous-time reservoir without having to solve all the extra interface issues for SDEs, DDEs, and other problem types.

2. How does each input sample u_k enter the dynamics? A natural approach would be to define an interpolated input signal u(t) from the discrete data and use it directly as a forcing term in the dynamical system.

3. A sampling rule: The system evolves in continuous time, but reservoir computing still needs a discrete state r_k for each input step. So we need to decide what part of the trajectory becomes the reservoir state.

   Here, the simplest rule is terminal-state sampling: after evolving over one interval, use the final state x(t_k + Δt) as the reservoir state r_k. This is enough for a first version, and can later be extended.

An idea of the high level API could be something of the sorts:

res = SciMLProblemReservoir(
    prob,
    args...;
    sampler = TerminalStateSampling(),
    kwargs...
)

where args and kwargs are forwarded to the solve similarly to the models in DiffEqFlux. The workflow in ReservoirComputing.jl could look something like

using DifferentialEquations
using ReservoirComputing

function ctesn!(dx, x, p, t)
    u = p.input(t)
    dx .= tanh.(p.W * x .+ p.Win * u .+ p.b)
end

#define x0, tspan and p here

prob = ODEProblem(ctesn!, x0, tspan, p)

res = SciMLProblemReservoir(
    prob;
    sampler = TerminalStateSampling(),
    saveat = ts,
    solver = Tsit5(),
)

# now the reservoir can go into a ReservoirComputer

rc = ReservoirComputer(
    res;
    LinearReadout(size(W, 1) => out_dims),
)

Of course there is a lot of plumbing to take care of, for instance dispatch and adapt the collectstates function, but I think this should be more or less the approach.

All naming in the quick example is subject to changes