Introduce a noiseless predict and project (noiseless measurement)

[THargreaves]

In a target tracking setting, we need to perform data association (deciding which measurement came from which data point). See here.

This is typically done by taking each x_t, pushing it through the transition dynamics (without noise!), and then projecting it to the space of y_{t+1} (which can be viewed as a noiseless measurement). These unnoised, projected x_{t+1} can then be compared to the observations and associated.

StoneSoup has a sensible way of doing this (a Boolean flag for whether to include noise). It should be simple to refactor our code to include this. The only difficulty could be defining SSMProblems.distribution since this will be degenerate. Maybe that's just not worth worrying about for now and the user can just define simulate directly if they need data association.

[THargreaves]

As a side note (@charlesknipp this is the example I couldn't remember), splitting up the project/measure/observe into a noiseless + noise part is useful for efficiency.

In the the Lévy SSM, when you are performing closed form inference on the driving NVM scale, you need to use the noiseless projection statistics Hm + c, H'PH. For maximal performance, it would be nice to create these first using the noiseless projection, cache them (with a custom callback) and then add the noise.

This is only a minor optimisation on the scale of the whole algorithm but provides a nice motivation for why these could be useful.

[charlesknipp]

I have a few random thoughts on this subject:

• Intuitively, a change like this implies that dynamic/observation processes must be defined by both a deterministic and a stochastic function. This may be worth consideration, but I'm not sure how much of the interface would need to change.
• In @FredericWantiez's APF, he got around this by using x -> mean(SSMProblems.distribution(model.dyn, step, x)), which only works for models designed with this constructor in mind, but may be an unobtrusive start to this PR.
• This is a design choice of LowLevelParticleFilters.jl in which noiseless draws are made by setting the dynamics noise to nothing within propagate_particles!. I personally don't care much for this interface since it makes simple nonlinear dynamics, like stochastic volatility, very unintuitive.

I'll think about this a little more, but it would be helpful to have some potentially difficult models just to ensure they remain tractable in our interface.

[charlesknipp]

Consider a very basic stochastic volatility process, with a local level trend.

struct LocalLevelTrend{T} <: LatentDynamics{Vector{T}}
    γ::Vector{T}
end

Define the deterministic projection, which is just identity in this case.

function SSMProblems.project(
    proc::LocalLevelTrend{T}, step::Integer, state::Vector{T},
) where {T<:Real}
    return state
end

We can use the deterministic projection to then define the SSMProblems.distribution method more abstractly.

function SSMProblems.distribution(
    proc::LocalLevelTrend{T}, step::Integer, state::Vector{T},
) where {T<:Real}
    new_state = SSMProblems.project(proc, step, state)
    return MvNormal(new_state, PDMats.PDiagMat([exp(state[2]) proc.γ]))
end

This example is easy to implement, but some features prevent a rigorous understanding of how limiting this change could be. UCSV has additive noise in the states and no stochastic endogenous elements (all stochastic variables are latent variables).

It is annoying to have to force the user to write an additional function, but this ultimately isn't very intrusive.

[THargreaves]

That's a nice example to think about, thanks.

I think something complicated is that the projection is a function of both the dynamics and the observation process. E.g. for a linear-Gaussian model, the projection is governed by H, c which are part of the OP. But I don't think it makes sense to have a projection defined abstractly for an observation process since it depends what your state is.

It is annoying to have to force the user to write an additional function, but this ultimately isn't very intrusive.

I completely agree. For me, the key objective is to implement this in such a way that these extensions are optional. Like if you just want an SSM filtered, you define distribution; if you then want tracking, you'll get an error at first, and then you can break that up into the two necessary functions.

I think this demo script from Stone Soup is a great starting point to base our ideas off. It all seems very clean.

[charlesknipp]

If I understand correctly, I think we should define the noiseless projection in three parts:

1. SSMProblems.project(proc::LatentDynamics, step, state) from $X_{t-1}$ to $X_{t}$
2. SSMProblems.project(proc::ObservationProcess, step, state) from $X_{t}$ to $Y_{t}$
3. SSMProblems.project(model::StateSpaceModel, step, state) which combines (1.) and (2.) mapping $X_{t-1}$ to $Y_{t}$

It's likely that we almost never use (2.) on it's own, but (1.) would be quite useful to more intuitively define the auxiliary particle filter.

The only difficulty could be defining SSMProblems.distribution since this will be degenerate

There's also a good chance I'm misunderstanding something. I'm unsure as to why this would affect SSMProblems.distribution since this isn't part of the model; unless there's a forward looking characteristic in the latent dynamics.