class: middle, center, title-slide
Lecture 5: State-space models
Prof. Gilles Louppe
g.louppe@uliege.be
???
-
Add data assimilation
-
Brownian motion: more simply, it is a gaussian process
-
The SDE is an Ito SDEs
-
Application: Black-Scholes model
-
EnKF? Unscented KF?
class: middle
.center.width-70[
]
.center[Today's case study: .bold[tracking] the location of a wolf
over time using noisy GPS observations.]
class: middle
class: middle
.center[
]
We previously defined latent variable models as probabilistic models that explain observed data
class: middle
.center.width-10[
]
What if the system evolves over time and we have a sequence of observations
class: middle
To model a dynamical system, we can first assume a discretization of time and introduce a sequence of latent variables
In this context, a .bold[state-space model] is a latent variable model that explains a sequence
class: middle
.center[
]
In a Markovian state-space model, the latent variables form a Markov chain
The observations are conditionally independent given the latent variables,
class: middle
.center.width-10[
]
We want to track the location of a wild animal (e.g., a wolf) over time using noisy GPS observations.
Assumptions:
- The animal has a home location (den, nest) at
$\mu \in \mathbb{R}^2$ . - The animal moves according to a random walk with drift towards the home location.
- The GPS observations are noisy measurements of the animal's true location.
- Time is discretized regularly every
$\Delta t$ time units.
class: middle
.center[]
State-space model:
- States
$z_t \in \mathbb{R}^2$ represent the true location of the animal at time$t$ . They evolve as a random walk with drift towards the home location$\mu$ . $$ p(z_t | z_{t-1}) = \mathcal{N}(z_t | z_{t-1} - \kappa (z_{t-1} - \mu) \Delta t, \sigma^2 \Delta t I), $$ where$\kappa > 0$ is the strength of attraction to the home location and$\sigma^2$ is the variance of the random walk. - Observations
$x_t \in \mathbb{R}^2$ represent noisy GPS measurements of the animal's location at time$t$ . They relate to the states via $$ p(x_t | z_t) = \mathcal{N}(x_t | z_t, R), $$ where$R$ is the observation noise covariance.
class: middle
.center.width-70[
]
.center[Example of trajectory and observations
generated from the discrete-time state-space model (
class: middle
Given a state-space model and a sequence of observations
- Prediction:
$p(z_{t+k} | x_{1:t})$ for$k \geq 1$ . - Filtering:
$p(z_t | x_{1:t})$ . - Smoothing:
$p(z_t | x_{1:T})$ .
class: middle
The Bayes filter is a recursive algorithm for estimating the filtering distributions
class: middle
.italic[Proof.] The filtering distribution can be derived in two steps:
- Prediction: Push the filtering distribution from the previous time step through the transition model to obtain a prediction of the current state. That is,
$$p(z_t | x_{1:t-1}) = \int p(z_t | z_{t-1}) p(z_{t-1} | x_{1:t-1}) dz_{t-1}.$$ - Update: Condition on the new observation to obtain the filtering distribution,
$$p(z_t | x_{1:t}) = \frac{p(x_t | z_t) p(z_t | x_{1:t-1})}{p(x_t | x_{1:t-1})},$$ where the marginal likelihood$p(x_t | x_{1:t-1})$ is given by$$p(x_t | x_{1:t-1}) = \int p(x_t | z_t) p(z_t | x_{1:t-1}) dz_t.$$
class: middle
Once we have computed the filtering distributions
class: middle
The Bayes smoother computes the smoothing distributions
class: middle
.italic[Proof.] The joint distribution
Marginalizing over
class: middle
.center.width-10[
]
Although the Bayes filter and Bayes smoother provide a general framework for inference in state-space models, they are .bold[rarely tractable in practice] as they involve integrals that are difficult to compute.
Further assumptions on the transition and observation models are required for closed-form solutions.
class: middle
A linear Gaussian state-space model (LGSSM) is a state-space model where both the transition and observation models are linear Gaussian. That is,
$$\begin{aligned}
p(z_t | z_{t-1}) &= \mathcal{N}(z_t | A z_{t-1}, Q), \\
p(x_t | z_t) &= \mathcal{N}(x_t | H z_t, R),
\end{aligned}$$
where
.success[If the prior distribution
class: middle
The .bold[Kalman filter] provides a closed-form expression for the filtering distributions in linear Gaussian state-space models.
At each time step
class: middle
.italic[Proof.] By recursion, assume that at time step
For the prediction step, we have
$$\begin{aligned}
p(z_t | x_{1:t-1}) &= \int p(z_{t-1} | x_{1:t-1}) p(z_t | z_{t-1}) dz_{t-1} \\
&= \int \mathcal{N}(z_{t-1} | m_{t-1}, P_{t-1}) \mathcal{N}(z_t | A z_{t-1}, Q) dz_{t-1} \\
&= \int \mathcal{N}\left(\begin{pmatrix} z_{t-1} \\ z_t \end{pmatrix} | \begin{bmatrix} m_{t-1} \\ A m_{t-1} \end{bmatrix}, \begin{bmatrix} P_{t-1} & P_{t-1} A^T \\ A P_{t-1} & A P_{t-1} A^T + Q \end{bmatrix}\right) dz_{t-1} \\
&= \mathcal{N}(z_t | m^-_t, P^-_t),
\end{aligned}$$
where
class: middle
For the update step, we join the prediction distribution with the observation model, $$\begin{aligned} p\left(\begin{pmatrix} z_t \\ x_t \end{pmatrix} | x_{1:t-1} \right) &= p(z_t | x_{1:t-1}) p(x_t | z_t) \\ &= \mathcal{N}\left(\begin{pmatrix} z_t \\ x_t \end{pmatrix} | \begin{bmatrix} m^-_t \\ H m^-_t \end{bmatrix}, \begin{bmatrix} P^-_t & P^-_t H^T \\ H P^-_t & H P^-_t H^T + R \end{bmatrix}\right). \end{aligned}$$
Therefore, the filtering distribution is given by the conditional distribution
???
Intuition:
Mean update:
-
$H m^-_t$ is the predicted observation based on the predicted state. -
$x_t - H m^-_t$ is the innovation or measurement residual, i.e., the difference between the actual observation and the predicted observation. - The Kalman gain
$K_t$ determines how much we adjust our prediction based on the new observation.
Covariance update:
- If the observation noise
$R$ is small compared to the prediction uncertainty$P^-_t$ , then$K_t$ approaches$H^{-1}$ (if$H$ is invertible), and we rely heavily on the new observation. - Conversely, if
$R$ is large, then$K_t$ approaches zero, and we rely more on our prediction.
class: middle
.center.width-70[
]
.center[Mean estimate of the wolf's trajectory using the Kalman filter.]
class: middle
.center.width-70[
]
.center[Filtering distribution at each time step using the Kalman filter.]
class: middle
The smoothing distributions
The parameters can be computed recursively using the .bold[Rauch-Tung-Striebel smoother] equations (proof omitted for brevity),
$$\begin{aligned}
C_t &= P_t A^T (P^-_{t+1})^{-1}, \\
m^s_t &= m_t + C_t (m^s_{t+1} - m^-_{t+1}), \\
P^s_t &= P_t + C_t (P^s_{t+1} - P^-_{t+1}) C_t^T,
\end{aligned}$$
for
???
XXX Check Sarkka's book for the full derivation. Consider adding it for completeness.
class: middle
.center.width-70[
]
.center[Mean estimate of the wolf's trajectory using the Kalman smoother.]
class: middle
.center.width-70[
]
.center[Smoothing distribution at each time step using the Kalman smoother.]
class: middle
Hidden Markov models
A hidden Markov model (HMM) is a state-space model where all variables are discrete and the transition and observation models are categorical distributions. That is,
$$\begin{aligned}
p(z_t=j | z_{t-1}=i) &= A_{i, j}, \\
p(x_t=k | z_t=j) &= B_{j, k},
\end{aligned}$$
where
.success[If the prior distribution
class: middle
Example: Wolf behavior modeling
- States
$z_t \in {1, ..., K}$ represent the behavior of the animal at time$t$ (e.g., resting, foraging, traveling). - Observations
$x_t \in {1, ..., M}$ represent discrete measurements related to the animal's behavior (e.g., GPS speed categories, activity levels). - Transition model
$p(z_t | z_{t-1})$ captures the probabilities of switching between different behaviors. - Observation model
$p(x_t | z_t)$ captures the probabilities of observing certain measurements given the animal's behavior.
class: middle
The .bold[forward algorithm] provides a closed-form expression for the filtering distributions in hidden Markov models.
At each time step
class: middle
The smoothing distributions
The smoothing parameters are then given by
class: middle
.success[Both linear Gaussian state-space models and hidden Markov models are special cases of state-space models where exact inference is tractable.]
.alert[However, they are limited in their expressiveness and may not capture the complexity of real-world dynamical systems.]
class: middle
class: middle
.center.width-10[
]
We have so far assumed that time is discretized regularly with a fixed time step
However,
- physical processes are often more naturally modeled in .bold[continuous time];
- observations may be collected at .bold[irregular time intervals], triggered by events rather than a clock, or at multiple time scales.
class: middle
In discrete-time state-space models, we considered transition models of the form
Shuffling the terms, we get
class: middle
Omitting
The solution of this ODE with initial condition
class: middle
- If
$z(0) > \mu$ , then$z(t)$ decreases towards$\mu$ as$t$ increases. - If
$z(0) < \mu$ , then$z(t)$ increases towards$\mu$ as$t$ increases. - Solution:
$$z(t) = \mu + (z(0) - \mu) e^{-\kappa t}.$$
.alert[This is deterministic: given
???
The decay is 'exponential' because the difference
... although the system approaches the equilibrium point
class: middle
To add stochasticity to the ODE, we need a continuous-time stochastic process that can model random noise.
We can model the noise term
-
$B(0) = 0$ . -
$B(t)$ has independent increments: for$0 \leq s < t$ ,$B(t) - B(s) \sim \mathcal{N}(0, t-s)$ . -
$B(t)$ is continuous in$t$ .
class: middle
Adding Brownian motion to the ODE, we get a .bold[stochastic differential equation] (SDE)
More rigorously, Brownian motion is nowhere differentiable and the notation
???
The "differential form" means that the change in
class: middle
For more generality, we can extend
We can also introduce a .bold[diffusion term]
In this form, the SDE describes the infinitesimal change in the state
- The drift term
$f(z(t), t) dt$ represents the deterministic change in the state. - The diffusion term
$g(z(t), t) dB(t)$ represents the stochastic change in the state due to Brownian motion.
???
Teaser: this equation is the basis of modern generative models such as .bold[diffusion models] used in image synthesis (e.g., DALL-E 2, Stable Diffusion).
class: middle
Recall our animal movement example in discrete time:
In continuous time, this becomes the SDE
This process is known as the .bold[Ornstein-Uhlenbeck process], which describes a mean-reverting behavior with Gaussian noise.
.success[The discrete-time model is the Euler-Maruyama discretization of the OU process with step size
class: middle
.center.width-70[
]
.center[Example of continuous trajectory generated from the Ornstein-Uhlenbeck process.
(This is the true trajectory used throughout the lecture.)]
class: middle
In continuous-time state-space models, the observation model can be defined as a conditional distribution
This is similar to the discrete-time case, except that observations can be collected at .bold[irregular time intervals]
class: middle
Linear Gaussian continuous-time state-space models are continuous-time analogs of linear Gaussian state-space models.
They are defined by linear SDEs for the state dynamics and linear Gaussian observation models,
$$\begin{aligned}
dz(t) &= A z(t) dt + Q^{1/2} dB(t), \\
x(t_i) &\sim \mathcal{N}(x(t_i) | H z(t_i), R),
\end{aligned}$$
where
class: middle
Filtering and smoothing distributions
Both now correspond to stochastic processes over continuous time rather than sequences over discrete time steps.
class: middle
.bold[Continuous-time] is natural when:
- Observations at irregular intervals
- Physical/mechanistic interpretation important
- Parameters have continuous-time meaning (rates, time constants)
.bold[Discrete-time] is practical when:
- Regular sampling
- Computational simplicity preferred
- No strong mechanistic model
.italic[Common approach: Model in continuous time for interpretability, discretize for computation.]
class: end-slide, center count: false
The end.