Inference Methods

The task of estimating a sequence of sources $\mathbf{s}(1),\dots,\mathbf{s}(T)$ given a sequence of observations $\mathbf{x}(1),\dots,\mathbf{x}(T)$ and the model is called inference. In case $\mathbf{f}$ and $\mathbf{g}$ in Eqs. (1) and (2) are linear, the state can be inferred analytically with an algorithm called the Kalman filter [3]. In a filter phase, evidence from the past is propagated forward, and in a smoothing phase, evidence from the future is propagated backwards. Only the most recent state can be inferred using the Kalman filter, otherwise the algorithm should be called the Kalman smoother. In [4], the Kalman filter is extended for blind source separation from time-varying noisy mixtures.

The idea behind iterated extended Kalman smoother [3] (IEKS) is to linearise the mappings $\mathbf{f}$ and $\mathbf{g}$ around the current state estimates using the first terms of the Taylor series expansion. The algorithm alternates between updating the state estimates by Kalman smoothing and renewing the linearisation. When the system is highly nonlinear or the initial estimate is poor, the IEKS may diverge.

The iterative unscented Kalman smoother [5,6] (IUKS) replaces the local linearisation of IEKS by a deterministic sampling technique. The sampled points are propagated through the nonlinearities, and a Gaussian distribution is fitted to them. The use of nonlocal information improves convergence and accuracy at the cost of doubling the computational complexity. Still there is no guarantee of convergence.

A recent variant called backward-smoothing extended Kalman filter [8] searches for the maximum a posteriori solution to the filtering problem by a guarded Gauss-Newton method. It increases the accuracy further and guarantees convergence at the cost of about hundredfold increase in computational burden.

Particle filter [9] uses a set of particles or random samples to represent the state distribution. It is a Monte Carlo method developed especially for sequences. The particles are propagated through nonlinearities and there is no need for linearisation nor iterating. Given enough particles, the state estimate approaches the true distribution. Combining the filtering and smoothing directions is not straightforward but there are alternative methods for that. In [10], particle filters are used for non-stationary ICA.