Update Rules

In the previous section we derived all the equations needed for the computation of the cost function. Given the posterior means and variances and discrete posterior probabilities , we can compute the cost function which measures the quality of the approximation of the posterior pdf of the unknown variables. Any standard optimisation algorithm could be used for minimising the cost function, but it is sensible to utilise the particular form of the function. Due to lack of space, we shall only outline the update rules but a more detailed description can be found in [4].

Let us denote
*C* = *C*_{q} + *C*_{p}, where *C*_{q} is the part originating
from the expectation of
and *C*_{p} is the part
originating from expectation of
.
We shall
see how it is possible to derive efficient fixed point algorithms for
and
assuming that we have computed the
gradients of *C*_{p} with respect to the current estimates of
and
.

Since *C*_{q} has a term
for each
whose posterior is approximated by Gaussian ,
solving for
yields an update
rule for
:

Now suppose is roughly quadratic with respect to :

(33) |

Then

(34) |

and hence the derivatives with respect to and would be

= | (35) | ||

= | (36) |

As

(37) |

Since this update rule makes a quadratic approximation of the cost function