Assume is a weight in one of the MLP networks and we have evaluated the partial derivatives and . The variance is easy to update with a fixed point update rule derived by setting the derivative to zero

By looking at the form of the cost function for Gaussian terms, we can find an approximation for the second derivatives with respect to the means as [34]

This allows using an approximate Newton's iteration to update the mean

There are some minor corrections to these update rules as explained in [34].

Antti Honkela 2001-05-30