The basic setting for updating the hidden states is the same as above for the network weights. The correlations between consecutive states cause some changes to the formulas and require new ones for adaptation of the correlation coefficients. All the feedforward computations use the marginal variances which are not actual variational parameters. This affects the derivatives with respect to the other parameters of the state distribution. Let us use the notation to mean that the part of the cost function is considered to be a function of the intermediate variables in addition to the variational parameters. This and Equation (5.48) yield following rules for evaluating the derivatives of the true cost function:
The term in the above equations cannot be evaluated directly, but requires again the use of new intermediate variables. This leads to the recursive formula
The term is easy to evaluate from Equation (6.33), and it gives
Equations (6.38) and (6.40) yield a fixed point update rule for :
The fixed point update rule of the variances can be solved from Equation (6.37):
The update rule for the means is similar to that of the weights in Equation (6.36) but it includes a correction which tries to compensate the simultaneous updates of the sources. The correction is explained in detail in .