next up previous
Next: Fixed-point iteration for the Up: Updating for the Gaussian Previous: Updating for the Gaussian

Newton iteration for the mean $ m$

The Newton iteration for $ m$ is obtained by

$\displaystyle m_{i+1}$ $\displaystyle = m_i - \frac{\partial {\cal C}(m_i,v_i) / \partial m_i} {\partial^2 {\cal C}(m_i,v_i) / \partial m_i^2}$    
  $\displaystyle = m_i - \frac{M + 2 V (m_i - m_0) + E \exp (m + v/2)} {2V + E \exp(m+v/2)}.$ (55)

The Newton iteration converges in one step if the second derivative remains constant. The step is too short if the second derivative decreases and too long if the second derivative increases. For stability, it is better to take too short than too long steps.

In this case, the second derivative always decreases if the mean $ m$ decreases and vice versa. For stability it is therefore useful to restrict the growth of $ m$ because it is consistently over-estimated.


Tapani Raiko 2006-08-28