Updating the posterior distribution

The posterior distribution $q(s)$ of a latent Gaussian node can be updated as follows.

1. The distribution $q(s)$ affects the terms of the cost function ${\cal C}_s$ arising from the variable $s$ itself, namely ${\cal C}_{s,p}$ and ${\cal C}_{s,q}$, as well as the ${\cal C}_p$ terms of the children of $s$, denoted by ${\cal C}_{\operatorname{ch}(s),p}$. The gradients of the cost ${\cal C}_{\operatorname{ch}(s),p}$ with respect to $\left< s \right>$, $\mathrm{Var}\left\{s\right\}$, and $\left< \exp s \right>$ are computed according to Equations (13-16).

2. The terms in ${\cal C}_p$ which depend on $\overline{s}$ and $\widetilde{s}$ can be shown (see Appendix B.2) to be of the form ³

   $${\cal C}_p={\cal C}_{s,p}+{\cal C}_{\operatorname{ch}(s),p}=M\overline{s} + V[(\overline{s}- \overline{s}_ )^2 + \widetilde{s}] + E \left< \exp s \right>,$$ (17)

   
   
   where

   $$M = \frac{\partial {\cal C}_{p} }{\partial \overline{s}} <tex2htm... ...partial {\cal C}_{p} }{ \partial \widetilde{s}} <tex2html_comment_mark>146 \: , E = \frac{\partial {\cal C}_{p} }{ \partial \left< \exp s \right> }. %c$$ (18)

   
   

3. The minimum of ${\cal C}_s$ = ${\cal C}_{s,p} + {\cal C}_{s,q} + {\cal C}_{\operatorname{ch}(s),p}$ is solved. This can be done analytically if $E = 0$, corresponding to the case of so-called free-form solution (see Lappal-Miskin00 for details):

   $$\overline{s}_ = \overline{s}_ - \frac{M}{2V}\: , \hspace{7mm} \widetilde{s}_ = \frac{1}{2V}.$$ (19)

   
   
   Otherwise the minimum is obtained iteratively. Iterative minimisation can be carried out efficiently using Newton's method for the posterior mean $\overline{s}$ and a fixed-point iteration for the posterior variance $\widetilde{s}$. The minimisation procedure is discussed in more detail in Appendix A.