Cost Function of the Gaussian Variable

A node called the Gaussian variable was presented in Section . The part of the cost function C defined in () that corresponds to the variable is derived here. The posterior approximations of s, m and v are mutually independent

q(s,m, v) = q(s)q(m)q(v) (10.1)

and they are of the form

$$q(s)= \operatorname{N}\left(s;\overline{s},\widetilde{s}\right).$$ (10.2)

The first part C_p defined in () is addressed first.

 

$$- \left< \ln p\left(s \mid m,v\right) \right>$$ C_s,p = (10.3)

$$-\left< \ln \operatorname{N}\left(s;m,\exp(-v)\right) \right>$$ = (10.4)

$$-\left< \ln \left[\left(2\pi \exp(-v)\right)^{-1/2} \exp \frac{-(s-m)^2}{2\exp(-v)}\right] \right>$$ = (10.5)

$$-\left< \ln \left[2\pi \exp(-v)\right]^{-1/2} \right> -\left< \ln \exp \left[-\frac{1}{2}(s-m)^2\exp v\right] \right>$$ = (10.6)

$$\frac{1}{2}\ln 2\pi - \frac{1}{2}\left< v \right> + \frac{1}{2}\left< (s-m)^2 \right>\left< \exp v \right>.$$ = (10.7)

We still need to compute the expectation of (s-m)²

 

$$\left< (s-m)^2 \right> \left< s^2-2sm+m^2 \right>$$ = (10.8)

$$\left< s^2 \right> -2\left< sm \right>+\left< m^2 \right>$$ = (10.9)

$$\overline{s}^2 + \widetilde{s} -2\overline{s}\left< m \right>+\left< m \right>^2+\mathrm{Var}\left\{m\right\}$$ = (10.10)

$$\left(\overline{s}-\left< m \right>\right)^2+\widetilde{s}+\mathrm{Var}\left\{m\right\}.$$ = (10.11)

Substituting $\left< (s-m)^2 \right>$ in () by () yields ():

$$C_{s,p} = \frac{1}{2}\left\{ \left< \exp v \right> \left[\le... ...\widetilde{s} \right] - \left< v \right> + \ln 2\pi\right\} .$$ (10.12)

The second part of the cost function C_q defined in () is

$$\left< \ln q(s) \right>$$ C_s,q = (10.13)

$$\left< \ln \operatorname{N}\left(s;\overline{s},\widetilde{s}\right) \right>$$ = (10.14)

$$\left< \ln \left[(2\pi \widetilde{s})^{-1/2}\exp \frac{-(s-\overline{s})^2}{2\widetilde{s}}\right] \right>$$ = (10.15)

$$-\frac{1}{2} \ln 2\pi \widetilde{s} + \frac{-\left< (s-\overline{s})^2 \right>}{2\widetilde{s}}$$ = (10.16)

$$-\frac{1}{2} \ln 2\pi \widetilde{s} + \frac{-\widetilde{s}}{2\widetilde{s}}$$ = (10.17)

$$-\frac{1}{2} \ln 2\pi e \widetilde{s},$$ = (10.18)

which is the form used in Equation ().