Addition

Addition and multiplication nodes can be used e.g. for constructing affine transformations between the variables. Denoting the inputs by s_k, the output is $\sum_k s_{k}$ for an addition node. The mean, variance and expected exponential of the addition node are

   

$$\left< s_{1}+s_{2} \right> \left< s_{1} \right> + \left< s_{2} \right>$$ = (4.24)

$$\mathrm{Var}\left\{s_{1}+s_{2}\right\} \mathrm{Var}\left\{s_{1}\right\} + \mathrm{Var}\left\{s_{2}\right\}$$ = (4.25)

$$\left< \exp(s_{1}+s_{2}) \right> \left< \exp s_{1} \right> \left< \exp s_{2} \right>$$ = (4.26)

assuming that the variables s_k are independent. The derivatives of the cost function propagate to the inputs using the chain rule applied to each of the Equations (), () and (). They are

$$\frac{\partial C}{\partial \left< s_1 \right>} \frac{\partial C}{\partial \left< s_1 + s_2 \right>} \frac{\parti... ...tial \left< s_1 \right>} = \frac{\partial C}{\partial \left< s_1 + s_2 \right>}$$ = (4.27)

$$\frac{\partial C}{\partial \mathrm{Var}\left\{s_1\right\}} \frac{\partial C}{\partial \mathrm{Var}\left\{s_1 + s_2\right\}} ... ...s_1\right\}} = \frac{\partial C}{\partial \mathrm{Var}\left\{s_1 + s_2\right\}}$$ = (4.28)

$$\frac{\partial C}{\partial \left< \exp s_1 \right>} \frac{\partial C}{\partial \left< \exp (s_1+s_2) \right>} \frac{\partial \left< \exp (s_1+s_2) \right>}{\partial \left< \exp (s_1) \right>}$$ = (4.29)

$$\left< \exp s_2 \right>\frac{\partial C}{\partial \left< \exp (s_1+s_2) \right>}.$$ =

The equations for larger sums are obtained by induction, e.g. s₁+s₂+s₃=(s₁+s₂)+s₃.