Next: The likelihood term Up: Evaluating the cost function Previous: Evaluating the cost function   Contents

### The terms originating from the parameters

Assuming the parameters are and the approximation is of the form

 (6.2)

the terms of Equation (6.1) originating from the parameters can be written as

 (6.3)

In the case of Dirichlet distributions one in the previous equation must of course consist of a vector of parameters for the single distribution.

There are two different kinds of parameters in , those with Gaussian distribution and those with a Dirichlet distribution. In the Gaussian case the expectation over gives the negative entropy of a Gaussian, , as derived in Equation (A.5) of Appendix A.

The expectation of can also be evaluated using the formulas of Appendix A. Assuming

 (6.4)

where and , the expectation becomes

 (6.5)

where we have used the results of Equations (A.4) and (A.6).

For Dirichlet distributed parameters, the procedure is similar. Let us assume that the parameter , and . Using the notation of Appendix A, the negative entropy of the Dirichlet distribution , , can be evaluated as in Equation (A.14) to yield

 (6.6)

The special function required in these terms is , where is the gamma function. The psi function is also known as the digamma function and it can be efficiently evaluated numerically for example using techniques described in [4]. The term is a normalising constant of the Dirichlet distribution as defined in Appendix A.

The expectation of can be evaluated similarly

 (6.7)

Next: The likelihood term Up: Evaluating the cost function Previous: Evaluating the cost function   Contents
Antti Honkela 2001-05-30