next up previous
Next: Diagonal Gaussian ensemble Up: Ensemble Learning for Independent Previous: Model selection

Model for the measurements

  The measurements vectors $\{x(t)\}$ are assumed to be generated by a linear mapping A from mutually independent source signals $\{s(t)\}$and additive Gaussian noise $\{v(t)\}$.

x(t) = As(t) + v(t)

The components vi(t) of the noise are assumed to have means bi and variances $e^{2\sigma_i}$. Another way to put this is to say that x(t) has Gaussian distribution with mean As(t)+b and diagonal covariance with components $e^{2\sigma_i}$. Each component Aij of the linear mapping is assumed to have zero mean and unit variance.

The distribution of each source signal is a mixture of Gaussians (MOG).

\begin{displaymath}
p(s_i(t) \vert c_i, S_i, \gamma_i) = \frac{\sum_j e^{c_{ij}}{\cal G}(s_i(t); S_{ij},
 e^{2\gamma_{ij}})} {\sum_j e^{c_{ij}}}\end{displaymath}

The parameters cij are the logarithms of mixture coefficients, S the means and $\gamma$ the logarithms of the standard deviations of the Gaussians[*] (here ${\cal G}(a; b,
c)$ denotes a Gaussian distribution over a with mean b and variance c).

The distributions of parameters cij, Sij, $\gamma_{ij}$,bi and $\sigma_i$ are ${\cal G}(c_{ij}; 0, e^{2\alpha})$,${\cal G}(S_{ij}; 0, e^{2\epsilon})$, ${\cal G}(\gamma_{ij}; \Gamma, e^{2\delta})$,${\cal G}(b_i; B, e^{2\epsilon})$ and ${\cal G}(\sigma_i; \Sigma, e^{2\eta}
)$.

The prior distribution of the hyperparameters $\alpha$, $\epsilon$,$\Gamma$, $\delta$, B, $\beta$, $\Sigma$ and $\eta$ is assumed to be uniform in the area of reasonable values for the hyperparameters.

To summarise: the eight hyperparameters are assigned flat prior pdfs. The distributions of other parameters are defined hierarchically from these using Gaussian distributions each parametrised by the mean and the logarithm of the standard deviation. The joint pdf of $\{x(t),
s(t), A, b, \sigma, c, S, \gamma, \alpha, \epsilon, \Gamma, \delta, B,
\beta, \Sigma, \eta\}$ is simply the product of the independent pdfs.


next up previous
Next: Diagonal Gaussian ensemble Up: Ensemble Learning for Independent Previous: Model selection
Harri Lappalainen
7/10/1998