Linear independent factor analysis

In many instances there exist several nodes which have quite similar role in the chosen structure. Assuming that $i^$th such node corresponds to a scalar variable $y_i$, it is convenient to use the vector ${\bf y}$ = $(y_1,y_2,\ldots,y_n)^T$ to jointly denote all the corresponding scalar variables $y_1,y_2,\ldots,y_n$. This notation is used in Figures 5 and 6 later on. Hence we represent the scalar source nodes corresponding to the variables $s_i(t)$ using the source vector ${\mathbf{s}}(t)$, and the scalar nodes corresponding to the observations $x_i(t)$ using the observation vector ${\mathbf{x}}(t)$.

The addition and multiplication nodes can be used for building an affine transformation

$${\bf x}(t) = {\bf As}(t) + {\mathbf{a}} + {\bf n}_x(t)$$ (32)

from the Gaussian source nodes ${\mathbf{s}}(t)$ to the Gaussian observation nodes ${\mathbf{x}}(t)$. The vector ${\mathbf{a}}$ denotes the bias and vector ${\bf n}_x(t)$ denotes the zero-mean Gaussian noise in the Gaussian node ${\mathbf{x}}(t)$. This model corresponds to standard linear factor analysis (FA) assuming that the sources $s_i(t)$ are mutually uncorrelated; see for example ICABook01.

If instead of Gaussianity it is assumed that each source $s_i(t)$ has some non-Gaussian prior, the model (32) describes linear independent factor analysis (IFA). Linear IFA was introduced by Attias99, who used variational Bayesian learning for estimating the model except for some parts which he estimated using the expectation-maximisation (EM) algorithm. Attias used a mixture-of-Gaussians source model, but another option is to use the variance source to achieve a super-Gaussian source model. Figure 5 depicts the model structures for linear factor analysis and independent factor analysis.

Figure 5: Model structures for linear factor analysis (FA) (left) and independent factor analysis (IFA) (right).