The algorithm

The initial step in source separation, using the method described in this article, is whitening, or sphering. This projection of the data is used to achieve the uncorrelation between the solutions found, which is a prerequisite of statistical independence hyvar. The whitening can as well be seen to ease the separation of the independent signals kar+. It may be accomplished by PCA projection: , with $E\{ {\bf vv}^T \} = I$. The whitening matrix ${\bf V}$ is given by

$${\bf V} = {\bf \Lambda}^{-1/2} {\bf \Xi}^T,$$

where ${\bf \Lambda} = \mathrm{diag} [\lambda (1),\ldots, \lambda(M) ]$ is a diagonal matrix with the eigenvalues of the data covariance matrix $E\{ {\bf x} {\bf x}^T \}$, and ${\bf \Xi}$ a matrix with the corresponding eigenvectors as its columns.

Consider a linear combination $y = {\bf w}^T{\bf v}$ of a sphered data vector ${\bf v}$, with $\Vert {\bf w} \Vert = 1$. Then $E \{ y^2 \} = 1$ and $kurt(y) = E\{y^4\} - 3$, whose gradient with respect to ${\bf w}$ is $4E \{ {\bf v}({\bf w}^T{\bf v})^3 \}$.

Based on this, hyvar introduced a simple and efficient fixed-point algorithm for computing ICA, calculated over sphered zero-mean vectors ${\bf v}$, that is able to find one of the rows of the separating matrix ${\bf B}$ (noted ${\bf w}$) and so identify one independent source at a time -- the corresponding independent source can then be found using Eq. . This algorithm, a gradient descent over the kurtosis, is defined for a particular k as

1.

Take a random initial vector ${\bf w}_0$ of unit norm. Let l = 1.

2.

Let ${\bf w}_l = E\{{\bf v}({\bf w}_{l-1}^T{\bf v})^3\} - 3{\bf w}_{l-1}$. The expectation can be estimated using a large sample of ${\bf v}_k$ vectors (say, 1,000 vectors).

3.

Divide ${\bf w}_l$ by its norm (e.g. the Euclidean norm $\Vert{\bf w} \Vert = \sqrt{\sum_i \mathrm{w}_i^2}$ ).

4.

If $\vert{\bf w}_l^T {\bf w}_{l-1} \vert$ is not close enough to 1, let l = l+1 and go back to step 2. Otherwise, output the vector ${\bf w}_l$.

In order to estimate more than one solution, and up to a maximum of M, the algorithm may be run as many times as required. It is, nevertheless, necessary to remove the information contained in the solutions already found, to estimate each time a different independent component. This can be achieved, after the fourth step of the algorithm, by simply subtracting the estimated solution $\hat{s} = {\bf w}^T{\bf v}$ from the unsphered data ${\bf x}_k$. As the solution is defined up to a multiplying constant, the subtracted vector must be multiplied by a vector containing the regression coefficients over each vector component of ${\bf x}_k$.