EM Algorithm

The EM algorithm for solving PCA [7] iterates updating $\mathbf{A}$ and $\mathbf{S}$ alternately. When either of these matrices is fixed, the other one can be obtained from an ordinary least-squares problem. The algorithm alternates between the updates

$$\mathbf{S}\leftarrow (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\ma... ...quad \mathbf{A}\leftarrow \mathbf{X}\mathbf{S}^T(\mathbf{S}\mathbf{S}^T)^{-1}.$$ (5)

This iteration is especially efficient when only a few principal components are needed, that is $c\ll d$ [7].