Factor analysis is widely used in the social sciences and there is an extensive literature describing various methods for estimating the factors and then rotating them in order to yield solutions which have simple interpretations [122,110,35,25,5]. Principal component analysis (PCA) is often used as the first stage for finding the factors [63,58]. We shall call PCA and related methods signal transformation (ST) approaches. These methods define a criterion on the factors and then find a transformation from observations to factors which optimises the criterion. In PCA, the criterion is that the resulting factors are uncorrelated and have maximal variance. A further requirement of orthogonal A is required to make the problem well determined.
In many cases ST approaches yield a fair approximation to the result which would be gained by the Bayesian approach to estimating the factors. Almost always, however, the ST methods provide only point estimates. From the point of view of this thesis, reference [9] is relevant because it uses ensemble learning for estimating the posterior probabilities of the factors and the mapping A.
There are several criteria for the rotation. Varimax, one of the most popular rotation criteria, aims at maximising the sparsity of the matrix A. In general, the goal of the rotation is to find a meaningful interpretation of the resulting factors.