Advances in independent component analysis with applications to data mining

Doctoral thesis of Ella Bingham, 2003

Abstract:

This thesis considers the problem of finding latent structure in high 
dimensional data. It is assumed that the observed data are generated by
unknown latent variables and their interactions. The task is to find
these latent variables and the way they interact, given the observed data
only. It is assumed that the latent variables do not depend on each other
but act independently. 

A popular method for solving the above problem is independent
component analysis (ICA). It is a statistical method for expressing a
set of multidimensional observations as a combination of unknown
latent variables that are statistically independent of each other.
Starting from ICA, several methods of estimating the latent structure
in different problem settings are derived and presented in this
thesis. An ICA  algorithm for analyzing complex valued signals is given;
 a way of using ICA in the context of regression is discussed; and
an ICA-type algorithm is used for analyzing the topics in dynamically changing
text data. In addition to ICA-type methods, two algorithms are given for
estimating the latent structure in binary valued data. Experimental results
are given on all of the presented methods. 

Another, partially overlapping problem considered in this thesis is
dimensionality reduction.  Empirical validation is given on a
computationally simple method called random projection: it does not
introduce severe distortions in the data.  It is also proposed that
random projection could be used as a preprocessing method prior to
ICA, and experimental results are shown to support this claim.

This thesis also contains several literature surveys on various aspects
of finding the latent structure in high dimensional data.