Introduction

This paper presents a nonlinear independent component analysis (ICA) algorithm. In much of the ICA research the mapping from the sources s(t) to the observations x(t) has been assumed to be linear, but here we consider a more general model where the observations are assumed to be generated by a nonlinear mapping f from the sources as shown in (1).

 

x(t) = f(s(t)) + n(t) (1)

The observations are assumed to be corrupted by Gaussian noise n(t). The sources are assumed to be generated by an i.i.d. process and the distribution of each of the sources is modelled by a mixture of Gaussians (MOG) and the nonlinear mapping f is modelled by a multi-layer perceptron (MLP) network.

It is well known that the problem of determining the nonlinearity f is indeterminate, that is, there exists an infinite number of different nonlinearities which can produce observations with the same distribution from some independent sources [2]. The Bayesian approach does not suffer from the indeterminacy because a posterior probability can be assigned to all the nonlinear models. Here we use ensemble learning which is a computationally efficient approximation to the full Bayesian treatment [5]. Although each model will have some probability of being responsible for generating the data, typically almost all probability is concentrated on a very small subset of models. It suffices to approximate this high probability region of the posterior probability of the models. In ensemble learning, a parametric approximation is fitted to the posterior probability.

The model structure is presented in Sect. 2. The cost function and details about how to efficiently evaluate it are discussed in Sect. 3. Finally, the learning algorithm is introduced in Sect. 4. This paper is accompanied by [3] which demonstrates the feasibility of the method in simulations with artificial and natural data sets and discusses other existing nonlinear ICA algorithms as well as the limitations and possible extensions of the algorithm described here.