Using MLP networks as generative models was proven feasible in simulations with artificial data. The network was able to retrieve the original inputs which had generated the data. The difficulty of the problem is apparent from the results obtained with the linear ICA-model. Bayesian learning was used for solving the indeterminacy of the unknown mapping.
The Bayesian approach is particularly valuable for unsupervised learning due to its robustness against overlearning and the ability to compare models. Other techniques, such as cross-validation, are available for supervised learning but they are not applicable for unsupervised learning.
The Bayesian approach was implemented using ensemble learning, which is an efficient method for approximating the posterior distributions. Its main advantage over the traditional Laplace's method is that the search for good models is focused on those areas of the model space which occupy large probability mass, as opposed to searching for large probability densities.