Harri Valpola, Tapani Raiko and Juha Karhunen¹

Building Blocks for Hierarchical Latent Variable Models

Date: October 1, 2001

Helsinki University of Technology, Neural Networks Research Centre
P.O.Box 5400, FIN-02015 HUT, Espoo, Finland
E-mail: {Harri.Valpola, Tapani.Raiko, Juha.Karhunen}@hut.fi
URL: http://www.cis.hut.fi/projects/ica/bayes/

Abstract:

We introduce building blocks from which a large variety of latent variable models can be built. The blocks include continuous and discrete variables, summation, addition, nonlinearity and switching. Ensemble learning provides a cost function which can be used for updating the variables as well as optimising the model structure. The blocks are designed to fit together and to yield efficient update rules. Emphasis is on local computation which results in linear computational complexity. We propose and test a structure with a hierachical nonlinear model for variances and means.