next up previous
Next: Feedforward equations Up: Using an MDL-based cost Previous: Measuring the description length

Computing the expectation of the description length for neural networks

In this section we shall describe a method for computing the expectation of the description length for neural networks in an efficient feedforward process. We shall first write down the feedforward equations for the outputs of the neurons and the description length, and then take expectations of the equations. This results in converted feedforward equations, which automatically yield the expected description length. It is then easy to use backpropagation to adapt the parameters. For the sake of simplicity, we shall only consider supervised learning with multilayer perceptrons (MLP), although the method is general and can be applied to practically any type of neural network, both supervised and unsupervised learning.



 

Harri Lappalainen
5/19/1998