In everyday life people solve extremely complex information processing tasks without even noticing it. The brain handles the vast amount of sensory information with such ease that it is sometimes difficult to recognise the difficulty of the task. When people started making intelligent computer programmes they concentrated on what they thought to be the highest brain function: logical reasoning. It was assumed that sensory information could be quite easily processed to a form, which could be manipulated using logical operations. When people tried to make programmes able to process sensory information -- for example recognise objects from pictures or words from speech -- they soon realised that the very first levels of information processing are far from trivial.
Even the simplest animals can solve many difficult sensory processing problems much better than our programmes. No man-made machine has yet been able to outperform even a simple fly, not to talk about animals with more complex nervous system like fish or frog. It is unthinkable to reach the capacity of the human brain in the near future.
The amount of calculations performed by the fastest contemporary computers does, in fact, compare to that of the nervous system of a fly. Still we are pitifully far from building machines that could work in natural environment as well as the flies. So it is not that much the speed that is lacking but knowledge about how the calculations should be done. It seems reasonable to hope that studying the information processing principles underlying the capacity of biological brain would teach us how to built more intelligent computer programmes, and naturally it would be interesting per se to know more about the brain.
There are many approaches to the brain study, e.g. psychological and anatomical studies, biophysical studies of neurons, measurements of neural activity in behaving animals, information theory, and simulations with artificial neural networks. Ideally all of these lines of study support each other, but they have justification to exist irrespective of the others.
The artificial neural networks can model the brain with very different levels of abstraction. Some models describe the physical and chemical processes of individual neurons, while others model only the abstract information processing principles. In this work, I have adopted the latter approach. I have developed a computationally efficient method for learning so called sparse codes.