Contents

• Contents
• Introduction
  • Aim of the Thesis
  • Problem Settings
  • Structure and Contributions of the Thesis
• Extensions of Factor Analysis
  • Linear Models
    • Factor Analysis
    • Principal Component Analysis
    • Independent Component Analysis
  • Nonlinear Models
    • Mixtures of Linear Models
    • Nonlinear Component Analysis
    • Self-Organising Map
    • Supervised Learning Tasks
    • Nonlinear Factor Analysis
    • Nonlinear Dynamical Factor Analysis
    • Hierarchical Nonlinear Factor Analysis
  • Sparse Coding
  • Learning Criteria
• Bayesian Inference and Ensemble Learning
  • Bayesian Probability Theory
    • Bayes Rule
    • Marginalisation Principle
    • Approximations
  • Ensemble Learning
    • Factorial Approximation
    • Hierarchical Models
    • Connection to coding
    • Model Selection
  • Generalisation
    • An Example on Polynomial Fitting
    • Example Where Point Estimates Fail
    • Role of Prior Information
• Building Blocks for Hierarchical Nonlinear Factor Analysis
  • Gaussian Variables
    • Update Rule
    • First Example
  • Addition
  • Multiplication
  • Gaussian Variable with Nonlinearity
    • Nonlinearities
    • Update Rule
  • Form of the Cost Function
• Hierarchical Nonlinear Factor Analysis with Variance Modelling
  • Variance Neurons
  • Formulation of the Model Structure
    • Main Structure
    • Hierarchical Prior for the Weights
    • Hierarchical Prior for the Sources
    • Comparison of the Notation
  • Simple Example
• Learning Algorithm
  • Initialisation
  • Adjustment
    • Linear Computational Complexity
  • Learning the Structure
    • Pruning
    • Regeneration
  • Avoiding Nonglobal Minima
    • Rebooting
    • Special States
• Bars Problem
  • Learning Procedure
  • Results
• Experiments with Image Data
  • Learning Procedure
  • Results
• Discussion
• Bibliography
• Cost Function of the Gaussian Variable
• Update Rule of the Nonlinear Node