Learning is extremely important for control of complex systems Astrom01. Nonlinear control is difficult even in the case that the system dynamics are known. If the dynamics are not known, the traditional approach is to make a model of the dynamics (system identification) and then try to control the simulated model (nonlinear model predictive control). The model learned from data is of course not perfect, but these imperfections are often ignored. The modern view of control sees feedback as a tool for uncertainty management murr+03, but managing it already in the modelling might have advantages. For instance, the controller can avoid regions where the confidence in model is not high enough Kocijan03.
The idea of learning probabilistic nonlinear state-space models for control is not new. The theory and different phenomena are already well covered in BarShalom81. What has changed, though, is the range of models that can be used in practice, due to developments in Bayesian learning theory and computer performance. The issue remains challenging, i.e. investigation in Bemporad00 tells that observability and controllability properties of these systems cannot be easily deduced.
Recently, Rosenqvist05 presented a method for system identification and control in nonlinear state-space models. The nonlinearities are modelled as piecewise linear (or affine). The system identification is based on the prediction error method. In probability theory, this corresponds to a maximum likelihood estimate assuming a Gaussian process noise. Our method continues this work by applying more sophisticated methods from the machine learning community.
Our method of choice, nonlinear dynamical factor analysis (NDFA) by Valpola02NC is a state-of-the-art tool for finding nonlinear state-space models with variational Bayesian learning. In NDFA, the parameters, the states, and the observations are real-valued vectors that are modelled with parametrised probability distributions. Uncertainties from noisy observations and model imperfections are thus taken explicitly into account. Variational learning has many benefits compared to the maximum likelihood method. It is less prone to overfitting and can be used for selecting the model structure, e.g. the dimensionality of the state space.
An earlier version of our method is described in Raiko05IJCNN from the neural-networks point of view. At the time we could not reasonable study the efficiency of the methods due to problems with the available inference algorithms. In this paper, we use a novel algorithm Raiko06ICA tailored for our purposes.
The rest of the paper is structured as follows: In Section 2, the nonlinear state-space estimator is reviewed and in Section 3 its use as a controller is presented. After experiments in Section 4 matters are discussed and concluded.