next up previous contents
Next: Approximations Up: Bayesian probability theory Previous: Structure among unknown variables   Contents


Decision theory

Decision theory (see book by Dean and Wellman, 1991) was first formulated by Blaise Pascal in the 17th century. When faced with a number of actions $ A$, each with possibly more than one possible outcome $ B$ with different probabilities $ P(B\mid A)$, the rational choice is the action that gives the highest expected value $ U(A)$:

$\displaystyle U(A) = \int_B U(A,B)P(B\mid A),$ (2.4)

where $ U(A,B)$ is the value of action $ A$ producing the outcome $ B$. Daniel Bernoulli refined the idea in the 18th century. In his solution, he defines a utility function and computes expected utility rather than expected financial value. For example, people tend to insure their property even though the expected financial value of the decision is negative (after all, insurance business must be profitable for the insurance companies). This is explained by the fact that in case of losing one's whole property, every euro is more important (has more utility).

Bayesian decision theory (see book by Bernardo and Smith, 2000) works in situations where the outcomes of actions are unknown but one still has to make decisions. One simply chooses the action with highest expected utility over the predictive distribution of outcomes. Bayesian decision theory does not just give a way to make actions but it is actually the only coherent way of acting. This can be shown using a so-called Dutch book argument (Resnik, 1987). A Dutch book is a set of odds and bets which guarantees a profit, no matter what the outcome of the gamble. A Dutch book can never be made against a Bayesian decision maker, and if decisions are such that a Dutch book cannot be made against the decision maker, the decisions can always be interpreted to be made by a Bayesian decision maker.

Decision theory has a clear connection to control theory. (Dean and Wellman, 1991) The control that minimises a certain cost functional is called the optimal control (see book by Kirk, 2004). When the control cost is used as the negative utility $ -U(\cdot)$, decision theory provides optimal control, even under uncertainty (stochastic control). Section 4.3.2 and Publication IV apply Bayesian decision theory for control in nonlinear state-space models.


next up previous contents
Next: Approximations Up: Bayesian probability theory Previous: Structure among unknown variables   Contents
Tapani Raiko 2006-11-21