Bayes' rule

From the product rule it is possible to derive Bayes' rule:

$$P(A \vert B C) = P(A \vert C) P(B \vert A C) / P(B \vert C) \, .$$ (1)

If we assume that A is one of several explanations for the new observation, B is the new observation and C summarises all prior assumptions and experience, we notice that Bayes' rule tells how the learning system should update its beliefs as it receives a new observation.

Before making the observation B, the learning system knows only C, but afterwards it knows BC, that is, it knows ``B and C''. Bayes' rule then tells how the learning system should adapt P(A | C)into P(A | BC) in response to the observation. In order for Bayes' rule to be useful, the explanation A needs to be such that together with the prior assumptions and experience C it fixes the probability P(B | AC).

Usually P(A | C) is called the prior probability and P(A | BC) the posterior probability. It should be noted, however, that this distinction is relative to the observation; the posterior probability for one observation is the prior probability for the next observation.