Conjugate priors

For a given class of likelihood functions $p(\boldsymbol{X}\vert \boldsymbol{\theta})$, the class $\mathcal{P}$ of priors $p(\boldsymbol{\theta})$ is called a conjugate if the posterior $p(\boldsymbol{\theta}\vert \boldsymbol{X})$ is of the same class $\mathcal{P}$.

This is a very useful property if the class $\mathcal{P}$ consists of a set of probability densities with the same functional form. In such a case the posterior distribution will also have the same functional form. For instance the conjugate prior for the mean of a Gaussian distribution is Gaussian. In other words, if the prior of the mean and the functional form of the likelihood are Gaussian, the posterior of the mean will also be Gaussian [16].