Due to limitations of computational and storage capacity, it is practically always impossible to take into account all the available knowledge about minor details which possibly have some impact on the observation. It is then reasonable to ignore the details and model only their net effect, which manifests itself in minor fluctuations not predictable from the things included in the model. These fluctuations are called noise.
For real valued parameters, the standard choice of noise model is the Gaussian distribution. The central limit theorem states, roughly, that if a very large number of very small independent fluctuations are summed linearly, then the distribution of the resulting total fluctuation will be Gaussian. In reality, neither the requirement of all the fluctuations being small nor the linearity of summation are perfectly met, but Gaussian noise models are used anyway because they are mathematically very convenient.