For those looking for an introduction to Bayesian probability theory
I can recommend the unfinished manuscript by Edwin Jaynes, Probability Theory : The Logic of
Science, which is available in the net.
Bayesian methods have been applied to neural networks e.g. by MacKay, Neal, Bishop, Barber, Schottky and Frey.