For an introduction to Bayesian methods, you can look at the slides for my NIPS*2004 tutorial: Postscript or PDF.
Here are some papers that touch on fundamental issues in Bayesian inference:
Wang, C. and Neal, R. M. (2012) ``Gaussian process regression with heteroscedastic or non-Gaussian residuals'', Technical Report, 19 pages: abstract, pdf.Li, L. and Neal, R. M. (2008) ``Compressing parameters in Bayesian high-order models with application to logistic sequence models'', Bayesian Analysis, vol. 3, pp. 793-822: abstract, pdf.
Li, L., Zhang, J., and Neal, R. M. (2008) ``A method for avoiding bias from feature selection with application to naive Bayes classification models'', Bayesian Analysis, vol. 3, pp. 171-196: abstract, pdf.
Shahbaba, B. and Neal, R. M. (2005) ``Improving classification when a class hierarchy is available using a hierarchy-based prior'', Technical Report No. 0510, Dept. of Statistics, 11 pages: abstract, postscript, pdf.
Neal, R. M. (2001) ``Transferring prior information between models using imaginary data'', Technical Report No. 0108, Dept. of Statistics, University of Toronto, 29 pages: abstract, postscript, pdf, associated software.
Neal, R. M. (2001) ``Defining priors for distributions using Dirichlet diffusion trees'', Technical Report No. 0104, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf, associated software.
Neal, R. M. (1997) ``Monte Carlo implementation of Gaussian process models for Bayesian regression and classification'', Technical Report No. 9702, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated reference, associated software.
Neal, R. M. (1996) Bayesian Learning for Neural Networks, Lecture Notes in Statistics No. 118, New York: Springer-Verlag: blurb, associated references, associated software.
Neal, R. M. (1994) ``Priors for infinite networks'', Technical Report CRG-TR-94-1, Dept. of Computer Science, University of Toronto, 22 pages: abstract, postscript, pdf, associated reference.
Neal, R. M. (1993) Probabilistic inference using Markov chain Monte Carlo methods, Technical Report CRG-TR-93-1, Dept. of Computer Science, University of Toronto, 144 pages: abstract, contents, postscript, pdf.
Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.