Radford Neal's Publications

Book
Refereed research papers
Refereed research notes
Refereed tutorial/review papers
Refereed conference papers
Unrefereed conference papers
Book chapters
Ph.D. and M.Sc. theses
Technical reports
Unpublished manuscripts

Refereed research papers

Shestopaloff, A. Y. and Neal, R. M. (2017) ``Sampling latent states for high-dimensional non-linear state space models with the embedded HMM method'', Bayesian Analysis, 26 pages: abstract, journal page.

Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. (2014) ``Split Hamiltonian Monte Carlo'', Statistics and Computing, vol. 24, pp. 339-349.

Tarasov, L., Dykeb, A. S., Neal, R. M., and Peltier, W. R. (2011) ``A data-calibrated distribution of deglacial chronologies for the North American ice complex from glaciological modeling'', Earth and Planetary Science Letters, available online, 5 October 2011, doi:10.1016/j.epsl.2011.09.010.

Shahbaba, B. and Neal, R. M. (2009) ``Nonlinear Models Using Dirichlet Process Mixtures'', Journal of Machine Learning Research, vol. 10, pp. 1829-1850: abstract, pdf, associated references.

Li, L. and Neal, R. M. (2008) ``Compressing parameters in Bayesian high-order models with application to logistic sequence models'', Bayesian Analysis, vol. 3, pp. 793-822: abstract, pdf.

Li, L., Zhang, J., and Neal, R. M. (2008) ``A method for avoiding bias from feature selection with application to naive Bayes classification models'', Bayesian Analysis, vol. 3, pp. 171-196: abstract, pdf.

Sun, S., Greenwood, C. M. T., and Neal, R. M. (2007) ``Haplotype inference using a Bayesian hidden Markov model'', Genetic Epidemiology, vol. 31, pp. 937-948: abstract.

Jain, S. and Neal, R. M. (2007) ``Splitting and merging components of a nonconjugate Dirichlet process mixture model'' (with discussion), Bayesian Analysis, vol. 2, pp 445-472, posted online 2007-01-23: pdf (discussion is in other URLs).

Listgarten, J., Neal, R. M., Roweis, S. T., Wong, P., and Emili, A. (2007) ``Difference detection in LC-MS data for protein biomarker discovery'', Bioinformatics, vol. 23, pp. e198-e204.

Shahbaba, B. and Neal, R. M. (2007) ``Improving classification when a class hierarchy is available using a hierarchy-based prior, Bayesian Analysis, vol. 2, pp. 221-238: pdf.

Shahbaba, B. and Neal, R. M. (2006) ``Gene function classification using Bayesian models with hierarchy-based priors'', BMC Bioinformatics, 7:448, 9 pages: abstract, pdf, html, associated references.

Jain, S. and Neal, R. M. (2004) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Journal of Computational and Graphical Statistics, vol. 13, pp. 158-182: abstract, associated references.

Neal, R. M. (2003) ``Slice sampling'' (with discussion), Annals of Statistics, vol. 31, pp. 705-767: abstract, text from online journal site, associated references, associated software, more associated software.

Neal, R. M. (2001) ``Annealed importance sampling'', Statistics and Computing, vol. 11, pp. 125-139: abstract, associated references.

Diaconis, P., Holmes, S., and Neal, R. M. (2000) ``Analysis of a nonreversible Markov chain sampler'', Annals of Applied Probability, vol. 10, pp. 726-752: abstract, associated reference.

Neal, R. M. (2000) ``Markov chain sampling methods for Dirichlet process mixture models'', Journal of Computational and Graphical Statistics, vol. 9, pp. 249-265: abstract, associated references, associated software.

Moffat, A., Neal, R. M., and Witten, I. H. (1998) ``Arithmetic coding revisited'', ACM Transactions on Information Systems, vol. 16, pp. 256-294: abstract, associated reference, associated software.

Neal, R. M. and Dayan, P. (1997) ``Factor analysis using delta-rule wake-sleep learning'', Neural Computation, vol. 9, pp. 1781-1803: abstract, associated references, associated software.

MacKay, D. J. C. and Neal, R. M. (1996) ``Near Shannon limit performance of low density parity check codes'', Electronics Letters, vol. 32, pp. 1645-1646. Reprinted with printing errors corrected in vol. 33, pp. 457-458: abstract.

Neal, R. M. (1996) ``Sampling from multimodal distributions using tempered transitions'', Statistics and Computing, vol. 6, pp. 353-366: abstract, associated references.

Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995) ``The ``wake-sleep'' algorithm for unsupervised neural networks'', Science, vol. 268, pp. 1158-1161: abstract, associated references.

Dayan, P., Hinton, G. E., Neal, R. M., and Zemel, R. S. (1995) ``The Helmholtz machine'', Neural Computation, vol. 7, pp. 1022-1037: abstract, associated reference.

Neal, R. M. (1994) ``An improved acceptance procedure for the hybrid Monte Carlo algorithm'', Journal of Computational Physics, vol. 111, pp. 194-203: abstract.

Neal, R. M. (1992) ``Connectionist learning of belief networks'', Artificial Intelligence, vol. 56, pp. 71-113: abstract, associated reference.

Witten, I. H., Birtwistle, G. M., Cleary, J. G., Hill, D. R., Levinson, D., Lomow, G. A., Neal, R., Peterson, M., Unger, B. W., and Wyvill, B. L. M. (1983) ``JADE: A distributed software prototyping environment'', ACM Operating Systems Review, vol. 7, no. 3.

Witten, I. H. and Neal, R. M. (1982) ``Using Peano curves for bilevel display of continuous-tone images'', IEEE Computer Graphics and Applications, vol. 2, no. 3.

Refereed research notes

Gilks, W. R., Neal, R. M., Best, N. G., and Tan, K. K. C. (1997) ``Corrigendum: Adaptive rejection Metropolis sampling'', Applied Statistics, vol. 46, pp. 541-542. [Note: Contrary to the impression given by the journal's failure to cite the original reference, I was not an author of the paper containing the error which this note corrects.]

Neal, R. M. (1993) ``Comments on `A theoretical analysis of Monte Carlo algorithms for the simulation of Gibbs random field images''', IEEE Transactions on Information Theory, vol. 39, p. 310.

Neal, R. M. (1992) ``Asymmetric parallel Boltzmann machines are belief networks'', Neural Computation, vol. 4, pp. 832-834.

Refereed tutorial/review papers

Witten, I. H., Neal, R. M., and Cleary, J. G. (1987) ``Arithmetic coding for data compression'', Communications of the ACM, vol. 30, pp. 520-540: abstract, associated software.

Refereed conference papers

Listgarten, J., Neal, R. M., Roweis, S. T., Puckrin, R., and Cutler, S. (2007) ``Bayesian detection of infrequent differences in sets of time series with shared structure'', in B. Scholkopf, et al (editors), Advances in Neural Information Processing Systems 19 (aka NIPS*2006), 8 pages.

Meeds, E., Ghahramani, Z., Neal, R., and Roweis, S. (2007) ``Modeling dyadic data with binary latent features'', in B. Scholkopf, et al (editors), Advances in Neural Information Processing Systems 19 (aka NIPS*2006), 8 pages.

Listgarten, J., Neal, R. M., Roweis, S. T., and Emili, A. (2005) ``Multiple alignment of continuous time series'', in L. K. Saul, et al (editors), Advances in Neural Information Processing Systems 17 (aka NIPS*2004), MIT Press, 8 pages: abstract, postscript, pdf.

Neal, R. M., Beal, M. J., and Roweis, S. T. (2004) ``Inferring state sequences for non-linear systems with embedded hidden Markov models'', in S. Thrun, et al (editors), Advances in Neural Information Processing Systems 16 (aka NIPS*2003), MIT Press, 8 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (2003) ``Density modeling and clustering using Dirichlet diffusion trees'', in J. M. Bernardo, et al. (editors) Bayesian Statistics 7, pp. 619-629: abstract, postscript, pdf, associated references, associated software.

Harvey, M. and Neal, R. M. (2000) ``Inference for Belief Networks Using Coupling From the Past'', in C. Boutilier and M. Goldszmidt (editors), Uncertainty in Artificial Intelligence: Proceedings of the Sixteenth Conference (2000), pp. 256-263: abstract, postscript, pdf, associated reference.

Neal, R. M. (1998) ``Regression and classification using Gaussian process priors'' (with discussion), in J. M. Bernardo, et al (editors) Bayesian Statistics 6, Oxford University Press, pp. 475-501: abstract, postscript (without discussion), pdf (without discussion), associated reference, associated software.

MacKay, D. J. C. and Neal, R. M. (1995) ``Good codes based on very sparse matrices'', in C. Boyd (editor) Cryptography and Coding: 5th IAM Conference, Lecture Notes in Computer Science No. 1025, pp. 100-111. Springer-Verlag: abstract.

Moffat, A., Neal, R., and Witten, I. H. (1995) ``Arithmetic coding revisited'', in J. A. Storer and M. Cohn (editors) Proceedings of the Fifth IEEE Data Compression Conference, pp. 202-211, Los Alamitos, California: IEEE Computer Society Press.

Neal, R. M. (1993) ``Bayesian learning via stochastic dynamics'', in C. L. Giles, S. J. Hanson, and J. D. Cowan (editors) Advances in Neural Information Processing Systems 5 (aka NIPS*1992), pp. 475-482, San Mateo, California: Morgan Kaufmann: abstract, postscript, pdf, associated references.

Unrefereed conference papers

Hinton, G. E., Dayan, P., To, A., and Neal, R. (1995) ``The Helmholtz machine through time'', in F. Fogelman-Soulie and R. Gallinari (editors) ICANN-95, pp. 483-490: abstract, associated reference.

Neal, R. M. (1992) ``Bayesian mixture modeling'', in C. R. Smith, G. J. Erickson, and P. O. Neudorfer (editors) Maximum Entropy and Bayesian Methods: Proceedings of the 11th International Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis, Seattle, 1991, pp. 197-211, Dordrecht: Kluwer Academic Publishers: abstract, associated references.

Neal, R. M., Lomow, G. A., Peterson, M., Unger, B. W., and Witten, I. H. (1984) ``Experience with an inter-process communication protocol in a distributed programming environment'', CIPS Session 84 Conference, Calgary, Alberta, May 1984.

Wyvill, B. L. M., Neal, R., Levinson, D., and Bramwell, B. (1984) ``JAGGIES: A distributed hierarchical graphics system'', CIPS Session 84 Conference, Calgary, Alberta, May 1984.

Book chapters

Neal, R. M. and Zhang, J. (2006) ``High dimensional classification with Bayesian neural networks and Dirichlet diffusion trees'', in I. Guyon, S. Gunn, M. Nikravesh, and L. A. Zadeh (editors) Feature Extraction: Foundations and Applications, Studies in Fuzziness and Soft Computing, Volume 207, Springer, pp. 265-295.

Neal, R. M. (2006) ``Classification with Bayesian neural networks'', in J. Quinonero-Candela, B. Magnini, I. Dagan, and F. D'Alche-Buc (editors) Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment, Lecture Notes in Computer Science no. 3944, Springer-Verlag, pp. 28-32.

Neal, R. M. (1998) ``Assessing relevance determination methods using DELVE'', in C. M. Bishop (editor), Neural Networks and Machine Learning, pp. 97-129, Springer-Verlag: abstract, associated references, postscript, pdf.

Neal, R. M. and Hinton, G. E. (1998) ``A view of the EM algorithm that justifies incremental, sparse, and other variants'', in M. I. Jordan (editor) Learning in Graphical Models, pp. 355-368, Dordrecht: Kluwer Academic Publishers: abstract, postscript, pdf.

Neal, R. M. (1998) ``Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation'', in M. I. Jordan (editor) Learning in Graphical Models, pp. 205-225, Dordrecht: Kluwer Academic Publishers: abstract, associated reference, postscript, pdf.

Ph.D. and M.Sc. Theses

Neal, R. M. (1980) ``An Editor for Trees'', MSc thesis, University of Calgary, 100 pages: abstract, pdf.

Technical reports

Neal, R. M. and Rosenthal, J. S. (2023) ``Efficiency of reversible MCMC methods: Elementary derivations and applications to composite methods'', Technical Report, 24 pages: abstract, pdf.

Neal, R. M. (2020) ``Non-reversibly updating a uniform [0,1] value for Metropolis accept/reject decisions'', Technical Report, 14 pages: abstract, pdf.

Shestopaloff, A. Y. and Neal, R. M. (2016) ``Sampling latent states for high-dimensional non-linear state space models with the embedded HMM method'', Technical Report, 21 pages: abstract, pdf.

Neal, R. M. (2015) ``Fast exact summation using small and large superaccumulators'', Technical Report, 22 pages: abstract, pdf, associated software.

Neal, R. M. (2015) ``Representing numeric data in 32 bits while preserving 64-bit precision'', Technical Report, 16 pages: abstract, pdf, associated software.

Shestopaloff, A. Y. and Neal, R. M. (2014) ``Efficient Bayesian inference for stochastic volatility models with ensemble MCMC methods'', Technical Report, 17 pages: abstract, pdf.

Shestopaloff, A. Y. and Neal, R. M. (2013) ``On Bayesian inference for the M/G/1 queue with efficient MCMC sampling'', Technical Report, 18 pages: abstract, pdf.

Wang, C. and Neal, R. M. (2013) ``MCMC methods for Gaussian process models using fast approximations for the likelihood'', Technical Report, 21 pages: abstract, pdf.

Shestopaloff, A. Y. and Neal, R. M. (2013) ``MCMC for non-linear state space models using ensembles of latent sequences, Technical Report, 18 pages: abstract, pdf.

Wang, C. and Neal, R. M. (2012) ``Gaussian process regression with heteroscedastic or non-Gaussian residuals'', Technical Report, 19 pages: abstract, pdf.

Neal, R. M. (2012) ``How to View an MCMC Simulation as a Permutation, with Applications to Parallel Simulation and Improved Importance Sampling'', Technical Report No. 1201, Dept. of Statistics, University of Toronto, 42 pages: abstract, postscript, pdf.

Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. (2011) ``Split Hamiltonian Monte Carlo'', arXiv:1106.5941v1.

Neal, R. M. (2010) ``MCMC using ensembles of states for problems with fast and slow variables such as Gaussian process regression'', Technical Report No. 1011, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated software.

Thompson, M. and Neal, R. M. (2010) ``Covariance-adaptive slice sampling'', Technical Report No. 1002, Dept. of Statistics, University of Toronto, 17 pages: abstract, postscript, pdf.

Shahbaba, B. and Neal, R. M. (2007) ``Nonlinear Models Using Dirichlet Process Mixtures'', Technical Report No. 0707, Dept. of Statistics, University of Toronto, 16 pages: abstract, postscript, pdf, associated references.

Li, L., Zhang, J., and Neal, R. M. (2007) ``A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models'', Technical Report No. 0705, Dept. of Statistics, University of Toronto, 21 pages: abstract, postscript, pdf.

Neal, R. M. (2006) ``Puzzles of anthropic reasoning resolved using full non-indexical conditioning'', Technical Report No. 0607, Dept. of Statistics, University of Toronto, 53 pages: abstract, postscript, pdf.

Shahbaba, B. and Neal, R. M. (2006) ``Gene function classification using Bayesian models with hierarchy-based priors'', Technical Report No. 0606, Dept. of Statistics, University of Toronto, 14 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (2005) ``Estimating ratios of normalizing constants using Linked Importance Sampling'', Technical Report No. 0511, Dept. of Statistics, University of Toronto, 37 pages: abstract, postscript, pdf, associated software.

Shahbaba, B. and Neal, R. M. (2005) ``Improving classification when a class hierarchy is available using a hierarchy-based prior'', Technical Report No. 0510, Dept. of Statistics, 11 pages: abstract, postscript, pdf.

Jain, S. and Neal, R. M. (2005) ``Splitting and merging components of a nonconjugate Dirichlet process mixture model'', Technical Report No. 0507, Dept. of Statistics, 37 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (2005) ``The short-cut Metropolis method'', Technical Report No. 0506, Dept. of Statistics, University of Toronto, 28 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2004) ``Taking bigger Metropolis steps by dragging fast variables'', Technical Report No. 0411, Dept. of Statistics, University of Toronto, 9 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2004) ``Improving asymptotic variance of MCMC estimators: Non-reversible chains are better'', Technical Report No. 0406, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf.

Neal, R. M. (2003) ``Markov chain sampling for non-linear state space models using embedded hidden Markov models'', Technical Report No. 0304, Dept. of Statistics, University of Toronto, 9 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (2002) ``Circularly-coupled Markov chain sampling'', Technical Report No. 9910 (revised), Dept. of Statistics, University of Toronto, 49 pages: abstract, postscript, pdf.

Neal, R. M. (2001) ``Transferring prior information between models using imaginary data'', Technical Report No. 0108, Dept. of Statistics, University of Toronto, 29 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2001) ``Defining priors for distributions using Dirichlet diffusion trees'', Technical Report No. 0104, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf, associated references, associated software.

Pinto, R. L. and Neal, R. M. (2001) ``Improving Markov chain Monte Carlo estimators by coupling to an approximating chain'', Technical Report No. 0101, Dept. of Statistics, University of Toronto, 13 pages: abstract, postscript, pdf.

Neal, R. M. (2000) ``Slice sampling'', Technical Report No. 2005, Dept. of Statistics, University of Toronto, 40 pages: abstract, postscript, pdf, associated references, associated software, more associated software.

Jain, S. and Neal, R. M. (2000) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Technical Report No. 2003, Dept. of Statistics, 32 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1998) ``Markov chain sampling methods for Dirichlet process mixture models'', Technical Report No. 9815, Dept. of Statistics, University of toronto, 17 pages: abstract, postscript, pdf, associated references, associated software.

Neal, R. M. (1998) ``Annealed importance sampling'', Technical Report No. 9805 (revised), Dept. of Statistics, University of Toronto, 25 pages: abstract, associated references, postscript, pdf.

Neal, R. M. (1997) ``Markov chain Monte Carlo methods based on `slicing' the density function'', Technical Report No. 9722, Dept. of Statistics, University of Toronto, 27 pages: abstract, postscript, pdf, associated references.

Diaconis, P., Holmes, S., and Neal, R. M. (1997) ``Analysis of a non-reversible Markov chain sampler'', Technical Report BU-1385-M, Biometrics Unit, Cornell University, 26 pages: abstract, associated reference, postscript, pdf.

Neal, R. M. (1997) ``Monte Carlo implementation of Gaussian process models for Bayesian regression and classification'', Technical Report No. 9702, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated reference, associated software.

Neal, R. M. and Dayan, P. (1996) ``Factor analysis using delta-rule wake-sleep learning'', Technical Report No. 9607, Dept. of Statistics, University of Toronto, 23 pages: abstract, postscript, pdf, associated references, associated software.

Neal, R. M. (1995) ``Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation'', Technical Report No. 9508, Dept. of Statistics, University of Toronto, 22 pages: abstract, associated reference, postscript, pdf.

Neal, R. M. (1994) ``Sampling from multimodal distributions using tempered transitions'', Technical Report No. 9421, Dept. of Statistics, University of Toronto, 22 pages: abstract, associated references, postscript, pdf.

Neal, R. M. (1994) ``Priors for infinite networks'', Technical Report CRG-TR-94-1, Dept. of Computer Science, University of Toronto, 22 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (1993) Probabilistic Inference Using Markov Chain Monte Carlo Methods, Technical Report CRG-TR-93-1, Dept. of Computer Science, University of Toronto, 144 pages: abstract, contents, postscript, pdf.

Neal, R. M. (1992) ``Bayesian training of backpropagation networks by the hybrid Monte Carlo method'', Technical Report CRG-TR-92-1, Dept. of Computer Science, University of Toronto, 21 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1990) ``Learning stochastic feedforward networks'', Technical Report CRG-TR-90-7, Dept. of Computer Science, University of Toronto, 34 pages: abstract, postscript, pdf, associated reference.

Unpublished manuscripts

Neal, R. M. (1989) ``The computational complexity of taxonomic inference'', unpublished manuscript, 18 pages: postscript, pdf.