## MCMC using Hamiltonian dynamics

**Radford M. Neal,
Dept. of Statistics and Dept. of Computer Science, University of Toronto**

Hamiltonian dynamics can be used to produce distant
proposals for the Metropolis algorithm, thereby avoiding the slow
exploration of the state space that results from the diffusive
behaviour of simple random-walk proposals. Though originating in
physics, Hamiltonian dynamics can be applied to most problems with
continuous state spaces by simply introducing fictitious "momentum"
variables. A key to its usefulness is that Hamiltonian dynamics
preserves volume, and its trajectories can thus be used to define
complex mappings without the need to account for a hard-to-compute
Jacobian factor - a property that can be exactly maintained even
when the dynamics is approximated by discretizing time. In this
review, I discuss theoretical and practical aspects of Hamiltonian
Monte Carlo, and present some of its variations, including using
windows of states for deciding on acceptance or rejection, computing
trajectories using fast approximations, tempering during the course of
a trajectory to handle isolated modes, and short-cut methods that
prevent useless trajectories from taking much computation time.

Chapter 5, pages 113 to 162, in the
Handbook of Markov Chain Monte Carlo,
edited by Steve Brooks, Andrew Gelman, Galin L. Jones, and Xiao-Li Meng,
Chapman & Hall / CRC Press, 2011.

First online version
posted 5 March 2010: postscript, pdf.
This version does not differ
stubstantively from the final version
at the handbook website.

Also available from arXiv.org.

There are R programs
that accompany this review paper.