Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Radford M. Neal, Dept. of Statistics and Dept. of Computer Science, University of Toronto

Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model are reviewed, and two new classes of methods are presented. One new approach is to make Metropolis-Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.

Technical Report No. 9815, Dept. of Statistics, University of Toronto (September 1998), 17 pages: postscript, pdf, associated software.

Associated references: A revised version of this technical report has appeared as the following paper:
Neal, R. M. (2000) ``Markov chain sampling methods for Dirichlet process mixture models'', Journal of Computational and Graphical Statistics vol. 9, pp. 249-265: abstract, associated references, associated software.
The following technical report describes a more elaborate algorithm based on split-merge proposals:
Jain, S. and Neal, R. M. (2000) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Technical Report No. 2003, Dept. of Statistics (July 2000), 32 pages: abstract, postscript, pdf, associated references.
Some earlier work of mine on models equivalent to Dirichlet process mixtures is described in the following technical report:
Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.