Splitting and Merging Components of a Nonconjugate Dirichlet Process Mixture Model

Sonia Jain, Division of Biostatistics and Bioinformatics, Dept. of Family and Preventive Medicine, University of California at San Diego
Radford M. Neal, Dept. of Statistics and Dept. of Computer Science, University of Toronto

The inferential problem of associating data to mixture components is difficult when components are nearby or overlapping. We introduce a new split-merge Markov chain Monte Carlo technique that efficiently classifies observations by splitting and merging mixture components of a nonconjugate Dirichlet process mixture model. Our method, which is a Metropolis-Hastings procedure with split-merge proposals, samples clusters of observations simultaneously rather than incrementally assigning observations to mixture components. Split-merge moves are produced by exploiting properties of a restricted Gibbs sampling scan. A simulation study compares the new split-merge technique to a nonconjugate version of Gibbs sampling and an incremental Metropolis-Hastings technique. The results demonstrate the improved performance of the new sampler. We illustrate the utility of our technique as an unsupervised clustering method using real data.

Technical Report No. 0507, Dept. of Statistics, University of Toronto (August 2005), 37 pages: postscript, pdf.

Associated references: A revised version of this technical report has been published as follows:
Jain, S. and Neal, R. M. (2007) ``Splitting and merging components of a nonconjugate Dirichlet process mixture model'' (with discussion), Bayesian Analysis, vol. 2, pp 445-472, posted online 2007-01-23: pdf (discussion is in other URLs).
Earlier work along these lines for conjugate models was reported in the the following technical report:
Jain, S. and Neal, R. M. (2000) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Technical Report No. 2003, Dept. of Statistics (July 2000), 32 pages: abstract, postscript, pdf, associated references.

The following earlier paper by Radford Neal contains related work:
Neal, R. M. (1998) ``Markov chain sampling methods for Dirichlet process mixture models'', Technical Report No. 9815, Dept. of Statistics, University of toronto, 17 pages: abstract, postscript, pdf, associated references, associated software.