JEFFREY MILLER 
Duke University
“Combinatorial Stochastic Processes for Variable-Dimension Models”
ABSTRACT
Many of the commonly-used nonparametric models are infinite-dimensional, such as Dirichlet or Pitman-Yor process mixtures (DPMs/PYMs), Hierarchical Dirichlet processes (HDPs), and Indian buffet processes (IBPs). A less common but very natural type of Bayesian nonparametric model is constructed by taking a family of finite-dimensional models and putting a prior on the dimension — that is, using a variable-dimension model. The standard approach to inference in such models is reversible jump MCMC, however, it can be nontrivial to design good reversible jump moves, particularly in high-dimensional settings. We have found that, in fact, the variable-dimension approach gives rise to combinatorial stochastic processes that closely parallel those of DPMs, HDPs, and IBPs, and consequently, inference in certain variable-dimension models can be done in much the same way as for their infinite-dimensional counterparts. Further, there are certain advantages: cleaner clusters/topics/features (no tendency to make tiny superfluous groups), more control over the distribution of the number of clusters/topics/features, consistency for the dimension (e.g., the number of components), and conceptual simplicity.
