We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Variational Inference in Nonconjugate Models.
- Authors
Chong Wang; Blei, David M.
- Abstract
Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest--like the correlated topic model and Bayesian logistic regression--are nonconjugate. In these models, mean-field methods cannot be directly applied and practitioners have had to develop variational algorithms on a case-by-case basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world data sets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression.
- Subjects
MATHEMATICAL statistics; MEAN field theory; APPROXIMATION theory; PROBABILITY theory; MATHEMATICAL optimization; MULTIVARIATE analysis; LOGISTIC regression analysis
- Publication
Journal of Machine Learning Research, 2013, Vol 14, Issue 4, p1005
- ISSN
1532-4435
- Publication type
Article