Nonparametric Involutive Markov Chain Monte CarloRemote
A challenging problem in probabilistic programming is to develop inference algorithms that work for arbitrary programs in a universal probabilistic programming language (PPL). The density functions defined by such programs, which may use stochastic branching and recursion, are in general nonparametric, in that they express models on an infinite-dimensional parameter space. However standard inference algorithms usually target distributions with a fixed finite number of parameters. We present the nonparametric involutive Markov chain Monte Carlo (NP-iMCMC) framework for designing MCMC inference algorithms for nonparametric models expressible in a universal PPL. Building on the unifying involutive MCMC framework, we show how a wide range of MCMC algorithms (including Metropolis-Hasting, Gibbs sampling and Hamiltonian Monte Carlo) can be systematically generalised to nonparametric models, and presented as particular instantiations of the NP-iMCMC framework. Such a unifying framework makes it easy to derive and prove the correctness of new nonparametric extensions of existing MCMC algorithms. Preliminary empirical results suggest that strengths of the MCMC algorithms carry onto its nonparametric extension. Moreover, a recent empirical study on nonparametric HMC, which is an instance of NP-iMCMC, has demonstrated significant performance improvements over existing general MCMC approaches.
Slides (Lafi2022_Carol_Mak_Nonparametric Involutive MCMC (1).pdf) | 782KiB |
Sun 16 JanDisplayed time zone: Eastern Time (US & Canada) change
15:05 - 16:20 | |||
15:05 18mTalk | Towards Denotational Semantics of AD for Higher-Order, Recursive, Probabilistic LanguagesRemote LAFI Alexander K. Lew Massachusetts Institute of Technology, USA, Mathieu Huot Oxford University, Vikash K. Mansinghka MIT File Attached | ||
15:23 18mTalk | A Language and Smoothed Semantics for Convergent Stochastic Gradient DescentRemote LAFI File Attached | ||
15:42 18mTalk | Nonparametric Involutive Markov Chain Monte CarloRemote LAFI Carol Mak University of Oxford, Fabian Zaiser University of Oxford, C.-H. Luke Ong University of Oxford File Attached | ||
16:01 18mTalk | Rigorous Approximation of Posterior Inference for Probabilistic ProgramsRemote LAFI Fabian Zaiser University of Oxford, Raven Beutner CISPA Helmholtz Center for Information Security, Germany, C.-H. Luke Ong University of Oxford File Attached |