MCMC Explained: The Statistical Method That Transformed Bayesian Inference

Markov Chain Monte Carlo (MCMC) was first developed at Los Alamos in 1953 when Metropolis and colleagues simulated a liquid-gas equilibrium system, realizing they only needed a Markov chain sharing the target's equilibrium distribution. Hastings generalized this approach in 1970, giving rise to the Metropolis-Hastings algorithm, while Geman et al. introduced the Gibbs sampler in 1984 as a special case. The method remained largely outside mainstream statistics until Gelfand et al. brought it to the broader Bayesian community in 1990, after which MCMC quickly became central to Bayesian inference. At its core, a Markov chain is a sequence of random variables where each state depends only on the immediately preceding one, characterized by its initial distribution and transition probabilities. The Metropolis-Hastings-Green algorithm, further generalized by Green in 1995, constructs transition distributions that preserve a specified equilibrium, with reversibility serving as a key sufficient condition for stationarity.