Laplace Approximation Gives Trained Neural Networks Uncertainty Estimates Without Retraining
Standard neural networks produce single point predictions without indicating how confident they are, and full Bayesian inference is usually too costly for large models. The Laplace approximation offers a practical alternative by using the curvature of the loss around a trained model's optimum to form a Gaussian posterior over its weights. This approach requires no retraining, sampling loops, or new architectures, relying instead on a second-order analysis of an existing solution. The Hessian's inverse supplies the posterior covariance, so parameters the data pins down closely receive narrow uncertainty while poorly determined ones receive wide uncertainty. The method is a local approximation whose reliability depends on whether a Gaussian fits the region around the mode, so its assumptions should be checked rather than taken for granted.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in