How Gaussian Process Classification Extends GP Regression to Discrete Labels

Gaussian Process Classification (GPC) generalizes Gaussian Process Regression to handle problems where outputs are discrete class labels rather than continuous values. Unlike regression, classification cannot use a Gaussian likelihood, making exact inference infeasible and requiring approximate inference methods instead. The approach places a GP prior over a latent function, which is then passed through a logistic (sigmoid) function to produce valid class probabilities between 0 and 1. A key challenge is that the latent function itself is never directly observed — only inputs and class labels are available — so the latent function must be integrated out during inference. GPC builds on the same conceptual framework as logistic regression, extending it to a non-parametric setting in the same way GPR extends linear regression.