Discriminative vs Generative Models: How They Differ in What They Learn

In supervised machine learning, both discriminative and generative models solve the same prediction problem — mapping an input to an output — but they do so by modeling fundamentally different probability relationships. Discriminative models learn the conditional probability of an output given an input, p(y|x), and predict by finding the most probable output directly from that relationship. Generative models take the opposite approach, learning how inputs are distributed for each possible output, p(x|y), and combining that with the prior probability of each output to make a prediction via Bayes' rule. Support Vector Machines are a common example of discriminative models, while Gaussian Mixture Models and Bayesian Networks represent the generative approach. The key distinction is not classification versus data generation, but rather which underlying probability structure each model learns in order to arrive at its prediction.