How Backpropagation Works: Manual Gradient Calculation Verified by PyTorch

A developer walks through computing gradients by hand for a two-layer neural network using only scalar values and the chain rule. The network consists of one input, a hidden neuron with a ReLU activation, an output neuron, and a squared-error loss function with four trainable parameters. Starting from a forward pass that produces a loss of 25, the backward pass traces gradients step by step, arriving at values of 30, 30, 20, and 10 for w1, b1, w2, and b2 respectively. The same network is then built in PyTorch using autograd, which produces identical gradient values after a single call to loss.backward(). The exercise illustrates how gradients flow back through layers by multiplying local derivatives at each node, a process that scales to any network depth.