How PyTorch's autograd computes every gradient automatically

Backpropagation done by hand works for a tiny network but does not scale to models with millions of weights. PyTorch instead records each operation of the forward pass as a graph node as the code runs, with every result keeping a link to the function that produced it. Calling L.backward() then walks this graph in reverse, applying each node's local derivative to the gradient passed down from above. Gradients accumulate in buckets attached to the weights that requested them, and when a value feeds several paths the contributions are summed. For the worked example, the resulting gradients match the values obtained by hand.