Proximal Methods: Why Splitting a Nonsmooth Problem Beats Solving It Directly

A DEV Community article explains why machine learning problems combining a smooth loss with a nonsmooth penalty are often handled by proximal methods rather than gradient descent. The approach isolates the nonsmooth term and solves it exactly through a proximal operator, which generalises a gradient step. The piece covers how splitting algorithms such as ISTA, FISTA, and ADMM use this operator, with the lasso as a key example. It notes that subgradient methods converge slowly and fail to deliver exact sparsity or feasibility. The article also discusses the practical limitations of these methods.