Flat L2 regularisation matches EWC's Task A retention but cripples new-task learning

A registered experiment on Permuted MNIST tested whether EWC's Fisher-weighted penalty is needed to prevent catastrophic forgetting, or whether a uniform L2 penalty works as well. At a weak setting, flat L2 came within 0.86 points of EWC's Task A accuracy, but it cost 2.70 points on Task B, over six times EWC's cost, so no setting met the pre-registered fair-budget criterion. At a stronger setting, flat L2 matched EWC's Task A accuracy of 97.63% exactly. However, its Task B accuracy fell to 18.42%, compared with EWC's 97.43%, showing the network was largely unable to learn the new task. The author concludes that Fisher weighting is not needed to match retention, but it is needed to retain that performance without sacrificing the ability to learn new tasks.