Butterfly adapter fine-tunes certified networks while preserving their robustness guarantee
Certified classifiers give a proof that predictions stay unchanged within a radius around each input, and that radius shrinks when fine-tuning increases the network's Lipschitz constant. Because clinics have only a few thousand labelled images, certified networks trained on ImageNet must usually be fine-tuned, and standard methods such as LoRA and full fine-tuning raised the constant several-fold in the authors' experiments. The proposed butterfly adapter multiplies each frozen weight matrix by a trained orthogonal matrix, which keeps the spectral norm unchanged for any parameter values. The method is implemented in the open-source eigenkit library, where a demonstration script shows the butterfly ratio staying at 1.000 while LoRA's ratio rises to 2.764 with random factors. In tests with dermatologists at the University of Trieste on trichoscopic scalp images, the butterfly approach kept certified accuracy at a perturbation level where the other methods performed worse.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in