Verifying AI Training Method SDAR Requires Three-Arm Study, Not Just Final Accuracy

A technical deep-dive into the SDAR (Supervised Distillation with Adaptive Regularization) method highlights that its reported gains over plain GRPO — roughly 9.4% on ALFWorld and 10.2% on WebShop — are only half the story. The method's core claim is that a gated distillation mechanism prevents the training instability caused by noisy teacher rejections in naive GRPO+OPSD setups. Properly verifying this requires three parallel training runs: a plain GRPO baseline, an ungated teacher-distillation arm, and the full SDAR system. Per-turn loss variance, not final task success rate, is identified as the critical stability metric that distinguishes SDAR from its alternatives. The author notes that the high computational cost of running all three arms on AWS — particularly the unstable middle arm — is why the work remains a design blueprint rather than a completed benchmark.