Why research thresholds should be set by the question, not by a product roadmap

A reader asked how a kill-gate should work for a social robot, which raised the broader issue of who gets to define success for research. The author argues the threshold must come from the research question itself, written before knowing whether a product or client depends on the result. In an EWC-versus-L2 comparison, a single retention pass hid a cost difference that a product-shaped gate would have obscured. The same risk appears in silent failures, such as a recognizer that never fired for one language yet showed a clean average. The tradeoff is slower timelines and claims that sometimes must adapt to unfavorable results, but the author considers this the only worthwhile form of publishable measurement.