UI tests passed on a broken banking app until bugs were deliberately injected
The author built ChaosBank, a banking app with 135 switchable bugs, to test whether a 34-test UI suite could detect real faults. On the clean build every test passed, which showed only that the app worked, not that the tests would catch errors. Switching on deliberate bugs one at a time revealed that nine tests never failed and 18 of the 23 money-related bugs survived the full suite. A portfolio test, for example, only checked that numbers appeared on screen, so a loss displayed as a gain went unnoticed. The author then rewrote tests to assert specific values, such as a minus sign on a losing holding, and debugged SwiftUI accessibility identifiers that were obscuring the elements being tested.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in