Developer flags self-asserted test steps and untestable surfaces in E2E run
A developer running end-to-end tests on a sandbox project reported a 10/10 pass rate, but noted that one of the ten steps could not be externally verified — it was explicitly labelled as self-asserted rather than silently marked green. The untestable property concerned whether the binary correctly refuses unknown operation names, a check that is impossible to automate because the command interface has no argument slot through which a bad operation name can be passed. The developer argues this is a product observation, not a test limitation: if something cannot be tested from outside, it often also cannot be used from outside, and both gaps share the same fix. The remaining nine steps were validated across two independently created sandboxes, with all 46 output lines matching byte-for-byte, made possible by deriving environment identifiers from content rather than random allocation. An additional hour was lost to Git Bash silently rewriting POSIX paths before they reached the program, a transformation that produced no error and caused failures far from the actual source.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in