When passing tests still let a software bug slip through undetected
A developer maintaining qbofile, a browser-based accounting file converter, discovered that green unit tests can mask incorrect logic when both sides of an assertion are generated by the same flawed code. The bug involved converting transaction split amounts from IIF format, used by QuickBooks Desktop, to QIF format, read by Quicken and GnuCash, where a sign convention error would cause both sides of the test equation to flip simultaneously, keeping tests green. Mutation testing confirmed the assertions were wired to the code but could not determine whether the underlying logic was correct. The developer resolved the uncertainty by building a purpose-designed QIF file and importing it into GnuCash, verifying the split totals were accurate — turning a confident assumption into a confirmed measurement. Separately, a real defect was uncovered during the review process, unrelated to signs and involving how IIF payee and memo fields map to QIF's distinct P and M fields.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in