Why checking for empty strings after Unicode normalization can silently drop payee data
A developer building a bank-file conversion tool discovered a subtle bug where Unicode normalization (NFKD) converted fullwidth Chinese punctuation into standard ASCII punctuation, leaving a non-empty string even after all meaningful characters were lost. Because the code checked for an empty string to flag fully lost payee names, transactions with names like '北京,上海' slipped through as a false 'partial loss' warning instead of a 'complete loss' alert. This meant QBO files were imported with correct dates and amounts but blank payee fields, with no accurate warning shown to users. The fix replaced the empty-string check with a regex test for at least one alphanumeric character, correctly identifying strings containing only leftover punctuation as fully emptied. The case illustrates a broader pitfall: wherever Unicode normalization precedes an emptiness check, the real question should be whether any meaningful content survives, not merely whether any characters remain.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in