Reject malformed UTF-8 in CSV imports before parsing with TextDecoder fatal mode

·1 views

A CSV file can have correct columns yet contain corrupted text, because damage can occur before the parser ever sees the data. JavaScript's TextDecoder silently replaces invalid byte sequences with the U+FFFD replacement character by default, which can alter values like lookup keys without warning. Setting the fatal option to true makes decoding throw an error instead, allowing import workflows to reject damaged bytes. The article recommends checking file size before reading, keeping prior data and showing an actionable error on failure, and avoiding automatic retries under other encodings. It also notes that strict decoding only confirms byte validity, not correct encoding or valid CSV, so syntax and domain checks must follow separately.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why CSV delimiter detection should be able to report ambiguity

A small file like 'id,name;label' can be read as two columns under either a comma or a semicolon, so picking the first delimiter in a list silently hides an unresolved choice. The article recommends parsing the text separately with each candidate delimiter and accepting only results that pass syntax and table-shape checks, such as quoting, escaped quotes, embedded newlines, and consistent field counts. A sample function selects the widest valid candidate and otherwise returns a 'needs-choice' status listing the tied delimiters. The author notes that this widest-candidate rule is only a heuristic, that a known export format or expected header schema gives stronger evidence, and that one-column files are valid rather than errors. After a user picks a delimiter, the importer should reparse the original text, show the headers and sample rows, and keep existing data until the choice is confirmed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Supervisor Uses FalkorDB Graph Memory to Track Agent Failures and Verification

A developer team built AI Supervisor, a control plane for coordinating AI coding agents on multi-step missions. They found that storing isolated logs made it hard to see how failures, supervisory decisions, and handoffs relate to one another. To address this, they selected FalkorDB as a graph memory layer alongside SQLite, modeling entities such as tasks, executions, decisions, and verification evidence with explicit relationships. A key design choice is that an agent's completion claim is not treated as verified; dependent tasks stay blocked until their prerequisite verification is accepted. The article presents this as a broader lesson that agent coordination requires explicit state and evidence rather than optimistic status messages.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Ripple uses a graph database to flag company documents left out of date by policy changes

A company that shortens its refund window from 30 to 14 days can leave help pages, saved AI replies and workflows still promising the old terms. Ripple, a graph-powered company knowledge tool built with FalkorDB and Gemini for the Graph Hacks 2026 hackathon, addresses this by mapping relationships between policies, documents, claims, workflows and agent replies. When a policy changes, it traces active dependencies up to six hops away and shows an evidence path explaining why each source is affected. Owners can then review and repair the flagged material. The project treats the graph database as its core store, with SQLite used only for local authentication records.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Stack Intel Aims to Help Startups Make Better Use of Existing Tools and Resources

A DEV Community article reflects on the widely cited claim that 90% of startups fail, questioning both the statistic and why the startup ecosystem keeps producing new tools without helping companies use what they already have. The author notes that failure figures depend heavily on definitions, citing U.S. Bureau of Labor Statistics establishment-survival data that ranges from 49.8% to 57.3% across selected cohorts. The piece argues that startups face tight constraints on time, money, and attention while relevant resources such as cloud startup programs or existing internal tools often go unnoticed. The project behind the essay, Stack Intel, is presented as an exploration of whether an intelligence system could help companies discover and apply available technology and opportunities, not as a cure for startup failure.

0 comments Read more at DEV Community