Postgres migration passed in staging but locked a production checkout table for six minutes
A schema migration that ran in 40 milliseconds in an empty staging database caused a six-minute outage on the checkout table in production. The migration added a column, backfilled it with an unbatched UPDATE, and then set NOT NULL, which together rewrote millions of rows and held an exclusive lock throughout. The author explains that the staging environment, built from migration history rather than real data, could not reveal these costs. The recommended fixes include a production-shaped staging dataset, batched backfills, adding constraints in non-blocking steps, and setting lock and statement timeouts to surface problems early.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in