How a Python Temp File Leak Crashed CI Servers and the Fix Made It Worse

A nightly CI build failure led engineers to discover thousands of undeleted temporary directories in /tmp, each containing hundreds of megabytes of Doxygen XML output. The root cause was a missing cleanup call whenever the parsing pipeline aborted with an exception. An initial fix using Python's tempfile.TemporaryDirectory relied on the garbage collector to delete folders automatically, which appeared to work at first. However, circular references in the parser objects delayed GC collection, causing temp directories to accumulate and eventually triggering a massive synchronous disk deletion that blocked the main thread and caused service timeouts. The team concluded that delegating heavy resource cleanup to Python's garbage collector is an anti-pattern, and that explicit lifecycle management of temporary folders was necessary.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in