Why Node.js background workers should combine events and polling, not rely on one
Using setInterval for Node.js background jobs creates two hidden problems: it fires on a fixed schedule regardless of how long each run takes, causing overlapping executions that can exhaust database connection pools, and it issues thousands of idle queries daily even when there is no work to do. Chaining setTimeout from the end of each work cycle eliminates overlap by making the delay a gap between runs rather than a fixed clock. Event-driven mechanisms like Postgres LISTEN/NOTIFY or Redis pub/sub remove idle query costs and reduce job latency from seconds to milliseconds, but they offer no delivery guarantee — a missed notification means a job silently never runs. The recommended pattern combines both approaches: events handle the fast path while a slow poll, running perhaps once a minute, acts as a safety net to catch anything the event missed. This hybrid gives low latency on healthy days while preserving the reliability guarantee that pure event-driven systems lack.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in