Capping concurrent async calls in Node.js with a 15-line promise pool
Promise.all starts every call at once, which can overwhelm a provider's rate limits or exhaust sockets and memory when processing thousands of items. The article proposes a concurrency limit that runs at most N calls at a time, starting the next as soon as one finishes. It uses a dependency-free approach where N lanes pull from a shared index, keeping results in input order and converting failures into values so one error does not discard the batch. The author notes that for LLM APIs the limit should be paired with a token-bucket rate limiter, while CPU-bound work needs worker threads instead. Retries should live inside the worker and cancellation can be handled with an AbortSignal, and libraries like p-limit offer the same behavior.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in