Tracking LLM spend per customer in Node.js using AsyncLocalStorage
A common problem is that an OpenAI invoice shows total spending but not which customer generated it, which matters for flat-priced plans. The article proposes attaching a customer ID to the request's async context once, through middleware, instead of threading it through every function. Developers can wrap the OpenAI client so each call reads that context and records its cost, with feature tags added via nested context scopes. Background jobs can start their own context tied to the relevant customer. Costs are converted to dollars at write time using a dated price table, with attention to cached input tokens billed at a lower rate.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in