Tenant-Attributed LLM Summaries in Node.js: Batch Jobs vs Streaming Requests

The article argues that the choice between batch and streaming LLM processing for large document summaries should follow the tenant's workflow, with both transports writing to a shared usage ledger. Large imports or bulk refreshes are best handled as asynchronous batch jobs, while an editor awaiting a single result benefits from a streamed chat completion. Regardless of transport, each document, chunk, retry, and partial result should stay attributable to one tenant, recording fields such as tenant ID, document identity, prompt version, input size, mode, and job state. The author stresses creating the local record and counting tokens before submission, chunking oversized inputs, and keeping provider acceptance distinct from the product's own tenant commitments. Cancellations, retries, and disconnects are to be modeled as explicit state transitions, with idempotency rules to prevent duplicate jobs.