Guide to Handling Amazon Bedrock Throttling in Production LLM Workloads

Throttling is a routine condition for teams running large-scale LLM traffic on Amazon Bedrock, signaled by ThrottlingException or TooManyRequestsException when request or token rates exceed model-specific regional quotas. The article argues that resolving it requires a layered strategy rather than a single configuration change. Key steps include checking Service Quotas to distinguish adjustable soft limits from fixed hard limits, applying exponential backoff with full jitter to retries, and capping concurrency. It also recommends circuit breakers, separate handling for model and client timeouts, and cross-region inference profiles, with provisioned throughput reserved as a last resort. The author further notes that setting realistic maxTokens values avoids over-reserving token quota that can trigger early throttling.