SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

How OpenAI Handles 29,000 ChatGPT Messages Per Second With Boring Engineering

0
·1 views

OpenAI reported crossing 900 million weekly active users in February 2026, with approximately 2.5 billion messages processed daily — roughly 29,000 every second. Incoming requests first hit a global router that directs users to the nearest regional data center based on location, available compute, and hardware requirements. Within each region, hundreds of application servers handle authentication, quota checks, and conversation history retrieval, with load balancers distributing traffic to prevent overload. Surprisingly, the database powering this scale is standard PostgreSQL, configured with one primary write instance and nearly 50 read replicas spread across regions. This architecture works because ChatGPT's workload is heavily read-oriented, allowing the system to deliver responses in low double-digit millisecond latency with five-nines availability.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer Tests 8 Chrome Extensions to Reveal How Host Permissions Are Displayed

A developer behind eight Chrome extensions under Legacy Tools examined how different host permission declarations appear to users in Chrome's extensions manager. Across four declaration methods — broad host_permissions, fixed-site matches, optional grants, and activeTab — the Permissions section appeared blank in seven of the eight tested extensions. The developer found that content_scripts matches entries carry the same access weight as explicit host_permissions, yet produce identical user-facing warnings, meaning a blank permissions list does not indicate a harmless extension. One extension declaring thirty URL patterns displayed only fifteen in the UI due to deduplication, showing that a shorter visible list does not reflect narrower actual coverage. The findings highlight that permission declarations function as user-facing UI rather than mere technical configuration, and that activeTab is worth the added design effort for limiting access.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Amazon Bedrock Offers Prompt Management and Flows to Tame LLM App Complexity

Developers building large language model applications often embed prompts directly in code, making updates cumbersome and version tracking difficult. Amazon Bedrock addresses this with two features — Prompt Management and Prompt Flows — that treat prompts as versioned AWS resources with their own identifiers and configurations. Prompt Management allows teams to iterate on prompts independently of application code, while production systems reference stable, pinned versions. Prompt Flows provides a serverless orchestration layer for building multi-step LLM pipelines using a graph of nodes, eliminating the need for custom glue code. Together, the features enable cost-conscious routing — for example, sending only technical queries to more expensive models like Claude while handling general ones with cheaper alternatives.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Microsoft Expands MAI Playground With Image, Voice, Transcription and Reasoning Models

Microsoft has launched an expanded MAI Playground as a limited-preview environment where users can test its growing family of in-house AI models. The platform currently includes models covering image generation, transcription, voice, and reasoning capabilities under a single interface. Access remains restricted, with MAI-Image-2 API availability limited to select customers such as early adopter WPP, while broader developer access via Azure AI Foundry is planned for a later stage. Microsoft has positioned the Playground as a public testing layer rather than a production-ready API, cautioning users that outputs should be treated as experimental. Pricing and full technical specifications for the models have not yet been publicly disclosed.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

One Developer Passed All 12 AWS Certifications in a Year by Starting with the Hardest

A developer completed all 12 AWS certifications between August 2024 and 2025 by deliberately reversing the conventional study order, beginning with the Solutions Architect Professional — widely regarded as the most difficult exam. The strategy was based on the idea that clearing the broadest exam first makes every subsequent certification feel like a narrower subset, building psychological momentum rather than eroding it. Cloud Practitioner, typically the starting point for most candidates, was taken last as a near-victory lap, though it still required dedicated preparation for its unique topic areas. Total exam fees ran between approximately $2,000 and $2,700 before tax, though AWS's 50% discount voucher system — awarded upon each pass — significantly reduced cumulative costs. The developer also highlighted shifting from passive study materials to practice exams as the exam date neared, and treating rest as a deliberate part of the preparation plan.

0 comments Read more at DEV Community
How OpenAI Handles 29,000 ChatGPT Messages Per Second With Boring Engineering · ShortSingh