Load tests show t3code's AI streaming holds below 50 users but degrades past 100
A developer benchmarked pingdotgg/t3code's streaming AI completion setup using k6 on an AWS t3.xlarge instance, ramping from 10 to 100 concurrent users over five minutes. Measured metrics included time-to-first-token, token throughput, SSE connection drop rate, and Next.js server memory use. Performance stayed stable under 30 users, with drops at about 2% near 50 to 80 users, mostly from client-side abort races. At 100 users, p95 time-to-first-token exceeded one second, connection drops reached 8.4%, and memory passed 1GB due to SSE connection bookkeeping overhead. The author recommends deploying as-is below 50 users and adding a Redis-backed pub/sub layer for larger teams.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in