Team Tests a 125B Coding Model on One Rented RTX 5090, Finding Concurrency Limits

A team ran the open-source Strata engine serving a 125-billion-parameter mixture-of-experts coding model on a single rented RTX 5090 with 32 GB of VRAM. Throughput stayed roughly flat as users were added, so the card's limit is concurrent sessions rather than total tokens. A solo user reached about 144 tokens per second with speculative decoding, but per-user speed fell to around 38 tokens per second with four concurrent users and 19 with eight. Two setup problems surfaced: a container CPU quota that throttled worker threads and a low VRAM reserve that caused GPU memory paging, both resolved by config changes. The authors note that the quality benchmark figure comes from the model card rather than their own testing.