SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

NVIDIA MPS on Amazon EC2 Cuts ASR Inference Costs by 75% at Scale

0
·1 views

A collaborative effort by AWS, NVIDIA, and Heidi demonstrates how NVIDIA Multi-Process Service (MPS) can reduce automatic speech recognition inference costs by 75% on Amazon EC2. The setup runs on g6e.4xlarge and g7e.4xlarge instances equipped with NVIDIA L40S GPUs, using a three-container architecture built around the NVIDIA Triton Inference Server. Unlike time-slicing or MIG partitioning, MPS allows multiple CUDA clients to share a single GPU context concurrently without requiring application rewrites. The benchmark identifies an optimal operating point where mean latency stays below 650 ms and p99 latency remains under 1,000 ms, balancing throughput gains against acceptable response times. The findings highlight that maximising GPU utilisation through concurrent execution, rather than running a single model instance per GPU, offers a practical cost-efficiency improvement for production-scale speech recognition workloads.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer finds six tests that passed without actually testing anything

A developer discovered six separate instances in one project where automated tests reported success without genuinely verifying the properties they were designed to check. Examples included a database restore script that confirmed an empty database's schema as valid, a health-check running at high volume instead of the low-volume condition it was meant to simulate, and a bucketing test whose assertion and input were the same hardcoded constant. Even after identifying the pattern and writing a rule to prevent it, the developer immediately wrote another hollow test, highlighting that awareness alone does not prevent the mistake. The author concludes that catching such tests requires a mechanical discipline: for every guard, explicitly identify what input would make it pass without the target property holding, and confirm that input appears as a failing negative case in the test suite.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

More Context Can Make AI Answers Worse, Not Better, Research Shows

AI models advertise large context windows of up to one million tokens, but their effective working memory is significantly smaller in practice. As more text is added to a prompt, the model's attention — which is fixed and must be divided across all tokens — becomes diluted, reducing answer quality. Research, including the 'Lost in the Middle' study by Liu et al., found that information placed in the middle of long prompts is retrieved far less accurately than content near the beginning or end. Real-world data compounds the problem, as multiple similar or conflicting document versions make it harder for the model to identify the correct source. Experts recommend sending fewer but higher-quality chunks — typically three to five — and placing key instructions at the top and bottom of a prompt to improve reliability.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Dev ships Three.js iOS game in 17 days after four silent, hard-to-debug failures

A developer successfully published a Three.js game to the iOS App Store in 17 days across 247 commits, running inside a WKWebView via Capacitor with no network calls. Four critical issues nearly derailed the project, each failing silently without errors or logs, making each one resemble a different problem entirely. Offscreen rendering ran at 2.5 minutes per frame due to Chrome defaulting to a software rasterizer; switching to the Metal GPU backend brought performance to 38–60 fps with a single flag change. On iOS, WebKit silently kills the GPU process without firing the standard webglcontextlost event, requiring additional detectors — including a draw-call floor check — to reliably catch and recover from context loss. A StoreKit timing issue and a world-tiling modulo mismatch were among the other silent failures that demanded careful workarounds before the app could ship.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds EventMocha, a Multi-Agent AI Tool for End-to-End Event Campaign Automation

A developer created EventMocha, a multi-agent AI assistant designed to automate the full workflow of event planning and promotion, submitted as an entry to the AllThingsAgentic Hackathon. The tool uses Google's Agent Development Kit alongside Gemini 2.5 and 3.7 Flash models to generate HTML promotional copy, visual posters via Imagen, and video teasers via Veo within a single session. It also automates Google Workspace tasks by creating Calendar events, Gmail drafts, and Google Tasks through the Model Context Protocol. A live voice interface powered by the Gemini Live API allows users to interact with the system in real time using bidirectional conversation. The platform also supports targeted revisions, preserving previously approved content while updating only the specific assets a user requests to change.

0 comments Read more at DEV Community
NVIDIA MPS on Amazon EC2 Cuts ASR Inference Costs by 75% at Scale · ShortSingh