Three AI services run concurrently on 24GB GPU, using 17.8GB VRAM

·2 views

A developer tested running multiple AI services simultaneously on a single NVIDIA RTX 3090 GPU with 24GB of VRAM. The setup included a Whisper ASR transcription service, a text-to-speech model, and an embedding model for a week. The combined services used an average of 17.8GB VRAM, with peaks reaching 23.4GB during simultaneous processing bursts. The experiment demonstrated that practical multi-tenant AI workloads require less VRAM than theoretical estimates suggest. The GPU maintained stability without spilling to system RAM despite nearing its capacity limit during peak usage.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Software Development Article Advises Caution When Refactoring Shift Overlap Code

A DEV Community article details a specific method for refactoring code that calculates shift overlaps and breaks. The author warns that small edits can unintentionally change the program's behavior, making it appear as a code cleanup when it is not. The core advice is to first establish a verified baseline of current behavior before extracting helper functions. The article provides concrete coding examples and instructions for placement within a disposable repository. The goal is to create a smaller function while preserving all existing logic around edge cases like midnight wraps and break credits.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

FastAPI JWT Authentication Guide Demonstrates Five-Endpoint API

A developer published a complete guide for implementing JWT authentication in FastAPI. The tutorial provides a single-file, 118-line solution with five API endpoints for login, user data, and protected routes. It utilizes modern libraries including PyJWT for token handling and Argon2 for password hashing. The guide emphasizes practical security by demonstrating common failure cases like expired or invalid tokens. It is tested with Python 3.12 and FastAPI 0.128.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS Secrets Manager helps production applications securely manage credentials

Hardcoding credentials like database passwords in source code creates security risks, especially when code is shared or deployed. Services like AWS Secrets Manager provide secure storage for sensitive data such as API keys and database connection strings. This approach separates configuration from application code, preventing exposure in repositories or Docker images. Applications retrieve credentials at runtime with appropriate IAM permissions, enabling secure access without embedding secrets. Centralized secret management also simplifies credential updates through rotation workflows.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer creates screen time app that forces outdoor photo break to unlock sites

A developer has built a Chrome extension called Outside Leash to manage personal screen time habits. The tool limits daily access to sites like YouTube and Reddit to 20 minutes before locking the tab. To regain access for an additional 30 minutes, the user must take a photo with their phone matching a generated quest, such as finding a specific colored object. The system uses a local AI model, Gemma 4, running entirely on the user's laptop to verify the photos without cloud services. The project was created for the Hacktoberfest Open-Source AI Challenge.

0 comments Read more at DEV Community