Google Gemma 4 Runs on AWS SageMaker NVIDIA T4 at 80% of L4 Speed

·1 views

A developer has successfully deployed Google's Gemma 4 language model on Amazon SageMaker using an NVIDIA T4 GPU, the smallest GPU instance the platform offers. Benchmarks show the T4 decodes at 77–82% the speed of the newer NVIDIA L4, while producing identical outputs. The deployment required a custom patch to work around a shared memory limitation in vLLM 0.30.0 that normally prevents Gemma 4 from running on older Turing-architecture GPUs like the T4. Among the tested model sizes — 2B, 4B, 12B, and 26B — the 12B variant is the largest that fits within the T4's 16 GB of memory. Python-based MCP tools were also developed to simplify management of the vLLM-hosted deployment on SageMaker.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AWS offers secure EC2 management without opening SSH port

AWS Systems Manager Session Manager provides an alternative to SSH for managing Linux EC2 instances. It allows administrators to establish interactive sessions without exposing the inbound SSH port 22. The method relies on an agent installed on the instance and administrator IAM permissions. All session requests are handled through AWS services, removing the need for a direct network path to the instance. This approach enhances security by eliminating the requirement for an open inbound port on the instance's security group.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Node.js Express Service Adds Early Image Size Limits to Block Oversized Uploads

A technical guide recommends implementing early size checks for image uploads in Node.js Express applications. The method involves reading image metadata before full processing to compare dimensions against set limits. This prevents expensive decoding of oversized files, such as identity documents in fintech services. The approach uses two separate limits: one for file size during transfer and another for pixel dimensions during processing. Early rejection reduces server load and provides clear feedback when uploads exceed configured thresholds.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer creates interactive Netflix catalog explorer using Python and open data

A developer built an interactive data visualization application to explore Netflix's catalog using a public dataset. The tool was created with Python, pandas, Plotly, and Streamlit, analyzing 8,807 titles including movies and TV shows. It features summary indicators, charts by release year and content type, and breakdowns by country and genre. The project demonstrates connecting exploratory data analysis with reproducible deployment while acknowledging the dataset represents a historical snapshot.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Laravel Horizon's built-in alerting limited, additional monitoring tools required

Laravel Horizon's only native alert is 'LongWaitDetected', which triggers when a queue's estimated wait time exceeds a configurable threshold. This alert repeats every five minutes during the condition but never sends a recovery notification, and it fails if Horizon itself is down. Gauges from packages like spatie/laravel-prometheus can monitor if Horizon is running or paused, while counters are needed to track failure rates and job latency for actionable alerts. Some issues, like expired job reservations, do not generate any metrics and require direct checks of the queue state.

0 comments Read more at DEV Community