Single TPU Chip Runs Gemma 4 AI Agent Backend at Under $0.11 per Million Tokens

·1 views

A developer has published a detailed build log showing that Google's Gemma 4 E2B model can be self-hosted on a single Google Cloud TPU v5e chip using vLLM, achieving 1,496 output tokens per second at roughly $0.107 per million output tokens. The setup costs approximately $0.58 per hour on spot pricing and delivers 8.02 ms per-token latency, making it capable of supporting 8 to 16 concurrent lightweight AI agents. The guide covers three provisioning models — spot, on-demand, and flex-start — noting that flex-start is the only option that automatically stops billing after a set duration. A key finding is that flex-start provisioning for a v5litepod-1 instance is only accepted in the us-west4-a zone, with other zones rejecting the configuration at the API level regardless of available quota. The author also highlights that four of their initial performance predictions were disproved by actual benchmarks, calling those discrepancies the most instructive part of the exercise.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds Open-Source Security Framework to Detect Malicious AI Agent Skills

A developer has released 'agent-skills-guard', a static analysis framework designed to detect security threats hidden inside AI agent skill files used by tools like Claude and GitHub Copilot. The framework scans entire skill definition files, including metadata fields like descriptions, catching injected instructions that silently direct agents to leak data without user awareness. In testing, the tool successfully flagged two high-severity prompt-injection phrases embedded solely within a skill's description field, with no malicious code present elsewhere. Detection rules are stored in a separate JSON file rather than hardcoded, making it easier for users to extend the scanner with custom patterns. The developer acknowledged two current limitations: the tool cannot yet detect skills crafted to over-trigger through persuasive but non-malicious wording, and it has no mechanism to alert users when a previously approved skill is silently updated after installation.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Screenshots Are the Wrong Input When Using AI to Convert Figma Designs to Code

Design-to-code teams commonly export Figma frames as PNG screenshots and feed them into AI tools like Claude or Cursor, but this approach produces unreliable results. A screenshot strips away the structured layer tree that encodes layout intent, component types, variant properties, and spacing tokens — leaving the AI to infer rather than read. This inference is inherently lossy: the model may reconstruct code that looks visually correct but is semantically wrong, using hardcoded pixel values instead of design tokens or incorrect component variants. Tools like figmascope offer an alternative by exporting a structured context bundle — including typed intermediate representations, design tokens, and component inventories — that AI agents can parse directly. The core issue is not AI model capability but the quality of input provided, since even a capable model can only guess when given pixel data instead of structured design information.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Stop Serverless Cron Jobs From Failing Silently or Running Twice

Migrating scheduled tasks to serverless functions eliminates single-server dependency but introduces new reliability risks, including silent failures and duplicate executions. Because platforms like EventBridge, SQS, and Azure Queue all use at-least-once delivery, the same job can trigger more than once, potentially double-charging customers or sending duplicate emails. Developers must build idempotency into their function code using a stable, event-derived key and an atomic check-and-set database operation to safely skip duplicate runs. Tracking job completion state — not just whether an event was received — helps detect half-finished executions caused by mid-run crashes. Dead-letter queues are also recommended to catch poison messages and prevent silent data loss when a job repeatedly fails.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Lumabri experiment aims to run large AI models across peer-to-peer networks

A developer has launched Lumabri, an experimental project that distributes the workload of running large language models across multiple ordinary computers in a peer-to-peer network. Inspired by earlier work on Colibrì, which explored running large LLMs on a single consumer machine, Lumabri treats a swarm of networked computers as a shared pool of disk, compute, and model resources. The system is particularly suited to Mixture-of-Experts models, where only a fraction of parameters are activated per token, allowing small activations to be sent to peers that already hold the relevant model weights. The developer is experimenting with security and reliability features including SHA-256 verification, signed model state, peer verification, and failover mechanisms. Lumabri is an early-stage personal experiment built without datacenter infrastructure, and the source code is publicly available on GitHub.

0 comments Read more at Hacker News