On-Demand vs Spot GPUs: How Teams Should Actually Decide Where to Run AI Inference

·1 views

As GPU cloud capacity tightens and demand surges — highlighted by Amazon crossing a $3 trillion valuation partly on AI cloud growth — teams running AI inference face a critical cost decision between on-demand and spot GPU instances. While spot instances can cost 60–70% less than on-demand, they carry the risk of sudden interruption, making the headline discount misleading without accounting for workload type. A practical framework categorizes workloads into three buckets: batch jobs that can checkpoint and resume safely on spot, real-time inference behind load balancers that can tolerate spot only with careful multi-zone engineering, and latency-sensitive user-facing services that require the reliability of on-demand pricing. Hidden costs such as model cold-start times, the on-demand baseline most teams still maintain, and the engineering effort to manage spot fleets further erode the apparent savings. The key takeaway is that blended cost modeling — not the best-case spot discount — should drive GPU provisioning decisions.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Recurring csh Shell Bug Prompts Engineers to Build SSH Command Wrapper

In June 2026, database backups began failing on all Sakura Internet-hosted WordPress sites due to an SSH error traced back to a recurring shell compatibility issue. Sakura's default login shell, csh, cannot interpret bash syntax, and a newly added DB backup progress-monitoring feature had bypassed an earlier safeguard that was meant to prevent this exact problem. Developers first attempted a fix using base64 encoding via OpenSSL, but discovered that another hosting environment, heteml, lacked the required OpenSSL command entirely. The final solution was a Python wrapper function using shlex.quote() to pass scripts directly as arguments to /bin/sh -c, eliminating any dependency on external commands. The approach works regardless of the server's login shell, requires only that scripts be single-line, and has since been verified across multiple hosting environments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS CloudFormation Explained: IaC, Templates, Stacks and Change Sets

AWS CloudFormation is Amazon's native Infrastructure as Code service that allows engineers to define, provision, and manage cloud resources using YAML or JSON templates instead of manual console interactions. Before IaC, infrastructure was configured manually, making it difficult to reproduce environments consistently or track changes over time. With CloudFormation, a template acts as a blueprint that the service reads and executes to build identical infrastructure across any region or account. Key benefits include version control, auditability, drift detection, and the ability to integrate infrastructure changes into CI/CD pipelines. The service supports advanced features such as parameters, mappings, conditions, intrinsic functions, change sets, and stack sets for enterprise-scale deployments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Silent Cache Bug Fixed in npmx.dev That Froze npm Package Version Displays

A bug in npmx.dev, a browser for the npm registry, was causing package version comparisons to display outdated data — for example, showing tinyclip at version 0.0.1 instead of the actual latest 0.1.8. The root cause was traced to a composable called useCachedFetch, where the client branch hardcoded the browser fetch option cache: 'force-cache', which ignores server cache headers and serves a frozen local copy indefinitely. The npm registry already sends correct Cache-Control headers with a 300-second max-age and ETag support, meaning the app was simply not honoring them. The fix replaced 'force-cache' with 'default' in two locations, instructing the browser to follow standard HTTP caching rules instead. The change passed all 1,000-plus existing unit tests and was submitted as pull request #3156, resolving the originally reported issue #1832.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

TryHackMe CTF Writeup: XOR Crypto and PHP Filter Exploits Explained

A CTF writeup from TryHackMe's 'Overflow The Jackpot' challenge details solutions across multiple security categories including cryptography, web, and forensics. The crypto challenge 'B1t Recovery' used a 4-byte repeating XOR key generated via Python's os.urandom, which was trivially broken using a known-plaintext attack on the predictable 'THM{' flag prefix. By XORing the first four ciphertext bytes against the known prefix, the full key was recovered and the flag decrypted without any brute force. A second web challenge, 'Lost Fortune', involved a PHP application with a flawed path-traversal filter that blocked '../' sequences but failed to block PHP stream wrappers like 'php://filter', allowing file disclosure. The writeup concludes that repeating-key XOR is insecure for any real encryption, and recommends authenticated ciphers such as AES-GCM or ChaCha20-Poly1305 instead.

0 comments Read more at DEV Community