SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

GPU Cloud Costs Drop to 18% of On-Demand Price Using Interruptible Instances

0
·30 views

A developer reports paying between $0.55 and $0.70 per hour for GPU instances on interruptible capacity, compared to $3.04 per hour at on-demand rates — a saving of over 80%. Unlike CPU instances where discounts typically range from 55–75%, GPU families appear to qualify for discounts near the top of AWS's advertised 90% ceiling. However, interruptible pricing varies significantly by availability zone, and the cheapest zones often carry the highest risk of interruption and low capacity ratings. A harder constraint is the vCPU quota for interruptible GPU instances, which is capped at 64 per region and cannot easily be raised through self-service. This quota ceiling limits scaling flexibility and forces users to plan workloads around fixed instance shapes rather than adjusting capacity incrementally.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Running a 33B dense agent model on a single 24 GB GPU: offloading pitfalls and empty replies

Nanbeige4.3-33B, an Apache-2.0 agent coding model released by the Nanbeige team at BOSS Zhipin on 24 October, was self-hosted on one RTX 4090 using llama.cpp and bartowski's GGUF files. Because the model is dense, offloading layers to CPU RAM sharply slowed generation, dropping a 4-bit run to 1.6 tokens per second, while a smaller 3-bit file fitting fully on the GPU ran at about 56 tokens per second. The 3-bit file still ran acceptably on a 16 GB RTX 2000 Ada, but the 2-bit version gave a wrong answer to a basic arithmetic check, so it was ruled out. With thinking mode enabled by default, some long replies came back with empty content and the full text in reasoning_content, while tool calls stayed reliable. The authors traced this to the chat template leaving a thinking block open, and found that disabling thinking mode placed the full answer in the correct field.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Caching pip downloads saves under a second in GitHub Actions; source builds cost far more

Adding cache: pip to actions/setup-python caches pip's download directory, which on a fast runner saves less than a second per install. The bigger time cost comes when a dependency lacks a wheel for the platform and must be compiled, which took 10.9 seconds in one measurement and is also stored in the cache. Tests on a latchkey-small runner showed the pip download cache cut installation time from 7.7 to 6.9 seconds. Poetry's cache option instead restores the whole virtualenv, skipping the install step as well as the download.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Hospital staff chatbot built from training materials and exam answer keys

A team built a bilingual English and Arabic chatbot to answer staff questions about the InterSystems TrakCare hospital information system. Their training corpus came from scraping Moodle assessment print-outs, which contained correct answers, yielding 433 verified question-answer pairs. They also transcribed 43 training videos with an open speech-to-text model and distilled them into topic files. The system uses a hand-built BM25 retrieval engine with source-trust weighting and an Arabic normalization and clinical synonym layer rather than a vector database. The chatbot is available as a live demo on a training environment without patient data, running on free open models.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

cdkd Pairs with AWS Emulator to Run CDK Application Code Locally Without Redeploying

A DEV Community article explains how combining AWS Emulator, a local tool that emulates AWS APIs, with cdkd, a faster alternative CDK deployment tool, streamlines CDK development. cdkd automatically wires up the connection to the emulator and resolves environment variables from CDK code, removing the need for a .env file. Developers can mix targets, using real AWS for some services while writing other data to the emulator. Application code, such as Lambda functions, runs directly from source, so changes take effect without redeploying. The author demonstrates the setup with a stack of a Lambda function, DynamoDB table, S3 bucket, and API Gateway, using AWS Emulator v1.0.322 and cdkd v0.349.0.

0 comments Read more at DEV Community