Developer Builds 50-Line Python API Gateway to Unify Three Local LLM Machines

·1 views

A developer running local large language models (LLMs) across three machines — a Mac Mini, a Windows PC, and an Ubuntu server, each with Ollama installed — grew frustrated with hardcoding different IP addresses into every script. To solve this, they built a lightweight API gateway in approximately 50 lines of Python using Flask, which automatically routes requests to the correct machine based on the model specified. The gateway exposes a single endpoint, allowing all scripts to target one URL while the gateway handles model-to-machine mapping behind the scenes. If a machine goes offline, the gateway returns a 503 error rather than silently failing, and a health-check endpoint can be added to enable smarter routing. The developer notes the setup also makes infrastructure changes — such as moving models between machines — a one-line update rather than a multi-script overhaul.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Terraform Explained: How Infrastructure as Code Works from Init to Destroy

Terraform is an open-source infrastructure as code tool that allows teams to define, provision, and manage cloud or on-premises resources using configuration files instead of manual setup. These configuration files can be stored in version control systems like Git, enabling teams to track, review, and roll back infrastructure changes much like application code. A key component called the state file (terraform.tfstate) records all resources Terraform has created, preventing duplicate provisioning and enabling accurate updates. The core workflow involves four commands — fmt, validate, init, and plan — followed by terraform apply to create resources, and terraform destroy to tear them down. Remote state storage options such as AWS S3 or Google Cloud Storage allow entire teams to collaborate safely on shared infrastructure.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Six-gate checklist helps RPG devs validate core loop before adding content

A framework aimed at small RPG prototype developers proposes testing six essential gameplay gates before expanding a project with more rooms, enemies, or loot. The six gates cover player orientation, meaningful encounters, visible choices with trade-offs, state feedback, consequence continuity, and restart motivation. The author argues that most dungeon-crawler prototypes grow in scope prematurely, adding content before a single short run of three to five minutes is proven understandable and repeatable. Developers are advised to run the prototype once without explanation, identify the first point of confusion, fix only that issue, and repeat the process. The checklist is described as engine-neutral, applying equally to projects built in Godot, Unity, or any custom framework.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Correctly Set Alt Text for Decorative Images in WordPress

Most accessibility guides advise adding alt text to every image, but decorative images — such as dividers, background textures, and ornaments — should have an empty alt attribute so screen readers skip them entirely. Leaving the alt attribute out altogether is equally problematic, as some screen readers will then read the raw file name aloud to users. In WordPress's block editor, leaving the 'Alternative text' field blank correctly outputs alt="" in the HTML, which is the intended behavior. Icons that duplicate adjacent text labels are decorative and need empty alt text, while standalone clickable icons with no visible label are meaningful and require descriptive alt text. Editors can verify correct implementation by previewing a page and checking the source HTML to confirm decorative images display alt="" rather than a missing attribute.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How PgBouncer Pool Exhaustion Silently Kills API Performance

A technical deep dive explains why APIs can time out even when PostgreSQL appears idle, pointing to connection pool exhaustion at the PgBouncer layer as the true culprit. PgBouncer acts as a middleman between thousands of client connections and a limited number of Postgres backends, and the system breaks down when transactions hold connections longer than expected. Common triggers include slow downstream API calls made mid-transaction, lock waits, and runaway transactions — patterns that cause a queue of waiting clients to build rapidly. In transaction pooling mode, the problem is especially deceptive because the pool appears elastic, leading teams to wrongly assume Postgres's max_connections is the binding limit. Developers are advised to monitor the cl_waiting and sv_active metrics via PgBouncer's SHOW POOLS command and to use Little's Law to size pools correctly rather than simply raising default_pool_size.

0 comments Read more at DEV Community