LLM Routing in 2026: How to Choose Between Managed Gateways and Self-Hosted Libraries

·8 views

The choice between managed API gateways and self-hosted routing libraries for large language models depends primarily on who controls the infrastructure rather than the number of supported models. LiteLLM, which documents support for over 100 providers, suits teams running their own GPU clusters, though it requires engineers to handle updates, secrets, and scaling themselves. OpenRouter, which raised $113 million in a Series B round at a $1.3 billion valuation, offers a single API key abstracting multiple vendors, making it strong for model discovery but at an added cost margin over direct provider calls. Managed options like Portkey, Cloudflare AI Gateway, and Vercel AI Gateway prioritize observability and operational simplicity, but tie users to their respective ecosystems. A lightweight local-first client category also exists for users seeking immediate usability without server setup, though it lacks observability dashboards, custom integration hooks, and transparent upstream provider disclosure.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How DBAs Safely Identify and Remove Unused SQL Server Indexes

Unused database indexes accumulate over time as application features change, queries are replaced, or workloads evolve, leaving tables burdened with indexes that no longer serve their original purpose. While SQL Server's sys.dm_db_index_usage_stats can flag indexes with no recorded reads, experts caution that usage counters reset on server restart and some indexes only support periodic workloads like month-end reporting or annual audits. Even an unread index continues to impose a performance cost, as SQL Server must update every relevant index on each insert, update, or delete operation, increasing write overhead and maintenance workload. A 2023 DBA Stack Exchange discussion highlighted cases where tables had accumulated over 70 indexes, illustrating how incremental fixes can linger long after the original problem is resolved. The recommended approach is to gather evidence across multiple business cycles, remove indexes one at a time, monitor workload impact, and maintain rollback scripts before making any permanent changes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer uses Claude AI to rebuild portfolio, stresses human judgment remains essential

A developer rebuilt their personal portfolio using Claude Code, an AI coding assistant, while retaining full control over all key design and technical decisions. The project used a modern stack including TypeScript, React 19, Next.js 16, Tailwind CSS 4, and Framer Motion, deployed on Vercel. Rather than jumping straight to new features, the developer prioritized an audit of SEO, accessibility, and performance before making any changes. Accessibility improvements included pause controls for carousels and properly sized touch targets, while SEO work covered canonical metadata, a sitemap, and Google Search Console verification. The finished site scored near-perfect Lighthouse ratings, and the developer emphasized that AI tools require human direction, measurement, and critical review to be genuinely effective.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Docker Compose Can Create Reproducible AI Agent Evaluation Environments

Inconsistent AI agent evaluation results between local and CI environments are often caused by differences in tool responses, database state, clock settings, and dependency versions rather than the model itself. A Docker Compose-based evaluation lab can eliminate much of this accidental variability by providing a versioned, controlled environment for replaying synthetic test cases. The setup uses Compose profiles to separate core services from optional components like fault injectors, telemetry collectors, and local model runners, keeping baseline runs lightweight. Container images are pinned by digest, dependencies locked, and a JSON manifest records all attempted conditions — including seed, clock, prompt, and policy revisions — for each evaluation run. Structured evidence artifacts are written to a mounted directory after each run, making behavioral regressions easier to identify and distinguish from infrastructure-level changes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

VMware Quietly Retreats from SmartNIC-Based Distributed Firewall Sales

Broadcom VP Umesh Mahajan confirmed at VMware Explore this month that VMware has stepped back from SmartNIC-based network offload, the technology underpinning its Distributed Services Engine's distributed firewall. VMware has stopped actively selling the SmartNIC-resident distributed firewall, which was designed to enforce micro-segmentation at line rate without consuming host CPU resources. While the Distributed Services Engine itself remains part of VMware Cloud Foundation and continues to be supported, the core commercial proposition has been withdrawn. Broadcom's lifecycle documentation further confirms that Network Introspection for Security, a related SmartNIC security capability, will be discontinued after the final NSX 4.2.x release or by October 11, 2027. The retreat reflects a market reality where the hardware dependency, added firmware lifecycle, and new failure domains the technology introduced failed to justify its value broadly enough to sustain sales.

0 comments Read more at DEV Community