Four Self-Hosted AI Inference Orchestrators Reviewed and Compared in 2026

·4 views

A technical comparison of four self-hosted AI inference orchestration tools — LocalAI, exo, GPUStack, and vLLM — has been published by Nexlab. The review evaluates how each platform handles local AI model deployment and inference workloads. Self-hosted inference tools have gained traction as developers and organizations seek greater control over AI infrastructure without relying on cloud providers. The article aims to help engineers choose the right orchestration solution based on their specific hardware and deployment needs.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Microsoft Foundry's Code Interpreter Actually Runs Python Inside a Sandbox

Microsoft Foundry's Agent Service includes a Code Interpreter tool that spins up an isolated Python container to perform deterministic computations, rather than relying on a language model to reason through calculations. The system addresses a core weakness of large language models, which are prone to errors in arithmetic, string manipulation, and large dataset operations. When an agent invokes the tool, Foundry lazily provisions a sandboxed container, stages files through Azure Storage, executes the code, and returns verified output with no network access to external infrastructure. Three key components govern the system: the tool definition, the container environment, and the session, which is a billable, time-limited runtime unit. Understanding the distinction between auto-managed and explicitly managed containers, as well as the difference between prompt agents and hosted agents, is critical for developers deploying the feature at production scale.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Atlassian Forge JSM Portal Panel Silent Submit Bug Traced to Wrong Payload Shape

Developers using Atlassian's Forge platform encountered a silent failure when submitting data via the jiraServiceManagement:portalRequestCreatePropertyPanel module, where issue properties returned as undefined without any error or warning. The root cause was an incorrectly shaped payload passed to the view.submit() call, which the @forge/bridge package's loosely typed submit signature — accepting any object — failed to catch at compile time. Atlassian's own manifest-reference page documents the required shape: an object with a fields array of key-value objects and a required isValid boolean. Developers are advised to consult this documentation before writing their submit handler, as the package types alone will not flag a malformed payload. A separate, still-unresolved challenge involves populating a real Jira custom field from the stored property value after submission.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OKX Governance Layer Rated 7/10 Risk in DeFi Security Audit

A DeFi security research team has published a governance attack surface review of OKX, which manages approximately $31.2 billion in total value locked across Ethereum and Layer 2 networks. The audit identified nine distinct attack vectors spanning the protocol's on-chain proposal, voting, and execution pipeline, as well as its off-chain processes including multi-sig coordination and oracle feeds. Among the most critical findings are flash-loan-driven voting exploits, where borrowed OKB tokens could be used to pass malicious proposals within a single transaction. Analysts also flagged that the top 10 OKB holders control roughly 30% of supply, enabling potential whale collusion, while timelock windows as short as zero to 30 minutes leave insufficient time for community or auditor intervention. The report assigns an overall risk score of 7 out of 10 and calls for stronger timelock safeguards, improved multi-sig key management, and measures to reduce voting power concentration.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OpenZFS bug causes mass segfaults and silent outages; fixed in versions 2.4.4 and 2.3.9

A kernel-level arithmetic flaw in OpenZFS versions before 2.4.4 and 2.3.9 caused widespread, seemingly random process crashes on affected hosts. The bug in zfs_fillpage() triggered an unsigned integer underflow when a file was truncated while an mmap page fault was in progress, causing the kernel to zero-fill large swaths of physical memory belonging to unrelated processes. The resulting crash of systemd's PID 1, combined with its default CrashAction=freeze setting, left affected machines completely unresponsive for hours while still technically running. From the outside, the server appeared powered off — no SSH, no HTTP responses — though journal logs confirmed it had been silently failing the entire time. OpenZFS released fixes on 21 August 2026 in versions 2.4.4 and 2.3.9, and administrators can limit outage duration by setting CrashAction=reboot in a systemd drop-in file.

0 comments Read more at DEV Community