ExecuTorch MLX Delegate Runs Qwen3 Up to 4.52x Faster Than PyTorch MPS on Apple Silicon

·1 views

A developer tested Apple Silicon inference speeds using ExecuTorch 1.3.1's experimental MLX delegate, released May 18, 2026, with the Qwen3-0.6B language model. Benchmarks showed decode throughput of 41.8 tokens per second with PyTorch MPS BF16, rising to 134.8 with MLX BF16 and 188.9 with MLX INT4. The INT4 quantized model was 4.52 times faster than the PyTorch MPS baseline and produced a file 71.8% smaller than its BF16 equivalent. However, 4-bit quantization altered the generated text output in two out of three test prompts, indicating a trade-off between speed and output fidelity. The MLX delegate, which routes PyTorch computation graphs to Apple's MLX framework for GPU execution, remains experimental and subject to API changes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Modern AI Agents Work: Harness, Tools, Skills, Memory and MCP Explained

Modern AI agents have evolved beyond simple chatbots into systems capable of reasoning, planning, and autonomously completing complex tasks using multiple integrated components. A production AI agent combines a language model with a harness, tools, skills, memory, and Model Context Protocol (MCP) integrations, each serving a distinct role. The agent harness acts as the execution layer, coordinating task management, tool access, security, and memory retrieval to bridge the gap between reasoning and real-world action. Tools allow agents to interact with external systems like databases and APIs, while MCP provides a standardized protocol for discovering and connecting to those systems without custom-built integrations for each one. Skills add a higher layer of reusable workflows and domain knowledge, enabling agents to handle sophisticated, multi-step tasks efficiently.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer adds unit-price billing and alias matching to payapp and reconmatch tools

A developer has released updates to two open-source financial utilities, payapp and reconmatch, addressing limitations users encountered after initial releases. Payapp version 0.13 introduces unit-price schedule support, allowing quantity multiplied by unit price calculations alongside quantity overrun validation and journal-style workpapers for contractor and owner reporting. Reconmatch versions 0.12.1 and 0.13 add regex-based transaction filters, an alias mapping feature that rewrites bank tokens to book names for cleaner reconciliation, and a new month-end workflow guide. Both tools remain offline and rely solely on Python's standard library, with no external dependencies or live bank feed integration. The updated projects are available on GitHub under the developer's repositories for payapp and reconmatch.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OmniRoute AI Gateway Fallbacks Must Be Tested for Semantic Consistency

OmniRoute is an MIT-licensed AI gateway that routes requests across multiple providers with quota-aware fallback and token compression, but a successful HTTP response does not guarantee consistent behavior across models. When a fallback model activates, differences in tool-call formatting, system-role handling, context limits, and streaming order can silently alter workflow outcomes. Developers are advised to pin specific provider and model IDs, validate exact tool-call fields, test context boundary preservation, and ensure mid-stream failovers emit explicit retry or route-change events rather than merging partial responses. Each fallback event should carry a request ID, route revision, provider identity, and terminal status so consumers can deduplicate correctly. If a fallback model lacks a required capability, the recommended approach is to fail with a typed error rather than return a lower-quality but syntactically valid answer that could cause downstream state mutations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Agent Repo Lacks Role Handoff Cards, Risking Unclear Accountability

The open-source repository msitarzewski/agency-agents offers MIT-licensed AI agent roles with distinct personas such as frontend specialist and product manager. While named roles are easy to understand, the project currently has no structured mechanism to document what authority or responsibility transfers when work moves between agents. A contributor proposes a formal handoff card system that would record who passed work, to whom, why, what evidence was gathered, and what authority is being requested. The proposal also outlines a research protocol to test whether users can correctly identify role boundaries and distinguish read-only reviews from actions that mutate state. The author notes this is a design proposal and has not been validated against the repository's actual interface or tested with real users.

0 comments Read more at DEV Community