d3-turbo: 26B-parameter model answers many questions from one read, outpacing larger rivals on throughput

·6 views

The vLLM Semantic Router team released d3-turbo on Hugging Face on 19 October 2026, a 26-billion-parameter model with a 460-million-parameter image encoder under an Apache 2.0 license. Rather than generating free-form text, it returns probability distributions over options the user predefines, such as routing a customer ticket or flagging urgency. Its sibling d3-flash re-reads the input for every question, while d3-turbo reads once and answers many, which the author's benchmarks on a single AMD MI325X show becomes faster beyond about four questions. At 255 questions, the reported median latency was 2.12 seconds for d3-turbo versus 32.94 seconds for d3-flash, though d3-flash was quicker for a single question. The model supports choice, yes/no, score, box, and tracking tasks, and accepts up to 131,072 tokens of context when long-context mode is enabled.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Daily app budgets and a visible override counter outperformed blocking for one writer's phone habits

A developer who struggled with full blocks, accountability partners, and cold-turkey attempts over 2024 found each method removed access without addressing the underlying reasons for phone use. Their approach instead assigns per-app daily time budgets, with messaging and maps left unlimited, and prompts once at the limit to either continue or do an intended task first. Each override is recorded in a visible counter without punishment or streaks. Logging from 1 to 21 October reportedly cut late-night phone use from 90 to 12 minutes per night, daily opens from 41 to 19, and weekly overrides from 23 to 6. The writer has released the tool, ChoiceField, as a free iOS app and suggests a week of manual logging as a starting point.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

LLM API 장애에도 서비스가 멈추지 않도록 하는 6가지 설계 원칙

LLM API를 제품에 연결하는 일은 개발 단계보다 운영 단계에서 훨씬 까다로운데, 응답 지연, 네트워크 오류, Rate Limit, 서버 오류 등 다양한 실패가 언제든 발생할 수 있다. 이 글은 이런 실패를 전제로 한 호출 구조를 위해 여섯 가지 원칙을 제시한다. 핵심은 요청 목적에 맞는 타임아웃 설정, 오류 유형에 따른 선별적 재시도, 지수 백오프를 통한 부하 분산이다. 또한 주 모델이 실패할 때 보조 모델이나 규칙 기반 로직으로 전환하는 폴백, 반복 요청에 대한 캐시, 장애 중인 API에 대한 반복 호출을 막는 서킷 브레이커도 함께 다룬다. 마지막으로 응답 시간, 모델별 오류, 재시도 횟수 등을 기록하고 모니터링해야 안정적으로 운영할 수 있다고 강조한다.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Guía paso a paso para instalar Tailscale en un homelab Linux

Este procedimiento explica cómo instalar el cliente de Tailscale en un equipo Linux de un homelab, autenticarlo en la tailnet y verificar su funcionamiento. Como referencia se usa la release v1.104.1, aunque las fuentes no detallan cómo instalar esa versión concreta, por lo que los pasos deben tomarse como el procedimiento general de la documentación. Se requiere un gestor de paquetes compatible, acceso con sudo y el módulo de kernel tun, que puede cargarse manualmente o al arrancar. La instalación se realiza con el script oficial de una línea, copiado de la web de Tailscale, y se autentica con tailscale up, comprobando luego el dispositivo en la consola de administración. Finalmente, la guía recoge opciones como SSH, la gestión de caducidad de claves y las herramientas tailscale ip y tailscale status para revisar el estado.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OpenAI Acquires Orchestra Bio to Help Trace Drug Development Delays

OpenAI announced its October 20 acquisition of Orchestra Bio, describing a use case for coordinating drug development. In its example, a failed manufacturing batch could delay a clinical trial, and Orchestra's agents would help identify affected activities and timelines while scientists and program leaders make the decisions. OpenAI also said it plans to combine Orchestra with its Rosalind product to link scientific findings with studies, decisions and next steps. The company has not published a controlled evaluation or quantified time savings for this workflow. The article also offers a proposed developer design for recording impact claims as inspectable, correctable and withdrawable records, though it is not Orchestra's actual architecture.

0 comments Read more at DEV Community