Sapling RCM switches search router from chat model to Jev, cutting latency and cost

·2 views

Sapling RCM, a medical billing app, added record search where users press ⌘K and query across eight tables, and a small model routes each query to the relevant tables. The team compared Jev, a non-chat probability model from TypeSafe served via OpenRouter, against Gemini 3.5 Flash-Lite. Jev's median routing call took 466 ms versus 900 ms for Gemini, and it cost about $23 per million searches compared with $170. Jev missed results on 3 of 27 searches, while Gemini missed on 6 of 30. The approach asks the model what kind of text was entered, such as a person's name or claim number, and maps those types to tables in application code.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Vox Republishes 2014 Article on FBI Letter Sent to Martin Luther King Jr.

A 2014 Vox article, shared on Hacker News, examines a letter the FBI sent to Martin Luther King Jr. The letter was reportedly written in an effort to persuade him to take his own life. The source material provided here contains only the article link and discussion metadata, so the letter's full contents and context are not described. Readers are directed to the Vox article for the complete text and historical background. The item gained modest attention on Hacker News, with 49 points and 6 comments.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Exposing a local MCP server: tunnels are easy, access control is the hard part

A DEV Community article argues that making a local MCP server reachable is the simple step, while deciding who can call it and revoking access without redeploying takes most of the effort. It explains that stdio transport only works for one person on one machine, so remote use needs HTTP or SSE transport. The author outlines four needs: a stable address, transport, authentication, and revocation. Five approaches are compared, including ngrok, Cloudflare Tunnel with Access, exposing a port directly, and moving the server to a VPS. The piece ends by describing an outbound-relay model in which the server connects out to a relay and nothing listens on the local machine.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Prewarming a Voice Agent Cut Model Costs but Not First-Reply Latency

A voice agent team found that prewarming the LLM before calls connected accounted for nearly 60% of their model spend, even though many calls ended before any conversation began. Most calls failed at voicemail, no answer, or within a turn or two, and retries made up about a third of volume with a high failure rate. They disabled prewarm for retries, and latency for prewarmed and non-prewarmed calls turned out nearly identical, with p50 around 1.2 seconds in both cases. A controlled test reproduced the same result, and prompt caching was already hitting about 98% on prewarm replies, so the prompt was warm regardless. The finding suggested the expected speed benefit was not materializing, prompting a closer look at where the delay actually came from.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Ghost in the Terminal: A Local AI Storyteller Built for Hacktoberfest

A developer has created Ghost in the Terminal, a command-line program that answers questions in the voice of a spooky ghost. It is built as an entry for the Hacktoberfest Open-Source AI Challenge and is designed to tell eerie stories about abandoned servers and forgotten programmers. The app runs the Gemma 3 1B open-weight model locally through Ollama, using Python to communicate with Ollama's local API. Because it runs on the user's own machine, no cloud AI API key is needed and prompts stay on the device. The project's source code is available on GitHub for others to download and modify.

0 comments Read more at DEV Community