SaaS automotive AI assistant cut response time from 30 seconds to under 3 via pipeline redesign

·1 views

A US-based automotive SaaS company was struggling with an AI customer assistant that took up to 30 seconds to respond while searching a library of roughly 40,000 PDF documents totalling over 120 GB. Engineers found the bottleneck was not the language model itself but four upstream inefficiencies: retrieving too many passages, sending oversized prompts, checking user permissions after retrieval, and performing no caching between sessions. The team rebuilt the system as a custom retrieval-augmented generation pipeline on Google Vertex AI with Gemini, adding agentic orchestration and voice capabilities. The key architectural change was placing intent detection first, so the system identified the type of request before deciding what to retrieve, which documents the user was authorised to see, and how much context to pass to the model. The redesign brought response times down to under three seconds while also reducing costs and preventing potential data-permission leaks.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Validate Address Data at Entry Point to Prevent Downstream System Errors

Incorrect or incomplete address data entered by users can quickly spread into billing, fulfillment, and mailing systems if not caught early. Experts recommend validating data as close to the point of entry as possible, rather than relying solely on batch cleanup processes that leave a window for errors to propagate. One effective approach uses real-time API-based validation, where a submitted address is checked and standardized before being saved to the database. This architecture keeps validation logic separate from core application code and supports use cases like ecommerce checkouts, CRM systems, and customer registration forms. Treating data quality as part of the application workflow — rather than an afterthought — reduces downstream cleanup and improves the reliability of connected systems over time.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

IoT Botnets Still Exploit Open Telnet and Default Passwords a Decade After Mirai

China's National Network and Information Security Notification Centre issued a notice in August 2026 identifying five active cross-border botnet families — Mirai, CondiBot, Gafgyt, TBot, and SoftBot — all gaining initial access through Telnet or SSH brute force against factory-default credentials and unpatched older vulnerabilities. Internet scanning tools like ZoomEye show that a significant number of devices continue to expose Telnet on port 23 to the open internet, forming the recruitment pool these botnets rely on. Analysts note that raw counts of reachable Telnet services overstate actual vulnerable devices, as the figures include honeypots, reconfigured network equipment, and lab systems hosted in cloud environments. Three structural factors keep the problem persistent: devices outlive vendor support cycles, ownership and maintenance responsibilities are often unclear in small organisations, and remote-access interfaces enabled for convenience are rarely disabled post-setup. While newer botnets like Dysphoria have adopted advanced techniques such as blockchain-based command infrastructure to resist takedowns, the initial entry method has remained unchanged since the original Mirai outbreak in 2016.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

APC Recommends 'model: inherit' to Keep AI Agent Definitions Portable

A design guideline for APC (a repository-owned context layer for AI agents) recommends using 'model: inherit' in agent definition files rather than hardcoding a specific AI model. The approach separates project knowledge — such as an agent's role, skills, and responsibilities — from operational details like provider accounts and available models, which can vary between contributors. When a model is set to 'inherit', the runtime resolves the actual model using its own configured defaults, rather than forcing every user who clones the repository to meet a potentially unavailable or unaffordable model requirement. Hardcoding a vendor-specific model is only advised when the task genuinely requires it, such as when a workflow depends on a model-specific capability agreed upon by the team. Project maintainers are encouraged to periodically audit agent files and replace incidental model choices with 'inherit', documenting any justified exceptions clearly.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Solana's 90% Rent Cut Leaves Millions of Old Accounts Holding Surplus Funds

Solana is reducing account storage costs by 90% across five phased steps, meaning accounts funded before each price reduction now hold more lamports than required for rent exemption. The byte price has already dropped from 6,960 to 5,080 lamports since September 2026, with three more reductions expected in the Agave 4.4 release. Account holders can reclaim the surplus without closing their accounts using program-specific withdrawal instructions, provided the account retains at least the current rent-exempt minimum. The recoverable amounts are modest per account — ranging from roughly 0.00039 SOL for a nonce account to 0.024 SOL for a vote account at the final step — but can become significant for wallets, validators, or programs managing thousands of accounts. Only accounts created before each price step are eligible; all newly created accounts are funded at the current lower rate and have no surplus to reclaim.

0 comments Read more at DEV Community