Why LLM Apps Succeed in Demos but Break Down in Production

·1 views

Developers building LLM-powered apps often test with carefully crafted queries and ideal conditions, masking critical weaknesses that only surface when real users interact with the system. Common failure points include vague or malformed user inputs, prompt injection attacks, and slow response times that frustrate users even if average latency appears acceptable. Unexpected costs can also spiral quickly, with API bills reaching thousands of dollars monthly when caching and model routing are not implemented. Edge cases such as large file uploads, stale embeddings, and multilingual queries can crash systems that handled demo scenarios flawlessly. Experts recommend building diverse evaluation sets, load testing before launch, implementing semantic caching, and logging every failure to progressively harden the application for production use.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Google Replaces Direct URLs in Search Results with Redirect Links

Google has changed how URLs appear in its search results, moving away from providing direct links to websites. Instead of linking users straight to a destination page, Google now routes clicks through its own redirect system. This shift means users can no longer see or access a clean, direct URL simply by hovering over or clicking a search result. The change has drawn attention from developers and privacy-conscious users who prefer transparent, untracked links. Google has not publicly detailed the full reasoning behind the move, though it likely supports better click tracking and data collection.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Why AI Coding Tools Need Transparent Pricing and Task-Based Model Selection

A software developer argues that AI coding agents lack the cost transparency and accountability that other trades take for granted, often trapping users in a sunk-cost cycle of iterative prompting without clear delivery guarantees. Unlike traditional software marketplaces where engineers were rated publicly and priced honestly, current AI tools normalize vague estimates and unpredictable outcomes. The author also raises concerns about widening access inequality, noting that frontier model hardware grows more expensive rather than more affordable over time. To address this, the developer proposes an 'intelligence ladder' — a framework that maps tasks to specific model requirements and cost estimates before work begins. After a year of development, a working version of this system is now ready to be shared publicly.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

pdlc-skills Adds Three Tools to Track Project Status, Change Impact, and Trends

The pdlc development lifecycle framework offers three visibility tools — a statusline, /pdlc-status, and /pdlc-retro — designed to monitor software projects across different timescales. The statusline displays a feature's current stage, next step, run mode, test results, and time spent in the current stage in a single compact line. The /pdlc-status command verifies project data against changelogs and git tags, flagging features that have stalled without shipping. A relation-mapping tool, /pdlc-relate, automatically identifies how features connect through six relationship types by scanning requirements text for dependency language. All three tools read from shared state files under docs/.pdlc-state/, meaning the accuracy of their output depends entirely on how well those files are maintained.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Better AI Answers Come From Cutting Context, Not Adding More

A software developer argues that reducing what goes into an LLM's context window typically improves output quality more than expanding it. Because language models attend to all provided tokens equally, irrelevant material competes with useful information and can anchor the model on wrong details. Content buried in the middle of a long context also receives less model attention than material placed at the start or end, meaning placement matters as much as inclusion. The author recommends a workflow of generous retrieval followed by aggressive trimming — removing full files, stale conversation history, and boilerplate — keeping only the task, the directly relevant facts, and the desired output format. This approach simultaneously improves answer accuracy and lowers token costs, making it one of the few optimizations that benefits both quality and budget.

0 comments Read more at DEV Community