Three Questions That Decide When to Use AI vs. Deterministic Code in Software
A software developer building an LLM-powered support agent has outlined a practical framework for deciding which components should use AI and which should rely on deterministic code. The approach rests on three questions: whether the answer is a fact or a judgment, what a wrong answer would cost, and whether a test can pin the correct output today. High-stakes decisions like refund eligibility are handled by rule-based software, while tasks like interpreting user intent are delegated to the language model. Retrieval is treated as a hybrid case, where the AI decides when to search but a hard software contract governs how the search function is invoked. Critically, the risk classification of every action is defined statically in code rather than determined at runtime by the model, preventing prompt injection from shifting decision boundaries.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in