Why Forcing Determinism on AI Agents Is the Wrong Goal, Experts Argue

·3 views

A founder building agentic AI infrastructure argues that the core weakness of large language models — their inherent probabilistic nature — cannot be fixed by layering on RAG, vector databases, or agentic optimizations. The article notes that nearly 90% of recent YC-backed startups are vertical AI companies stacking solutions on top of foundation models, yet none achieve true determinism. Traditional software produces predictable outputs from fixed inputs, while generative AI models produce variable results by mathematical design. The author contends that industry band-aids like retrieval-augmented generation reduce hallucinations but do not eliminate them, and that each added agentic step introduces more probabilistic decisions. The piece calls on builders to pursue auditability and transparency as realistic goals rather than chasing the commercially appealing but technically impossible promise of deterministic AI behavior.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How a dedicated Elasticsearch index cut autocomplete latency from 5s to milliseconds

A developer building autocomplete over 100 million documents discovered that slow first-call response times of around 5 seconds were caused by a cold-start pipeline, including JIT compilation, HTTP/TLS connection pooling, and loading Lucene segments into memory. Initial benchmarks run on a local dataset were misleading, as database enrichment dominated response times locally, while on the full production-scale environment it was the Elasticsearch aggregation query itself that became the bottleneck. Prefix match, Edge N-gram, and Search As You Type patterns all required a costly terms aggregation to deduplicate results, which proved unacceptably slow at scale. The investigation highlighted that testing autocomplete performance requires production-scale data volumes to identify the true bottleneck. The findings pointed toward adopting a dedicated index strategy to shift work to index-time rather than query-time, dramatically reducing search latency.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Silent data-loss bug fixed in Zulip's Microsoft Teams importer before it struck

A developer contributing to Zulip, an open-source team chat server, identified a subtle batching bug in its Microsoft Teams data importer that had never triggered an error in production. The flaw involved a Python generator that yielded a shared list object, then called `.clear()` on it before the next batch, meaning any consumer retaining multiple batches would silently receive the same overwritten list. In a test dataset, this caused the total message count to collapse from 29 to 24 with no exception raised. The fix, submitted as pull request zulip/zulip#39814, replaces the clear-and-refill approach with a fresh list per batch, matching the contract used by the standard library's own `itertools.batched`. A regression test was also added to catch the data-loss scenario with a concrete assertion.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How LISP's REPL Transformed Human-Computer Interaction in the 1960s

The Read-Eval-Print Loop (REPL) originated from LISP in the 1960s, developed by AI researchers at MIT who needed to test expressions interactively rather than submit entire programs via batch processing. Before REPL, programmers wrote code on punch cards, submitted them to operators, and waited hours or days to see results and discover bugs. REPL compressed that feedback cycle from days to seconds by reading input, evaluating it, printing the result, and looping back for the next command. The term 'read-eval-print loop' was formally documented in the MACLISP Reference Manual by David Moon in 1974. The concept later became foundational to Python's >>> prompt, Node.js console, Jupyter Notebooks, and browser developer tools, fundamentally shifting programming from batch commands to interactive conversation.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Audit and Restrict an AI Coding Agent's Network Access in CI Pipelines

AI coding agents in development environments simultaneously access source code, credentials, and network connections, creating a risk that prompt-injection attacks hidden in READMEs or dependencies could turn the agent into a data exfiltration tool. Most development teams focus on tool-approval controls but neglect outbound network restrictions, leaving a critical gap at the firewall layer. A proposed mitigation involves building a reproducible egress regression fixture — a minimal sandboxed environment that enforces an explicit allowlist of permitted network destinations such as model API endpoints and package registries. The fixture uses a shell script to verify that allowlisted hosts remain reachable while denying access to cloud metadata endpoints, simulated attacker hosts, and paste sites. This network-layer control is designed to hold even if the agent's tool-approval logic is bypassed, and can be integrated as an enforceable check in CI pipelines running on Docker, VMs, or cloud sandboxes.

0 comments Read more at DEV Community