SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

AI vs. Code: Why High-Frequency Logic Should Stay Deterministic

0
·34 views

A software experiment building an LLM-powered support agent found that routing high-frequency decisions through AI models carries a steep hidden cost. Running a simple refund eligibility check 50,000 times daily via Claude Sonnet costs roughly $29,200 a year, compared to fractions of a cent for the same logic written as a Java method. The cost gap between an AI call and a deterministic code check is approximately one billion to one. The author argues the real boundary is not about capability but frequency: AI earns its cost for one-off judgement calls like parsing natural language, while rule-based decisions that run per request, per retry, or per rule evaluation belong in code. The finding suggests that what initially looks like a correctness design decision also turns out to be the economically rational one.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Seccomp Filters Should Validate Architecture Before Checking Syscall Numbers

A seccomp policy can appear complete yet leave an alternate syscall path open, because filters see only a syscall number and architecture identifier rather than semantic function names. The same integer can mean different things under different ABIs, so a policy checking only the syscall number can be bypassed. Architecture validation should therefore occur at the start of the filter, as shown in Linux seccomp(2) documentation, with unknown architectures rejected. The article recommends choosing default actions matching the risk level, maintaining separate filters and tests for each ABI, and failing closed when instructions cannot be classified. Seccomp is also limited because it cannot inspect pointer contents, so it is usually paired with namespaces, file permissions, and capabilities.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Perfect Annotation Agreement Can Hide Sparse Labelling Coverage

A single 100% agreement figure can make two annotation files appear more complete than they are. In the bundled partial example, only one of twelve records is labelled by both sides, so conditional agreement reads 100% while both-labelled coverage is just 1/12. The metric counts only records labelled by both sides, so it ignores the remaining records entirely. The article's review tool therefore checks that both checkpoints share the same records, taxonomy, and order before comparing them, and it keeps manual decisions separate from the original annotations. It also stresses that agreement is not accuracy and that the two annotation roles are not verified as independent people.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Guide Outlines C# Techniques for High-Throughput RAG Pipelines and Vector Database Tuning

A DEV Community article presents an architectural blueprint for building high-throughput agentic and retrieval-augmented generation pipelines in C#. It covers vector database tuning, including IVF cluster parameters such as nlist and nprobe, alongside HNSW indexing. The piece explains how Span, ReadOnlySpan, and pooled memory buffers can reduce garbage collection overhead during heavy vector operations. It also includes a SIMD-accelerated cosine similarity example and describes a two-tier L1/L2 caching model to limit redundant similarity queries. The author reports benchmark results indicating a reported throughput gain of about 4.8x over default settings when HNSW tuning and caching are combined.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds Neural Network Basics in Plain JavaScript to Understand How AI Learns

A JavaScript developer set out to understand how AI models learn after realising that using AI tools was not enough to explain their inner workings. Rather than relying on frameworks like TensorFlow or PyTorch, they chose to implement the fundamentals from scratch in plain JavaScript. They began with a simple task of teaching a program to predict the sum of two numbers from examples, illustrating the core idea of supervised learning. The article then explains how a neuron combines weighted inputs with a bias, and how a loss function measures prediction error so the model can adjust itself. The author presents the approach as a step toward eventually building a GPT-style model.

0 comments Read more at DEV Community