Developer adds a lightweight gate to cut LLM costs on a Hebrew stock-review chatbot

·2 views

A developer running telavivstocks.com, a Hebrew dashboard for Tel Aviv Stock Exchange stocks, found that their AI assistant ran a full embedding, vector search and generation for every question, including off-topic ones. To fix this, they placed a small decision model, TypeSafe's Jev, in front of the expensive model to decide whether it should be called at all. The gate classifies each question by intent and checks for prompt-injection attempts, returning probabilities that can be compared against a threshold. The developer estimates the cost at about two cents per thousand questions, and wrote the gate instructions in English while keeping visitors' Hebrew questions unchanged.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Argues Tool Engineering Will Replace Prompt Engineering as AI Skill of Choice

A developer writer contends that prompt engineering, once a prized skill, is becoming obsolete as language models grow more capable. He points to three drivers: models now grasp plain-language requests without elaborate scaffolding, very large context windows reduce the need for compression tricks, and fine-tuning and retrieval-augmented generation solve problems that prompt hacks once addressed. He proposes a successor discipline, Tool Engineering, centered on building, selecting, and orchestrating the tools AI agents use to act in the real world. The piece argues that a model without tools can reason but cannot take actions such as booking a hotel or running a database query. Developers who keep investing in prompt techniques, he warns, risk becoming increasingly irrelevant.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Recording Free Inference Calls in a Ledger Before Making Them

The article argues that free inference lanes fail less on quality than on accounting, since unrecorded spend cannot be subtracted from a budget. It proposes placing a ledger in front of the lane and recording each reservation before the upstream call is made. Because workers can die mid-call and retries can double spend, the ledger must survive crashes without manual cleanup. The author describes a lease-based design where each job holds an open budget row that is committed or released, with a sweeper expiring stale leases. The implementation uses only the Python standard library and a single SQLite file, suitable for a laptop or small always-on machine.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Google Unveils Gemini 4 Argon With 1M-Token Output, Rolling Out Gradually

Google DeepMind announced Gemini 4 Argon on September 30, 2026, positioning it for complex software engineering, knowledge work, and defensive cybersecurity. Its standout specification is an output limit of up to 1 million tokens, raised from 64K, which could support long multi-step tasks. Access is initially limited to trusted cyber defenders and testers, with broader developer and enterprise availability still to come. Google reports a 77.9% score on DeepSWE v1.1 along with an introductory API price of $2 per million input tokens and $10 per million output tokens. The article cautions that these are publisher-reported figures, not a controlled head-to-head comparison with OpenAI's GPT-6.1 Sol.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Log Model Call Metadata Instead of Payloads to Reduce Data Leak Risk

The article argues that AI features often leak sensitive data through logs rather than request bodies, since logging receives less security scrutiny. It proposes logging only a "shape" record for each model call, including a payload digest, byte size, field names, latency, and attempt count, instead of message content. Encryption is deemed insufficient because logs are decrypted when searched, exported, or shared in support bundles. The approach is enforced with a CI test that scans log fixtures for disallowed fields and sensitive patterns such as emails, bearer tokens, and private keys. The author also recommends including a deliberately leaky fixture to confirm the scanner actually detects leaks.

0 comments Read more at DEV Community