SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Smolagents sandbox bug caused agents to retry valid Python code in endless loop

0
·1 views

A developer discovered a bug in smolagents' LocalPythonExecutor sandbox where valid Python dict unpacking syntax using ** caused repeated failures with a misleading error message. The sandbox incorrectly tried to evaluate the None key used internally by Python's AST to signal a spread operation, triggering an 'InterpreterError: NoneType is not supported' message even though the model's code contained no None values. Because the error message pointed to a nonexistent problem, the AI agent could not correct its code and retried the identical valid syntax multiple times, wasting LLM calls and step budget. The root cause was that the executor walked dictionary keys blindly without checking for None spread markers before calling evaluate_ast. The fix involved processing key-value pairs together, treating a None key as an instruction to merge a mapping rather than as an expression to evaluate.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

How one dev team built a single AI tutor routing 60+ features across five subjects

A development team built a multi-subject AI tutoring platform covering general academics, Indian competitive exams like JEE and NEET, language learning up to CEFR C1, career coaching, and an arena mode. Rather than letting each feature manage its own model calls, the team centralized all AI-backed features into a single registry with defined pricing, model chains, and fallback logic. A cost audit revealed that 'thinking tokens' were the biggest expense, prompting a full architectural redesign of the routing layer. The team also discovered a critical flaw where mastery scores never updated because quiz triggers relied on undocumented user-typed keywords, which was fixed by adding explicit UI controls and a shared progress engine. Key engineering lessons included treating response caching as an allowlist, issuing automatic refunds when cheaper fallback models respond, and ensuring errors surface visibly to users rather than silently rendering blank screens.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds AI Memory Agent With Intentional Forgetting, Benchmarks It Against Naive Systems

A developer built an AI memory agent called Synapse for the Global AI Hackathon Series on Qwen Cloud, designed to selectively forget outdated information rather than storing everything indefinitely. Unlike conventional approaches that treat all data equally in a vector database, Synapse assigns each memory a decaying salience score based on importance, recall frequency, and elapsed time. The system uses two different decay rates — around 72 hours for episodic details and 30 days for stable facts like preferences — so it forgets casual conversation while retaining meaningful information. A background process periodically consolidates repeated mentions into single memories and detects contradictions, automatically retiring stale facts when newer ones supersede them. The developer benchmarked the system against a naive vector-store approach using real Qwen Cloud API calls throughout, with no mocked responses, to validate the claims in practice.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Engineering team cuts RAG pipeline latency 40% using Bayesian search and hybrid retrieval

A development team rebuilt their retrieval-augmented generation (RAG) pipeline from scratch after standard production deployments proved inadequate for legal, API, and support-ticket use cases. The team replaced fixed 512-token chunking with document-type-specific strategies, including recursive, semantic, and agentic chunking, achieving recall@10 scores between 91% and 97%. They combined vector search with BM25 and cross-encoder reranking using Reciprocal Rank Fusion, which improved relevance correlation from roughly 0.75 to 0.92. A Bayesian-informed search optimization was also applied, ultimately reducing overall query latency by 40%. The team published their configurations and Python code to help other engineers replicate the tunable retrieval pipeline.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How One Team Built Automated LLM Evaluation That Catches 92% of Hallucinations

A development team discovered critical gaps in their AI quality process after deploying a RAG-based customer support assistant that served hallucinated responses to over 500 users before the issue was detected. The failures included fabricated billing policies and rate-limit figures pulled from a competitor's documentation, exposing the team's reliance on informal, manual testing. In response, they built a production-grade evaluation pipeline featuring a suite of automated judges assessing faithfulness, instruction-following, schema validity, safety, and domain accuracy. The system integrates directly into CI/CD workflows, blocking code merges that degrade output quality and enabling regression detection in real time. The team reports the pipeline now catches 92% of hallucinations before deployment, replacing subjective human review with structured, metric-driven evaluation.

0 comments Read more at DEV Community