Agent Picks Wrong Tool in 50-Tool Menus, Accuracy Drops to 55%

·6 views

A developer tested whether giving an AI agent more tools makes it worse at selecting the correct one. Using a fixed model and 20 scenarios, accuracy fell from 80% with 5 or 10 tools to 75% with 25 and 55% with 50. Most errors at the largest menu size were calls to tools unrelated to the request, with one tool, find_contact, absorbing most of them. The author also found that accuracy relative to random guessing improved as menus grew, so the raw drop reflects more options rather than a model that simply got worse. The findings align with prior research reporting similar declines in tool-selection accuracy as tool counts increase.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

DotNetPip Brings Python Dependency Management into .NET MSBuild Pipelines

A developer has released DotNetPip, an open-source MSBuild integration that manages Python dependencies directly within C# build pipelines. The tool addresses the tedium of setting up CPython runtimes, restoring pip packages and virtual environments, and handling module search paths and native binaries when calling Python from C#. Developers declare required Python packages in their .csproj file instead of relying on external setup scripts. Running dotnet build triggers custom MSBuild tasks that resolve these dependencies automatically, after the DotNetPip package is added from NuGet. The project is hosted on GitHub, and its author invites feedback, edge-case reports, and contributions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Checklist for Sideloading Android Apps: Permissions, Signing and Version History

Installing Android apps from outside the Play Store can help with regional availability or older devices, but it removes the store's safety checks and shifts responsibility to the user. The article advises questioning each permission request and asking whether the app's core function truly requires it, especially for sensitive access like contacts, SMS, or device administration. It recommends confirming that a new version is signed with the same key as the installed app and that the app has a clear source for future security updates. Users should also review version history and changelogs for gaps, sudden jumps, or missing fixes, and check the target Android version and package name against the official release. The piece concludes that careful checks can reduce risk while keeping the flexibility of installing apps outside the store.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Exposed windows_exporter endpoints call for a host inventory and tighter access controls

The Windows exporter publishes host metrics for Prometheus and commonly listens on port 9182. A reachable endpoint can reveal operating-system details, service states, process names, and resource usage, so it should be treated as internal telemetry rather than a public page. Search results from November 5, 2026 showed six port matches and twenty body matches, but the article stresses these are different observations and do not confirm a population of exposed hosts. Recommended safeguards include limiting enabled collectors, restricting textfile collector directories, binding the listener to monitoring interfaces, and allowing only the Prometheus server through the firewall. Defenders are advised to maintain an inventory of owned hosts running the exporter, including their host roles and configuration details.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Third-party Claude API proxies may fake prompt caching savings, study warns

A developer who audited more than a dozen third-party API relays reports that many advertise Claude prompt caching but fail to deliver it. Some proxies rotate requests across keys without session affinity, so cache writes repeat and users pay higher rates. Others strip cache control blocks and bill standard rates, or inject fake cache-read numbers into usage data. Even small changes to the system prompt can invalidate caches, and the article notes caching loses money if hit rates fall below about 22%. The author recommends a simple two-request test script and demanding per-request transparency from providers, and discloses that they run a competing routing gateway.

0 comments Read more at DEV Community