Study of 17,000 AI Coding Agent Runs Reveals Tool Preferences of Claude, Codex and Cursor

·3 views

A study conducted by Armature Tech analyzed approximately 17,000 runs across three major AI coding agents — Claude, Codex, and Cursor — to determine which tools each system prefers to install and use. The research aimed to provide empirical data on the real-world behavior of these agents during coding tasks. Findings were published on the Armature Tech blog, offering developers insight into the default tendencies of popular AI coding tools. Such data can help engineering teams make more informed decisions when choosing or configuring AI-assisted development environments.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How One AI Firm Cut Model Spend by Routing Tasks Away From GPT-4o

An AI company discovered that GPT-4o was handling 77% of its production traffic while consuming 97% of its total model spend, revealing a costly imbalance in how tasks were assigned to language models. The disparity arose not from deliberate choices but from three systemic forces: demos built on the strongest model becoming permanent defaults, no per-task cost visibility in monthly bills, and asymmetric blame that punished cheap-model failures but never questioned frontier-model overuse. To address this, the company introduced a written routing policy that reserves frontier models for open-ended, high-stakes, or judgment-heavy tasks, while directing structured, verifiable work to cheaper alternatives. The key distinction driving the policy is whether a task executes an existing plan — which cheaper models handle well — or requires deciding the plan, where errors are costly to detect and reverse. The company also cautions that the right success metric is cost per completed task, not cost per call, since a cheaper model requiring multiple retries can negate its savings.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Confidence-Gated Image Recognition System Using Five Specialized AI Models

A developer has built TargetV1, a personal image recognition pipeline that uses five specialized AI models — DINOv3, SAM, Moondream2, CLIP, and BLIP — each assigned a distinct task rather than relying on a single model to handle everything. The system is designed to acknowledge uncertainty and self-verify instead of producing confident but incorrect outputs. Development hit an early roadblock when PyTorch lacked compiled CUDA kernels for the new NVIDIA Blackwell GPU architecture, which was resolved by installing a newer PyTorch build targeting the cu128 runtime. A second setback arose from manually cloning model repositories, which caused Windows path conflicts, corrupted weight downloads, and mismatched layer names, problems that were ultimately bypassed by switching to Hugging Face's transformers library. The project highlights practical challenges developers face when working with cutting-edge hardware and fragmented model tooling outside mainstream tutorials.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Neve Language Aims to Unify High-Level and Low-Level Deep Learning Development

A developer known as No Saved DATA has introduced Neve, a programming language designed to handle the full deep learning stack — from preprocessing to GPU kernels — within a single high-level syntax inspired by Python. The project was motivated by limitations in PyTorch and Python's Global Interpreter Lock, which force researchers to rely on C, C++, or Rust for performance-critical tasks. Neve runs on an LLVM JIT backend and already delivers performance competitive with NumPy, OpenBLAS, and Python's SentencePiece in early benchmarks. The language currently supports automatic differentiation, parallel data workers, SIMD code, and a GPU kernel interface, though the complete deep learning framework is still months away. Documentation and source code are publicly available, and the developer is actively seeking community feedback via Discord and GitHub.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Six MIT-licensed AI workflow kits released with built-in evidence verification

A software project called Software Sausage has released six open-source AI workflow kits under the MIT license, available at version v0.17.0 on GitHub. Each kit includes a README, an editable evidence ledger, and a dependency-free shell verifier, with all 17 verifiers confirmed passing at release. The kits address tasks such as coding agent oversight, prompt evaluation, document parsing, UI bug reproduction, and dependency updates. Developers note the checks verify that required files and fields exist, making these structural rather than performance guarantees. Newly added workflows are explicitly marked as unbenchmarked until measured results are formally published.

0 comments Read more at DEV Community