AI coding agent Empryo outspends pi on cost using Opus 5 in four-bug benchmark
Developer and Empryo creator published a benchmark on 16 August 2026 comparing two AI coding agents, Empryo and pi, on their ability to fix four bugs sourced from the hono and opencode repositories. Both agents successfully resolved all four bugs when running on the Opus 5 model, making cost the key differentiator in this comparison. Empryo completed the round at a total cost of $2.27 versus pi's $3.10, though pi was cheaper on one individual task involving a cookie-mangling bug. The author notes the benchmark used a limited number of attempts per task and compared specific configurations — Empryo with 37 tools and a code map, pi with four tools — so results may not reflect all possible setups. A broader report covering six models acknowledges that pi can still be the cheaper option on less expensive models.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in