SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Terminal-Bench 4.0: Top AI Models Differ by 0.3% in Score but Nearly 2x in Cost

0
·1 views

The Terminal-Bench 4.0 leaderboard, published in early September 2026, shows GPT-6 Astra via Codex leading at 58.2% and Claude Fable 5.1 via Claude Code close behind at 57.9%. Despite the near-identical scores, a full benchmark run costs roughly $3,300 for Astra compared to approximately $6,200 for Fable 5.1. The margin between the two models falls within their overlapping statistical uncertainty ranges, meaning neither can be declared a clear winner. At the other extreme, Sonnet 5 scored just 12.4% while costing $9,600 per run — nearly three times the price of the top model for a fraction of the performance. Analysts suggest developers should weigh cost-per-point and score uncertainty, not just leaderboard rank, when selecting a model for production use.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Unit 42 Links Latin America Intrusions to LLM-Assisted Attacks and SOCKS5 Relay Tool

Palo Alto Networks' Unit 42 has documented separate intrusion campaigns targeting organizations in Mexico and Brazil, attributed to threat clusters CL-CRI-1131 and CL-CRI-1163. Attackers gained initial access through phishing emails carrying resume-disguised attachments, after which multiple remote access trojans were deployed on compromised Windows endpoints. The threat actors repeatedly attempted to deploy SockTz, a reverse SOCKS5 relay tool, cycling through nine versions over roughly two hours, suggesting iterative troubleshooting possibly aided by large language models. On compromised systems, attackers made multiple attempts to extract sensitive credential files, including SAM and NTDS.dit, using Volume Shadow Copies and batch scripts with destination write-permission checks. Unit 42 also observed activity coordinating data exfiltration toward a specific IP address, though successful file transmission and SockTz connection establishment were not explicitly confirmed.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Tutorial: How to Build a Structured Amazon Product Research Workflow Using Codex

A developer tutorial published on DEV Community outlines how to use OpenAI's Codex to build a disciplined Amazon product research workflow with built-in approval gates. The system processes product candidates through hard filters — covering price range, monthly sales, review count, rating, and landed cost share — before any scoring takes place. Candidates with missing data are routed to a 'verify' queue, while those failing non-negotiable rules are outright rejected, ensuring no scoring model can override critical thresholds. Surviving candidates receive a weighted score across demand, competition, margin, and quality dimensions, with all weights and thresholds treated as versioned, auditable decisions. The tutorial emphasizes human approval checkpoints for actions that carry legal, financial, or operational risk, distinguishing safe automation from decisions requiring human oversight.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Lenexus Builds Dependency-Mapping Platform to Model Business Risk and Failure Costs

Lenexus, developed by MJ Corp, is a platform that maps every system, supplier, employee, and AI model a company relies on into a single interactive dependency graph. The tool aims to answer a question most organizations struggle with: when something fails, what else breaks and what does it cost. Key features include failure simulation with financial impact estimates, a financial decision simulator, and a document intelligence module that extracts entities and relationships from unstructured text for human review. Notably, the platform separates its deterministic risk-scoring engine from its AI explanation layer, meaning analysis quality can improve as better language models become available. However, Lenexus remains a demo product without publicly confirmed paying customers, and its performance on real-world, incomplete data has yet to be tested.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

12-Year-Old in Tamil Nadu to Host 3-Day Online Coding Festival from a Budget Phone

A 12-year-old developer from Tamil Nadu, India, is organizing KODA Online Fest 2026, a three-day online coding festival scheduled for September 12–14. The event will be run entirely from a POCO C55 smartphone and features three tracks: a Build Sprint, a Math Kata Hour, and a Beginner Dojo, culminating in an awards day. The organizer previously attempted a similar event that received no submissions after it was buried inside a chat app, prompting a rebuild as a standalone, easier-to-access platform. The festival's volunteer team includes two senior engineers from China and a 14-year-old youth co-host. The organizer says the core lesson from the failed attempt was to make participation as simple as possible, offering a single link, multiple tracks, and public recognition for contributors.

0 comments Read more at DEV Community
Terminal-Bench 4.0: Top AI Models Differ by 0.3% in Score but Nearly 2x in Cost · ShortSingh