SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

JetBrains Launches Kotlin AI Benchmark; Token Efficiency Varies 12x Across Top Agents

0
·3 views

JetBrains has released the Kotlin Benchmark, an official leaderboard that evaluates AI coding agents on 105 real engineering tasks drawn from active open-source Kotlin repositories. Claude Code with Opus 4.7 xhigh topped the leaderboard with an 85.7% resolution rate, followed closely by JetBrains Junie and OpenAI Codex at 81.9%. However, the benchmark reveals a striking 12x gap in token consumption per solved task across the top 20 configurations, meaning some lower-ranked setups solve nearly as many tasks at a fraction of the cost. The benchmark follows SWE-bench methodology, verifying solutions in containerized environments using each repository's own tests, with no self-reporting. JetBrains has published the datasets, test harnesses, and GitHub repository openly, aiming to fill a gap left by generic benchmarks that previously underserved the Kotlin ecosystem.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Dev Tutorial: Adding PostgreSQL Persistence to a NestJS Choose-Your-Own-Adventure API

A developer series called 'Grimoire API' is building a choose-your-own-adventure backend using NestJS, with Part 2 focusing on data persistence across server restarts. The tutorial introduces PostgreSQL paired with TypeORM as the database layer, run locally via Docker Compose for easy setup and teardown. A PlayerProgress entity is created to track each user's current story page and XP, mapping directly to a database table through TypeORM decorators. Notably, the design deliberately omits a stored 'level' column, deriving it from XP at runtime to prevent data inconsistencies. The module wires everything together using NestJS's dependency injection pattern, making the player progress repository available through TypeOrmModule.forFeature.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How Sentry was configured to show user journey funnels, not just stack traces

A development team found that despite having Sentry installed and tracking errors, no one could determine whether checkout failures stemmed from the payment step, coupon logic, or a tracking pixel. The core problem was that installation was mistaken for instrumentation — raw stack traces were being captured without context about what the user was actually doing. The team restructured their error events to include domain, phase, and action fields, allowing engineers and product managers to filter by user action before ever opening a stack trace. This change made it possible to distinguish revenue-threatening failures from post-sale noise within the same dashboard. As a result, checkout conversion data became visible alongside error data, aligning engineering alerts directly with product OKRs.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How fixing checkout timeouts during influencer campaigns lifted conversions by 10%

A developer on a high-traffic infoproduct checkout platform discovered that conversion rates were dropping during influencer campaigns because the micro frontend architecture was forcing users' browsers to download excessive JavaScript before the payment button could load. The root cause was identified using Sentry and Datadog observability tools, which revealed that payment-step JS was timing out on mobile devices under peak traffic — sessions were dying before the POST request could complete. The team addressed the problem by implementing server-side rendering via an existing BFF layer, loading JavaScript on demand, and optimising CDN delivery so only the current step's code loaded upfront. Results were measured not through Lighthouse lab scores but through backoffice sales data cross-referenced with Sentry and Datadog metrics, confirming a 10% increase in conversions within the month. The engineer credited the outcome to aligning observability tooling with product OKRs and real sales metrics, and noted the result contributed to their promotion to Staff Engineer II.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer proposes 'Live TUI' term for async, multi-pane terminal app interfaces

A developer has proposed the term 'Live TUI' to describe terminal user interfaces where individual panes can refresh and show progress independently without blocking the rest of the workspace. The concept draws an analogy to AJAX in web development, where partial page updates keep the surrounding interface usable during background requests. A demo application called hqtui-demo has been released to illustrate the combination of live state, independent pane updates, and local progress indicators. The author acknowledges that asynchronous terminal interfaces are not new, and makes no claim to inventing the pattern — only to naming the combined experience. The proposal invites feedback on whether 'Live TUI' effectively communicates this design expectation to developers building terminal applications.

0 comments Read more at DEV Community