FrontierHarness Eval Shows 17x Cost Variation Across 9 AI Benchmark Harnesses

·1 views

FrontierHarness Eval is a newly shared evaluation tool that tests a single AI model across nine different benchmark harnesses. The project reveals that the cost per passing result can vary by as much as 17 times depending on which harness is used. This highlights significant inefficiencies in how AI models are evaluated, even when the underlying model remains constant. The tool was shared on Hacker News as a community project, attracting early discussion around evaluation methodology and cost optimization.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Unicode Math Symbols, Not Fonts, Power Every Copy-Paste Style Generator

A developer building a free copy-paste font tool called Fonti discovered that stylised text shared on social media does not use actual fonts at all. Characters that appear bold or italic are in fact distinct Unicode symbols from the Mathematical Alphanumeric Symbols block, ranging from U+1D400 to U+1D7FF and introduced in Unicode 3.1 in 2001. Originally created so mathematicians could distinguish visually similar symbols with different meanings, these characters were later repurposed by social media users for decorative bios and profiles. Converting text is therefore a mathematical offset calculation, not font rendering, though some characters require exception handling because older Unicode blocks already contained certain styled letters. The discovery reshaped the developer's entire approach, confirming that the tool is essentially a lookup table requiring only a small JavaScript function alongside otherwise static HTML.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Regulation and Liability May Be the Best Job Security for Developers in the AI Era

A career analyst mapped 46 computing roles using data from regulators and central banks to identify which tech jobs are most vulnerable to automation. The research found that jobs protected by liability, regulatory oversight, restricted data, or closed systems are far less susceptible to AI substitution. High-stakes fields like avionics and medical device software require certified human sign-off, making full automation legally and practically difficult. Recent court cases — including a US lawsuit against UnitedHealth over an AI discharge tool and a Canadian ruling against Air Canada for a misleading chatbot — show courts are increasingly holding companies liable for algorithmic decisions. The analysis suggests developers who work outside these protective barriers, such as those building low-stakes marketing websites, face the greatest near-term risk of being replaced by AI.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

ThoughtDAG's Session Atlas Turns Local AI Agent Logs into Editable Graph Mirrors

ThoughtDAG has launched Session Atlas, a desktop feature that imports local AI agent sessions from tools like Codex and Claude Code into editable graph-based mirrors grouped by project. Unlike conventional agent memory systems, Atlas maintains a strict separation between the original session record, the user's curated workspace, and the context sent to future model requests. The original session file is never modified; users can edit, reorganize, or annotate mirrored nodes independently without altering the source history. Atlas also monitors active local sessions and incrementally appends new turns, respecting user edits and avoiding overwrites. The tool aims to give users explicit, transparent control over what context their AI agents receive, rather than relying on automatic or hidden memory retrieval.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds automated LinkedIn creator research pipeline for under $1 using Apify MCP

A developer has built an end-to-end LinkedIn creator research workflow using the Apify Model Context Protocol (MCP) server integrated with Claude Code. The pipeline chains three tools — a LinkedIn posts scraper, a comments scraper, and a website content crawler — allowing an AI agent to discover and run each tool within a single session. In under 18 minutes and for approximately $0.69, the workflow produced a content playbook, an ideal customer profile derived from comment analysis, and a CSV of highly engaged commenters. The Apify MCP server exposes Actors as callable tools via a configurable URL, eliminating the need for manual data transfers between tabs or applications. The author notes the same prompt can be redirected at any public LinkedIn profile, including competitors or one's own account.

0 comments Read more at DEV Community