Article URL: https://spectrum.ieee.org/mexico-olinia-car-electric-vehicle Comments URL: https://news.ycombinator.com/item?id=49112368 Points: 3 # Comments: 0
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
OpenAI reported that enabling two Responses API settings — retained reasoning and compaction — raised GPT-5.6 Sol's score on the ARC-AGI-3 public benchmark from 13.3% to 38.3%. The initial evaluation harness prevented the model from carrying forward reasoning across steps of the benchmark's interactive 2D puzzle tasks, limiting its performance. Alongside the score improvement, the new configuration also reduced output token usage by approximately six times. OpenAI emphasized that this outcome highlights how API configuration, not just model capability, can significantly influence both agent performance and token efficiency. The findings serve as a practical reminder that benchmark scores reflect the combined effect of the model, its settings, and the evaluation setup rather than model weights alone.
A data modelling guide published on DEV Community argues that the most common error in business reporting is not poor chart design but mismatched data grain — the unit that each row in a dataset represents. Mixing invoice-level, line-level, payment, and financial-summary data in a single table causes totals to multiply and filters to answer the wrong question. The article illustrates the risk with a concrete example: joining a £1,000 invoice total across five line items and summing the result incorrectly yields £5,000. Analysts are advised to state the reporting question in plain language first, then select the appropriate grain — such as one row per invoice, per product line, or per payment — before building any visualisation. The guide also recommends keeping separate fact tables for each natural grain and documenting non-additive measures to ensure dashboards remain accurate when data is refreshed.
A widely cited AI memory benchmark, LOCOMO, has a significant methodological gap: its published evaluation harness skips all 446 adversarial questions in Category 5, which are designed to test whether a system can correctly abstain from answering. The harness also instructs the AI judge to 'COMMIT AND ANSWER' and never say 'not specified,' effectively eliminating any measurement of calibrated abstention. The LLM judge used for scoring has been found by an independent audit to accept roughly 63% of deliberately wrong answers, and LOCOMO's own answer key contains errors that cap the realistic maximum honest score at around 93.6. This means a recently reported score of 92.5 — touted as near-perfect memory performance — sits at the ceiling of a flawed benchmark, not at the ceiling of actual capability. The core issue is not fraud but misrepresentation: the metric measures answer accuracy on answerable questions only, yet is routinely interpreted as an overall measure of memory system quality.
As Go projects grow beyond simple examples, keeping all code in a single main.go file makes applications harder to navigate and maintain. The solution is to reorganize code into separate packages, each with a distinct responsibility such as data models, HTTP handlers, and server startup. In this approach, a models package holds data structures like the Task struct, while a handlers package manages HTTP request logic, importing from models as needed. This pattern, known as separation of concerns, makes it easier to locate code, fix bugs, and add new features over time. The restructured project uses only Go's standard library, making it accessible to developers learning idiomatic Go project organization.
Discussion (0)
Log in to join the discussion and vote.
Log in