SShortSingh.
Back to feed

Define Your Data Grain Before Picking a Chart, Analysts Warned

0
·1 views

A data modelling guide published on DEV Community argues that the most common error in business reporting is not poor chart design but mismatched data grain — the unit that each row in a dataset represents. Mixing invoice-level, line-level, payment, and financial-summary data in a single table causes totals to multiply and filters to answer the wrong question. The article illustrates the risk with a concrete example: joining a £1,000 invoice total across five line items and summing the result incorrectly yields £5,000. Analysts are advised to state the reporting question in plain language first, then select the appropriate grain — such as one row per invoice, per product line, or per payment — before building any visualisation. The guide also recommends keeping separate fact tables for each natural grain and documenting non-additive measures to ensure dashboards remain accurate when data is refreshed.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

RE-call vs Mem0: No-LLM memory system trades accuracy for cost and privacy

Developer benchmarks comparing RE-call, an open-source memory system that never calls an LLM, against Mem0 show Mem0 outperforms RE-call on temporal reasoning tasks, scoring 0.567 versus 0.408 on the BEAM benchmark. The gap exists because Mem0 distills memories into clean summaries at ingest using an LLM, while RE-call stores raw conversation turns, causing date-related queries to retrieve wrong instances of the same fact. However, RE-call's no-LLM design means zero ingest cost compared to Mem0's $7.29 and 2.6 million tokens spent processing the same workload. RE-call also completed ingestion in 67 seconds versus Mem0's 288 seconds, and keeps all data within the user's own database. The author concludes that neither system is universally better — the right choice depends on whether a user prioritises accuracy on time-sensitive queries or prefers lower cost, faster ingest, and local data control.

0
ProgrammingDEV Community ·

OpenAI Urges Clearer Reporting of AI Benchmark Conditions and Setup Variables

OpenAI has published new guidance calling on researchers, evaluators, and buyers to treat AI benchmark scores as measurements tied to specific testing conditions rather than fixed capability limits. The company argues that factors such as the evaluation harness, compute budget, available tools, and memory management can significantly alter observed model performance, particularly on complex multi-step tasks. OpenAI distinguishes between different types of evaluation claims — including strong elicitation, controlled comparison, and safeguard robustness — noting that each requires different evidence and cannot be used interchangeably. The guidance recommends that benchmark reports disclose the full setup behind a score so that comparisons between models remain meaningful and reliable. OpenAI cited its own GPT-5.5 cyber-range evaluations as an example, where performance visibly improved when the harness used context compaction across extended tasks.

0
ProgrammingDEV Community ·

How to Actually Verify If an AI Crawler Is Legitimate, Not Just Labeled

AI crawlers like GPTBot identify themselves via user-agent strings, but these can be easily faked by anyone in seconds, making them unreliable as sole proof of identity. Verification requires independently confirming a crawler's claimed identity using methods the operator controls, such as reverse DNS lookup, published IP ranges, or cryptographic HTTP message signatures. Reverse DNS remains the most robust method, while IP-range checks are the most widely supported, used by 20 of 41 documented crawlers including OpenAI's GPTBot. Cryptographic signing via HTTP Message Signatures is emerging as the most secure approach, with infrastructure support from Cloudflare, Akamai, and AWS, though operator-side adoption remains limited. The stakes have grown significantly by 2026, as verified crawler identity now governs real privileges like search representation, pay-per-crawl access, and bot-challenge exemptions.

0
ProgrammingDEV Community ·

AI memory systems cannot reliably detect what they don't know, new benchmark shows

A developer investigating AI memory agent reliability found that the ability to abstain from unanswerable questions is not a single capability but varies with how far removed a question is from its supporting evidence. A new benchmark was built with questions generated at controlled 'excision distances,' revealing that discrimination between answerable and unanswerable queries improves sharply as more context is deleted, from near-chance accuracy at close range to near-perfect at full topic removal. This finding explains why two public benchmarks, LOCOMO and BEAM, appeared to contradict each other: both were measuring different points on the same hidden curve and reporting them as a flat capability score. Further analysis showed that for roughly two-thirds of test questions, deleting the gold evidence caused no change in retrieval scores, meaning the correct answer was never the top-ranked result in the first place. The author concludes that abstention failures in AI memory systems are structural rather than tunable, and that benchmark comparisons are misleading without specifying the difficulty distance of unanswerable questions.

Define Your Data Grain Before Picking a Chart, Analysts Warned · ShortSingh