SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

How Entity Resolution Cleans Messy Name Data, Using Billboard Charts as a Test Case

0
·1 views

Entity resolution is the process of identifying which database records refer to the same real-world person, company, or entity — a common challenge wherever humans enter names. A practical guide published on DEV Community walks through a five-step approach using Billboard Hot 100 chart data spanning 1958 to the present. The exercise found that Elvis Presley alone appears under seven distinct credit strings, meaning raw record counts can undercount an artist's catalog by nearly half. Across 11,275 unique chart credits, up to 43% contain joining words like 'Featuring,' 'With,' or '&,' making normalization rules a core requirement rather than an edge-case fix. The guide also documents two cases where cleaning rules misfired — such as generating the fake artist name '2Pac Duet' — illustrating why every rule must be tested against real data before being applied at scale.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

dbt Semantic Layer, Cube, and AtScale Compared for Enterprise Metric Governance

Three leading enterprise semantic layer tools — dbt Semantic Layer, Cube, and AtScale — each take a distinct architectural approach to defining and serving business metrics. dbt Semantic Layer treats metrics as version-controlled YAML code, Cube acts as a headless API layer in front of data, and AtScale focuses on OLAP-style aggregate acceleration. While all three handle standard dashboard and analyst use cases competently, they share critical gaps around unmodelled queries, row-level authorisation, and reproducing historical metric definitions. These limitations become especially significant when AI agents or regulated business environments require dynamic intent resolution and auditable, entitlement-aware query execution. Evaluators are advised to test beyond basic metric retrieval and focus on ambiguous queries, multi-user entitlements, and point-in-time metric reproducibility before committing to a platform.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Each Claude Code subagent carries ~436k tokens of fixed overhead before doing any work

A developer team running a weekly AI-powered product review pipeline discovered that spawning each Claude Code subagent incurs roughly 436,000 tokens of fixed overhead, regardless of the actual task size. This overhead stems from the initial context load at spin-up and a cache write on the final turn — not from the content being processed. In a controlled test, splitting a review task across three subagents cost nearly three times more than using a single agent, while the embedded product content itself accounted for only about 46,000 tokens per agent — a 9.5-to-1 ratio of fixed cost to payload. The finding challenges a common assumption that agent costs scale with content size, showing instead that the number of agents is the primary cost driver. The team recommends merging agents with overlapping roles and reserving separate subagents only for genuinely independent tasks that cannot be handled within a single context.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer builds serverless iOS app that maps live earthquakes on a 3D globe

A developer has created Earthquake: Live Seismic Monitor, an iOS app that visualizes recent seismic activity on an interactive 3D globe using real-time public data from the USGS GeoJSON feed. The app requires no custom backend, as it polls USGS directly and updates the scene by comparing new data against what is already cached on the device. Each earthquake is represented by a marker whose size and color encode its magnitude and depth, allowing users to quickly distinguish between major shallow quakes and minor deep ones. To handle offline scenarios honestly, the app caches the last successful feed and displays the source timestamp rather than implying instantaneous updates. The project highlights how reliable open government datasets can power a full consumer app without the need for any developer-managed server infrastructure.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

AI Coding Assistants Know the React Key Rule But Routinely Ignore It in Practice

AI coding assistants can correctly explain why using array indexes as React keys is problematic, yet frequently generate exactly that pattern in real codebases. The issue is not a knowledge gap — models have encountered this React best practice thousands of times during training. The problem arises because generating code token-by-token under multiple simultaneous decisions differs from directly recalling a known rule. Index keys tend to appear when data lacks an obvious unique identifier, when list rendering is a small part of a larger request, or in early-stage placeholder components that often ship unchanged. This highlights a broader gap between an AI model possessing information and reliably applying it during active code generation.

0 comments Read more at DEV Community
How Entity Resolution Cleans Messy Name Data, Using Billboard Charts as a Test Case · ShortSingh