SShortSingh.
Back to feed

How Profiling a 60-Million-Row Query Exposed a Caching Problem, Not a Query Problem

0
·1 views

A developer investigating a slow admin dashboard endpoint assumed a GROUP BY query over 60 million rows was the core problem, but profiling revealed it completed in just 567 milliseconds. The real issue was that the database performed a full aggregation and sort of nearly 60,000 member groups on every single request, regardless of which page was being viewed. Because the LIMIT clause only applies after the entire result set is sorted, the query cost remained identical whether loading page 1 or page 500. With three leaderboard tabs refreshed frequently by multiple admins, the repeated load threatened the same Aurora cluster serving around 900,000 mobile users. Standard fixes like indexing or cursor-based pagination were ruled out since the sort key was a computed aggregate, pointing toward caching as the appropriate architectural solution.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer Builds Self-Hosted QR Code Tracker Using Go and SQLite

A developer has created QrStamp, a lightweight, self-hosted web application for generating and tracking QR codes without relying on third-party services. The project is built using Go for the backend, SQLite for embedded zero-configuration data storage, and vanilla web technologies for the frontend. QR codes are generated on the fly as PNG files using the go-qrcode library, while scan counts are tracked and updated in real time via SQLite transactions. The application features a dark-mode UI and real-time scan metrics, and runs as a single portable file. The developer released the project as a practical exercise for sharpening backend development skills in Go.

0
ProgrammingDEV Community ·

Why vi.mock in Vitest silently fails and how to fix it

A common Vitest pitfall causes mocks to silently fail when developers reference variables defined outside the vi.mock factory. This happens because Vitest hoists vi.mock calls to the top of the file during transformation, before any imports or variable declarations are executed. As a result, any variable referenced inside the factory does not yet exist at runtime, causing the mock to break without a clear error. The fix is to use vi.hoisted to create mock values in the same hoisted phase, or to define them directly inside the factory itself. Alternatively, developers can use vi.mocked inside individual tests to apply mock behavior without relying on shared variables.

0
ProgrammingDEV Community ·

How One Dev Team Uses a Hybrid Git Workflow to Manage Client Projects

A software development team has shared the Git branching strategy it uses to keep client codebases stable and production-ready. The workflow combines elements of Git Flow and Trunk-Based Development, built around three long-lived branches: main, staging, and short-lived feature branches. All code changes must go through pull requests with automated CI checks and at least one senior developer review before merging into staging via squash commits. Clients review and approve features in the staging environment before any code is promoted to the main production branch. Releases are tagged with semantic versioning, and hotfixes follow a separate naming convention to avoid disrupting the main release line.

0
ProgrammingDEV Community ·

How DNS Translates Domain Names Into IP Addresses Explained

Every internet-connected device is identified by a unique IP address, but users access websites through human-readable domain names, making a translation system necessary. The Domain Name System (DNS) handles this by converting domain names like example.com into the corresponding IP addresses that computers use to communicate. When a browser needs to resolve a domain, it first checks its local cache before sending a query to a recursive resolver, typically operated by an ISP or public services like Google or Cloudflare. The recursive resolver then queries a chain of DNS servers — including root nameservers, Top Level Domain (TLD) nameservers, and authoritative nameservers — until it retrieves the correct IP address. DNS caching at various stages speeds up this process by storing previously resolved results and reducing the number of network requests needed.

How Profiling a 60-Million-Row Query Exposed a Caching Problem, Not a Query Problem · ShortSingh