Dev fixes memory-killing build step by replacing redundant rebuilds with record lookups

·3 views

A software developer noticed their project's acceptance-check step was repeatedly killed by the OS out-of-memory killer four times over two days. Investigation revealed the step, named a 'read', was actually triggering a full recompile of all 15 project units just to extract pass/fail booleans — duplicating work already done by prior control runs. Each control run had already stored its result alongside the exact commit hash it ran against, making re-execution unnecessary. The fix replaced the costly rebuilds with a simple comparison of stored commit identifiers, adding a 'stale' counter to flag outdated records instead. The developer concluded that any observational step consuming resources disproportionate to mere data-gathering is likely re-deriving facts that already exist.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How One Engineer Eliminated Per-Service Deploy Pipelines Using a Single Manifest

Software engineer Anton, working on migrating a PHP monolith to Go microservices, describes how his team unified deployment across all services by centralising configuration in a single manifest file. A silent Go linker bug — where an invalid -X flag produces no error yet leaves binaries stamped with a default 'dev' version — corrupted audit trails and distributed traces without triggering any alerts. The issue was compounded because the flawed flag existed in both the Makefile and Dockerfile, meaning fixing only one source caused local and production builds to silently diverge. To prevent recurrence, the team built a verification step that compiles binaries with a known probe version string and fails the build if that string cannot be found in the output. The broader takeaway is that a single authoritative manifest, combined with automated checks over written rules, made consistent and reliable deployments possible across every service.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

90% of a Service's Metrics Require Zero Instrumentation Code, Analysis Shows

A software engineer analysed the full metrics output of a production service and found 67 total metric records in its snapshot. Of those, 59 came automatically from the platform runtime and only 6 were written by the service itself. Metrics covering liveness, readiness, build version, broker delivery, Postgres pool state, and Go runtime internals all appeared without a single line of custom instrumentation. The engineer stores metrics as a diffable snapshot file rather than relying on a live scrape endpoint, making changes trackable over time. The finding challenges the common assumption that meaningful observability requires deliberate instrumentation effort from developers.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Overly broad config hashing caused 15 units to invalidate from a 6-line edit

A software project used SHA-256 hashes of an entire shared configuration file to verify whether individual units were up to date. When a developer added just six lines describing one new component, all fifteen units were marked stale and required full re-runs, consuming nearly an hour of machine time. The root cause was a mismatch between what the hash actually guaranteed — that no byte in the file had changed — and what engineers assumed it meant: that only their unit's relevant inputs had changed. The fix involved redefining the pin to hash each unit's own configuration block plus genuinely shared sections, rather than the whole file. The episode highlights that hash granularity is a design contract, and a check that is technically correct but disproportionately costly can quietly erode engineering practices over time.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Salesforce Data Cloud Deployments Can Fail Even When Source and Target Look Identical

A developer working on a Salesforce Data Cloud deployment encountered repeated failures despite both source and target environments appearing structurally identical. Investigation revealed that while the same components existed in both environments, their configurations, relationships, and dependency chains were not fully aligned. Data Cloud's interconnected architecture — spanning connectors, data streams, data lake objects, mappings, and identity resolution — means a single misconfigured dependency can break the entire deployment pipeline. The developer adopted a layer-by-layer validation checklist, correcting mismatched relationships and mappings before each redeployment attempt. The key takeaway is that a successful deployment requires matching component configuration and dependencies, not just component presence.

0 comments Read more at DEV Community