SShortSingh.
Back to feed

How Linux fork() Uses Copy-on-Write and Why It Can Exhaust Memory

0
·10 views

When a Linux process calls fork(), the kernel does not immediately duplicate physical memory; instead, it copies the page table hierarchy and marks all entries as read-only. Any subsequent write by either the parent or child process triggers a hardware page fault, prompting the kernel to allocate a new physical page and copy the data. This Copy-on-Write mechanism keeps fork() fast under low-write conditions but becomes costly when workloads involve heavy, sustained writes to large memory regions. Systems running in-memory databases like Redis can experience write amplification, TLB thrashing, and uncontrolled memory growth during background snapshot operations. In extreme cases, the cascade of micro-allocations exhausts available RAM and activates the kernel's Out-of-Memory killer, terminating processes unexpectedly.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingHacker News ·

Foremerge Detects Intent Conflicts Between Parallel AI Coding Agents Before Merge

GPTree has released Foremerge, an open-source coordination layer designed to catch architectural conflicts when multiple AI coding agents work simultaneously on the same code repository. Unlike Git, which only flags line-level merge conflicts, Foremerge requires each agent to declare its intended scope and operation before making changes, enabling deterministic conflict detection without relying on AI judgment. The tool ships as a single Rust binary with a CLI and MCP server, and integrates with popular AI coding tools like Claude Code, Codex, and Cursor in about 30 seconds. The team tested the system with up to 98 parallel agents on one repository and recorded zero undetected conflicts, though a blind spot involving class-level versus method-level scope claims was identified and is being addressed. Foremerge is available under the Apache-2.0 license via its GitHub repository or a one-line install script.

0
ProgrammingDEV Community ·

Classification-Only AI Models Challenge LLMs for Bulk Data Labeling in Lakehouses

Teams using large language models for text classification in data pipelines face high costs and slow performance, particularly when processing tens of millions of rows in Apache Iceberg lakehouses. In September 2026, a company called TypeSafe AI released Jev, a model designed exclusively for classification tasks such as labeling, scoring, or returning yes/no probabilities with calibrated confidence. Unlike LLMs, which generate output token by token and often return fragile JSON, dedicated classification models complete tasks in a single forward pass, cutting both latency and expense. Several open-source projects quickly emerged to replicate Jev's interface on self-hostable models. The article explores how these two model types can be used together within an Iceberg lakehouse, using Dremio's SQL AI functions as one practical example.

0
ProgrammingDEV Community ·

Terraform vs OpenTofu: How HCL Brings Infrastructure-as-Code to the Cloud

On August 10, 2023, HashiCorp switched Terraform's license from MPL 2.0 to the Business Source License, prompting a coalition of companies to launch OpenTofu, an open-source fork hosted under the Linux Foundation. Terraform uses HashiCorp Configuration Language (HCL) to let teams declare the desired end state of servers, networks, and databases as versioned text files, rather than configuring them manually through a cloud console. Its core engine compares the declared state against the current real-world state and calculates only the changes needed, making the process idempotent and repeatable. A directed acyclic graph automatically resolves resource dependencies during planning, while remote backends with state locking prevent conflicting changes from multiple team members. Reusable modules allow the same configuration to deploy identical environments, such as staging and production, by simply swapping variable values.