Whoz Splits 530M-Document MongoDB Collection to Cut Backup and Restore Times

·1 views

Whoz, a SaaS platform for professional services talent management, faced mounting operational strain after its MongoDB worklog collection grew to 530 million documents and 32 GB, with backups taking nearly an hour and restores up to four hours. The core data unit, called a worklog, records time spent by consultants on projects, and rising client numbers combined with finer-grained user entries were accelerating growth year over year. Engineers evaluated three solutions — MongoDB sharding, WiredTiger storage tiering to S3, and application-level partitioning — before ruling out the first two as either overengineered or unsupported. Sharding was dismissed because the problem was data locality rather than query throughput, while MongoDB confirmed that WiredTiger's internal tiering capabilities are not exposed for general use. The team ultimately opted for application-level collection splitting, separating historical worklogs from recent ones while keeping both transparently accessible to the application.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AI Observability Explained: Why Traditional Monitoring Fails for AI Systems

Traditional software monitoring assumes identical inputs produce identical outputs, but AI models can return two different answers to the same prompt without triggering any errors. AI observability addresses this gap by recording every detail of a request — including the prompt, model version, token usage, cost, latency, and output quality — not just uptime metrics. Standard infrastructure tools have no way to flag failures like a factually wrong but grammatically perfect response, a silent model version change, or poor retrieval results from a vector search. Distributed tracing breaks a single AI request into individual steps called spans, allowing engineers to pinpoint whether a bad answer stems from flawed retrieval, a poor prompt, or the model itself. Without this level of visibility, a failing AI feature can appear perfectly healthy on a conventional monitoring dashboard.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Tune Linux IPC Message Queues for WLOADCTL Scheduling Workloads

WLOADCTL, a distributed scheduling platform, relies on Linux Inter-Process Communication (IPC) mechanisms — including message queues, shared memory, and semaphores — for communication between components on the same node. Default Linux IPC settings can prove insufficient in high-volume environments, causing message queue errors or bottlenecks. Administrators can inspect current limits using the ipcs -l command and, if needed, raise kernel parameters such as msgmni, msgmax, and msgmnb in /etc/sysctl.conf to increase queue capacity. After applying changes with sysctl -p, a restart of affected WLOADCTL nodes is required since updated kernel IPC settings only apply to newly created resources. Stale IPC resources left behind by abnormal process termination can be identified via ipcs and manually removed using the ipcrm utility.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

TypeScript 6.0 Enforces Strict Function Types by Default, Breaking Existing Callbacks

TypeScript 6.0 enables strictFunctionTypes by default, introducing contravariant parameter checking that can cause widespread compile failures in codebases that previously relied on bivariant behavior. The change means callbacks accepting a base type where a derived type is expected will no longer compile, breaking patterns that worked silently in earlier versions. A key distinction exists between method signatures, which retain bivariant checking for compatibility, and function properties, which now enforce contravariance. Teams are advised against silencing errors with 'any' or type assertions, as doing so undermines type safety and can reintroduce runtime bugs. Recommended migration paths include widening parameter types to unions, using method syntax where bivariance is intentional, or applying generic constraints to preserve assignability safely.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

CarSegNet Uses Bounded Trimap Bands to Refine Auto Segmentation Mattes

A developer working on CarSegNet, a vehicle image segmentation system, has detailed a runtime approach that restricts AI-based edge refinement to a narrow 'trimap band' around uncertain mask boundaries. The method prevents a neural refinement model from altering confidently segmented areas, addressing a common problem where improving one edge inadvertently damages another. The pipeline combines SAM for semantic masking with an NSJ alpha refiner that only operates within the uncertain strip, leaving the rest of the frame untouched. To handle cases where the prior mask is confidently wrong — such as missing antennas or filled wheel openings — the system adds an image-based edge term that widens the band where the photograph shows structure the mask does not. The design prioritizes cheap, inspectable cached-matte runs and avoids loading large segmentation checkpoints unless the execution path explicitly requires them.

0 comments Read more at DEV Community