Perfect Annotation Agreement Can Hide Sparse Labelling Coverage

·2 views

A single 100% agreement figure can make two annotation files appear more complete than they are. In the bundled partial example, only one of twelve records is labelled by both sides, so conditional agreement reads 100% while both-labelled coverage is just 1/12. The metric counts only records labelled by both sides, so it ignores the remaining records entirely. The article's review tool therefore checks that both checkpoints share the same records, taxonomy, and order before comparing them, and it keeps manual decisions separate from the original annotations. It also stresses that agreement is not accuracy and that the two annotation roles are not verified as independent people.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Claude Fable 5.2 Tops Dual-Arm Robotics Benchmark Without Task-Specific Training

Anthropic's general-purpose model Claude Fable 5.2 reportedly scored a 64.9% average success rate on the RoboDojo dual-arm manipulation benchmark, up from 27.6% for its predecessor and ahead of Opus 5.5 and GPT-6 Astra. Rather than relying on a dedicated robot policy, the model plans actions directly from camera input, reasoning about the scene and adjusting when grips slip or objects shift. Official real-robot demonstrations reportedly included solving a Rubik's cube, unscrewing a bottle cap, and drawing with a pen, none of which the model was specifically trained on. The company also says the model handled hardware design tasks, producing PCB layouts, enclosure models and firmware for three working devices. The benchmark is simulated, and the model depends on clear visual input and existing robot controllers for low-level motor control.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Connector Engineering Ch.05 Explores Waterproofing Design for Interface Seals

This chapter from the STATERA Library's Connector Engineering series examines how interface seals keep water, dust, and contaminants out of connectors at the boundary where housings meet. It explains that effective sealing relies on controlled, continuous compression of an elastomeric gasket rather than on rubber alone, since both insufficient and excessive compression cause problems. The text covers how seal cross-section, elastomer choice such as silicone, EPDM, or fluoroelastomers, and hardness affect performance under environmental and mechanical conditions. It also stresses that housing geometry, mold quality, and tolerance stack-up must be designed together with the seal to maintain adequate pressure across the full range of manufacturing variation.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

IP Code Explained: Dust and Water Ratings for Connector Environmental Protection

The IP Code, defined by IEC 60529, classifies enclosure protection against hazardous parts, solid objects, and water ingress, offering engineers a shared vocabulary for connector specification. Its two numerals rate solid-particle and water protection separately, so they must be assessed independently rather than merged into one durability score. The first digit covers access restriction and dust exclusion, with IP5X permitting limited non-interfering dust and IP6X requiring a dust-tight design. The second digit addresses water exposure, ranging from dripping and spraying to jets, temporary immersion, and continuous immersion. The article notes that ratings such as IP67 and IP68 carry specific test conditions, so engineers should verify declared depth, duration, and configuration rather than relying on the marking alone.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

D-Bus Policy Governs Messages, Not the Authorization Logic Behind Them

D-Bus is widely used for local service communication on desktops and increasingly in embedded and administrative systems. Its bus daemon policy decides which messages are delivered, but it cannot judge whether a receiving method performs a safe operation. The article argues that a narrow policy still depends on the service exposing a narrow API, and that applications must enforce their own authorization after receiving a message. It recommends inventorying exported interfaces, separating system and session bus trust domains, and treating policy files as reviewable configuration. The author also notes that signals can leak sensitive data, so services should limit what they emit.

0 comments Read more at DEV Community