Developer finds benchmark error erased 20% speed lead for homemade language over Rust and Zig
A developer building 'machin', a Go-flavored language that compiles to C, discovered a flawed benchmarking method had falsely shown a 20–25% speed advantage over Rust and Zig in an integer loop test. The harness ran all trials for each language sequentially, meaning the last language tested always suffered from CPU thermal throttling, skewing results unfairly. After fixing the benchmark to interleave runs, the claimed win collapsed to a statistical tie within a 3% margin. The developer removed the misleading claim from the project's README and documentation, and updated the harness to avoid declaring winners within that noise band. Machin does hold genuine advantages over the two languages, including compile-time deadlock detection and a significantly smaller binary size of around 14 kB versus Rust's 335 kB.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in