40-Year-Old IEEE 754 Rounding Trick Causes Overflow Bugs in Tenstorrent AI Chips

A developer working on Tenstorrent's tt-metal ML framework discovered a critical floating-point overflow bug in the chip's SFPU math unit, which handles transcendental functions like exp and log via polynomial approximations. The bug surfaced in computation chains such as exp(x) * exp(-x), where intermediate results exceeded the representable floating-point range even though the final mathematical result should equal 1. A secondary issue involved a decades-old IEEE 754 rounding technique — adding 0.5 before truncation — which interacted unexpectedly with the SFPU's internal precision during fp16/bf16 to fp32 conversions, producing small but measurable deviations from PyTorch reference outputs. The overflow fix involved restructuring the computation graph using algebraic identities to avoid storing oversized intermediate values. The findings highlight how hardware-level numerical quirks in AI accelerators can silently produce incorrect results, underscoring the importance of thorough edge-case testing against reference implementations.