DataFusion on the Apple Silicon GPU: One Optimizer Rule, the Same SQL, Sorts 6.9x to 28.8x Faster

Apache DataFusion will run a physical optimizer rule you hand it, after its own. The rule sees the finished physical plan and may replace any node with an operator of its own; DataFusion runs what comes back and never looks at the SQL again. ArrowMetal 0.4.0, released on 2 October, is one such rule for the Apple silicon GPU, the Rust crate datafusion-arrowmetal. Register it on a SessionContext and EXPLAIN shows a MetalExec where SortExec was. The SQL is unchanged, the answer is DataFusion's, and a report says which nodes it took, which it left, and why.