Researchers derive a FlashAttention kernel from about 25 lines of SQL

A new demo shows that a FlashAttention-style kernel can be automatically derived from roughly 25 lines of SQL that simply express standard attention. The approach relies on three SQL extensions under development: xarray-sql, which represents petabyte-scale cloud array datasets as relational tables; ddx, which adds JAX-style automatic differentiation to SQL; and einfold, a modular query planner for fast tensor contractions. The authors argue that separating logical tensor-program layers from physical execution layers is both more correct and more scalable. They frame this as a relational take on the 'Bitter Lesson' in machine learning and invite collaborators to join their Discord to explore the intersection of database theory and ML systems.