Developer builds B+ tree disk storage engine, discovers why fixed-size records fall short

A web developer documenting their systems programming journey set out to understand how real databases like MySQL and PostgreSQL store data on disk. Starting from a simple in-memory key-value store in C, they progressively built up to a B+ tree implementation after learning it underpins most production databases. When attempting to add disk persistence, they initially copied their in-memory layout using fixed-size records and a header struct, but quickly decided this did not reflect how real databases operate. Switching to variable-length records introduced new challenges around locating and binary-searching keys efficiently without costly data shuffling on every insert. The developer ultimately discovered the slot-directory page design, where fixed-size metadata slots are stored at the front of a page and variable-length records grow inward from the back, as the standard solution used in real storage engines.