Recording Free Inference Calls in a Ledger Before Making Them

The article argues that free inference lanes fail less on quality than on accounting, since unrecorded spend cannot be subtracted from a budget. It proposes placing a ledger in front of the lane and recording each reservation before the upstream call is made. Because workers can die mid-call and retries can double spend, the ledger must survive crashes without manual cleanup. The author describes a lease-based design where each job holds an open budget row that is committed or released, with a sweeper expiring stale leases. The implementation uses only the Python standard library and a single SQLite file, suitable for a laptop or small always-on machine.