Cost Model Estimates Token Speed for 276B AI Model Streamed Directly from SSD

A developer built a roughly 250-line cost model to predict how fast a 276B-parameter mixture-of-experts AI model, Inkling-Small, could run on a 24 GB Mac mini by streaming weights from an SSD rather than loading them into RAM. Unlike dense models, mixture-of-experts architectures activate only a subset of weights per token, making it feasible to read required weights from disk on demand. The cost model takes only configuration files as input and derives byte counts to estimate per-token data transfer, without downloading any model weights. It was validated against two independent artifacts, though a later correction revealed one gate had passed due to two offsetting errors rather than accurate modelling. The key finding was that total parameter count matters little; per-token cost is driven by the number of active experts, layers, and expert size, with only 10% of Inkling-Small's experts fitting in 24 GB of RAM.