Inkling 2 Mini: 26.4B looped transformer runs on 16GB local RAM, weighing against cloud APIs
Thinking Machines Lab released Inkling 2 Mini under the Apache 2.0 license, a 26.4B-parameter model with a 1M-token context window. Its architecture loops the middle 12 of 24 layers three times using shared weights, so it stores only the 26.4B weights while computing roughly 48 layers' worth of depth per token. This keeps memory requirements low, making the model practical on a 16GB laptop via quantized GGUF or on a 32GB Mac through Ollama at zero per-token cost. Cloud APIs remain better suited to high-concurrency traffic, strict latency targets, and fine-tuning, since the looped layers demand more compute per token under heavy batching.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in