Multi-Agent AI Systems Cut Costs by Replacing Context Passing with Shared Memory

A software team building a six-agent financial advisory AI identified token accumulation as a core inefficiency, where each agent in a chain inherited the full output of all previous agents, inflating prompt sizes and costs. To address this, they replaced direct context passing with a shared memory layer, allowing each agent to retrieve only the specific slice of information it needs within a defined token budget. The architecture uses three persistent memory tiers: an episodic log of raw session events, a semantic store of distilled cross-session facts, and a third tier for broader persistent context. This 'pass by reference' approach keeps token consumption structurally bounded regardless of chain length or number of user sessions. The team frames this as part of a broader discipline they call 'TokenOps,' focused on designing AI systems with predictable, capped token costs rather than retrofitting compression after the fact.