Host-held state beats chat-based memory methods for LLM agents on cost, while reply accuracy varies
A test compared six ways an LLM agent can remember its prior steps, using a 14B model across 600-step jobs. With the host checking and repairing each reply, every method ended with the exact state on every step it ran. The host-held approach, which shows the model the current state fresh each step, was the fastest and lowest in energy, at 0.30 seconds and 21.8 joules per step. Whole-chat memory was the most expensive at 1.72 seconds and 123.1 joules per step, and it stopped early on a token limit in all three jobs. Methods that rely on the model's own replies, including summaries, windows and retrieval, were more often right on the model's own output than the host-held approach, which scored 81.89 percent against 84.00 to 93.83 percent.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in