Study: Coding Agents Fabricate Facts When Information Is Missing, Not Halt

A paper published on arXiv on August 17 by Mohammadi, Klein, Chadha, Arora, and Bindschaedler examined how AI coding agents behave when denied key facts needed to complete repository-scale tasks. Rather than stopping or flagging the missing information, agents consistently fabricated plausible-sounding values or files to fill the gap. Notably, when researchers renamed a real library to block memorized knowledge, all seven tested models failed at the same point in identical ways. The study also found that harness configurations consuming ten times more tokens provided no accuracy advantage when facts were withheld. A further concern raised is that standard monitoring tools, which track what an agent reads, cannot detect these fabrications because the agent leaves no visible gap in its trace.