Researchers identify internal 'workspace' in LLMs that drives reasoning, separate from fluency

Researchers have identified a small, relatively organized internal region in large language models — dubbed J-Space — that appears to function like a mental scratchpad for concepts during reasoning. This workspace is believed to be distinct from the neural pathways that handle grammar, style, and fluent text generation, which seem to operate more automatically. Experiments show that directly substituting one concept for another within this internal space — without altering the prompt — can change the model's output, suggesting reasoning is guided by manipulable internal representations. Crucially, when this workspace region is disrupted or removed, models can continue generating fluent, confident-sounding text while losing much of their reasoning capacity. For developers, the key takeaway is that linguistic fluency is not a reliable indicator of correctness, and understanding this separation could help address hallucinations and inconsistent behavior in LLM-based products.