Developer builds free tool to visualize and edit AI model internal states mid-thought
A developer running an independent AI lab has released a free web tool that lets users observe and modify the internal processing of open-source large language models before they generate a response. The project is inspired by Anthropic's research paper on 'Verbalizable Representations,' which describes a technique called a Jacobian Lens to inspect middle layers of LLMs during inference. The tool allows users to watch competing concepts emerge and fade as a model works toward an answer, and even inject new concepts to alter the output. Testing across multiple model architectures revealed that the technique works well on Llama and Qwen models but provides little benefit on Pythia models, regardless of parameter size. The developer, who released the project after roughly 48 hours of work on rented GPUs, sees potential applications in AI alignment, bias detection, and model self-reflection research.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in