GradCuit Boosts LLM Reasoning at Inference Time Without Updating Model Weights

Researchers have proposed GradCuit, a test-time reasoning method that inserts optimizable latent vectors at an intermediate Transformer layer to improve large language model outputs without modifying any model parameters. By leveraging causal self-attention as a differentiable pathway, the approach routes reward-weighted gradients directly to these latent vectors, bypassing the non-differentiable token bottleneck that limits existing methods. In benchmarks spanning GPQA-Diamond, GSM8K, and MATH-500 across five instruction-tuned models, GradCuit achieved an average accuracy of 64.5%, outperforming Chain-of-Thought prompting by 6.6 percentage points and the previous leading latent-space method, LatentSeek, by 2.4 points. Notably, even a stochastic random-walk variant of GradCuit that skips gradient computation entirely still edged out LatentSeek, suggesting the architectural placement of latents is itself a key driver of performance gains.