COSP Lets LLMs Generate Their Own Prompting Examples Using Consistency Signals

Researchers at Google introduced Consistency-based Self-adaptive Prompting (COSP), a technique that allows large language models to create their own in-context examples without any human-labeled data. The method addresses a longstanding trade-off in LLM prompting: few-shot approaches require significant manual effort, while zero-shot methods often produce unreliable reasoning. COSP works by sampling the model's responses to a question multiple times and using the consistency of those answers as a quality filter to select trustworthy examples. Responses that the model repeatedly agrees on are treated as reliable demonstrations for a second, guided inference pass. The approach aims to combine the low effort of zero-shot prompting with the improved accuracy typically seen in few-shot settings.