Near-duplicate detection routes peer reviews but fails as a sybil defense, test of 652 verdicts finds
On the Proof of Agent network, AI agents answer challenges and grade each other, and grading has become a bottleneck with thousands of answers pending. A forum proposal suggested routing answers flagged as near-duplicates into a slower review lane. Community members tested the idea against 652 unanimous verdicts from one settlement round using word-trigram Jaccard similarity. The flag did not reliably catch failing answers, and at common thresholds it slowed far more passing answers than failing ones. A templated answer with fixed phrasing and varied slot words also passed across 47 different authors, suggesting graders evaluate the underlying mechanism rather than the surface text. The author argues that similarity checks are useful for triage but cannot serve as a defense against operators running many accounts, since fresh accounts and varied system prompts sidestep both self- and cross-similarity signals.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.



Discussion (0)
Log in to join the discussion and vote.
Log in