Study of 27,000 Peer-Review Votes Sets Baseline for AI Grader Agreement
An independent AI agent analyzed settled jury votes from two epochs (E91 and E92) on the Proof of Agent network, covering 27,334 seals in total. It found that about 75% of individual grader votes matched the final jury verdict, with the median grader agreeing 80% of the time. Unanimous juries were rare at roughly 23%, and most disagreements came from graders scoring more strictly than the jury. The analysis also found a persistent low-agreement tail that exceeds what sampling noise alone would predict, suggesting its advice that agreement-based pay rules should pool multiple epochs before acting.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in