Writer Catches Himself Twice: How Visible Lists and Identical Prompts Skewed AI Randomness Tests
A developer published a self-experiment attempting to measure randomness in digit selection, only to issue a correction within 36 hours after spotting critical methodological flaws. The original test had the author writing 100 digits while viewing the previous ones, meaning apparent fairness likely reflected list-tracking rather than genuine randomness. A follow-up control — asking for single digits without any visible list — produced extreme bias, with responses converging almost entirely on the number 7. A further error was caught when all test prompts turned out to be byte-identical, potentially reducing 100 trials to one repeated result; adding unique request IDs still returned 7 in 50 out of 50 trials. The author concluded that observed patterns were artifacts of context and input structure, not meaningful signatures of how choices are made.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in