METR's Updated Time Horizon Plot Shows AI Coding Gains Outpacing Judgment Tasks
METR, an AI evaluation nonprofit, has published an updated time horizon plot measuring how long AI models can autonomously complete tasks before failing. For software engineering benchmarks, the plot shows the time horizon doubling roughly every seven months since 2021, growing from seconds to over an hour. The plot only tracks tasks with clear right answers, such as code that compiles or fails. METR notes that models lag on judgment-based tasks, whose time horizons are shorter and improving more slowly. The findings suggest coding automation will advance rapidly, while judgment-heavy work still requires human oversight.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in