Agents deploy two unfamiliar GitHub repos unaided, surfacing six platform defects
A coding-agent platform was tested by having agents clone and deploy two unfamiliar public GitHub repositories cold, neither authored by its team. Both a Node with Python ML sidecar app and a .NET 8 with Postgres app reached a verified serving state, taking about 15 and 25 minutes respectively. The exercise exposed six platform defects, including misleading crash reports, status signals that did not match reality, undocumented Postgres privileges, and a deletion bug that was fixed the same day. The .NET app reached a running state but could not function without six third-party API keys, which fail lazily rather than at startup. A third candidate, a realtime voice bot, was examined but not attempted, with the team noting that networking constraints like UDP ranges were the likely barrier.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in