AegisCrawler open-sources governance-first browser scraping architecture

Developers behind AegisCrawler have open-sourced a web scraping framework designed to address long-term maintenance failures rather than just initial scraper creation. A Chrome extension records user interactions and DOM snapshots, which are processed by a deterministic rule generator to produce versioned, human-readable YAML scraping rules. Each rule must pass a replay verification step and receives a checksum-linked version history before it can be activated, ensuring full auditability. Optional LLM assistance is available strictly to propose rule patches, which require human approval and a security scan before taking effect — the model is never a live runtime dependency. The backend, built in Go with SQLite as a single binary, manages execution through lease-based job queuing, capped retries, tiered rate limiting, and per-site circuit breakers tied to immutable approved rule versions.