One developer built a 38.5M-domain open search engine on a 2 GB server

A solo developer launched cl0q, a free, open-source search engine that does not log searches or require accounts, beginning its first crawl in April 2025. Running on a modest 2 GB VPS, the project grew to index 38.5 million domains and 24.8 million pages, with 8.5 million domains added in June alone. Key challenges included avoiding subdomain traps, managing chronic disk space shortages, and resolving database contention between the crawler and search functions. Certificate Transparency logs proved a valuable discovery tool, surfacing roughly 1,700 new domains per minute at peak. The developer's long-term target is 100 million domains, with storage now offloaded to a remote host connected via Tailscale.