PHP log parser handles millions of lines at fixed 44.5 MB memory using cube aggregation

A development team built a custom PHP log parser for Laravel to answer granular questions about which web crawlers fetched which URLs, on which dates, and with what HTTP status codes. The team abandoned GoAccess because its pre-aggregated report panels cannot be cross-referenced, making it impossible to determine, for example, whether Googlebot received any 404 errors. Their parser supports 13 log formats, including variations of the common log format, Cloudflare JSON, Caddy JSON, and W3C-style IIS logs, all normalized to UTC at parse time. Benchmarks showed the parser processing 500,000 lines at a peak memory of just 44.5 MB — a figure that remains constant regardless of file size, since no data accumulates in memory during processing. Results are stored in a PostgreSQL table keyed on a multi-dimensional cube of project, date, hour, bot, URL, and status code, enabling flexible cross-querying of log data.