Misconfigured sudo logging silently consumed 10 GB by duplicating nightly database backups
A disk-usage alert on a Chatwoot production server revealed that /var/log/sudo-io had grown to 5.9 GB and was adding roughly 654 MB per week. Investigation showed the server had fewer sudo sessions than any other host in the fleet, yet its per-session log size was 8.3 times heavier than comparable machines. The root cause was a nightly offsite backup job that read database dumps through sudo, causing sudoers' log_output feature to write a full byte-for-byte copy of each dump into the audit log. Other hosts in the same fleet had already exempted these backup-read commands via /etc/sudoers.d/iolog-exemptions, but the Chatwoot host did not, because exemption files had been configured manually on each server with no consistency checks. The fix was to apply scoped command exemptions that suppress only the byte stream for known file-read operations while preserving session metadata, and to roll the exemptions out uniformly across all four logging hosts.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in