Docker daemon.json log rotation (max-size/max-file) does NOT prevent unbounded disk growth when containers are created with --rm and ephemeral names (e.g. backup jobs, CI runners, batch tasks). Each new container gets its own fresh log budget. A script that spawns containers like docker run --rm --name job-$(date +%s) ... bypasses the per-container cap entirely — a verbose process (e.g. borg serve) can write 62G+ per invocation, and if the job re-runs, each invocation adds another 62G. The daemon.json rotation only constrains each individual container's log files, not aggregate disk usage across containers. On ZFS with Docker's zfs storage driver, this fills the pool to 0B AVAIL, wedging all services on the same pool.
For ephemeral containers that produce large stdout/stderr, disable Docker logging entirely at the container level:
docker run --rm --log-driver none ...This is the correct fix for backup jobs, batch processes, and any --rm container where logs aren't needed after exit. The daemon.json max-size/max-file settings are a false safety net here — they protect per-container but not per-invocation-pattern.
If you DO need logs from ephemeral containers, use --log-driver local with aggressive limits (max-size: 5m, max-file: 1) and monitor /var/lib/docker/containers/ total size externally via cron or node_exporter textfile collector.
Diagnostic: sudo find /var/lib/docker/containers/ -name '*.log' -size +100M -exec ls -lhS {} \; reveals which containers are the offenders.