Sanitized mirror from private repository - 2026-04-18 11:19:59 UTC

docs/guides/docker-log-rotation.md

@@ -0,0 +1,104 @@
+# Docker Log Rotation
+
+Prevents unbounded container log growth across all homelab hosts.
+Docker's default is no limit — a single chatty container can fill a disk.
+
+## Target Config
+
+```json
+{
+  "log-driver": "json-file",
+  "log-opts": {
+    "max-size": "10m",
+    "max-file": "3"
+  }
+}
+```
+
+10 MB × 3 files = max 30 MB per container.
+
+---
+
+## Linux Hosts (Ansible)
+
+Covers: **homelab-vm**, **concord-nuc**, **pi-5**, **matrix-ubuntu**
+
+```bash
+cd ansible/automation
+ansible-playbook -i hosts.ini playbooks/configure_docker_logging.yml
+```
+
+Dry-run first:
+```bash
+ansible-playbook -i hosts.ini playbooks/configure_docker_logging.yml --check
+```
+
+Single host:
+```bash
+ansible-playbook -i hosts.ini playbooks/configure_docker_logging.yml -e "host_target=homelab"
+```
+
+The playbook:
+1. Reads existing `daemon.json` (preserves existing keys)
+2. Merges in the log config
+3. Validates JSON
+4. Restarts the Docker daemon
+5. Verifies the logging driver is active
+
+### After running — recreate existing containers
+
+The daemon default only applies to **new** containers. Existing ones keep their old (unlimited) config until recreated:
+
+```bash
+# On each host, per stack:
+docker compose -f <compose-file> up --force-recreate -d
+```
+
+Or verify a specific container has the limit:
+```bash
+docker inspect <container> | jq '.[0].HostConfig.LogConfig'
+# Should show: {"Type":"json-file","Config":{"max-file":"3","max-size":"10m"}}
+```
+
+---
+
+## Synology Hosts (Not Applicable)
+
+**atlantis**, **calypso**, and **setillo** all use DSM's native `db` log driver (Synology Container Manager default). This driver stores container logs in an internal database managed by DSM — it does not produce json-file logs and does not support `max-size`/`max-file` options.
+
+**Do not change the log driver on Synology hosts.** Switching to `json-file` would break the Container Manager log viewer in DSM, and the `db` driver already handles log retention internally.
+
+To verify:
+```bash
+ssh atlantis "/var/packages/REDACTED_APP_PASSWORD/target/usr/bin/docker info 2>&1 | grep -i 'logging driver'"
+# Logging Driver: db  ← expected
+```
+
+---
+
+## Guava (TrueNAS SCALE)
+
+TrueNAS SCALE uses K3s (Kubernetes) as its primary app runtime — standard Docker daemon log limits don't apply to apps deployed through the UI. If you have standalone Docker containers on guava, apply the Linux procedure above via Ansible (`truenas-scale` host in inventory).
+
+---
+
+## Verification
+
+```bash
+# Check largest existing logs before rotation
+ssh <host> "sudo find /var/lib/docker/containers -name '*-json.log' -exec du -sh {} \; 2>/dev/null | sort -h | tail -10"
+
+# Check a container's effective log config
+docker inspect <name> | jq '.[0].HostConfig.LogConfig'
+
+# Check daemon logging driver
+docker info --format '{{.LoggingDriver}}'
+```
+
+---
+
+## What This Doesn't Do
+
+- **Does not truncate existing log files** — those are handled by the reactive `log_rotation.yml` playbook
+- **Does not apply to containers started before the daemon restart** — recreate them
+- **Does not configure per-container overrides** — individual services can still override in their compose with `logging:` if needed