homelab-optimized/docs/services/individual/zot.md

Zot — OCI Pull-Through Registry Cache

Overview

Zot is a single-container OCI registry running on Atlantis that acts as a pull-through cache for Docker Hub (docker.io).

Why: Docker Hub rate-limits unauthenticated pulls (100/6h per IP). After the first pull, any Docker Hub image is served instantly from local cache — no upstream latency on deploys, watchtower updates, or container restarts.

Host	Atlantis
Port	5050 (5000 was taken by nginx)
Web UI	http://100.83.230.112:5050
Metrics	http://100.83.230.112:5050/metrics
Compose	hosts/synology/atlantis/zot.yaml
Config	hosts/synology/atlantis/zot/config.json
Data	/volume2/metadata/docker2/zot/data/ on Atlantis

---

Scope — What Zot Caches

Zot caches Docker Hub images only.

Docker's registry-mirrors mechanism — the standard way to redirect pulls through a local cache — only intercepts pulls from the default registry (Docker Hub). When a compose file explicitly names a registry (e.g. lscr.io/linuxserver/sonarr, ghcr.io/immich-app/immich-server), Docker contacts that registry directly and bypasses the mirror entirely.

Image type	Example	Goes through Zot?
Unqualified Docker Hub	postgres:16, nginx:alpine, redis:7	✅ Yes
Explicit Docker Hub	docker.io/library/postgres:16	✅ Yes
LinuxServer	lscr.io/linuxserver/sonarr:latest	❌ No — direct
GitHub packages	ghcr.io/immich-app/immich-server:release	❌ No — direct
Quay	quay.io/prometheus/node-exporter:latest	❌ No — direct

What this means in practice: Official images (postgres, redis, nginx, alpine, mariadb, mosquitto, etc.) are cached. LinuxServer, Immich, Authentik, Tdarr, and all other explicitly-prefixed images are not.

Expanding scope (future option)

To cache lscr.io/ghcr.io/quay.io images, all compose files referencing those images would need to be rewritten to pull from Zot directly (e.g. 100.83.230.112:5050/linuxserver/sonarr:latest), and Zot's sync config updated to poll those registries. This is ~60 compose file changes across all hosts — deferred for now.

---

How It Works

docker pull postgres:16
       │
       ▼
Docker daemon checks registry-mirrors first
       │
       ▼
Zot (100.83.230.112:5050)
  ├── cached? → serve instantly from local disk
  └── not cached? → fetch from registry-1.docker.io, cache, serve

docker pull lscr.io/linuxserver/sonarr:latest
       │
       ▼
Docker sees explicit registry prefix → bypasses mirror
       │
       ▼
lscr.io (direct, not cached)

All registries are configured with onDemand: true — nothing is pre-downloaded. Images are only cached when first requested, then served locally forever after (until GC removes unreferenced blobs after 24h).

Docker Hub note: Docker Hub does not support catalog listing, so poll mode cannot be used with it. On-demand only is correct.

---

Storage

Images are stored deduplicated at /volume2/metadata/docker2/zot/data/. GC runs every 24h to remove blobs no longer referenced by any cached manifest.

---

Per-Host Mirror Configuration

Each Docker host has been configured with Zot as a registry mirror. This is a Docker daemon setting — done once per host, not managed by Portainer.

Status

Host	Configured	Method
Atlantis	✅ Done (manual)	DSM Container Manager → Registry → Settings → Mirror → http://localhost:5050
Calypso	✅ Done (manual)	DSM Container Manager → Registry → Settings → Mirror → http://100.83.230.112:5050
homelab-vm	✅ Done	/etc/docker/daemon.json
NUC	✅ Done	/etc/docker/daemon.json
Pi-5	✅ Done	/etc/docker/daemon.json

daemon.json format (Linux hosts)

{
  "registry-mirrors": ["http://100.83.230.112:5050"],
  "log-driver": "json-file",
  "log-opts": { "max-size": "10m", "max-file": "3" }
}

After editing, restart Docker: sudo systemctl restart docker

---

Adding Credentials (optional)

Without credentials, public Docker Hub images pull fine but rate-limits apply (100 pulls/6h per IP). With a Docker Hub account: 200/hr per account.

To add credentials, create this file directly on Atlantis (never in git):

cat > /volume2/metadata/docker2/zot/credentials.json << 'EOF'
{
  "registry-1.docker.io": {
    "username": "your-dockerhub-username",
    "password": "your-dockerhub-token"  // pragma: allowlist secret
  }
}
EOF

Then uncomment the credentials volume mount in zot.yaml and add "credentialsFile": "/etc/zot/credentials.json" to the sync block in config.json. Restart the stack via Portainer.

---

Verifying It Works

# Zot health check
curl http://100.83.230.112:5050/v2/
# Returns empty body with HTTP 200

# View cached images
curl http://100.83.230.112:5050/v2/_catalog

# Pull a Docker Hub image on any configured host, then check catalog
docker pull alpine:latest
curl http://100.83.230.112:5050/v2/_catalog
# "library/alpine" should appear

---

Known Limitations

• lscr.io / ghcr.io / quay.io images bypass the cache — Docker's mirror mechanism only intercepts unqualified (Docker Hub) pulls. See Scope section above.
• Port 5050 — port 5000 was already in use by nginx on Atlantis.
• No TLS — Zot is internal-only (LAN + Tailscale). Not exposed via NPM.
• No authentication — read-only, internal network access only.

---

Prometheus Integration

Zot exposes metrics at /metrics. Add to Prometheus scrape config:

- job_name: 'zot'
  static_configs:
    - targets: ['100.83.230.112:5050']
  metrics_path: '/metrics'