185 lines
5.7 KiB
Markdown
185 lines
5.7 KiB
Markdown
# Zot — OCI Pull-Through Registry Cache
|
|
|
|
## Overview
|
|
|
|
Zot is a single-container OCI registry running on Atlantis that acts as a
|
|
pull-through cache for Docker Hub (`docker.io`).
|
|
|
|
**Why:** Docker Hub rate-limits unauthenticated pulls (100/6h per IP). After the
|
|
first pull, any Docker Hub image is served instantly from local cache — no upstream
|
|
latency on deploys, watchtower updates, or container restarts.
|
|
|
|
| | |
|
|
|---|---|
|
|
| **Host** | Atlantis |
|
|
| **Port** | 5050 (5000 was taken by nginx) |
|
|
| **Web UI** | http://100.83.230.112:5050 |
|
|
| **Metrics** | http://100.83.230.112:5050/metrics |
|
|
| **Compose** | `hosts/synology/atlantis/zot.yaml` |
|
|
| **Config** | `hosts/synology/atlantis/zot/config.json` |
|
|
| **Data** | `/volume2/metadata/docker2/zot/data/` on Atlantis |
|
|
|
|
---
|
|
|
|
## Scope — What Zot Caches
|
|
|
|
**Zot caches Docker Hub images only.**
|
|
|
|
Docker's `registry-mirrors` mechanism — the standard way to redirect pulls through
|
|
a local cache — only intercepts pulls from the **default registry (Docker Hub)**.
|
|
When a compose file explicitly names a registry (e.g. `lscr.io/linuxserver/sonarr`,
|
|
`ghcr.io/immich-app/immich-server`), Docker contacts that registry directly and
|
|
bypasses the mirror entirely.
|
|
|
|
| Image type | Example | Goes through Zot? |
|
|
|---|---|---|
|
|
| Unqualified Docker Hub | `postgres:16`, `nginx:alpine`, `redis:7` | ✅ Yes |
|
|
| Explicit Docker Hub | `docker.io/library/postgres:16` | ✅ Yes |
|
|
| LinuxServer | `lscr.io/linuxserver/sonarr:latest` | ❌ No — direct |
|
|
| GitHub packages | `ghcr.io/immich-app/immich-server:release` | ❌ No — direct |
|
|
| Quay | `quay.io/prometheus/node-exporter:latest` | ❌ No — direct |
|
|
|
|
**What this means in practice:** Official images (`postgres`, `redis`, `nginx`,
|
|
`alpine`, `mariadb`, `mosquitto`, etc.) are cached. LinuxServer, Immich, Authentik,
|
|
Tdarr, and all other explicitly-prefixed images are not.
|
|
|
|
### Expanding scope (future option)
|
|
|
|
To cache `lscr.io`/`ghcr.io`/`quay.io` images, all compose files referencing those
|
|
images would need to be rewritten to pull from Zot directly (e.g.
|
|
`100.83.230.112:5050/linuxserver/sonarr:latest`), and Zot's sync config updated to
|
|
poll those registries. This is ~60 compose file changes across all hosts — deferred
|
|
for now.
|
|
|
|
---
|
|
|
|
## How It Works
|
|
|
|
```
|
|
docker pull postgres:16
|
|
│
|
|
▼
|
|
Docker daemon checks registry-mirrors first
|
|
│
|
|
▼
|
|
Zot (100.83.230.112:5050)
|
|
├── cached? → serve instantly from local disk
|
|
└── not cached? → fetch from registry-1.docker.io, cache, serve
|
|
|
|
docker pull lscr.io/linuxserver/sonarr:latest
|
|
│
|
|
▼
|
|
Docker sees explicit registry prefix → bypasses mirror
|
|
│
|
|
▼
|
|
lscr.io (direct, not cached)
|
|
```
|
|
|
|
All registries are configured with `onDemand: true` — nothing is pre-downloaded.
|
|
Images are only cached when first requested, then served locally forever after
|
|
(until GC removes unreferenced blobs after 24h).
|
|
|
|
**Docker Hub note:** Docker Hub does not support catalog listing, so poll mode
|
|
cannot be used with it. On-demand only is correct.
|
|
|
|
---
|
|
|
|
## Storage
|
|
|
|
Images are stored deduplicated at `/volume2/metadata/docker2/zot/data/`.
|
|
GC runs every 24h to remove blobs no longer referenced by any cached manifest.
|
|
|
|
---
|
|
|
|
## Per-Host Mirror Configuration
|
|
|
|
Each Docker host has been configured with Zot as a registry mirror.
|
|
This is a Docker daemon setting — done once per host, not managed by Portainer.
|
|
|
|
### Status
|
|
|
|
| Host | Configured | Method |
|
|
|---|---|---|
|
|
| Atlantis | ✅ Done (manual) | DSM Container Manager → Registry → Settings → Mirror → `http://localhost:5050` |
|
|
| Calypso | ✅ Done (manual) | DSM Container Manager → Registry → Settings → Mirror → `http://100.83.230.112:5050` |
|
|
| homelab-vm | ✅ Done | `/etc/docker/daemon.json` |
|
|
| NUC | ✅ Done | `/etc/docker/daemon.json` |
|
|
| Pi-5 | ✅ Done | `/etc/docker/daemon.json` |
|
|
|
|
### daemon.json format (Linux hosts)
|
|
|
|
```json
|
|
{
|
|
"registry-mirrors": ["http://100.83.230.112:5050"],
|
|
"log-driver": "json-file",
|
|
"log-opts": { "max-size": "10m", "max-file": "3" }
|
|
}
|
|
```
|
|
|
|
After editing, restart Docker: `sudo systemctl restart docker`
|
|
|
|
---
|
|
|
|
## Adding Credentials (optional)
|
|
|
|
Without credentials, public Docker Hub images pull fine but rate-limits apply
|
|
(100 pulls/6h per IP). With a Docker Hub account: 200/hr per account.
|
|
|
|
To add credentials, create this file **directly on Atlantis** (never in git):
|
|
|
|
```bash
|
|
cat > /volume2/metadata/docker2/zot/credentials.json << 'EOF'
|
|
{
|
|
"registry-1.docker.io": {
|
|
"username": "your-dockerhub-username",
|
|
"password": "your-dockerhub-token" // pragma: allowlist secret
|
|
}
|
|
}
|
|
EOF
|
|
```
|
|
|
|
Then uncomment the credentials volume mount in `zot.yaml` and add
|
|
`"credentialsFile": "/etc/zot/credentials.json"` to the `sync` block in
|
|
`config.json`. Restart the stack via Portainer.
|
|
|
|
---
|
|
|
|
## Verifying It Works
|
|
|
|
```bash
|
|
# Zot health check
|
|
curl http://100.83.230.112:5050/v2/
|
|
# Returns empty body with HTTP 200
|
|
|
|
# View cached images
|
|
curl http://100.83.230.112:5050/v2/_catalog
|
|
|
|
# Pull a Docker Hub image on any configured host, then check catalog
|
|
docker pull alpine:latest
|
|
curl http://100.83.230.112:5050/v2/_catalog
|
|
# "library/alpine" should appear
|
|
```
|
|
|
|
---
|
|
|
|
## Known Limitations
|
|
|
|
- **lscr.io / ghcr.io / quay.io images bypass the cache** — Docker's mirror
|
|
mechanism only intercepts unqualified (Docker Hub) pulls. See Scope section above.
|
|
- **Port 5050** — port 5000 was already in use by nginx on Atlantis.
|
|
- **No TLS** — Zot is internal-only (LAN + Tailscale). Not exposed via NPM.
|
|
- **No authentication** — read-only, internal network access only.
|
|
|
|
---
|
|
|
|
## Prometheus Integration
|
|
|
|
Zot exposes metrics at `/metrics`. Add to Prometheus scrape config:
|
|
|
|
```yaml
|
|
- job_name: 'zot'
|
|
static_configs:
|
|
- targets: ['100.83.230.112:5050']
|
|
metrics_path: '/metrics'
|
|
```
|