homelab-optimized/docs/guides/deploy-new-service-gitops.md

# Deploying a New Service via GitOps

*Last Updated: March 7, 2026*

This guide walks through every step needed to go from a bare `docker-compose.yml` file to a
live, Portainer-managed container that auto-deploys on every future `git push`. It covers the
complete end-to-end flow: writing the compose file, wiring it into the repo, adding it to
Portainer, and verifying the CI pipeline fires correctly.

---

## How the pipeline works

```
You write a compose file
        │
        ▼
git push to main
        │
        ▼
Gitea CI runs portainer-deploy.yml
        │  detects which files changed
        │  matches them against live Portainer stacks
        ▼
Portainer redeploys matching stacks
        │
        ▼
Container restarts on the target host
        │
        ▼
ntfy push notification sent to your phone
```

Every push to `main` that touches a file under `hosts/**` or `common/**` triggers this
automatically. You never need to click "redeploy" in Portainer manually once the stack is
registered.

---

## Prerequisites

- [ ] SSH access to the target host (or Portainer UI access to it)
- [ ] Portainer access: `http://192.168.0.200:10000`
- [ ] Git push access to `git.vish.gg/Vish/homelab`
- [ ] A `docker-compose.yml` (or `.yaml`) for the service you want to run

---

## Step 1 — Choose your host

Pick the host where the container will run. Use this table:

| Host | Portainer Endpoint ID | Best for |
|---|---|---|
| **Atlantis** (DS1823xs+) | `2` | Media, high-storage services, primary NAS workloads |
| **Calypso** (DS723+) | `443397` | Secondary media, backup services, Authentik SSO |
| **Concord NUC** | `443398` | DNS (AdGuard), Home Assistant, network services |
| **Homelab VM** | `443399` | Monitoring, dev tools, lightweight web services |
| **RPi 5** | `443395` | IoT, uptime monitoring, edge sensors |

The file path you choose in Step 2 determines which host Portainer deploys to — they must match.

---

## Step 2 — Place the compose file in the repo

Clone the repo if you haven't already:

```bash
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
```

Create your compose file in the correct host directory:

```
hosts/synology/atlantis/      ← Atlantis
hosts/synology/calypso/       ← Calypso
hosts/physical/concord-nuc/   ← Concord NUC
hosts/vms/homelab-vm/         ← Homelab VM
hosts/edge/rpi5-vish/         ← Raspberry Pi 5
```

For example, deploying a service called `myapp` on the Homelab VM:

```bash
# create the file
nano hosts/vms/homelab-vm/myapp.yaml
```

---

## Step 3 — Write the compose file

Follow these conventions — they're enforced by the pre-commit hooks:

```yaml
# myapp — one-line description of what this does
# Port: 8080
services:
  myapp:
    image: vendor/myapp:1.2.3          # pin a version, not :latest
    container_name: myapp
    restart: unless-stopped            # always use unless-stopped, not always
    security_opt:
      - no-new-privileges:true
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Los_Angeles
      - SOME_SECRET=${MYAPP_SECRET}    # secrets via Portainer env vars, not plaintext
    volumes:
      - /home/homelab/docker/myapp:/config
    ports:
      - "8080:8080"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
```

**Key rules:**

| Rule | Why |
|---|---|
| `restart: unless-stopped` | Allows `docker stop` for maintenance without immediate restart |
| `no-new-privileges:true` | Prevents container from gaining extra Linux capabilities |
| Pin image versions | Renovate Bot will open a PR when a new version is available; `:latest` gives you no control |
| Secrets via `${VAR}` | Never commit real passwords or tokens — set them in Portainer's stack environment UI |
| 2-space indentation | `yamllint` will block the commit otherwise |

If your service needs a secret, use variable interpolation and set the value in Portainer later (Step 6):

```yaml
environment:
  - API_KEY=${MYAPP_API_KEY}
  - DB_PASSWORD="REDACTED_PASSWORD"
```

---

## Step 4 — Validate locally before pushing

The pre-commit hooks run this automatically on `git commit`, but you can run it manually first:

```bash
# Validate compose syntax
docker compose -f hosts/vms/homelab-vm/myapp.yaml config

# Run yamllint
yamllint -c .yamllint hosts/vms/homelab-vm/myapp.yaml

# Scan for accidentally committed secrets
detect-secrets scan hosts/vms/homelab-vm/myapp.yaml
```

If `docker compose config` returns clean YAML with no errors, you're good.

---

## Step 5 — Commit and push

```bash
git add hosts/vms/homelab-vm/myapp.yaml
git commit -m "feat: add myapp to homelab-vm

Brief description of what this service does and why."
git push origin main
```

The pre-commit hooks will run automatically on `git commit`:

- `yamllint` — checks indentation and syntax
- `docker-compose-check` — validates the compose file parses correctly
- `detect-secrets` — blocks commits containing passwords or tokens

If any hook fails, fix the issue and re-run `git commit`.

---

## Step 6 — Add the stack to Portainer

This is a one-time step per new service. After this, every future `git push` will
auto-redeploy the stack without any manual Portainer interaction.

1. Open Portainer: `http://192.168.0.200:10000`
2. In the left sidebar, select the correct **endpoint** (e.g. "Homelab VM")
3. Click **Stacks** → **+ Add stack**
4. Fill in the form:

| Field | Value |
|---|---|
| **Name** | `myapp-stack` (lowercase, hyphens, no spaces) |
| **Build method** | `Git Repository` |
| **Repository URL** | `https://git.vish.gg/Vish/homelab` |
| **Repository reference** | `refs/heads/main` |
| **Authentication** | Enable → username `vish`, password = "REDACTED_PASSWORD" token |
| **Compose path** | `hosts/vms/homelab-vm/myapp.yaml` |
| **GitOps updates** | ✅ Enable (toggle on) |

5. If your compose file uses `${VAR}` placeholders, scroll down to **Environment variables** and add each one:

| Variable | Value |
|---|---|
| `MYAPP_API_KEY` | `your-actual-key` |
| `MYAPP_DB_PASSWORD` | `your-actual-password` |

6. Click **Deploy the stack**

Portainer pulls the file from Gitea, runs `docker compose up -d`, and the container starts.

> **Note on GitOps updates toggle:** Enabling this makes Portainer poll Gitea every 5 minutes
> for changes. However, the CI pipeline (`portainer-deploy.yml`) handles redeployment on push
> much faster — the toggle is useful as a fallback but the CI is the primary mechanism.

---

## Step 7 — Verify the CI pipeline fires

After your initial push (Step 5), check that the CI workflow ran:

1. Go to `https://git.vish.gg/Vish/homelab/actions`
2. You should see a `portainer-deploy.yml` run triggered by your push
3. Click into it — the log should show:

```
Changed files (1):
  hosts/vms/homelab-vm/myapp.yaml

Checking 80 GitOps stacks for matches...

Deploying (GitOps): myapp-stack  (stack=XXX)
  File: hosts/vms/homelab-vm/myapp.yaml
  ✓ deployed successfully

==================================================
Deployed (1): myapp-stack
```

If the run shows "No stacks matched the changed files — nothing deployed", it means the
compose file path in Portainer doesn't exactly match the path in the repo. Double-check the
**Compose path** field in Portainer (Step 6, step 4) — it must be identical, including the
`hosts/` prefix.

---

## Step 8 — Verify the container is running

On the Homelab VM (which is the machine you're reading this on):

```bash
docker ps --filter name=myapp
docker logs myapp --tail 50
```

For other hosts, SSH in first:

```bash
ssh calypso
sudo /usr/local/bin/docker ps --filter name=myapp
```

Or use Portainer's built-in log viewer: **Stacks** → `myapp-stack` → click the container name → **Logs**.

---

## Step 9 — Test future auto-deploys work

Make a trivial change (add a comment, bump an env var) and push:

```bash
# edit the file
nano hosts/vms/homelab-vm/myapp.yaml

git add hosts/vms/homelab-vm/myapp.yaml
git commit -m "chore: test auto-deploy for myapp"
git push origin main
```

Watch `https://git.vish.gg/Vish/homelab/actions` — a new `portainer-deploy.yml` run should
appear within 10–15 seconds, complete in under a minute, and the container will restart with
the new config.

---

## Common problems

### "No stacks matched the changed files"

The path stored in Portainer doesn't match the file path in the repo.

- In Portainer: **Stacks** → your stack → **Editor** tab → check the **Compose path** field
- It must exactly match the repo path, e.g. `hosts/vms/homelab-vm/myapp.yaml`
- Note: All Portainer stacks use canonical `hosts/` paths — ensure the Compose path field matches exactly (e.g. `hosts/synology/calypso/myapp.yaml`)

---

### "Conflict. The container name is already in use"

A container with the same `container_name` already exists on the host from a previous manual deploy or a different stack.

```bash
# Find and remove it
docker rm -f myapp

# Then re-trigger: edit any line in the compose file and push
```

Or via Portainer API:
```bash
curl -X DELETE \
  -H "X-API-Key: $PORTAINER_TOKEN" \
  "http://192.168.0.200:10000/api/endpoints/443399/docker/containers/$(docker inspect --format '{{.Id}}' myapp)?force=true"
```

---

### Pre-commit hook blocks the commit

**yamllint indentation error** — you have 4-space indent instead of 2-space. Fix with:
```bash
# Check which lines are wrong
yamllint -c .yamllint hosts/vms/homelab-vm/myapp.yaml
```

**detect-secrets blocks a secret** — you have a real token/password in the file. Move it to a `${VAR}` placeholder and set the value in Portainer's environment variables instead.

**docker-compose-check fails** — the compose file has a syntax error:
```bash
docker compose -f hosts/vms/homelab-vm/myapp.yaml config
```

---

### Portainer shows HTTP 500 on redeploy

Usually a docker-level error — check the full error message in the CI log or Portainer stack events. Common causes:

- Port already in use on the host → change the external port mapping
- Volume path doesn't exist → create the directory on the host first
- Image pull failed (private registry, wrong tag) → verify the image name and tag

---

## Checklist

- [ ] Compose file placed in correct `hosts/<host>/` directory
- [ ] Image pinned to a specific version (not `:latest`)
- [ ] `restart: unless-stopped` set
- [ ] Secrets use `${VAR}` placeholders, not plaintext values
- [ ] `docker compose config` passes with no errors
- [ ] `git push` to `main` succeeded
- [ ] Stack added to Portainer with correct path and environment variables
- [ ] CI run at `git.vish.gg/Vish/homelab/actions` shows successful deploy
- [ ] `docker ps` on the target host confirms container is running
- [ ] Future push triggers auto-redeploy (tested with a trivial change)

---

## Related guides

- [Add New Subdomain](add-new-subdomain.md) — wire up a public URL via Cloudflare + NPM
- [Renovate Bot](renovate-bot.md) — how image version update PRs work
- [Portainer API Guide](../admin/PORTAINER_API_GUIDE.md) — managing stacks via API
- [Add New Service Runbook](../runbooks/add-new-service.md) — extended checklist with monitoring, backups, SSO