Vish/homelab-optimized
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sanitized mirror from private repository - 2026-03-18 10:31:50 UTC
Some checks failed
Documentation / Build Docusaurus (push) Failing after 21m3s
Details
Documentation / Deploy to GitHub Pages (push) Has been skipped
Details

Browse Source
This commit is contained in:
Gitea Mirror Bot
2026-03-18 10:31:50 +00:00
commit 8e49624d78
1221 changed files with 304405 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

367
docs/guides/deploy-new-service-gitops.md Normal file
View File

@@ -0,0 +1,367 @@
# Deploying a New Service via GitOps
*Last Updated: March 7, 2026*
This guide walks through every step needed to go from a bare `docker-compose.yml` file to a
live, Portainer-managed container that auto-deploys on every future `git push`. It covers the
complete end-to-end flow: writing the compose file, wiring it into the repo, adding it to
Portainer, and verifying the CI pipeline fires correctly.
---
## How the pipeline works
```
You write a compose file
│
▼
git push to main
│
▼
Gitea CI runs portainer-deploy.yml
│ detects which files changed
│ matches them against live Portainer stacks
▼
Portainer redeploys matching stacks
│
▼
Container restarts on the target host
│
▼
ntfy push notification sent to your phone
```
Every push to `main` that touches a file under `hosts/**` or `common/**` triggers this
automatically. You never need to click "redeploy" in Portainer manually once the stack is
registered.
---
## Prerequisites
- [ ] SSH access to the target host (or Portainer UI access to it)
- [ ] Portainer access: `http://192.168.0.200:10000`
- [ ] Git push access to `git.vish.gg/Vish/homelab`
- [ ] A `docker-compose.yml` (or `.yaml`) for the service you want to run
---
## Step 1 — Choose your host
Pick the host where the container will run. Use this table:
| Host | Portainer Endpoint ID | Best for |
|---|---|---|
| **Atlantis** (DS1823xs+) | `2` | Media, high-storage services, primary NAS workloads |
| **Calypso** (DS723+) | `443397` | Secondary media, backup services, Authentik SSO |
| **Concord NUC** | `443398` | DNS (AdGuard), Home Assistant, network services |
| **Homelab VM** | `443399` | Monitoring, dev tools, lightweight web services |
| **RPi 5** | `443395` | IoT, uptime monitoring, edge sensors |
The file path you choose in Step 2 determines which host Portainer deploys to — they must match.
---
## Step 2 — Place the compose file in the repo
Clone the repo if you haven't already:
```bash
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
```
Create your compose file in the correct host directory:
```
hosts/synology/atlantis/ ← Atlantis
hosts/synology/calypso/ ← Calypso
hosts/physical/concord-nuc/ ← Concord NUC
hosts/vms/homelab-vm/ ← Homelab VM
hosts/edge/rpi5-vish/ ← Raspberry Pi 5
```
For example, deploying a service called `myapp` on the Homelab VM:
```bash
# create the file
nano hosts/vms/homelab-vm/myapp.yaml
```
---
## Step 3 — Write the compose file
Follow these conventions — they're enforced by the pre-commit hooks:
```yaml
# myapp — one-line description of what this does
# Port: 8080
services:
myapp:
image: vendor/myapp:1.2.3 # pin a version, not :latest
container_name: myapp
restart: unless-stopped # always use unless-stopped, not always
security_opt:
- no-new-privileges:true
environment:
- PUID=1000
- PGID=1000
- TZ=America/Los_Angeles
- SOME_SECRET=${MYAPP_SECRET} # secrets via Portainer env vars, not plaintext
volumes:
- /home/homelab/docker/myapp:/config
ports:
- "8080:8080"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 20s
```
**Key rules:**
| Rule | Why |
|---|---|
| `restart: unless-stopped` | Allows `docker stop` for maintenance without immediate restart |
| `no-new-privileges:true` | Prevents container from gaining extra Linux capabilities |
| Pin image versions | Renovate Bot will open a PR when a new version is available; `:latest` gives you no control |
| Secrets via `${VAR}` | Never commit real passwords or tokens — set them in Portainer's stack environment UI |
| 2-space indentation | `yamllint` will block the commit otherwise |
If your service needs a secret, use variable interpolation and set the value in Portainer later (Step 6):
```yaml
environment:
- API_KEY=${MYAPP_API_KEY}
- DB_PASSWORD="REDACTED_PASSWORD"
```
---
## Step 4 — Validate locally before pushing
The pre-commit hooks run this automatically on `git commit`, but you can run it manually first:
```bash
# Validate compose syntax
docker compose -f hosts/vms/homelab-vm/myapp.yaml config
# Run yamllint
yamllint -c .yamllint hosts/vms/homelab-vm/myapp.yaml
# Scan for accidentally committed secrets
detect-secrets scan hosts/vms/homelab-vm/myapp.yaml
```
If `docker compose config` returns clean YAML with no errors, you're good.
---
## Step 5 — Commit and push
```bash
git add hosts/vms/homelab-vm/myapp.yaml
git commit -m "feat: add myapp to homelab-vm
Brief description of what this service does and why."
git push origin main
```
The pre-commit hooks will run automatically on `git commit`:
- `yamllint` — checks indentation and syntax
- `docker-compose-check` — validates the compose file parses correctly
- `detect-secrets` — blocks commits containing passwords or tokens
If any hook fails, fix the issue and re-run `git commit`.
---
## Step 6 — Add the stack to Portainer
This is a one-time step per new service. After this, every future `git push` will
auto-redeploy the stack without any manual Portainer interaction.
1. Open Portainer: `http://192.168.0.200:10000`
2. In the left sidebar, select the correct **endpoint** (e.g. "Homelab VM")
3. Click **Stacks** → **+ Add stack**
4. Fill in the form:
| Field | Value |
|---|---|
| **Name** | `myapp-stack` (lowercase, hyphens, no spaces) |
| **Build method** | `Git Repository` |
| **Repository URL** | `https://git.vish.gg/Vish/homelab` |
| **Repository reference** | `refs/heads/main` |
| **Authentication** | Enable → username `vish`, password = "REDACTED_PASSWORD" token |
| **Compose path** | `hosts/vms/homelab-vm/myapp.yaml` |
| **GitOps updates** | ✅ Enable (toggle on) |
5. If your compose file uses `${VAR}` placeholders, scroll down to **Environment variables** and add each one:
| Variable | Value |
|---|---|
| `MYAPP_API_KEY` | `your-actual-key` |
| `MYAPP_DB_PASSWORD` | `your-actual-password` |
6. Click **Deploy the stack**
Portainer pulls the file from Gitea, runs `docker compose up -d`, and the container starts.
> **Note on GitOps updates toggle:** Enabling this makes Portainer poll Gitea every 5 minutes
> for changes. However, the CI pipeline (`portainer-deploy.yml`) handles redeployment on push
> much faster — the toggle is useful as a fallback but the CI is the primary mechanism.
---
## Step 7 — Verify the CI pipeline fires
After your initial push (Step 5), check that the CI workflow ran:
1. Go to `https://git.vish.gg/Vish/homelab/actions`
2. You should see a `portainer-deploy.yml` run triggered by your push
3. Click into it — the log should show:
```
Changed files (1):
hosts/vms/homelab-vm/myapp.yaml
Checking 80 GitOps stacks for matches...
Deploying (GitOps): myapp-stack (stack=XXX)
File: hosts/vms/homelab-vm/myapp.yaml
✓ deployed successfully
==================================================
Deployed (1): myapp-stack
```
If the run shows "No stacks matched the changed files — nothing deployed", it means the
compose file path in Portainer doesn't exactly match the path in the repo. Double-check the
**Compose path** field in Portainer (Step 6, step 4) — it must be identical, including the
`hosts/` prefix.
---
## Step 8 — Verify the container is running
On the Homelab VM (which is the machine you're reading this on):
```bash
docker ps --filter name=myapp
docker logs myapp --tail 50
```
For other hosts, SSH in first:
```bash
ssh calypso
sudo /usr/local/bin/docker ps --filter name=myapp
```
Or use Portainer's built-in log viewer: **Stacks** → `myapp-stack` → click the container name → **Logs**.
---
## Step 9 — Test future auto-deploys work
Make a trivial change (add a comment, bump an env var) and push:
```bash
# edit the file
nano hosts/vms/homelab-vm/myapp.yaml
git add hosts/vms/homelab-vm/myapp.yaml
git commit -m "chore: test auto-deploy for myapp"
git push origin main
```
Watch `https://git.vish.gg/Vish/homelab/actions` — a new `portainer-deploy.yml` run should
appear within 10–15 seconds, complete in under a minute, and the container will restart with
the new config.
---
## Common problems
### "No stacks matched the changed files"
The path stored in Portainer doesn't match the file path in the repo.
- In Portainer: **Stacks** → your stack → **Editor** tab → check the **Compose path** field
- It must exactly match the repo path, e.g. `hosts/vms/homelab-vm/myapp.yaml`
- Note: All Portainer stacks use canonical `hosts/` paths — ensure the Compose path field matches exactly (e.g. `hosts/synology/calypso/myapp.yaml`)
---
### "Conflict. The container name is already in use"
A container with the same `container_name` already exists on the host from a previous manual deploy or a different stack.
```bash
# Find and remove it
docker rm -f myapp
# Then re-trigger: edit any line in the compose file and push
```
Or via Portainer API:
```bash
curl -X DELETE \
-H "X-API-Key: $PORTAINER_TOKEN" \
"http://192.168.0.200:10000/api/endpoints/443399/docker/containers/$(docker inspect --format '{{.Id}}' myapp)?force=true"
```
---
### Pre-commit hook blocks the commit
**yamllint indentation error** — you have 4-space indent instead of 2-space. Fix with:
```bash
# Check which lines are wrong
yamllint -c .yamllint hosts/vms/homelab-vm/myapp.yaml
```
**detect-secrets blocks a secret** — you have a real token/password in the file. Move it to a `${VAR}` placeholder and set the value in Portainer's environment variables instead.
**docker-compose-check fails** — the compose file has a syntax error:
```bash
docker compose -f hosts/vms/homelab-vm/myapp.yaml config
```
---
### Portainer shows HTTP 500 on redeploy
Usually a docker-level error — check the full error message in the CI log or Portainer stack events. Common causes:
- Port already in use on the host → change the external port mapping
- Volume path doesn't exist → create the directory on the host first
- Image pull failed (private registry, wrong tag) → verify the image name and tag
---
## Checklist
- [ ] Compose file placed in correct `hosts/<host>/` directory
- [ ] Image pinned to a specific version (not `:latest`)
- [ ] `restart: unless-stopped` set
- [ ] Secrets use `${VAR}` placeholders, not plaintext values
- [ ] `docker compose config` passes with no errors
- [ ] `git push` to `main` succeeded
- [ ] Stack added to Portainer with correct path and environment variables
- [ ] CI run at `git.vish.gg/Vish/homelab/actions` shows successful deploy
- [ ] `docker ps` on the target host confirms container is running
- [ ] Future push triggers auto-redeploy (tested with a trivial change)
---
## Related guides
- [Add New Subdomain](add-new-subdomain.md) — wire up a public URL via Cloudflare + NPM
- [Renovate Bot](renovate-bot.md) — how image version update PRs work
- [Portainer API Guide](../admin/PORTAINER_API_GUIDE.md) — managing stacks via API
- [Add New Service Runbook](../runbooks/add-new-service.md) — extended checklist with monitoring, backups, SSO