Sanitized mirror from private repository - 2026-03-31 10:10:42 UTC
This commit is contained in:
Gitea Mirror Bot
271
docs/admin/secrets-management.md
Normal file
271
docs/admin/secrets-management.md
Normal file
@@ -0,0 +1,271 @@
|
||||
# Secrets Management Strategy
|
||||
|
||||
**Last updated**: March 2026
|
||||
**Status**: Active policy
|
||||
|
||||
This document describes how credentials and secrets are managed across the homelab infrastructure.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The homelab uses a **layered secrets strategy** with four components:
|
||||
|
||||
| Layer | Tool | Purpose |
|
||||
|-------|------|---------|
|
||||
| **Source of truth** | Vaultwarden | Store all credentials; accessible via browser + Bitwarden client apps |
|
||||
| **CI/CD secrets** | Gitea Actions secrets | Credentials needed by workflows (Portainer token, CF token, etc.) |
|
||||
| **Runtime injection** | Portainer stack env vars | Secrets passed into containers at deploy time without touching compose files |
|
||||
| **Public mirror protection** | `sanitize.py` | Strips secrets from the private repo before mirroring to `homelab-optimized` |
|
||||
|
||||
---
|
||||
|
||||
## Vaultwarden — Source of Truth
|
||||
|
||||
All credentials **must** be saved in Vaultwarden before being used anywhere else.
|
||||
|
||||
- **URL**: `https://vault.vish.gg` (or via Tailscale: `vault.tail.vish.gg`)
|
||||
- **Collection structure**:
|
||||
```
|
||||
Homelab/
|
||||
├── API Keys/ (OpenAI, Cloudflare, Spotify, etc.)
|
||||
├── Gitea API Tokens/ (PATs for automation)
|
||||
├── Gmail App Passwords/
|
||||
├── Service Passwords/ (per-service DB passwords, admin passwords)
|
||||
├── SMTP/ (app passwords, SMTP configs)
|
||||
├── SNMP/ (SNMPv3 auth and priv passwords)
|
||||
└── Infrastructure/ (Watchtower token, Portainer token, etc.)
|
||||
```
|
||||
|
||||
**Rule**: If a credential isn't in Vaultwarden, it doesn't exist.
|
||||
|
||||
---
|
||||
|
||||
## Gitea Actions Secrets
|
||||
|
||||
For credentials used by CI/CD workflows, store them as Gitea repository secrets at:
|
||||
`https://git.vish.gg/Vish/homelab/settings/actions/secrets`
|
||||
|
||||
### Currently configured secrets
|
||||
|
||||
| Secret | Used by | Purpose |
|
||||
|--------|---------|---------|
|
||||
| `GIT_TOKEN` | All workflows | Gitea PAT for repo checkout and Portainer git auth |
|
||||
| `PORTAINER_TOKEN` | `portainer-deploy.yml` | Portainer API token |
|
||||
| `PORTAINER_URL` | `portainer-deploy.yml` | Portainer base URL |
|
||||
| `CF_TOKEN` | `portainer-deploy.yml`, `dns-audit.yml` | Cloudflare API token |
|
||||
| `NPM_EMAIL` | `dns-audit.yml` | Nginx Proxy Manager login email |
|
||||
| `NPM_PASSWORD` | `dns-audit.yml` | Nginx Proxy Manager password |
|
||||
| `NTFY_URL` | `portainer-deploy.yml`, `dns-audit.yml` | ntfy notification topic URL |
|
||||
| `HOMARR_SECRET_KEY` | `portainer-deploy.yml` | Homarr session encryption key |
|
||||
| `IMMICH_DB_USERNAME` | `portainer-deploy.yml` | Immich database username |
|
||||
| `IMMICH_DB_PASSWORD` | `portainer-deploy.yml` | Immich database password |
|
||||
| `IMMICH_DB_DATABASE_NAME` | `portainer-deploy.yml` | Immich database name |
|
||||
| `IMMICH_JWT_SECRET` | `portainer-deploy.yml` | Immich JWT signing secret |
|
||||
| `PUBLIC_REPO_TOKEN` | `mirror-to-public.yaml` | PAT for pushing to `homelab-optimized` |
|
||||
| `RENOVATE_TOKEN` | `renovate.yml` | PAT for Renovate dependency bot |
|
||||
|
||||
### Adding a new Gitea secret
|
||||
|
||||
```bash
|
||||
# Via API
|
||||
TOKEN="your-gitea-pat"
|
||||
curl -X PUT "https://git.vish.gg/api/v1/repos/Vish/homelab/actions/secrets/MY_SECRET" \
|
||||
-H "Authorization: token $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"data": "actual-secret-value"}'
|
||||
```
|
||||
|
||||
Or via the Gitea web UI: Repository → Settings → Actions → Secrets → Add Secret.
|
||||
|
||||
---
|
||||
|
||||
## Portainer Runtime Injection
|
||||
|
||||
For secrets needed inside containers at runtime, Portainer injects them as environment variables at deploy time. This keeps credentials out of compose files.
|
||||
|
||||
### How it works
|
||||
|
||||
1. The compose file uses `${VAR_NAME}` syntax — no hardcoded value
|
||||
2. `portainer-deploy.yml` defines a `DDNS_STACK_ENV` dict mapping stack names to env var lists
|
||||
3. On every push to `main`, the workflow calls Portainer's redeploy API with the env vars from Gitea secrets
|
||||
4. Portainer passes them to the running containers
|
||||
|
||||
### Currently injected stacks
|
||||
|
||||
| Stack name | Injected vars | Source secret |
|
||||
|------------|--------------|---------------|
|
||||
| `dyndns-updater` | `CLOUDFLARE_API_TOKEN` | `CF_TOKEN` |
|
||||
| `dyndns-updater-stack` | `CLOUDFLARE_API_TOKEN` | `CF_TOKEN` |
|
||||
| `homarr-stack` | `HOMARR_SECRET_KEY` | `HOMARR_SECRET_KEY` |
|
||||
| `retro-site` | `GIT_TOKEN` | `GIT_TOKEN` |
|
||||
| `immich-stack` | `DB_USERNAME`, `DB_PASSWORD`, `DB_DATABASE_NAME`, `JWT_SECRET`, etc. | `IMMICH_DB_*`, `IMMICH_JWT_SECRET` |
|
||||
|
||||
### Adding a new injected stack
|
||||
|
||||
1. Add the secret to Gitea (see above)
|
||||
2. Add it to the workflow env block in `portainer-deploy.yml`:
|
||||
```yaml
|
||||
MY_SECRET: ${{ secrets.MY_SECRET }}
|
||||
```
|
||||
3. Read it in the Python block:
|
||||
```python
|
||||
my_secret = os.environ.get('MY_SECRET', '')
|
||||
```
|
||||
4. Add the stack to `DDNS_STACK_ENV`:
|
||||
```python
|
||||
'my-stack-name': [{'name': 'MY_VAR', 'value': my_secret}],
|
||||
```
|
||||
5. In the compose file, reference it as `${MY_VAR}` — no default value
|
||||
|
||||
---
|
||||
|
||||
## `.env.example` Pattern for New Services
|
||||
|
||||
When adding a new service that needs credentials:
|
||||
|
||||
1. **Never** put real values in the compose/stack YAML file
|
||||
2. Create a `.env.example` alongside the compose file showing the variable names with `REDACTED_*` placeholders:
|
||||
```env
|
||||
# Copy to .env and fill in real values (stored in Vaultwarden)
|
||||
MY_SERVICE_DB_PASSWORD="REDACTED_PASSWORD"
|
||||
MY_SERVICE_SECRET_KEY=REDACTED_SECRET_KEY
|
||||
MY_SERVICE_SMTP_PASSWORD="REDACTED_PASSWORD"
|
||||
```
|
||||
3. The real `.env` file is blocked by `.gitignore` (`*.env` rule)
|
||||
4. Reference variables in the compose file: `${MY_SERVICE_DB_PASSWORD}`
|
||||
5. Either:
|
||||
- Set the vars in Portainer stack environment (for GitOps stacks), or
|
||||
- Add to `DDNS_STACK_ENV` in `portainer-deploy.yml` (for auto-injection)
|
||||
|
||||
---
|
||||
|
||||
## Public Mirror Protection (`sanitize.py`)
|
||||
|
||||
The private repo (`homelab`) is mirrored to a public repo (`homelab-optimized`) via the `mirror-to-public.yaml` workflow. Before pushing, `.gitea/sanitize.py` runs to:
|
||||
|
||||
1. **Delete** files that contain only secrets (private keys, `.env` files, credential docs)
|
||||
2. **Delete** the `.gitea/` directory itself (workflows, scripts)
|
||||
3. **Replace** known secret patterns with `REDACTED_*` placeholders across all text files
|
||||
|
||||
### Coverage
|
||||
|
||||
`sanitize.py` handles:
|
||||
- All password/token environment variable patterns (`_PASSWORD=`, `_TOKEN=`, `_KEY=`, etc.)
|
||||
- Gmail app passwords (16-char and spaced `REDACTED_APP_PASSWORD` formats)
|
||||
- OpenAI API keys (`sk-*` including newer `sk-proj-*` format)
|
||||
- Gitea PATs (40-char hex, including when embedded in git clone URLs as `https://<token>@host`)
|
||||
- Portainer tokens (`ptr_` prefix)
|
||||
- Cloudflare tokens
|
||||
- Service-specific secrets (Authentik, Mastodon, Matrix, LiveKit, Invidious, etc.)
|
||||
- Watchtower token (`REDACTED_WATCHTOWER_TOKEN`)
|
||||
- Public WAN IP addresses
|
||||
- Personal email addresses
|
||||
- Signal phone numbers
|
||||
|
||||
### Adding a new pattern to sanitize.py
|
||||
|
||||
When you add a new service with a credential that `sanitize.py` doesn't catch, add a pattern to `SENSITIVE_PATTERNS` in `.gitea/sanitize.py`:
|
||||
|
||||
```python
|
||||
# Add to SENSITIVE_PATTERNS list:
|
||||
(
|
||||
r'(MY_VAR\s*[:=]\s*)["\']?([A-Za-z0-9_-]{20,})["\']?',
|
||||
r'\1"REDACTED_MY_VAR"',
|
||||
"My service credential description",
|
||||
),
|
||||
```
|
||||
|
||||
**Test the pattern before committing:**
|
||||
```bash
|
||||
python3 -c "
|
||||
import re
|
||||
line = 'MY_VAR=actual-secret-value'
|
||||
pattern = r'(MY_VAR\s*[:=]\s*)[\"\']?([A-Za-z0-9_-]{20,})[\"\']?'
|
||||
print(re.sub(pattern, r'\1\"REDACTED_MY_VAR\"', line))
|
||||
"
|
||||
```
|
||||
|
||||
### Verifying the public mirror is clean
|
||||
|
||||
After any push, check that `sanitize.py` ran successfully:
|
||||
|
||||
```bash
|
||||
# Check the mirror-and-sanitize workflow in Gitea Actions
|
||||
# It should show "success" for every push to main
|
||||
https://git.vish.gg/Vish/homelab/actions
|
||||
```
|
||||
|
||||
To manually verify a specific credential isn't in the public mirror:
|
||||
```bash
|
||||
git clone https://git.vish.gg/Vish/homelab-optimized.git /tmp/mirror-check
|
||||
grep -r "sk-proj\|REDACTED_APP_PASSWORD\|REDACTED_WATCHTOWER_TOKEN" /tmp/mirror-check/ || echo "Clean"
|
||||
rm -rf /tmp/mirror-check
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## detect-secrets
|
||||
|
||||
The `validate.yml` CI workflow runs `detect-secrets-hook` on every changed file to prevent new unwhitelisted secrets from being committed.
|
||||
|
||||
### Baseline management
|
||||
|
||||
If you add a new file with a secret that is intentionally there (e.g., `# pragma: allowlist secret`):
|
||||
|
||||
```bash
|
||||
# Update the baseline to include the new known secret
|
||||
detect-secrets scan --baseline .secrets.baseline
|
||||
git add .secrets.baseline
|
||||
git commit -m "chore: update secrets baseline"
|
||||
```
|
||||
|
||||
If `detect-secrets` flags a false positive in CI:
|
||||
1. Add `# pragma: allowlist secret` to the end of the offending line, OR
|
||||
2. Run `detect-secrets scan --baseline .secrets.baseline` locally and commit the updated baseline
|
||||
|
||||
### Running a full scan
|
||||
|
||||
```bash
|
||||
pip install detect-secrets
|
||||
detect-secrets scan > .secrets.baseline.new
|
||||
# Review diff before replacing:
|
||||
diff .secrets.baseline .secrets.baseline.new
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Scope
|
||||
|
||||
### What this strategy protects
|
||||
|
||||
- **Public mirror**: `sanitize.py` ensures no credentials reach the public `homelab-optimized` repo
|
||||
- **CI/CD**: All workflow credentials are Gitea secrets — never in YAML files
|
||||
- **New commits**: `detect-secrets` in CI blocks new unwhitelisted secrets
|
||||
- **Runtime**: Portainer env injection keeps high-value secrets out of compose files
|
||||
|
||||
### What this strategy does NOT protect
|
||||
|
||||
- **Private repo history**: The private `homelab` repo on `git.vish.gg` contains historical plaintext credentials in compose files. This is accepted risk — the repo is access-controlled and self-hosted. See [Credential Rotation Checklist](credential-rotation-checklist.md) for which credentials should be rotated.
|
||||
- **Portainer database**: Injected env vars are stored in Portainer's internal DB. Protect Portainer access accordingly.
|
||||
- **Container environment**: Any process inside a container can read its own env vars. This is inherent to the Docker model.
|
||||
|
||||
---
|
||||
|
||||
## Checklist for Adding a New Service
|
||||
|
||||
- [ ] Credentials saved in Vaultwarden first
|
||||
- [ ] Compose file uses `${VAR_NAME}` — no hardcoded values
|
||||
- [ ] `.env.example` created with `REDACTED_*` placeholders if using env_file
|
||||
- [ ] Either: Portainer stack env vars set manually, OR stack added to `DDNS_STACK_ENV` in `portainer-deploy.yml`
|
||||
- [ ] If credential pattern is new: add to `sanitize.py` `SENSITIVE_PATTERNS`
|
||||
- [ ] Run `detect-secrets scan --baseline .secrets.baseline` locally before committing
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Credential Rotation Checklist](credential-rotation-checklist.md)
|
||||
- [Gitea Actions Workflows](../../.gitea/workflows/)
|
||||
- [Portainer Deploy Workflow](../../.gitea/workflows/portainer-deploy.yml)
|
||||
- [sanitize.py](../../.gitea/sanitize.py)
|
||||
Reference in New Issue
Block a user