Sanitized mirror from private repository - 2026-03-17 11:49:57 UTC
This commit is contained in:
Gitea Mirror Bot
589
docs/admin/deployment.md
Normal file
589
docs/admin/deployment.md
Normal file
@@ -0,0 +1,589 @@
|
||||
# 🚀 Service Deployment Guide
|
||||
|
||||
**🟡 Intermediate Guide**
|
||||
|
||||
This guide covers how to deploy new services in the homelab infrastructure, following established patterns and best practices used across all 176 Docker Compose configurations.
|
||||
|
||||
## 🎯 Deployment Philosophy
|
||||
|
||||
### 🏗️ **Infrastructure as Code**
|
||||
- All services are defined in Docker Compose files
|
||||
- Configuration is version-controlled in Git
|
||||
- Ansible automates deployment and management
|
||||
- Consistent patterns across all services
|
||||
|
||||
### 🔄 **Deployment Workflow**
|
||||
```
|
||||
Development → Testing → Staging → Production
|
||||
↓ ↓ ↓ ↓
|
||||
Local PC → Test VM → Staging → Live Host
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Pre-Deployment Checklist
|
||||
|
||||
### ✅ **Before You Start**
|
||||
- [ ] Identify the appropriate host for your service
|
||||
- [ ] Check resource requirements (CPU, RAM, storage)
|
||||
- [ ] Verify network port availability
|
||||
- [ ] Review security implications
|
||||
- [ ] Plan data persistence strategy
|
||||
- [ ] Consider backup requirements
|
||||
|
||||
### 🎯 **Host Selection Criteria**
|
||||
|
||||
| Host Type | Best For | Avoid For |
|
||||
|-----------|----------|-----------|
|
||||
| **Synology NAS** | Always-on services, media, storage | CPU-intensive tasks |
|
||||
| **Proxmox VMs** | Isolated workloads, testing | Resource-constrained apps |
|
||||
| **Physical Hosts** | AI/ML, gaming, high-performance | Simple utilities |
|
||||
| **Edge Devices** | IoT, networking, lightweight apps | Heavy databases |
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker Compose Patterns
|
||||
|
||||
### 📝 **Standard Template**
|
||||
|
||||
Every service follows this basic structure:
|
||||
|
||||
```yaml
|
||||
version: '3.9'
|
||||
|
||||
services:
|
||||
service-name:
|
||||
image: official/image:latest
|
||||
container_name: Service-Name
|
||||
hostname: service-hostname
|
||||
|
||||
# Security hardening
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
user: 1026:100 # Synology user mapping (adjust per host)
|
||||
read_only: true # For stateless services
|
||||
|
||||
# Health monitoring
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
|
||||
# Restart policy
|
||||
restart: on-failure:5
|
||||
|
||||
# Resource limits
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
cpus: '0.5'
|
||||
reservations:
|
||||
memory: 256M
|
||||
|
||||
# Networking
|
||||
networks:
|
||||
- service-network
|
||||
ports:
|
||||
- "8080:80"
|
||||
|
||||
# Data persistence
|
||||
volumes:
|
||||
- /volume1/docker/service:/data:rw
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
|
||||
# Configuration
|
||||
environment:
|
||||
- TZ=America/Los_Angeles
|
||||
- PUID=1026
|
||||
- PGID=100
|
||||
env_file:
|
||||
- .env
|
||||
|
||||
# Dependencies
|
||||
depends_on:
|
||||
database:
|
||||
condition: service_healthy
|
||||
|
||||
# Supporting services (database, cache, etc.)
|
||||
database:
|
||||
image: postgres:15
|
||||
container_name: Service-DB
|
||||
# ... similar configuration
|
||||
|
||||
networks:
|
||||
service-network:
|
||||
name: service-network
|
||||
ipam:
|
||||
config:
|
||||
- subnet: 192.168.x.0/24
|
||||
|
||||
volumes:
|
||||
service-data:
|
||||
driver: local
|
||||
```
|
||||
|
||||
### 🔧 **Host-Specific Adaptations**
|
||||
|
||||
#### **Synology NAS** (Atlantis, Calypso, Setillo)
|
||||
```yaml
|
||||
# User mapping for Synology
|
||||
user: 1026:100
|
||||
|
||||
# Volume paths
|
||||
volumes:
|
||||
- /volume1/docker/service:/data:rw
|
||||
- /volume1/media:/media:ro
|
||||
|
||||
# Memory limits (conservative)
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
```
|
||||
|
||||
#### **Proxmox VMs** (Homelab, Chicago, Bulgaria)
|
||||
```yaml
|
||||
# Standard Linux user
|
||||
user: 1000:1000
|
||||
|
||||
# Volume paths
|
||||
volumes:
|
||||
- ./data:/data:rw
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
|
||||
# More generous resources
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 4G
|
||||
cpus: '2.0'
|
||||
```
|
||||
|
||||
#### **Physical Hosts** (Anubis, Guava)
|
||||
```yaml
|
||||
# GPU access (if needed)
|
||||
runtime: nvidia
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
|
||||
# High-performance settings
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 16G
|
||||
cpus: '8.0'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Directory Structure
|
||||
|
||||
### 🗂️ **Standard Layout**
|
||||
```
|
||||
/workspace/homelab/
|
||||
├── HostName/
|
||||
│ ├── service-name/
|
||||
│ │ ├── docker-compose.yml
|
||||
│ │ ├── .env
|
||||
│ │ ├── config/
|
||||
│ │ └── README.md
|
||||
│ └── service-name.yml # Simple services
|
||||
├── docs/
|
||||
└── ansible/
|
||||
```
|
||||
|
||||
### 📝 **File Naming Conventions**
|
||||
- **Simple services**: `service-name.yml`
|
||||
- **Complex services**: `service-name/docker-compose.yml`
|
||||
- **Environment files**: `.env` or `stack.env`
|
||||
- **Configuration**: `config/` directory
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Security Best Practices
|
||||
|
||||
### 🛡️ **Container Security**
|
||||
```yaml
|
||||
# Security hardening
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
- apparmor:docker-default
|
||||
- seccomp:unconfined # Only if needed
|
||||
|
||||
# User namespaces
|
||||
user: 1026:100 # Non-root user
|
||||
|
||||
# Read-only filesystem
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
- /var/tmp
|
||||
|
||||
# Capability dropping
|
||||
cap_drop:
|
||||
- ALL
|
||||
cap_add:
|
||||
- CHOWN # Only add what's needed
|
||||
```
|
||||
|
||||
### 🔑 **Secrets Management**
|
||||
```yaml
|
||||
# Use Docker secrets for sensitive data
|
||||
secrets:
|
||||
db_password:
|
||||
"REDACTED_PASSWORD" ./secrets/db_password.txt
|
||||
|
||||
services:
|
||||
app:
|
||||
secrets:
|
||||
- db_password
|
||||
environment:
|
||||
- DB_PASSWORD_FILE=/run/secrets/db_password
|
||||
```
|
||||
|
||||
### 🌐 **Network Security**
|
||||
```yaml
|
||||
# Custom networks for isolation
|
||||
networks:
|
||||
frontend:
|
||||
internal: false # Internet access
|
||||
backend:
|
||||
internal: true # No internet access
|
||||
|
||||
services:
|
||||
web:
|
||||
networks:
|
||||
- frontend
|
||||
- backend
|
||||
database:
|
||||
networks:
|
||||
- backend # Database isolated from internet
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Monitoring Integration
|
||||
|
||||
### 📈 **Health Checks**
|
||||
```yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
```
|
||||
|
||||
### 🏷️ **Prometheus Labels**
|
||||
```yaml
|
||||
labels:
|
||||
- "prometheus.io/scrape=true"
|
||||
- "prometheus.io/port=8080"
|
||||
- "prometheus.io/path=/metrics"
|
||||
- "service.category=media"
|
||||
- "service.tier=production"
|
||||
```
|
||||
|
||||
### 📊 **Logging Configuration**
|
||||
```yaml
|
||||
logging:
|
||||
driver: "json-file"
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
labels: "service,environment"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Deployment Process
|
||||
|
||||
### 1️⃣ **Local Development**
|
||||
```bash
|
||||
# Create service directory
|
||||
mkdir -p ~/homelab-dev/new-service
|
||||
cd ~/homelab-dev/new-service
|
||||
|
||||
# Create docker-compose.yml
|
||||
cat > docker-compose.yml << 'EOF'
|
||||
# Your service configuration
|
||||
EOF
|
||||
|
||||
# Test locally
|
||||
docker-compose up -d
|
||||
docker-compose logs -f
|
||||
```
|
||||
|
||||
### 2️⃣ **Testing & Validation**
|
||||
```bash
|
||||
# Health check
|
||||
curl -f http://localhost:8080/health
|
||||
|
||||
# Resource usage
|
||||
docker stats
|
||||
|
||||
# Security scan
|
||||
docker scout cves
|
||||
|
||||
# Cleanup
|
||||
docker-compose down -v
|
||||
```
|
||||
|
||||
### 3️⃣ **Repository Integration**
|
||||
```bash
|
||||
# Add to homelab repository
|
||||
cp -r ~/homelab-dev/new-service /workspace/homelab/TargetHost/
|
||||
|
||||
# Update documentation
|
||||
echo "## New Service" >> /workspace/homelab/TargetHost/README.md
|
||||
|
||||
# Commit changes
|
||||
git add .
|
||||
git commit -m "Add new-service to TargetHost"
|
||||
```
|
||||
|
||||
### 4️⃣ **Ansible Deployment**
|
||||
```bash
|
||||
# Deploy using Ansible
|
||||
cd /workspace/homelab/ansible
|
||||
ansible-playbook -i inventory.ini deploy-service.yml \
|
||||
--extra-vars "target_host=atlantis service_name=new-service"
|
||||
|
||||
# Verify deployment
|
||||
ansible atlantis -i inventory.ini -m shell \
|
||||
-a "docker ps | grep new-service"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Service-Specific Patterns
|
||||
|
||||
### 🎬 **Media Services**
|
||||
```yaml
|
||||
# Common media service pattern
|
||||
services:
|
||||
media-service:
|
||||
image: linuxserver/service:latest
|
||||
environment:
|
||||
- PUID=1026
|
||||
- PGID=100
|
||||
- TZ=America/Los_Angeles
|
||||
volumes:
|
||||
- /volume1/docker/service:/config
|
||||
- /volume1/media:/media:ro
|
||||
- /volume1/downloads:/downloads:rw
|
||||
ports:
|
||||
- "8080:8080"
|
||||
```
|
||||
|
||||
### 🗄️ **Database Services**
|
||||
```yaml
|
||||
# Database with backup integration
|
||||
services:
|
||||
database:
|
||||
image: postgres:15
|
||||
environment:
|
||||
- POSTGRES_DB=appdb
|
||||
- POSTGRES_USER=appuser
|
||||
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
|
||||
volumes:
|
||||
- db_data:/var/lib/postgresql/data
|
||||
- ./backups:/backups
|
||||
secrets:
|
||||
- db_password
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
|
||||
```
|
||||
|
||||
### 🌐 **Web Services**
|
||||
```yaml
|
||||
# Web service with reverse proxy
|
||||
services:
|
||||
web-app:
|
||||
image: nginx:alpine
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.webapp.rule=Host(`app.example.com`)"
|
||||
- "traefik.http.services.webapp.loadbalancer.server.port=80"
|
||||
volumes:
|
||||
- ./html:/usr/share/nginx/html:ro
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Deployment Checklist
|
||||
|
||||
### ✅ **Pre-Deployment**
|
||||
- [ ] Service configuration reviewed
|
||||
- [ ] Resource requirements calculated
|
||||
- [ ] Security settings applied
|
||||
- [ ] Health checks configured
|
||||
- [ ] Backup strategy planned
|
||||
- [ ] Monitoring integration added
|
||||
|
||||
### ✅ **During Deployment**
|
||||
- [ ] Service starts successfully
|
||||
- [ ] Health checks pass
|
||||
- [ ] Logs show no errors
|
||||
- [ ] Network connectivity verified
|
||||
- [ ] Resource usage within limits
|
||||
- [ ] Security scan completed
|
||||
|
||||
### ✅ **Post-Deployment**
|
||||
- [ ] Service accessible via intended URLs
|
||||
- [ ] Monitoring alerts configured
|
||||
- [ ] Backup jobs scheduled
|
||||
- [ ] Documentation updated
|
||||
- [ ] Team notified of new service
|
||||
- [ ] Performance baseline established
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Troubleshooting Deployment Issues
|
||||
|
||||
### 🔍 **Common Problems**
|
||||
|
||||
#### **Container Won't Start**
|
||||
```bash
|
||||
# Check logs
|
||||
docker-compose logs service-name
|
||||
|
||||
# Check resource constraints
|
||||
docker stats
|
||||
|
||||
# Verify image availability
|
||||
docker pull image:tag
|
||||
|
||||
# Check port conflicts
|
||||
netstat -tulpn | grep :8080
|
||||
```
|
||||
|
||||
#### **Permission Issues**
|
||||
```bash
|
||||
# Fix ownership (Synology)
|
||||
sudo chown -R 1026:100 /volume1/docker/service
|
||||
|
||||
# Fix permissions
|
||||
sudo chmod -R 755 /volume1/docker/service
|
||||
```
|
||||
|
||||
#### **Network Issues**
|
||||
```bash
|
||||
# Check network connectivity
|
||||
docker exec service-name ping google.com
|
||||
|
||||
# Verify DNS resolution
|
||||
docker exec service-name nslookup service-name
|
||||
|
||||
# Check port binding
|
||||
docker port service-name
|
||||
```
|
||||
|
||||
#### **Resource Constraints**
|
||||
```bash
|
||||
# Check memory usage
|
||||
docker stats --no-stream
|
||||
|
||||
# Check disk space
|
||||
df -h
|
||||
|
||||
# Monitor resource limits
|
||||
docker exec service-name cat /sys/fs/cgroup/memory/memory.limit_in_bytes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Update & Maintenance
|
||||
|
||||
### 📦 **Container Updates**
|
||||
```bash
|
||||
# Update single service
|
||||
docker-compose pull
|
||||
docker-compose up -d
|
||||
|
||||
# Update with Watchtower (automated)
|
||||
# Watchtower handles updates automatically for tagged containers
|
||||
```
|
||||
|
||||
### 🔧 **Configuration Changes**
|
||||
```bash
|
||||
# Apply configuration changes
|
||||
docker-compose down
|
||||
# Edit configuration files
|
||||
docker-compose up -d
|
||||
|
||||
# Rolling updates (zero downtime)
|
||||
docker-compose up -d --no-deps service-name
|
||||
```
|
||||
|
||||
### 🗄️ **Database Migrations**
|
||||
```bash
|
||||
# Backup before migration
|
||||
docker exec db-container pg_dump -U user dbname > backup.sql
|
||||
|
||||
# Run migrations
|
||||
docker-compose exec app python manage.py migrate
|
||||
|
||||
# Verify migration
|
||||
docker-compose exec app python manage.py showmigrations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Performance Optimization
|
||||
|
||||
### ⚡ **Resource Tuning**
|
||||
```yaml
|
||||
# Optimize for your workload
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 2G # Set based on actual usage
|
||||
cpus: '1.0' # Adjust for CPU requirements
|
||||
reservations:
|
||||
memory: 512M # Guarantee minimum resources
|
||||
```
|
||||
|
||||
### 🗄️ **Storage Optimization**
|
||||
```yaml
|
||||
# Use appropriate volume types
|
||||
volumes:
|
||||
# Fast storage for databases
|
||||
- /volume1/ssd/db:/var/lib/postgresql/data
|
||||
|
||||
# Slower storage for archives
|
||||
- /volume1/hdd/archives:/archives:ro
|
||||
|
||||
# Temporary storage
|
||||
- type: tmpfs
|
||||
target: /tmp
|
||||
tmpfs:
|
||||
size: 100M
|
||||
```
|
||||
|
||||
### 🌐 **Network Optimization**
|
||||
```yaml
|
||||
# Optimize network settings
|
||||
networks:
|
||||
app-network:
|
||||
driver: bridge
|
||||
driver_opts:
|
||||
com.docker.network.bridge.name: br-app
|
||||
com.docker.network.driver.mtu: 1500
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Next Steps
|
||||
|
||||
- **[Monitoring Setup](monitoring.md)**: Configure monitoring for your new service
|
||||
- **[Backup Configuration](backup.md)**: Set up automated backups
|
||||
- **[Troubleshooting Guide](../troubleshooting/common-issues.md)**: Common deployment issues
|
||||
- **[Service Categories](../services/categories.md)**: Find similar services for reference
|
||||
|
||||
---
|
||||
|
||||
*Remember: Start simple, test thoroughly, and iterate based on real-world usage. Every service in this homelab started with this basic deployment pattern.*
|
||||
Reference in New Issue
Block a user