589 lines
12 KiB
Markdown
589 lines
12 KiB
Markdown
# 🚀 Service Deployment Guide
|
||
|
||
**🟡 Intermediate Guide**
|
||
|
||
This guide covers how to deploy new services in the homelab infrastructure, following established patterns and best practices used across all 176 Docker Compose configurations.
|
||
|
||
## 🎯 Deployment Philosophy
|
||
|
||
### 🏗️ **Infrastructure as Code**
|
||
- All services are defined in Docker Compose files
|
||
- Configuration is version-controlled in Git
|
||
- Ansible automates deployment and management
|
||
- Consistent patterns across all services
|
||
|
||
### 🔄 **Deployment Workflow**
|
||
```
|
||
Development → Testing → Staging → Production
|
||
↓ ↓ ↓ ↓
|
||
Local PC → Test VM → Staging → Live Host
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 Pre-Deployment Checklist
|
||
|
||
### ✅ **Before You Start**
|
||
- [ ] Identify the appropriate host for your service
|
||
- [ ] Check resource requirements (CPU, RAM, storage)
|
||
- [ ] Verify network port availability
|
||
- [ ] Review security implications
|
||
- [ ] Plan data persistence strategy
|
||
- [ ] Consider backup requirements
|
||
|
||
### 🎯 **Host Selection Criteria**
|
||
|
||
| Host Type | Best For | Avoid For |
|
||
|-----------|----------|-----------|
|
||
| **Synology NAS** | Always-on services, media, storage | CPU-intensive tasks |
|
||
| **Proxmox VMs** | Isolated workloads, testing | Resource-constrained apps |
|
||
| **Physical Hosts** | AI/ML, gaming, high-performance | Simple utilities |
|
||
| **Edge Devices** | IoT, networking, lightweight apps | Heavy databases |
|
||
|
||
---
|
||
|
||
## 🐳 Docker Compose Patterns
|
||
|
||
### 📝 **Standard Template**
|
||
|
||
Every service follows this basic structure:
|
||
|
||
```yaml
|
||
version: '3.9'
|
||
|
||
services:
|
||
service-name:
|
||
image: official/image:latest
|
||
container_name: Service-Name
|
||
hostname: service-hostname
|
||
|
||
# Security hardening
|
||
security_opt:
|
||
- no-new-privileges:true
|
||
user: 1026:100 # Synology user mapping (adjust per host)
|
||
read_only: true # For stateless services
|
||
|
||
# Health monitoring
|
||
healthcheck:
|
||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||
interval: 30s
|
||
timeout: 10s
|
||
retries: 3
|
||
start_period: 60s
|
||
|
||
# Restart policy
|
||
restart: on-failure:5
|
||
|
||
# Resource limits
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 512M
|
||
cpus: '0.5'
|
||
reservations:
|
||
memory: 256M
|
||
|
||
# Networking
|
||
networks:
|
||
- service-network
|
||
ports:
|
||
- "8080:80"
|
||
|
||
# Data persistence
|
||
volumes:
|
||
- /volume1/docker/service:/data:rw
|
||
- /etc/localtime:/etc/localtime:ro
|
||
|
||
# Configuration
|
||
environment:
|
||
- TZ=America/Los_Angeles
|
||
- PUID=1026
|
||
- PGID=100
|
||
env_file:
|
||
- .env
|
||
|
||
# Dependencies
|
||
depends_on:
|
||
database:
|
||
condition: service_healthy
|
||
|
||
# Supporting services (database, cache, etc.)
|
||
database:
|
||
image: postgres:15
|
||
container_name: Service-DB
|
||
# ... similar configuration
|
||
|
||
networks:
|
||
service-network:
|
||
name: service-network
|
||
ipam:
|
||
config:
|
||
- subnet: 192.168.x.0/24
|
||
|
||
volumes:
|
||
service-data:
|
||
driver: local
|
||
```
|
||
|
||
### 🔧 **Host-Specific Adaptations**
|
||
|
||
#### **Synology NAS** (Atlantis, Calypso, Setillo)
|
||
```yaml
|
||
# User mapping for Synology
|
||
user: 1026:100
|
||
|
||
# Volume paths
|
||
volumes:
|
||
- /volume1/docker/service:/data:rw
|
||
- /volume1/media:/media:ro
|
||
|
||
# Memory limits (conservative)
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 1G
|
||
```
|
||
|
||
#### **Proxmox VMs** (Homelab, Chicago, Bulgaria)
|
||
```yaml
|
||
# Standard Linux user
|
||
user: 1000:1000
|
||
|
||
# Volume paths
|
||
volumes:
|
||
- ./data:/data:rw
|
||
- /etc/localtime:/etc/localtime:ro
|
||
|
||
# More generous resources
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 4G
|
||
cpus: '2.0'
|
||
```
|
||
|
||
#### **Physical Hosts** (Anubis, Guava)
|
||
```yaml
|
||
# GPU access (if needed)
|
||
runtime: nvidia
|
||
environment:
|
||
- NVIDIA_VISIBLE_DEVICES=all
|
||
|
||
# High-performance settings
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 16G
|
||
cpus: '8.0'
|
||
```
|
||
|
||
---
|
||
|
||
## 📁 Directory Structure
|
||
|
||
### 🗂️ **Standard Layout**
|
||
```
|
||
/workspace/homelab/
|
||
├── HostName/
|
||
│ ├── service-name/
|
||
│ │ ├── docker-compose.yml
|
||
│ │ ├── .env
|
||
│ │ ├── config/
|
||
│ │ └── README.md
|
||
│ └── service-name.yml # Simple services
|
||
├── docs/
|
||
└── ansible/
|
||
```
|
||
|
||
### 📝 **File Naming Conventions**
|
||
- **Simple services**: `service-name.yml`
|
||
- **Complex services**: `service-name/docker-compose.yml`
|
||
- **Environment files**: `.env` or `stack.env`
|
||
- **Configuration**: `config/` directory
|
||
|
||
---
|
||
|
||
## 🔐 Security Best Practices
|
||
|
||
### 🛡️ **Container Security**
|
||
```yaml
|
||
# Security hardening
|
||
security_opt:
|
||
- no-new-privileges:true
|
||
- apparmor:docker-default
|
||
- seccomp:unconfined # Only if needed
|
||
|
||
# User namespaces
|
||
user: 1026:100 # Non-root user
|
||
|
||
# Read-only filesystem
|
||
read_only: true
|
||
tmpfs:
|
||
- /tmp
|
||
- /var/tmp
|
||
|
||
# Capability dropping
|
||
cap_drop:
|
||
- ALL
|
||
cap_add:
|
||
- CHOWN # Only add what's needed
|
||
```
|
||
|
||
### 🔑 **Secrets Management**
|
||
```yaml
|
||
# Use Docker secrets for sensitive data
|
||
secrets:
|
||
db_password:
|
||
"REDACTED_PASSWORD" ./secrets/db_password.txt
|
||
|
||
services:
|
||
app:
|
||
secrets:
|
||
- db_password
|
||
environment:
|
||
- DB_PASSWORD_FILE=/run/secrets/db_password
|
||
```
|
||
|
||
### 🌐 **Network Security**
|
||
```yaml
|
||
# Custom networks for isolation
|
||
networks:
|
||
frontend:
|
||
internal: false # Internet access
|
||
backend:
|
||
internal: true # No internet access
|
||
|
||
services:
|
||
web:
|
||
networks:
|
||
- frontend
|
||
- backend
|
||
database:
|
||
networks:
|
||
- backend # Database isolated from internet
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 Monitoring Integration
|
||
|
||
### 📈 **Health Checks**
|
||
```yaml
|
||
healthcheck:
|
||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
||
interval: 30s
|
||
timeout: 10s
|
||
retries: 3
|
||
start_period: 60s
|
||
```
|
||
|
||
### 🏷️ **Prometheus Labels**
|
||
```yaml
|
||
labels:
|
||
- "prometheus.io/scrape=true"
|
||
- "prometheus.io/port=8080"
|
||
- "prometheus.io/path=/metrics"
|
||
- "service.category=media"
|
||
- "service.tier=production"
|
||
```
|
||
|
||
### 📊 **Logging Configuration**
|
||
```yaml
|
||
logging:
|
||
driver: "json-file"
|
||
options:
|
||
max-size: "10m"
|
||
max-file: "3"
|
||
labels: "service,environment"
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 Deployment Process
|
||
|
||
### 1️⃣ **Local Development**
|
||
```bash
|
||
# Create service directory
|
||
mkdir -p ~/homelab-dev/new-service
|
||
cd ~/homelab-dev/new-service
|
||
|
||
# Create docker-compose.yml
|
||
cat > docker-compose.yml << 'EOF'
|
||
# Your service configuration
|
||
EOF
|
||
|
||
# Test locally
|
||
docker-compose up -d
|
||
docker-compose logs -f
|
||
```
|
||
|
||
### 2️⃣ **Testing & Validation**
|
||
```bash
|
||
# Health check
|
||
curl -f http://localhost:8080/health
|
||
|
||
# Resource usage
|
||
docker stats
|
||
|
||
# Security scan
|
||
docker scout cves
|
||
|
||
# Cleanup
|
||
docker-compose down -v
|
||
```
|
||
|
||
### 3️⃣ **Repository Integration**
|
||
```bash
|
||
# Add to homelab repository
|
||
cp -r ~/homelab-dev/new-service /workspace/homelab/TargetHost/
|
||
|
||
# Update documentation
|
||
echo "## New Service" >> /workspace/homelab/TargetHost/README.md
|
||
|
||
# Commit changes
|
||
git add .
|
||
git commit -m "Add new-service to TargetHost"
|
||
```
|
||
|
||
### 4️⃣ **Ansible Deployment**
|
||
```bash
|
||
# Deploy using Ansible
|
||
cd /workspace/homelab/ansible
|
||
ansible-playbook -i inventory.ini deploy-service.yml \
|
||
--extra-vars "target_host=atlantis service_name=new-service"
|
||
|
||
# Verify deployment
|
||
ansible atlantis -i inventory.ini -m shell \
|
||
-a "docker ps | grep new-service"
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 Service-Specific Patterns
|
||
|
||
### 🎬 **Media Services**
|
||
```yaml
|
||
# Common media service pattern
|
||
services:
|
||
media-service:
|
||
image: linuxserver/service:latest
|
||
environment:
|
||
- PUID=1026
|
||
- PGID=100
|
||
- TZ=America/Los_Angeles
|
||
volumes:
|
||
- /volume1/docker/service:/config
|
||
- /volume1/media:/media:ro
|
||
- /volume1/downloads:/downloads:rw
|
||
ports:
|
||
- "8080:8080"
|
||
```
|
||
|
||
### 🗄️ **Database Services**
|
||
```yaml
|
||
# Database with backup integration
|
||
services:
|
||
database:
|
||
image: postgres:15
|
||
environment:
|
||
- POSTGRES_DB=appdb
|
||
- POSTGRES_USER=appuser
|
||
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
|
||
volumes:
|
||
- db_data:/var/lib/postgresql/data
|
||
- ./backups:/backups
|
||
secrets:
|
||
- db_password
|
||
healthcheck:
|
||
test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
|
||
```
|
||
|
||
### 🌐 **Web Services**
|
||
```yaml
|
||
# Web service with reverse proxy
|
||
services:
|
||
web-app:
|
||
image: nginx:alpine
|
||
labels:
|
||
- "traefik.enable=true"
|
||
- "traefik.http.routers.webapp.rule=Host(`app.example.com`)"
|
||
- "traefik.http.services.webapp.loadbalancer.server.port=80"
|
||
volumes:
|
||
- ./html:/usr/share/nginx/html:ro
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 Deployment Checklist
|
||
|
||
### ✅ **Pre-Deployment**
|
||
- [ ] Service configuration reviewed
|
||
- [ ] Resource requirements calculated
|
||
- [ ] Security settings applied
|
||
- [ ] Health checks configured
|
||
- [ ] Backup strategy planned
|
||
- [ ] Monitoring integration added
|
||
|
||
### ✅ **During Deployment**
|
||
- [ ] Service starts successfully
|
||
- [ ] Health checks pass
|
||
- [ ] Logs show no errors
|
||
- [ ] Network connectivity verified
|
||
- [ ] Resource usage within limits
|
||
- [ ] Security scan completed
|
||
|
||
### ✅ **Post-Deployment**
|
||
- [ ] Service accessible via intended URLs
|
||
- [ ] Monitoring alerts configured
|
||
- [ ] Backup jobs scheduled
|
||
- [ ] Documentation updated
|
||
- [ ] Team notified of new service
|
||
- [ ] Performance baseline established
|
||
|
||
---
|
||
|
||
## 🚨 Troubleshooting Deployment Issues
|
||
|
||
### 🔍 **Common Problems**
|
||
|
||
#### **Container Won't Start**
|
||
```bash
|
||
# Check logs
|
||
docker-compose logs service-name
|
||
|
||
# Check resource constraints
|
||
docker stats
|
||
|
||
# Verify image availability
|
||
docker pull image:tag
|
||
|
||
# Check port conflicts
|
||
netstat -tulpn | grep :8080
|
||
```
|
||
|
||
#### **Permission Issues**
|
||
```bash
|
||
# Fix ownership (Synology)
|
||
sudo chown -R 1026:100 /volume1/docker/service
|
||
|
||
# Fix permissions
|
||
sudo chmod -R 755 /volume1/docker/service
|
||
```
|
||
|
||
#### **Network Issues**
|
||
```bash
|
||
# Check network connectivity
|
||
docker exec service-name ping google.com
|
||
|
||
# Verify DNS resolution
|
||
docker exec service-name nslookup service-name
|
||
|
||
# Check port binding
|
||
docker port service-name
|
||
```
|
||
|
||
#### **Resource Constraints**
|
||
```bash
|
||
# Check memory usage
|
||
docker stats --no-stream
|
||
|
||
# Check disk space
|
||
df -h
|
||
|
||
# Monitor resource limits
|
||
docker exec service-name cat /sys/fs/cgroup/memory/memory.limit_in_bytes
|
||
```
|
||
|
||
---
|
||
|
||
## 🔄 Update & Maintenance
|
||
|
||
### 📦 **Container Updates**
|
||
```bash
|
||
# Update single service
|
||
docker-compose pull
|
||
docker-compose up -d
|
||
|
||
# Update with Watchtower (automated)
|
||
# Watchtower handles updates automatically for tagged containers
|
||
```
|
||
|
||
### 🔧 **Configuration Changes**
|
||
```bash
|
||
# Apply configuration changes
|
||
docker-compose down
|
||
# Edit configuration files
|
||
docker-compose up -d
|
||
|
||
# Rolling updates (zero downtime)
|
||
docker-compose up -d --no-deps service-name
|
||
```
|
||
|
||
### 🗄️ **Database Migrations**
|
||
```bash
|
||
# Backup before migration
|
||
docker exec db-container pg_dump -U user dbname > backup.sql
|
||
|
||
# Run migrations
|
||
docker-compose exec app python manage.py migrate
|
||
|
||
# Verify migration
|
||
docker-compose exec app python manage.py showmigrations
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 Performance Optimization
|
||
|
||
### ⚡ **Resource Tuning**
|
||
```yaml
|
||
# Optimize for your workload
|
||
deploy:
|
||
resources:
|
||
limits:
|
||
memory: 2G # Set based on actual usage
|
||
cpus: '1.0' # Adjust for CPU requirements
|
||
reservations:
|
||
memory: 512M # Guarantee minimum resources
|
||
```
|
||
|
||
### 🗄️ **Storage Optimization**
|
||
```yaml
|
||
# Use appropriate volume types
|
||
volumes:
|
||
# Fast storage for databases
|
||
- /volume1/ssd/db:/var/lib/postgresql/data
|
||
|
||
# Slower storage for archives
|
||
- /volume1/hdd/archives:/archives:ro
|
||
|
||
# Temporary storage
|
||
- type: tmpfs
|
||
target: /tmp
|
||
tmpfs:
|
||
size: 100M
|
||
```
|
||
|
||
### 🌐 **Network Optimization**
|
||
```yaml
|
||
# Optimize network settings
|
||
networks:
|
||
app-network:
|
||
driver: bridge
|
||
driver_opts:
|
||
com.docker.network.bridge.name: br-app
|
||
com.docker.network.driver.mtu: 1500
|
||
```
|
||
|
||
---
|
||
|
||
## 📋 Next Steps
|
||
|
||
- **[Monitoring Setup](monitoring.md)**: Configure monitoring for your new service
|
||
- **[Backup Configuration](backup.md)**: Set up automated backups
|
||
- **[Troubleshooting Guide](../troubleshooting/common-issues.md)**: Common deployment issues
|
||
- **[Service Categories](../services/categories.md)**: Find similar services for reference
|
||
|
||
---
|
||
|
||
*Remember: Start simple, test thoroughly, and iterate based on real-world usage. Every service in this homelab started with this basic deployment pattern.* |