Sanitized mirror from private repository - 2026-04-20 00:50:49 UTC

2026-04-20 00:50:49 +00:00
commit ca723d77b9
1441 changed files with 363888 additions and 0 deletions
--- a/docs/getting-started/30-Deployment-Guide.md
+++ b/docs/getting-started/30-Deployment-Guide.md
@@ -0,0 +1,743 @@
+# Deployment Guide
+
+## Overview
+
+This guide provides comprehensive instructions for deploying services in the homelab environment using GitOps principles with Docker Compose and Portainer. All deployments follow infrastructure-as-code practices with version control and automated workflows.
+
+## Deployment Architecture
+
+### GitOps Workflow
+```
+Developer ──▶ Git Repository ──▶ Portainer ──▶ Docker Compose ──▶ Running Services
+     │              │                │               │
+     │              │                │               └─▶ Health Checks
+     │              │                └─▶ Stack Management
+     │              └─▶ Configuration Validation
+     └─▶ Documentation Updates
+```
+
+### Repository Structure
+```
+homelab/
+├── hosts/
+│   ├── atlantis/           # Atlantis server configs
+│   ├── calypso/           # Calypso server configs
+│   ├── concord_nuc/       # Concord NUC configs
+│   ├── homelab_vm/        # Homelab VM configs
+│   └── raspberry-pi-5-vish/ # Raspberry Pi configs
+├── common/                # Shared configurations
+├── docs/                  # Documentation
+└── scripts/               # Automation scripts
+```
+
+## Prerequisites
+
+### Required Access
+- **Git Repository**: Read access to homelab repository
+- **Portainer Access**: Admin credentials for container management
+- **SSH Access**: Server administration capabilities
+- **Network Access**: Internal network connectivity
+
+### Required Tools
+```bash
+# Install required tools
+sudo apt update && sudo apt install -y \
+    git \
+    docker.io \
+    docker-compose \
+    curl \
+    wget \
+    vim
+
+# Verify installations
+git --version
+docker --version
+docker-compose --version
+```
+
+### Environment Setup
+```bash
+# Clone repository
+git clone https://git.vish.gg/Vish/homelab.git
+cd homelab
+
+# Set up environment variables
+export HOMELAB_ENV="production"
+export DOCKER_HOST="tcp://atlantis.vish.local:2376"
+export PORTAINER_URL="http://atlantis.vish.local:9000"
+```
+
+## Deployment Methods
+
+### Method 1: Portainer Stack Deployment (Recommended)
+
+#### Step 1: Access Portainer
+1. Navigate to [Portainer](http://atlantis.vish.local:9000)
+2. Login with admin credentials
+3. Select the appropriate endpoint
+
+#### Step 2: Create New Stack
+1. Go to **Stacks** → **Add Stack**
+2. Choose deployment method:
+   - **Git Repository** (recommended)
+   - **Upload** (for local files)
+   - **Web Editor** (for quick edits)
+
+#### Step 3: Configure Git Repository
+```yaml
+Repository URL: https://git.vish.gg/Vish/homelab.git
+Reference: refs/heads/main
+Compose Path: hosts/atlantis/service-name.yml
+```
+
+#### Step 4: Set Environment Variables
+```bash
+# Common variables
+PUID=1000
+PGID=1000
+TZ=America/New_York
+DOMAIN=vish.local
+
+# Service-specific variables
+SERVICE_PORT=8080
+SERVICE_DATA=/mnt/storage/service-name
+```
+
+#### Step 5: Deploy Stack
+1. Click **Deploy the Stack**
+2. Monitor deployment logs
+3. Verify service health
+
+### Method 2: Command Line Deployment
+
+#### Direct Docker Compose
+```bash
+# Navigate to service directory
+cd hosts/atlantis
+
+# Deploy service
+docker-compose -f service-name.yml up -d
+
+# Check status
+docker-compose -f service-name.yml ps
+
+# View logs
+docker-compose -f service-name.yml logs -f
+```
+
+#### Using Deployment Scripts
+```bash
+# Run deployment script
+./scripts/deploy-service.sh atlantis service-name
+
+# Bulk deployment
+./scripts/deploy-all.sh atlantis
+
+# Update existing service
+./scripts/update-service.sh atlantis service-name
+```
+
+### Method 3: Ansible Automation
+
+#### Playbook Deployment
+```bash
+# Deploy single service
+ansible-playbook -i inventory.ini ansible/deploy-service.yml \
+    -e target_host=atlantis \
+    -e service_name=plex
+
+# Deploy full stack
+ansible-playbook -i inventory.ini ansible/deploy-full-stack.yml \
+    -e target_host=atlantis
+
+# Update all services
+ansible-playbook -i inventory.ini ansible/update-all.yml
+```
+
+## Service Configuration
+
+### Docker Compose Template
+```yaml
+version: '3.8'
+
+services:
+  service-name:
+    image: organization/service:latest
+    container_name: service-name
+    restart: unless-stopped
+    
+    environment:
+      - PUID=${PUID:-1000}
+      - PGID=${PGID:-1000}
+      - TZ=${TZ:-UTC}
+      - SERVICE_CONFIG=${SERVICE_CONFIG}
+    
+    volumes:
+      - ${DATA_PATH}/config:/config
+      - ${DATA_PATH}/data:/data
+      - /etc/localtime:/etc/localtime:ro
+    
+    ports:
+      - "${SERVICE_PORT}:8080"
+    
+    networks:
+      - homelab
+    
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.service.rule=Host(`service.${DOMAIN}`)"
+      - "traefik.http.services.service.loadbalancer.server.port=8080"
+    
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 60s
+
+networks:
+  homelab:
+    external: true
+
+volumes:
+  service-config:
+    driver: local
+  service-data:
+    driver: local
+```
+
+### Environment Variables
+```bash
+# Create .env file
+cat > .env << EOF
+# User Configuration
+PUID=1000
+PGID=1000
+TZ=America/New_York
+
+# Network Configuration
+DOMAIN=vish.local
+SUBNET=192.168.10.0/24
+
+# Storage Configuration
+DATA_ROOT=/mnt/storage
+CONFIG_ROOT=/mnt/config
+BACKUP_ROOT=/mnt/backup
+
+# Service Configuration
+SERVICE_PORT=8080
+SERVICE_NAME=example-service
+SERVICE_VERSION=latest
+
+# Security Configuration
+SSL_CERT_PATH=/etc/ssl/certs
+SSL_KEY_PATH=/etc/ssl/private
+ADMIN_EMAIL=admin@vish.local
+EOF
+```
+
+## Server-Specific Deployments
+
+### Atlantis (Primary Server)
+```bash
+# Media services
+./deploy-service.sh atlantis plex
+./deploy-service.sh atlantis sonarr
+./deploy-service.sh atlantis radarr
+
+# Storage services
+./deploy-service.sh atlantis nextcloud
+./deploy-service.sh atlantis syncthing
+
+# Monitoring services
+./deploy-service.sh atlantis grafana
+./deploy-service.sh atlantis prometheus
+```
+
+### Calypso (Secondary Server)
+```bash
+# Development services
+./deploy-service.sh calypso gitea
+./deploy-service.sh calypso portainer
+
+# Authentication services
+./deploy-service.sh calypso authentik
+./deploy-service.sh calypso nginx-proxy-manager
+
+# Game servers
+./deploy-service.sh calypso minecraft
+./deploy-service.sh calypso satisfactory
+```
+
+### Concord NUC (Edge Server)
+```bash
+# Network services
+./deploy-service.sh concord adguard
+./deploy-service.sh concord pihole
+
+# IoT services
+./deploy-service.sh concord homeassistant
+./deploy-service.sh concord node-exporter
+
+# Media streaming
+./deploy-service.sh concord invidious
+./deploy-service.sh concord piped
+```
+
+### Homelab VM (Development)
+```bash
+# AI/ML services
+./deploy-service.sh homelab-vm ollama
+./deploy-service.sh homelab-vm openhands
+
+# Communication services
+./deploy-service.sh homelab-vm mattermost
+./deploy-service.sh homelab-vm signal-api
+
+# Testing services
+./deploy-service.sh homelab-vm test-environment
+```
+
+### Raspberry Pi (Monitoring)
+```bash
+# Monitoring services
+./deploy-service.sh raspberry-pi uptime-kuma
+./deploy-service.sh raspberry-pi glances
+
+# Lightweight services
+./deploy-service.sh raspberry-pi immich
+./deploy-service.sh raspberry-pi syncthing
+```
+
+## Network Configuration
+
+### Docker Networks
+```bash
+# Create homelab network
+docker network create \
+    --driver bridge \
+    --subnet=172.20.0.0/16 \
+    --gateway=172.20.0.1 \
+    homelab
+
+# Create monitoring network
+docker network create \
+    --driver bridge \
+    --subnet=172.21.0.0/16 \
+    --gateway=172.21.0.1 \
+    monitoring
+
+# List networks
+docker network ls
+```
+
+### Reverse Proxy Configuration
+```yaml
+# Nginx Proxy Manager
+version: '3.8'
+services:
+  nginx-proxy-manager:
+    image: jc21/nginx-proxy-manager:latest
+    container_name: nginx-proxy-manager
+    restart: unless-stopped
+    ports:
+      - "80:80"
+      - "443:443"
+      - "81:81"
+    volumes:
+      - ./data:/data
+      - ./letsencrypt:/etc/letsencrypt
+    environment:
+      DB_MYSQL_HOST: "db"
+      DB_MYSQL_PORT: 3306
+      DB_MYSQL_USER: "npm"
+      DB_MYSQL_PASSWORD: "npm"
+      DB_MYSQL_NAME: "npm"
+```
+
+## Storage Configuration
+
+### Volume Mapping
+```yaml
+# Standard volume structure
+volumes:
+  - ${DATA_ROOT}/service-name/config:/config
+  - ${DATA_ROOT}/service-name/data:/data
+  - ${MEDIA_ROOT}:/media:ro
+  - ${DOWNLOAD_ROOT}:/downloads
+  - /etc/localtime:/etc/localtime:ro
+```
+
+### Backup Integration
+```yaml
+# Backup-aware service
+services:
+  service-name:
+    # ... service configuration ...
+    volumes:
+      - service-data:/data
+      - backup-volume:/backup
+    
+    labels:
+      - "backup.enable=true"
+      - "backup.schedule=0 2 * * *"
+      - "backup.retention=30d"
+
+volumes:
+  backup-volume:
+    driver: local
+    driver_opts:
+      type: nfs
+      o: addr=backup-server.local,rw
+      device: ":/mnt/backup/service-name"
+```
+
+## Security Configuration
+
+### Container Security
+```yaml
+services:
+  secure-service:
+    # ... other configuration ...
+    
+    # Security options
+    security_opt:
+      - no-new-privileges:true
+    
+    # Read-only root filesystem
+    read_only: true
+    
+    # Temporary filesystem for writable areas
+    tmpfs:
+      - /tmp
+      - /var/tmp
+    
+    # User namespace
+    user: "${PUID}:${PGID}"
+    
+    # Capabilities
+    cap_drop:
+      - ALL
+    cap_add:
+      - CHOWN
+      - SETUID
+      - SETGID
+```
+
+### Network Security
+```yaml
+# Isolated network configuration
+networks:
+  frontend:
+    driver: bridge
+    internal: false
+  backend:
+    driver: bridge
+    internal: true
+
+services:
+  web-service:
+    networks:
+      - frontend
+      - backend
+  
+  database:
+    networks:
+      - backend
+```
+
+## Monitoring Integration
+
+### Health Checks
+```yaml
+services:
+  monitored-service:
+    # ... service configuration ...
+    
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 60s
+    
+    labels:
+      - "monitoring.enable=true"
+      - "monitoring.port=8080"
+      - "monitoring.path=/metrics"
+```
+
+### Logging Configuration
+```yaml
+services:
+  logged-service:
+    # ... service configuration ...
+    
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "10m"
+        max-file: "3"
+        labels: "service,environment"
+    
+    labels:
+      - "logging.enable=true"
+      - "logging.service=service-name"
+```
+
+## Deployment Validation
+
+### Pre-deployment Checks
+```bash
+#!/bin/bash
+# validate-deployment.sh
+
+echo "Validating deployment configuration..."
+
+# Check Docker Compose syntax
+docker-compose -f $1 config > /dev/null
+if [ $? -eq 0 ]; then
+    echo "✅ Docker Compose syntax valid"
+else
+    echo "❌ Docker Compose syntax error"
+    exit 1
+fi
+
+# Check required environment variables
+required_vars=("PUID" "PGID" "TZ" "DOMAIN")
+for var in "${required_vars[@]}"; do
+    if [ -z "${!var}" ]; then
+        echo "❌ Missing required variable: $var"
+        exit 1
+    else
+        echo "✅ Variable $var is set"
+    fi
+done
+
+# Check storage paths
+if [ ! -d "$DATA_ROOT" ]; then
+    echo "❌ Data root directory does not exist: $DATA_ROOT"
+    exit 1
+else
+    echo "✅ Data root directory exists"
+fi
+
+echo "✅ All validation checks passed"
+```
+
+### Post-deployment Verification
+```bash
+#!/bin/bash
+# verify-deployment.sh
+
+SERVICE_NAME=$1
+EXPECTED_PORT=$2
+
+echo "Verifying deployment of $SERVICE_NAME..."
+
+# Check container status
+if docker ps | grep -q $SERVICE_NAME; then
+    echo "✅ Container is running"
+else
+    echo "❌ Container is not running"
+    exit 1
+fi
+
+# Check port accessibility
+if curl -f http://localhost:$EXPECTED_PORT/health > /dev/null 2>&1; then
+    echo "✅ Service is responding on port $EXPECTED_PORT"
+else
+    echo "❌ Service is not responding on port $EXPECTED_PORT"
+fi
+
+# Check logs for errors
+if docker logs $SERVICE_NAME 2>&1 | grep -i error; then
+    echo "⚠️  Errors found in logs"
+else
+    echo "✅ No errors in logs"
+fi
+
+echo "✅ Deployment verification complete"
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### Container Won't Start
+```bash
+# Check container logs
+docker logs container-name
+
+# Check resource usage
+docker stats
+
+# Verify configuration
+docker-compose config
+
+# Check port conflicts
+netstat -tulpn | grep :8080
+```
+
+#### Permission Issues
+```bash
+# Fix ownership
+sudo chown -R $PUID:$PGID /mnt/storage/service-name
+
+# Check permissions
+ls -la /mnt/storage/service-name
+
+# Verify user mapping
+docker exec container-name id
+```
+
+#### Network Connectivity
+```bash
+# Test container networking
+docker exec container-name ping google.com
+
+# Check network configuration
+docker network inspect homelab
+
+# Verify DNS resolution
+docker exec container-name nslookup service.local
+```
+
+#### Storage Issues
+```bash
+# Check disk space
+df -h
+
+# Verify mount points
+mount | grep storage
+
+# Check RAID status
+cat /proc/mdstat
+```
+
+### Emergency Procedures
+
+#### Service Recovery
+```bash
+# Stop problematic service
+docker-compose -f service.yml down
+
+# Remove containers and volumes
+docker-compose -f service.yml down -v
+
+# Restore from backup
+./scripts/restore-service.sh service-name
+
+# Redeploy service
+docker-compose -f service.yml up -d
+```
+
+#### System Recovery
+```bash
+# Stop all services
+docker stop $(docker ps -q)
+
+# Clean up system
+docker system prune -a
+
+# Restart Docker daemon
+sudo systemctl restart docker
+
+# Redeploy critical services
+./scripts/deploy-critical.sh
+```
+
+## Automation Scripts
+
+### Deployment Automation
+```bash
+#!/bin/bash
+# deploy-service.sh
+
+HOST=$1
+SERVICE=$2
+COMPOSE_FILE="hosts/$HOST/$SERVICE.yml"
+
+if [ ! -f "$COMPOSE_FILE" ]; then
+    echo "Error: Compose file not found: $COMPOSE_FILE"
+    exit 1
+fi
+
+echo "Deploying $SERVICE on $HOST..."
+
+# Validate configuration
+docker-compose -f $COMPOSE_FILE config > /dev/null
+if [ $? -ne 0 ]; then
+    echo "Error: Invalid compose configuration"
+    exit 1
+fi
+
+# Deploy service
+docker-compose -f $COMPOSE_FILE up -d
+
+# Wait for service to be ready
+sleep 30
+
+# Verify deployment
+./scripts/verify-deployment.sh $SERVICE
+
+echo "Deployment complete: $SERVICE"
+```
+
+### Update Automation
+```bash
+#!/bin/bash
+# update-service.sh
+
+SERVICE=$1
+
+echo "Updating $SERVICE..."
+
+# Pull latest images
+docker-compose -f hosts/*/$(SERVICE).yml pull
+
+# Recreate containers
+docker-compose -f hosts/*/$(SERVICE).yml up -d
+
+# Clean up old images
+docker image prune -f
+
+echo "Update complete: $SERVICE"
+```
+
+## Best Practices
+
+### Configuration Management
+- Use environment variables for configuration
+- Store secrets in Docker secrets or external vaults
+- Version control all configuration files
+- Document all custom configurations
+
+### Resource Management
+- Set appropriate resource limits
+- Monitor resource usage
+- Plan for capacity growth
+- Implement resource quotas
+
+### Security Practices
+- Use non-root users in containers
+- Implement network segmentation
+- Regular security updates
+- Monitor for vulnerabilities
+
+### Backup Strategies
+- Automate backup processes
+- Test restore procedures
+- Implement versioned backups
+- Store backups offsite
+
+## Related Documentation
+
+- **[Service Categories](20-Service-Categories.md)** - Available services overview
+- **[Common Issues](40-Common-Issues.md)** - Troubleshooting guide
+- **[Ansible Automation](50-Ansible-Automation.md)** - Automated deployments
+- **[GitOps Guide](../admin/gitops-deployment-guide.md)** - GitOps workflows
+
+---
+
+*This deployment guide provides comprehensive instructions for deploying and managing services in the homelab environment using modern DevOps practices and tools.*