304 lines
11 KiB
Markdown
304 lines
11 KiB
Markdown
# Architecture Overview
|
|
|
|
## Infrastructure Design
|
|
|
|
This homelab implements a distributed, containerized architecture designed for high availability, scalability, and ease of management. The infrastructure follows GitOps principles with infrastructure-as-code practices.
|
|
|
|
## Network Architecture
|
|
|
|
### Physical Network
|
|
```
|
|
Internet
|
|
│
|
|
├── Router/Firewall (pfSense)
|
|
│ ├── Management VLAN (192.168.1.0/24)
|
|
│ ├── Server VLAN (192.168.10.0/24)
|
|
│ ├── IoT VLAN (192.168.20.0/24)
|
|
│ └── Guest VLAN (192.168.30.0/24)
|
|
│
|
|
└── Core Switch
|
|
├── Atlantis (192.168.10.10)
|
|
├── Calypso (192.168.10.20)
|
|
├── Concord NUC (192.168.10.30)
|
|
├── Homelab VM (192.168.10.40)
|
|
└── Raspberry Pi (192.168.10.50)
|
|
```
|
|
|
|
### Virtual Networks
|
|
- **Docker Networks**: Isolated container communication
|
|
- **VPN Tunnels**: Secure remote access via WireGuard
|
|
- **Tailscale Mesh**: Zero-trust network overlay
|
|
- **Cloudflare Tunnels**: Secure external access
|
|
|
|
## Server Architecture
|
|
|
|
### Atlantis (Primary Server)
|
|
**Role**: Main application server and storage
|
|
- **Hardware**: Dell PowerEdge R720
|
|
- **OS**: Ubuntu Server 22.04 LTS
|
|
- **Storage**: 12TB RAID-10 array
|
|
- **Services**: 40+ containerized applications
|
|
|
|
**Key Services**:
|
|
- Media Management (Plex, Sonarr, Radarr)
|
|
- File Storage (Nextcloud, Syncthing)
|
|
- Development Tools (GitLab, Portainer)
|
|
- Monitoring (Grafana, Prometheus)
|
|
|
|
### Calypso (Secondary Server)
|
|
**Role**: Backup services and specialized workloads
|
|
- **Hardware**: Custom build (AMD Ryzen)
|
|
- **OS**: Ubuntu Server 22.04 LTS
|
|
- **Storage**: 8TB RAID-1 array
|
|
- **Services**: 25+ containerized applications
|
|
|
|
**Key Services**:
|
|
- Authentication (Authentik)
|
|
- Game Servers (Minecraft, Satisfactory)
|
|
- Development (Gitea, CI/CD runners)
|
|
- Backup Services (Seafile, Immich)
|
|
|
|
### Concord NUC (Edge Computing)
|
|
**Role**: Edge services and IoT management
|
|
- **Hardware**: Intel NUC
|
|
- **OS**: Ubuntu Server 22.04 LTS
|
|
- **Storage**: 1TB NVMe SSD
|
|
- **Services**: 15+ lightweight applications
|
|
|
|
**Key Services**:
|
|
- Home Automation (Home Assistant)
|
|
- Network Services (AdGuard, Pi-hole)
|
|
- Media Streaming (Invidious, Piped)
|
|
- Monitoring (Node Exporter)
|
|
|
|
### Homelab VM (Development)
|
|
**Role**: Development and testing environment
|
|
- **Platform**: Proxmox VM
|
|
- **OS**: Ubuntu Server 22.04 LTS
|
|
- **Storage**: 500GB virtual disk
|
|
- **Services**: 30+ development tools
|
|
|
|
**Key Services**:
|
|
- AI/ML Tools (Ollama, OpenHands)
|
|
- Communication (Mattermost, Signal API)
|
|
- Testing Services (Various experimental apps)
|
|
- Monitoring (Alerting, NTFY)
|
|
|
|
### Raspberry Pi (Monitoring)
|
|
**Role**: Dedicated monitoring and lightweight services
|
|
- **Hardware**: Raspberry Pi 5
|
|
- **OS**: Raspberry Pi OS Lite
|
|
- **Storage**: 256GB microSD + USB storage
|
|
- **Services**: 5+ monitoring applications
|
|
|
|
**Key Services**:
|
|
- Uptime Monitoring (Uptime Kuma)
|
|
- System Monitoring (Glances)
|
|
- Photo Management (Immich)
|
|
- File Sharing (Samba)
|
|
|
|
## Container Architecture
|
|
|
|
### Orchestration Strategy
|
|
- **Docker Compose**: Primary orchestration tool
|
|
- **Portainer**: Web-based container management
|
|
- **Watchtower**: Automated container updates
|
|
- **GitOps**: Version-controlled deployments
|
|
|
|
### Container Patterns
|
|
```
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Application │ │ Database │ │ Storage │
|
|
│ Containers │ │ Containers │ │ Containers │
|
|
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
|
|
│ • Web Services │ │ • PostgreSQL │ │ • File Shares │
|
|
│ • APIs │ │ • MySQL │ │ • Object Store │
|
|
│ • Workers │ │ • Redis │ │ • Backup Vols │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
│ │ │
|
|
└───────────────────────┼───────────────────────┘
|
|
│
|
|
┌─────────────────┐
|
|
│ Proxy/LB │
|
|
│ Containers │
|
|
├─────────────────┤
|
|
│ • Nginx Proxy │
|
|
│ • Traefik │
|
|
│ • Cloudflare │
|
|
└─────────────────┘
|
|
```
|
|
|
|
## Storage Architecture
|
|
|
|
### Primary Storage (Atlantis)
|
|
- **RAID-10**: 4x 4TB drives for performance and redundancy
|
|
- **Hot Spare**: Additional drive for automatic replacement
|
|
- **Backup Target**: Weekly snapshots to external storage
|
|
|
|
### Secondary Storage (Calypso)
|
|
- **RAID-1**: 2x 4TB drives for redundancy
|
|
- **Backup Source**: Receives backups from other servers
|
|
- **Archive Storage**: Long-term data retention
|
|
|
|
### Distributed Storage
|
|
- **Syncthing**: Peer-to-peer file synchronization
|
|
- **Seafile**: Centralized file storage with versioning
|
|
- **Immich**: Photo management with AI features
|
|
- **Nextcloud**: Personal cloud storage
|
|
|
|
## Monitoring Architecture
|
|
|
|
### Metrics Collection
|
|
```
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
|
│ Node │ │ Container │ │ Application │
|
|
│ Exporter │───▶│ Advisor │───▶│ Metrics │
|
|
└─────────────┘ └─────────────┘ └─────────────┘
|
|
│ │ │
|
|
└───────────────────┼───────────────────┘
|
|
│
|
|
┌─────────────┐
|
|
│ Prometheus │
|
|
│ (Metrics) │
|
|
└─────────────┘
|
|
│
|
|
┌─────────────┐
|
|
│ Grafana │
|
|
│ (Dashboards)│
|
|
└─────────────┘
|
|
```
|
|
|
|
### Alerting Pipeline
|
|
```
|
|
Prometheus ──▶ Alertmanager ──▶ NTFY ──▶ Mobile/Desktop
|
|
│ │ │
|
|
│ ├──▶ Email ────┘
|
|
│ └──▶ Signal ───┘
|
|
│
|
|
└──▶ Uptime Kuma ──▶ Discord/Slack
|
|
```
|
|
|
|
## Security Architecture
|
|
|
|
### Network Security
|
|
- **Firewall Rules**: Strict ingress/egress controls
|
|
- **VPN Access**: WireGuard for remote connectivity
|
|
- **Zero Trust**: Tailscale mesh networking
|
|
- **SSL/TLS**: End-to-end encryption
|
|
|
|
### Application Security
|
|
- **Authentication**: Centralized with Authentik
|
|
- **Authorization**: Role-based access control
|
|
- **Secrets Management**: Docker secrets and environment files
|
|
- **Container Security**: Non-root users, read-only filesystems
|
|
|
|
### Data Security
|
|
- **Encryption at Rest**: LUKS disk encryption
|
|
- **Encryption in Transit**: TLS for all communications
|
|
- **Backup Encryption**: GPG-encrypted backups
|
|
- **Access Logging**: Comprehensive audit trails
|
|
|
|
## Deployment Architecture
|
|
|
|
### GitOps Workflow
|
|
```
|
|
Developer ──▶ Git Repository ──▶ CI/CD Pipeline ──▶ Container Registry
|
|
│ │
|
|
│ │
|
|
▼ ▼
|
|
Configuration ──▶ Portainer ──▶ Docker Compose ──▶ Containers
|
|
Files │ │
|
|
│ │ │
|
|
└───────────────┼──────────────────────────────┘
|
|
│
|
|
Monitoring & Alerting
|
|
```
|
|
|
|
### Continuous Deployment
|
|
- **Git-based**: All configurations in version control
|
|
- **Automated Testing**: Compose file validation
|
|
- **Rolling Updates**: Zero-downtime deployments
|
|
- **Rollback Capability**: Quick reversion to previous versions
|
|
|
|
## High Availability Design
|
|
|
|
### Service Redundancy
|
|
- **Load Balancing**: Nginx Proxy Manager
|
|
- **Health Checks**: Automated service monitoring
|
|
- **Failover**: Automatic service migration
|
|
- **Backup Services**: Secondary instances on different hosts
|
|
|
|
### Data Redundancy
|
|
- **RAID Arrays**: Hardware-level redundancy
|
|
- **Cross-server Backups**: Geographic distribution
|
|
- **Snapshot Schedules**: Point-in-time recovery
|
|
- **Offsite Backups**: Cloud storage integration
|
|
|
|
## Scalability Considerations
|
|
|
|
### Horizontal Scaling
|
|
- **Container Orchestration**: Easy service replication
|
|
- **Load Distribution**: Multiple server deployment
|
|
- **Database Clustering**: PostgreSQL/MySQL clusters
|
|
- **Storage Expansion**: Additional storage nodes
|
|
|
|
### Vertical Scaling
|
|
- **Resource Allocation**: Dynamic CPU/memory assignment
|
|
- **Storage Expansion**: RAID array growth
|
|
- **Network Upgrades**: 10GbE infrastructure
|
|
- **Hardware Refresh**: Regular equipment updates
|
|
|
|
## Technology Stack
|
|
|
|
### Core Technologies
|
|
- **Operating System**: Ubuntu Server 22.04 LTS
|
|
- **Containerization**: Docker & Docker Compose
|
|
- **Orchestration**: Portainer Community Edition
|
|
- **Reverse Proxy**: Nginx Proxy Manager
|
|
- **Monitoring**: Prometheus + Grafana stack
|
|
|
|
### Supporting Technologies
|
|
- **Version Control**: Git with Gitea
|
|
- **CI/CD**: Gitea Actions, Ansible
|
|
- **Backup**: Restic, rsync, custom scripts
|
|
- **Networking**: WireGuard, Tailscale, Cloudflare
|
|
- **Authentication**: Authentik, LDAP integration
|
|
|
|
## Performance Characteristics
|
|
|
|
### Expected Performance
|
|
- **Web Response**: < 200ms for local services
|
|
- **File Transfer**: 1Gbps+ within network
|
|
- **Database Queries**: < 50ms for typical operations
|
|
- **Container Startup**: < 30 seconds for most services
|
|
|
|
### Resource Utilization
|
|
- **CPU**: 20-40% average across servers
|
|
- **Memory**: 60-80% utilization with caching
|
|
- **Storage**: 70% capacity with growth planning
|
|
- **Network**: < 10% of available bandwidth
|
|
|
|
## Future Roadmap
|
|
|
|
### Short-term Improvements
|
|
- **Kubernetes Migration**: Container orchestration upgrade
|
|
- **Service Mesh**: Istio or Linkerd implementation
|
|
- **Observability**: Enhanced tracing and logging
|
|
- **Automation**: Expanded Ansible playbooks
|
|
|
|
### Long-term Vision
|
|
- **Edge Computing**: Additional edge nodes
|
|
- **AI/ML Integration**: GPU acceleration
|
|
- **Hybrid Cloud**: Public cloud integration
|
|
- **IoT Expansion**: Smart home integration
|
|
|
|
## Related Documentation
|
|
|
|
- [Prerequisites](04-Prerequisites.md) - Required knowledge and tools
|
|
- [Quick Start Guide](QUICK_START.md) - Deploy your first service
|
|
- [Infrastructure Documentation](../infrastructure/INFRASTRUCTURE_OVERVIEW.md)
|
|
- [Monitoring Setup](../admin/monitoring-setup.md)
|
|
|
|
---
|
|
|
|
*This architecture overview provides a comprehensive understanding of the homelab infrastructure design and implementation strategy.* |