Vish/homelab-optimized
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sanitized mirror from private repository - 2026-04-20 00:58:22 UTC
Some checks failed
Documentation / Build Docusaurus (push) Failing after 5m0s
Details
Documentation / Deploy to GitHub Pages (push) Has been skipped
Details

Browse Source
This commit is contained in:
Gitea Mirror Bot
2026-04-20 00:58:22 +00:00
commit 72afe8052c
1441 changed files with 363913 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
docs/getting-started/01-What-is-a-Homelab.md Normal file
View File

@@ -0,0 +1,164 @@
# What is a Homelab?
## Overview
A homelab is a personal computing environment where individuals can experiment, learn, and deploy various technologies and services in a controlled setting. It serves as a sandbox for testing new software, learning system administration, and hosting personal services.
## Why Build a Homelab?
### Learning and Skill Development
- **Hands-on Experience**: Practice with real hardware and software
- **Technology Exploration**: Test new tools and platforms safely
- **Career Development**: Build skills relevant to IT and DevOps roles
- **Certification Prep**: Practice environments for various certifications
### Personal Services
- **Self-hosted Applications**: Run your own cloud services
- **Data Privacy**: Keep personal data under your control
- **Custom Solutions**: Build applications tailored to your needs
- **Always Available**: 24/7 access to your services
### Cost Savings
- **Reduced Subscriptions**: Replace paid services with self-hosted alternatives
- **Hardware Utilization**: Make use of older hardware
- **Learning Investment**: Skills gained provide long-term value
## Common Homelab Components
### Hardware
- **Servers**: Dedicated machines for hosting services
- **Network Equipment**: Switches, routers, access points
- **Storage**: NAS devices, external drives
- **Monitoring**: UPS systems, environmental sensors
### Software
- **Virtualization**: Proxmox, VMware, Hyper-V
- **Containerization**: Docker, Kubernetes
- **Operating Systems**: Linux distributions, Windows Server
- **Applications**: Web servers, databases, monitoring tools
### Services
- **Media Management**: Plex, Jellyfin, Sonarr, Radarr
- **File Storage**: Nextcloud, Seafile, Syncthing
- **Monitoring**: Grafana, Prometheus, Uptime Kuma
- **Networking**: VPN servers, DNS, DHCP
## Homelab Types
### Beginner Setup
- Single computer or Raspberry Pi
- Basic services (file sharing, media server)
- Simple network configuration
- Learning-focused approach
### Intermediate Setup
- Multiple devices or virtual machines
- Network segmentation and VLANs
- Automated deployments
- Monitoring and alerting
### Advanced Setup
- Enterprise-grade hardware
- High availability configurations
- Complex networking (BGP, OSPF)
- Production-like environments
## Getting Started
### Planning Phase
1. **Define Goals**: What do you want to learn or achieve?
2. **Budget Planning**: Determine hardware and software costs
3. **Space Requirements**: Consider physical space and power
4. **Network Design**: Plan IP addressing and segmentation
### Implementation Phase
1. **Start Small**: Begin with basic services
2. **Document Everything**: Keep detailed notes and diagrams
3. **Backup Strategy**: Plan for data protection
4. **Security First**: Implement proper security measures
### Growth Phase
1. **Expand Gradually**: Add services based on needs
2. **Automate Processes**: Use configuration management
3. **Monitor Performance**: Track system health
4. **Share Knowledge**: Document and teach others
## Common Challenges
### Technical Challenges
- **Complexity Management**: Systems can become overwhelming
- **Hardware Failures**: Equipment will eventually fail
- **Security Concerns**: Proper hardening is essential
- **Performance Issues**: Resource constraints and bottlenecks
### Practical Challenges
- **Time Investment**: Maintenance requires ongoing effort
- **Power Consumption**: Electricity costs can add up
- **Noise Levels**: Server fans can be loud
- **Family Acceptance**: Balance hobby with household needs
## Best Practices
### Documentation
- **Network Diagrams**: Visual representation of infrastructure
- **Service Inventory**: List of all running services
- **Configuration Notes**: How services are configured
- **Troubleshooting Guides**: Common issues and solutions
### Security
- **Regular Updates**: Keep systems patched
- **Access Control**: Implement proper authentication
- **Network Segmentation**: Isolate services appropriately
- **Backup Verification**: Test restore procedures
### Monitoring
- **System Health**: CPU, memory, disk usage
- **Service Availability**: Uptime monitoring
- **Performance Metrics**: Response times and throughput
- **Alerting**: Notifications for issues
## This Homelab
This repository documents a comprehensive homelab setup featuring:
- **5 Physical Servers**: Atlantis, Calypso, Concord NUC, Homelab VM, Raspberry Pi
- **100+ Services**: Media management, development tools, monitoring
- **GitOps Workflow**: Infrastructure as code with automated deployments
- **Comprehensive Monitoring**: Grafana dashboards and alerting
### Key Features
- **Docker-based Deployments**: Containerized services with docker-compose
- **Automated Backups**: Regular data protection
- **Security Hardening**: VPN access, authentication, monitoring
- **High Availability**: Redundant services and failover capabilities
## Next Steps
1. **[Architecture Overview](03-Architecture-Overview.md)** - Understand the infrastructure design
2. **[Prerequisites](04-Prerequisites.md)** - Required knowledge and tools
3. **[Quick Start Guide](QUICK_START.md)** - Deploy your first service
4. **[Service Categories](../services/categories.md)** - Explore available services
## Resources
### Learning Materials
- [Homelab Subreddit](https://reddit.com/r/homelab)
- [Self-Hosted Awesome List](https://github.com/awesome-selfhosted/awesome-selfhosted)
- [Docker Documentation](https://docs.docker.com/)
- [Linux Academy](https://linuxacademy.com/)
### Hardware Vendors
- [Dell PowerEdge](https://www.dell.com/en-us/work/shop/servers-storage-and-networking/sf/poweredge-servers)
- [HP ProLiant](https://www.hpe.com/us/en/servers/proliant-servers.html)
- [Supermicro](https://www.supermicro.com/)
- [Raspberry Pi](https://www.raspberrypi.org/)
### Software Platforms
- [Proxmox VE](https://www.proxmox.com/en/proxmox-ve)
- [TrueNAS](https://www.truenas.com/)
- [pfSense](https://www.pfsense.org/)
- [Portainer](https://www.portainer.io/)
---
*This guide provides a foundation for understanding homelabs and serves as an introduction to the comprehensive setup documented in this repository.*

304
docs/getting-started/03-Architecture-Overview.md Normal file
View File

@@ -0,0 +1,304 @@
# Architecture Overview
## Infrastructure Design
This homelab implements a distributed, containerized architecture designed for high availability, scalability, and ease of management. The infrastructure follows GitOps principles with infrastructure-as-code practices.
## Network Architecture
### Physical Network
```
Internet
│
├── Router/Firewall (pfSense)
│ ├── Management VLAN (192.168.1.0/24)
│ ├── Server VLAN (192.168.10.0/24)
│ ├── IoT VLAN (192.168.20.0/24)
│ └── Guest VLAN (192.168.30.0/24)
│
└── Core Switch
├── Atlantis (192.168.10.10)
├── Calypso (192.168.10.20)
├── Concord NUC (192.168.10.30)
├── Homelab VM (192.168.10.40)
└── Raspberry Pi (192.168.10.50)
```
### Virtual Networks
- **Docker Networks**: Isolated container communication
- **VPN Tunnels**: Secure remote access via WireGuard
- **Tailscale Mesh**: Zero-trust network overlay
- **Cloudflare Tunnels**: Secure external access
## Server Architecture
### Atlantis (Primary Server)
**Role**: Main application server and storage
- **Hardware**: Dell PowerEdge R720
- **OS**: Ubuntu Server 22.04 LTS
- **Storage**: 12TB RAID-10 array
- **Services**: 40+ containerized applications
**Key Services**:
- Media Management (Plex, Sonarr, Radarr)
- File Storage (Nextcloud, Syncthing)
- Development Tools (GitLab, Portainer)
- Monitoring (Grafana, Prometheus)
### Calypso (Secondary Server)
**Role**: Backup services and specialized workloads
- **Hardware**: Custom build (AMD Ryzen)
- **OS**: Ubuntu Server 22.04 LTS
- **Storage**: 8TB RAID-1 array
- **Services**: 25+ containerized applications
**Key Services**:
- Authentication (Authentik)
- Game Servers (Minecraft, Satisfactory)
- Development (Gitea, CI/CD runners)
- Backup Services (Seafile, Immich)
### Concord NUC (Edge Computing)
**Role**: Edge services and IoT management
- **Hardware**: Intel NUC
- **OS**: Ubuntu Server 22.04 LTS
- **Storage**: 1TB NVMe SSD
- **Services**: 15+ lightweight applications
**Key Services**:
- Home Automation (Home Assistant)
- Network Services (AdGuard, Pi-hole)
- Media Streaming (Invidious, Piped)
- Monitoring (Node Exporter)
### Homelab VM (Development)
**Role**: Development and testing environment
- **Platform**: Proxmox VM
- **OS**: Ubuntu Server 22.04 LTS
- **Storage**: 500GB virtual disk
- **Services**: 30+ development tools
**Key Services**:
- AI/ML Tools (Ollama, OpenHands)
- Communication (Mattermost, Signal API)
- Testing Services (Various experimental apps)
- Monitoring (Alerting, NTFY)
### Raspberry Pi (Monitoring)
**Role**: Dedicated monitoring and lightweight services
- **Hardware**: Raspberry Pi 5
- **OS**: Raspberry Pi OS Lite
- **Storage**: 256GB microSD + USB storage
- **Services**: 5+ monitoring applications
**Key Services**:
- Uptime Monitoring (Uptime Kuma)
- System Monitoring (Glances)
- Photo Management (Immich)
- File Sharing (Samba)
## Container Architecture
### Orchestration Strategy
- **Docker Compose**: Primary orchestration tool
- **Portainer**: Web-based container management
- **Watchtower**: Automated container updates
- **GitOps**: Version-controlled deployments
### Container Patterns
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Application │ │ Database │ │ Storage │
│ Containers │ │ Containers │ │ Containers │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ • Web Services │ │ • PostgreSQL │ │ • File Shares │
│ • APIs │ │ • MySQL │ │ • Object Store │
│ • Workers │ │ • Redis │ │ • Backup Vols │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
│
┌─────────────────┐
│ Proxy/LB │
│ Containers │
├─────────────────┤
│ • Nginx Proxy │
│ • Traefik │
│ • Cloudflare │
└─────────────────┘
```
## Storage Architecture
### Primary Storage (Atlantis)
- **RAID-10**: 4x 4TB drives for performance and redundancy
- **Hot Spare**: Additional drive for automatic replacement
- **Backup Target**: Weekly snapshots to external storage
### Secondary Storage (Calypso)
- **RAID-1**: 2x 4TB drives for redundancy
- **Backup Source**: Receives backups from other servers
- **Archive Storage**: Long-term data retention
### Distributed Storage
- **Syncthing**: Peer-to-peer file synchronization
- **Seafile**: Centralized file storage with versioning
- **Immich**: Photo management with AI features
- **Nextcloud**: Personal cloud storage
## Monitoring Architecture
### Metrics Collection
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Node │ │ Container │ │ Application │
│ Exporter │───▶│ Advisor │───▶│ Metrics │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────────┼───────────────────┘
│
┌─────────────┐
│ Prometheus │
│ (Metrics) │
└─────────────┘
│
┌─────────────┐
│ Grafana │
│ (Dashboards)│
└─────────────┘
```
### Alerting Pipeline
```
Prometheus ──▶ Alertmanager ──▶ NTFY ──▶ Mobile/Desktop
│ │ │
│ ├──▶ Email ────┘
│ └──▶ Signal ───┘
│
└──▶ Uptime Kuma ──▶ Discord/Slack
```
## Security Architecture
### Network Security
- **Firewall Rules**: Strict ingress/egress controls
- **VPN Access**: WireGuard for remote connectivity
- **Zero Trust**: Tailscale mesh networking
- **SSL/TLS**: End-to-end encryption
### Application Security
- **Authentication**: Centralized with Authentik
- **Authorization**: Role-based access control
- **Secrets Management**: Docker secrets and environment files
- **Container Security**: Non-root users, read-only filesystems
### Data Security
- **Encryption at Rest**: LUKS disk encryption
- **Encryption in Transit**: TLS for all communications
- **Backup Encryption**: GPG-encrypted backups
- **Access Logging**: Comprehensive audit trails
## Deployment Architecture
### GitOps Workflow
```
Developer ──▶ Git Repository ──▶ CI/CD Pipeline ──▶ Container Registry
│ │
│ │
▼ ▼
Configuration ──▶ Portainer ──▶ Docker Compose ──▶ Containers
Files │ │
│ │ │
└───────────────┼──────────────────────────────┘
│
Monitoring & Alerting
```
### Continuous Deployment
- **Git-based**: All configurations in version control
- **Automated Testing**: Compose file validation
- **Rolling Updates**: Zero-downtime deployments
- **Rollback Capability**: Quick reversion to previous versions
## High Availability Design
### Service Redundancy
- **Load Balancing**: Nginx Proxy Manager
- **Health Checks**: Automated service monitoring
- **Failover**: Automatic service migration
- **Backup Services**: Secondary instances on different hosts
### Data Redundancy
- **RAID Arrays**: Hardware-level redundancy
- **Cross-server Backups**: Geographic distribution
- **Snapshot Schedules**: Point-in-time recovery
- **Offsite Backups**: Cloud storage integration
## Scalability Considerations
### Horizontal Scaling
- **Container Orchestration**: Easy service replication
- **Load Distribution**: Multiple server deployment
- **Database Clustering**: PostgreSQL/MySQL clusters
- **Storage Expansion**: Additional storage nodes
### Vertical Scaling
- **Resource Allocation**: Dynamic CPU/memory assignment
- **Storage Expansion**: RAID array growth
- **Network Upgrades**: 10GbE infrastructure
- **Hardware Refresh**: Regular equipment updates
## Technology Stack
### Core Technologies
- **Operating System**: Ubuntu Server 22.04 LTS
- **Containerization**: Docker & Docker Compose
- **Orchestration**: Portainer Community Edition
- **Reverse Proxy**: Nginx Proxy Manager
- **Monitoring**: Prometheus + Grafana stack
### Supporting Technologies
- **Version Control**: Git with Gitea
- **CI/CD**: Gitea Actions, Ansible
- **Backup**: Restic, rsync, custom scripts
- **Networking**: WireGuard, Tailscale, Cloudflare
- **Authentication**: Authentik, LDAP integration
## Performance Characteristics
### Expected Performance
- **Web Response**: < 200ms for local services
- **File Transfer**: 1Gbps+ within network
- **Database Queries**: < 50ms for typical operations
- **Container Startup**: < 30 seconds for most services
### Resource Utilization
- **CPU**: 20-40% average across servers
- **Memory**: 60-80% utilization with caching
- **Storage**: 70% capacity with growth planning
- **Network**: < 10% of available bandwidth
## Future Roadmap
### Short-term Improvements
- **Kubernetes Migration**: Container orchestration upgrade
- **Service Mesh**: Istio or Linkerd implementation
- **Observability**: Enhanced tracing and logging
- **Automation**: Expanded Ansible playbooks
### Long-term Vision
- **Edge Computing**: Additional edge nodes
- **AI/ML Integration**: GPU acceleration
- **Hybrid Cloud**: Public cloud integration
- **IoT Expansion**: Smart home integration
## Related Documentation
- [Prerequisites](04-Prerequisites.md) - Required knowledge and tools
- [Quick Start Guide](QUICK_START.md) - Deploy your first service
- [Infrastructure Documentation](../infrastructure/INFRASTRUCTURE_OVERVIEW.md)
- [Monitoring Setup](../admin/monitoring-setup.md)
---
*This architecture overview provides a comprehensive understanding of the homelab infrastructure design and implementation strategy.*

411
docs/getting-started/04-Prerequisites.md Normal file
View File

@@ -0,0 +1,411 @@
# Prerequisites
## Overview
Before diving into this homelab setup, ensure you have the necessary knowledge, tools, and hardware. This guide outlines the minimum requirements and recommended skills for successfully deploying and managing the infrastructure.
## Required Knowledge
### Essential Skills
- **Linux Administration**: Command line proficiency, file system navigation, package management
- **Networking Fundamentals**: TCP/IP, DNS, DHCP, VLANs, routing basics
- **Docker Basics**: Container concepts, docker-compose, image management
- **Git Version Control**: Repository management, branching, merging
### Recommended Skills
- **System Administration**: Service management, log analysis, troubleshooting
- **Security Practices**: SSH keys, firewall configuration, SSL/TLS certificates
- **Scripting**: Bash, Python, or similar for automation tasks
- **Monitoring**: Understanding metrics, alerting, and observability
### Learning Resources
- [Linux Journey](https://linuxjourney.com/) - Interactive Linux learning
- [Docker Official Tutorial](https://docs.docker.com/get-started/) - Container fundamentals
- [Networking Basics](https://www.cisco.com/c/en/us/solutions/small-business/resource-center/networking/networking-basics.html)
- [Git Handbook](https://guides.github.com/introduction/git-handbook/) - Version control basics
## Hardware Requirements
### Minimum Hardware
- **CPU**: 4 cores, 2.0GHz+ (x86_64 architecture)
- **RAM**: 8GB (16GB recommended)
- **Storage**: 500GB available space
- **Network**: Gigabit Ethernet connection
- **Power**: Uninterruptible Power Supply (UPS) recommended
### Recommended Hardware
- **CPU**: 8+ cores, 3.0GHz+ (Intel Xeon or AMD EPYC)
- **RAM**: 32GB+ with ECC support
- **Storage**: 2TB+ with RAID redundancy
- **Network**: 10GbE capable with managed switches
- **Power**: Enterprise UPS with network monitoring
### This Homelab Hardware
- **Atlantis**: Dell PowerEdge R720, 32GB RAM, 12TB RAID-10
- **Calypso**: Custom AMD Ryzen, 64GB RAM, 8TB RAID-1
- **Concord NUC**: Intel NUC, 16GB RAM, 1TB NVMe
- **Homelab VM**: Proxmox VM, 8GB RAM, 500GB virtual disk
- **Raspberry Pi**: Pi 5, 8GB RAM, 256GB microSD
## Software Requirements
### Operating System
- **Primary**: Ubuntu Server 22.04 LTS
- **Alternative**: Debian 12, CentOS Stream 9, Rocky Linux 9
- **Raspberry Pi**: Raspberry Pi OS Lite
### Core Software Stack
```bash
# Essential packages
sudo apt update && sudo apt install -y \
curl \
wget \
git \
vim \
htop \
net-tools \
openssh-server \
ufw \
fail2ban
```
### Docker Installation
```bash
# Install Docker Engine
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
# Add user to docker group
sudo usermod -aG docker $USER
```
### Git Configuration
```bash
# Configure Git
git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"
# Generate SSH key for Git
ssh-keygen -t ed25519 -C "your.email@example.com"
```
## Network Prerequisites
### Network Configuration
- **Static IP Addresses**: Servers should have static IPs
- **DNS Resolution**: Proper hostname resolution
- **Firewall Rules**: Appropriate port access
- **Time Synchronization**: NTP configuration
### Required Ports
| Service | Port | Protocol | Purpose |
|---------|------|----------|---------|
| SSH | 22 | TCP | Remote administration |
| HTTP | 80 | TCP | Web services |
| HTTPS | 443 | TCP | Secure web services |
| Docker API | 2376 | TCP | Docker remote API |
| Portainer | 9000 | TCP | Container management |
| Grafana | 3000 | TCP | Monitoring dashboards |
| Prometheus | 9090 | TCP | Metrics collection |
### Network Setup Example
```bash
# Configure static IP (Ubuntu/Netplan)
sudo vim /etc/netplan/00-installer-config.yaml
network:
version: 2
ethernets:
ens18:
dhcp4: false
addresses:
- 192.168.10.10/24
gateway4: 192.168.10.1
nameservers:
addresses:
- 192.168.10.1
- 8.8.8.8
# Apply configuration
sudo netplan apply
```
## Security Prerequisites
### SSH Security
```bash
# Generate SSH key pair
ssh-keygen -t ed25519 -f ~/.ssh/homelab_key
# Configure SSH client
cat >> ~/.ssh/config << EOF
Host atlantis
HostName 192.168.10.10
User homelab
IdentityFile ~/.ssh/homelab_key
Port 22
EOF
# Copy public key to servers
ssh-copy-id -i ~/.ssh/homelab_key.pub homelab@192.168.10.10
```
### Firewall Configuration
```bash
# Enable UFW firewall
sudo ufw enable
# Allow SSH
sudo ufw allow ssh
# Allow HTTP/HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Allow specific services
sudo ufw allow 9000/tcp # Portainer
sudo ufw allow 3000/tcp # Grafana
```
### SSL/TLS Certificates
- **Let's Encrypt**: Free SSL certificates for public domains
- **Self-signed**: For internal services
- **Certificate Management**: Automated renewal processes
## Storage Prerequisites
### Disk Configuration
```bash
# Check available disks
lsblk
# Create RAID array (example)
sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc
# Format and mount
sudo mkfs.ext4 /dev/md0
sudo mkdir /mnt/storage
sudo mount /dev/md0 /mnt/storage
# Add to fstab for persistence
echo '/dev/md0 /mnt/storage ext4 defaults 0 2' | sudo tee -a /etc/fstab
```
### Backup Strategy
- **Local Backups**: Regular snapshots to secondary storage
- **Remote Backups**: Offsite backup to cloud or remote location
- **Backup Testing**: Regular restore testing procedures
- **Retention Policy**: Define backup retention schedules
## Monitoring Prerequisites
### System Monitoring
```bash
# Install monitoring tools
sudo apt install -y \
htop \
iotop \
nethogs \
ncdu \
smartmontools
# Enable SMART monitoring
sudo systemctl enable smartd
sudo systemctl start smartd
```
### Log Management
```bash
# Configure log rotation
sudo vim /etc/logrotate.d/docker
/var/lib/docker/containers/*/*.log {
rotate 7
daily
compress
size=1M
missingok
delaycompress
copytruncate
}
```
## Development Environment
### Local Development Setup
```bash
# Install development tools
sudo apt install -y \
build-essential \
python3 \
python3-pip \
nodejs \
npm \
code
# Install useful Python packages
pip3 install --user \
docker-compose \
ansible \
requests \
pyyaml
```
### IDE Configuration
- **VS Code**: Remote SSH extension for server editing
- **Vim/Neovim**: Terminal-based editing with plugins
- **JetBrains**: Remote development capabilities
## Automation Prerequisites
### Ansible Setup
```bash
# Install Ansible
sudo apt install -y ansible
# Create inventory file
cat > inventory.ini << EOF
[homelab]
atlantis ansible_host=192.168.10.10
calypso ansible_host=192.168.10.20
concord ansible_host=192.168.10.30
[homelab:vars]
ansible_user=homelab
ansible_ssh_private_key_file=~/.ssh/homelab_key
EOF
# Test connectivity
ansible -i inventory.ini homelab -m ping
```
### CI/CD Prerequisites
- **Git Repository**: Version control for configurations
- **CI/CD Platform**: Gitea Actions, GitHub Actions, or GitLab CI
- **Container Registry**: Docker Hub or private registry
- **Deployment Keys**: SSH keys for automated deployments
## Backup and Recovery
### Backup Tools
```bash
# Install backup utilities
sudo apt install -y \
rsync \
restic \
borgbackup \
duplicity
# Configure restic repository
export RESTIC_REPOSITORY="/mnt/backup/restic"
export RESTIC_PASSWORD="REDACTED_PASSWORD"
restic init
```
### Recovery Planning
- **Documentation**: Detailed recovery procedures
- **Testing**: Regular disaster recovery drills
- **Offsite Storage**: Remote backup locations
- **Recovery Time Objectives**: Define acceptable downtime
## Validation Checklist
### Pre-deployment Checklist
- [ ] Hardware meets minimum requirements
- [ ] Operating system installed and updated
- [ ] Docker and Docker Compose installed
- [ ] Git configured with SSH keys
- [ ] Network connectivity verified
- [ ] Firewall rules configured
- [ ] SSH access working
- [ ] Storage properly configured
- [ ] Backup strategy implemented
- [ ] Monitoring tools installed
### Post-deployment Checklist
- [ ] All services accessible
- [ ] Monitoring dashboards functional
- [ ] Backup jobs running successfully
- [ ] Security hardening applied
- [ ] Documentation updated
- [ ] Team access configured
- [ ] Alerting rules tested
- [ ] Performance baselines established
## Common Issues and Solutions
### Docker Permission Issues
```bash
# Add user to docker group
sudo usermod -aG docker $USER
# Logout and login again
```
### Network Connectivity Problems
```bash
# Check network configuration
ip addr show
ip route show
systemctl status networking
# Test connectivity
ping 8.8.8.8
nslookup google.com
```
### Storage Issues
```bash
# Check disk space
df -h
du -sh /*
# Check RAID status
cat /proc/mdstat
sudo mdadm --detail /dev/md0
```
### Service Discovery Issues
```bash
# Check DNS resolution
nslookup service.local
dig service.local
# Check service status
docker ps
docker-compose ps
systemctl status docker
```
## Next Steps
Once prerequisites are met:
1. **[Quick Start Guide](QUICK_START.md)** - Deploy your first service
2. **[Architecture Overview](03-Architecture-Overview.md)** - Understand the design
3. **[Service Categories](../services/categories.md)** - Explore available services
4. **[GitOps Deployment](../admin/gitops-deployment-guide.md)** - Learn deployment workflows
## Support Resources
### Documentation
- [Infrastructure Overview](../infrastructure/INFRASTRUCTURE_OVERVIEW.md)
- [Troubleshooting Guide](../troubleshooting/README.md)
- [Security Guidelines](../security/README.md)
### Community
- [Homelab Subreddit](https://reddit.com/r/homelab)
- [Self-Hosted Community](https://reddit.com/r/selfhosted)
- [Docker Community](https://forums.docker.com/)
### Official Documentation
- [Docker Documentation](https://docs.docker.com/)
- [Ubuntu Server Guide](https://ubuntu.com/server/docs)
- [Ansible Documentation](https://docs.ansible.com/)
---
*Ensure all prerequisites are met before proceeding with the homelab deployment to avoid common setup issues and ensure a smooth installation process.*

295
docs/getting-started/20-Service-Categories.md Normal file
View File

@@ -0,0 +1,295 @@
# Service Categories
## Overview
This homelab hosts 100+ services across multiple categories, providing a comprehensive self-hosted infrastructure. Services are organized by function and deployed using Docker Compose with GitOps workflows.
## Media Management & Entertainment
### Core Media Services
- **[Plex](../services/individual/plex.md)** - Media server with transcoding
- **[Jellyfin](../services/individual/jellyfin.md)** - Open-source media server
- **[Emby](../services/individual/emby.md)** - Alternative media server
### Media Acquisition
- **[Sonarr](../services/individual/sonarr.md)** - TV show management
- **[Radarr](../services/individual/radarr.md)** - Movie management
- **[Lidarr](../services/individual/lidarr.md)** - Music management
- **[Readarr](../services/individual/readarr.md)** - Book management
- **[Prowlarr](../services/individual/prowlarr.md)** - Indexer management
### Download Clients
- **[qBittorrent](../services/individual/qbittorrent.md)** - BitTorrent client
- **[SABnzbd](../services/individual/sabnzbd.md)** - Usenet downloader
- **[JDownloader2](../services/individual/jdownloader2.md)** - Direct download manager
- **[yt-dlp](../services/individual/yt-dlp.md)** - YouTube downloader
### Media Processing
- **[Tdarr](../services/individual/tdarr.md)** - Media transcoding automation
- **[Handbrake](../services/individual/handbrake.md)** - Video transcoding
- **[MKVToolNix](../services/individual/mkvtoolnix.md)** - Video file manipulation
## Development & DevOps
### Version Control & CI/CD
- **[Gitea](../services/individual/gitea.md)** - Self-hosted Git service
- **[GitLab](../services/individual/gitlab.md)** - Complete DevOps platform
- **[Gitea Actions](../services/individual/gitea-actions.md)** - CI/CD automation
- **[Drone CI](../services/individual/drone.md)** - Container-native CI/CD
### Container Management
- **[Portainer](../services/individual/portainer.md)** - Docker management UI
- **[Dozzle](../services/individual/dozzle.md)** - Docker log viewer
- **[Watchtower](../services/individual/watchtower.md)** - Automated container updates
- **[Diun](../services/individual/diun.md)** - Docker image update notifications
### Development Tools
- **[Code Server](../services/individual/code-server.md)** - VS Code in browser
- **[Jupyter](../services/individual/jupyter.md)** - Interactive notebooks
- **[OpenHands](../services/individual/openhands.md)** - AI coding assistant
- **[Plane.so](../services/individual/plane.md)** - Project management
### Databases
- **[PostgreSQL](../services/individual/postgresql.md)** - Relational database
- **[MySQL](../services/individual/mysql.md)** - Popular database
- **[Redis](../services/individual/redis.md)** - In-memory data store
- **[InfluxDB](../services/individual/influxdb.md)** - Time-series database
## File Storage & Sync
### Cloud Storage
- **[Nextcloud](../services/individual/nextcloud.md)** - Personal cloud platform
- **[Seafile](../services/individual/seafile.md)** - File hosting platform
- **[ownCloud](../services/individual/owncloud.md)** - File sync and share
### File Synchronization
- **[Syncthing](../services/individual/syncthing.md)** - Peer-to-peer sync
- **[Resilio Sync](../services/individual/resilio.md)** - BitTorrent-based sync
- **[FreeFileSync](../services/individual/freefilesync.md)** - Folder comparison
### File Management
- **[FileBrowser](../services/individual/filebrowser.md)** - Web file manager
- **[Samba](../services/individual/samba.md)** - SMB/CIFS file sharing
- **[SFTP](../services/individual/sftp.md)** - Secure file transfer
## Monitoring & Observability
### Metrics & Dashboards
- **[Grafana](../services/individual/grafana.md)** - Visualization platform
- **[Prometheus](../services/individual/prometheus.md)** - Metrics collection
- **[InfluxDB](../services/individual/influxdb.md)** - Time-series database
- **[Telegraf](../services/individual/telegraf.md)** - Metrics agent
### System Monitoring
- **[Node Exporter](../services/individual/node-exporter.md)** - System metrics
- **[cAdvisor](../services/individual/cadvisor.md)** - Container metrics
- **[Glances](../services/individual/glances.md)** - System overview
- **[Netdata](../services/individual/netdata.md)** - Real-time monitoring
### Uptime & Alerting
- **[Uptime Kuma](../services/individual/uptime-kuma.md)** - Uptime monitoring
- **[Alertmanager](../services/individual/alertmanager.md)** - Alert routing
- **[NTFY](../services/individual/ntfy.md)** - Push notifications
- **[Gotify](../services/individual/gotify.md)** - Self-hosted notifications
### Log Management
- **[Loki](../services/individual/loki.md)** - Log aggregation
- **[Promtail](../services/individual/promtail.md)** - Log shipping
- **[Graylog](../services/individual/graylog.md)** - Log management
- **[Fluentd](../services/individual/fluentd.md)** - Log collection
## Networking & Security
### VPN & Remote Access
- **[WireGuard](../services/individual/wireguard.md)** - Modern VPN protocol
- **[OpenVPN](../services/individual/openvpn.md)** - Traditional VPN
- **[Tailscale](../services/individual/tailscale.md)** - Zero-config VPN
- **[Headscale](../services/individual/headscale.md)** - Self-hosted Tailscale
### Reverse Proxy & Load Balancing
- **[Nginx Proxy Manager](../services/individual/nginx-proxy-manager.md)** - Web-based proxy
- **[Traefik](../services/individual/traefik.md)** - Modern reverse proxy
- **[HAProxy](../services/individual/haproxy.md)** - Load balancer
- **[Cloudflare Tunnel](../services/individual/cloudflare-tunnel.md)** - Secure tunneling
### DNS & Network Services
- **[Pi-hole](../services/individual/pihole.md)** - Network-wide ad blocking
- **[AdGuard Home](../services/individual/adguard.md)** - DNS filtering
- **[Unbound](../services/individual/unbound.md)** - Recursive DNS resolver
- **[BIND9](../services/individual/bind9.md)** - Authoritative DNS
### Security Tools
- **[Authentik](../services/individual/authentik.md)** - Identity provider
- **[Authelia](../services/individual/authelia.md)** - Authentication server
- **[Fail2Ban](../services/individual/fail2ban.md)** - Intrusion prevention
- **[CrowdSec](../services/individual/crowdsec.md)** - Collaborative security
## Communication & Collaboration
### Chat & Messaging
- **[Mattermost](../services/individual/mattermost.md)** - Team communication
- **[Rocket.Chat](../services/individual/rocketchat.md)** - Open-source chat
- **[Matrix Synapse](../services/individual/matrix.md)** - Decentralized chat
- **[Signal API](../services/individual/signal-api.md)** - Signal messaging bridge
### Video Conferencing
- **[Jitsi Meet](../services/individual/jitsi.md)** - Video conferencing
- **[BigBlueButton](../services/individual/bigbluebutton.md)** - Web conferencing
- **[Jami](../services/individual/jami.md)** - P2P communication
### Email & Calendar
- **[Mailcow](../services/individual/mailcow.md)** - Email server suite
- **[Roundcube](../services/individual/roundcube.md)** - Webmail client
- **[Baikal](../services/individual/baikal.md)** - CalDAV/CardDAV server
- **[SOGo](../services/individual/sogo.md)** - Groupware server
## Productivity & Office
### Document Management
- **[Paperless-ngx](../services/individual/paperless-ngx.md)** - Document management
- **[Docuseal](../services/individual/docuseal.md)** - Document signing
- **[Stirling PDF](../services/individual/stirling-pdf.md)** - PDF manipulation
- **[OnlyOffice](../services/individual/onlyoffice.md)** - Office suite
### Note Taking & Knowledge
- **[Joplin Server](../services/individual/joplin.md)** - Note synchronization
- **[TiddlyWiki](../services/individual/tiddlywiki.md)** - Non-linear documentation
- **[DokuWiki](../services/individual/dokuwiki.md)** - File-based wiki
- **[BookStack](../services/individual/bookstack.md)** - Self-hosted wiki
### Project Management
- **[Plane.so](../services/individual/plane.md)** - Modern project management
- **[OpenProject](../services/individual/openproject.md)** - Project management suite
- **[Taiga](../services/individual/taiga.md)** - Agile project management
- **[Kanboard](../services/individual/kanboard.md)** - Kanban board
## Gaming & Entertainment
### Game Servers
- **[Minecraft](../services/individual/minecraft.md)** - Minecraft server
- **[Satisfactory](../services/individual/satisfactory.md)** - Satisfactory dedicated server
- **[Left 4 Dead 2](../services/individual/l4d2.md)** - L4D2 server
- **[Don't Starve Together](../services/individual/dont-starve.md)** - DST server
### Game Management
- **[PufferPanel](../services/individual/pufferpanel.md)** - Game server management
- **[Pterodactyl](../services/individual/pterodactyl.md)** - Game server panel
- **[AMP](../services/individual/amp.md)** - Application Management Panel
### Retro Gaming
- **[RetroArch](../services/individual/retroarch.md)** - Multi-emulator
- **[EmulationStation](../services/individual/emulationstation.md)** - Retro gaming frontend
- **[ROMM](../services/individual/romm.md)** - ROM management
## Utilities & Tools
### System Utilities
- **[Glances](../services/individual/glances.md)** - System monitoring
- **[Netdata](../services/individual/netdata.md)** - Real-time performance
- **[Speedtest](../services/individual/speedtest.md)** - Network speed testing
- **[IT Tools](../services/individual/it-tools.md)** - Developer utilities
### Backup & Recovery
- **[Duplicati](../services/individual/duplicati.md)** - Backup software
- **[Restic](../services/individual/restic.md)** - Fast backup program
- **[Borg Backup](../services/individual/borgbackup.md)** - Deduplicating backup
- **[Rclone](../services/individual/rclone.md)** - Cloud storage sync
### Network Tools
- **[Smokeping](../services/individual/smokeping.md)** - Network latency monitoring
- **[LibreSpeed](../services/individual/librespeed.md)** - Speed test server
- **[Iperf3](../services/individual/iperf3.md)** - Network performance testing
- **[Nmap](../services/individual/nmap.md)** - Network discovery
## AI & Machine Learning
### AI Platforms
- **[Ollama](../services/individual/ollama.md)** - Local LLM hosting
- **[OpenHands](../services/individual/openhands.md)** - AI coding assistant
- **[Perplexica](../services/individual/perplexica.md)** - AI search engine
- **[LlamaGPT](../services/individual/llamagpt.md)** - Self-hosted ChatGPT
### Machine Learning Tools
- **[Jupyter](../services/individual/jupyter.md)** - ML notebooks
- **[MLflow](../services/individual/mlflow.md)** - ML lifecycle management
- **[TensorBoard](../services/individual/tensorboard.md)** - ML visualization
- **[Weights & Biases](../services/individual/wandb.md)** - ML experiment tracking
## Finance & Personal Management
### Financial Management
- **[Firefly III](../services/individual/firefly.md)** - Personal finance manager
- **[Actual Budget](../services/individual/actual.md)** - Budgeting application
- **[GnuCash](../services/individual/gnucash.md)** - Accounting software
- **[Invoice Ninja](../services/individual/invoice-ninja.md)** - Invoicing platform
### Password Management
- **[Vaultwarden](../services/individual/vaultwarden.md)** - Bitwarden server
- **[Passbolt](../services/individual/passbolt.md)** - Team password manager
- **[KeeWeb](../services/individual/keeweb.md)** - Web-based password manager
## Social & Content
### Social Media
- **[Mastodon](../services/individual/mastodon.md)** - Decentralized social network
- **[Pleroma](../services/individual/pleroma.md)** - Lightweight social server
- **[Diaspora](../services/individual/diaspora.md)** - Distributed social network
### Content Aggregation
- **[FreshRSS](../services/individual/freshrss.md)** - RSS aggregator
- **[Miniflux](../services/individual/miniflux.md)** - Minimalist RSS reader
- **[Wallabag](../services/individual/wallabag.md)** - Read-later application
- **[Hoarder](../services/individual/hoarder.md)** - Bookmark manager
### Alternative Frontends
- **[Invidious](../services/individual/invidious.md)** - YouTube frontend
- **[Piped](../services/individual/piped.md)** - Privacy-friendly YouTube
- **[Redlib](../services/individual/redlib.md)** - Reddit frontend
- **[Proxitok](../services/individual/proxitok.md)** - TikTok frontend
## Deployment Information
### Server Distribution
- **Atlantis**: 40+ services (primary media and storage)
- **Calypso**: 25+ services (development and backup)
- **Concord NUC**: 15+ services (edge and IoT)
- **Homelab VM**: 30+ services (development and testing)
- **Raspberry Pi**: 5+ services (monitoring and lightweight)
### Resource Requirements
- **Total RAM**: 128GB across all servers
- **Total Storage**: 25TB+ with RAID redundancy
- **Network**: Gigabit with 10GbE backbone
- **Power**: 500W average consumption
### Management Tools
- **[Portainer](../services/individual/portainer.md)** - Container orchestration
- **[Watchtower](../services/individual/watchtower.md)** - Automated updates
- **[Grafana](../services/individual/grafana.md)** - Monitoring dashboards
- **[Uptime Kuma](../services/individual/uptime-kuma.md)** - Service monitoring
## Quick Access Links
### Most Used Services
- [Plex Media Server](http://atlantis.vish.local:32400)
- [Portainer Management](http://atlantis.vish.local:9000)
- [Grafana Dashboards](http://atlantis.vish.local:3000)
- [Gitea Repository](http://calypso.vish.local:3000)
- [Nextcloud Files](http://atlantis.vish.local:8080)
### Administrative Interfaces
- [Nginx Proxy Manager](http://calypso.vish.local:81)
- [Authentik SSO](http://calypso.vish.local:9000)
- [Uptime Kuma](http://raspberry-pi.vish.local:3001)
- [AdGuard Home](http://concord.vish.local:3000)
## Related Documentation
- **[Service Index](21-Service-Index.md)** - Alphabetical service listing
- **[Deployment Guide](30-Deployment-Guide.md)** - Service deployment procedures
- **[Common Issues](40-Common-Issues.md)** - Troubleshooting guide
- **[Ansible Automation](50-Ansible-Automation.md)** - Automated deployment
---
*This comprehensive service catalog provides an overview of all available services in the homelab infrastructure, organized by category for easy navigation and management.*

263
docs/getting-started/21-Service-Index.md Normal file
View File

@@ -0,0 +1,263 @@
# Service Index
## Alphabetical Service Listing
This index provides a comprehensive alphabetical listing of all services deployed in the homelab environment with quick access links and basic information.
## A
- **[Actual Budget](../services/individual/actual.md)** - Personal budgeting application
- **[AdGuard Home](../services/individual/adguard.md)** - Network-wide ad blocking and DNS filtering
- **[Alertmanager](../services/individual/alertmanager.md)** - Alert routing and management for Prometheus
- **[ArchiveBox](../services/individual/archivebox.md)** - Web page archiving and preservation
- **[Authentik](../services/individual/authentik.md)** - Identity provider and SSO solution
## B
- **[Baikal](../services/individual/baikal.md)** - CalDAV and CardDAV server
- **[BigBlueButton](../services/individual/bigbluebutton.md)** - Web conferencing platform
- **[BookStack](../services/individual/bookstack.md)** - Self-hosted wiki platform
- **[Borg Backup](../services/individual/borgbackup.md)** - Deduplicating backup program
## C
- **[cAdvisor](../services/individual/cadvisor.md)** - Container resource usage monitoring
- **[Calibre](../services/individual/calibre.md)** - E-book management and server
- **[Code Server](../services/individual/code-server.md)** - VS Code in the browser
- **[CrowdSec](../services/individual/crowdsec.md)** - Collaborative security engine
## D
- **[DashDot](../services/individual/dashdot.md)** - Modern server dashboard
- **[DokuWiki](../services/individual/dokuwiki.md)** - File-based wiki system
- **[Don't Starve Together](../services/individual/dont-starve.md)** - Game server
- **[Dozzle](../services/individual/dozzle.md)** - Real-time Docker log viewer
- **[Duplicati](../services/individual/duplicati.md)** - Cross-platform backup software
## E
- **[Emby](../services/individual/emby.md)** - Media server platform
- **[EmulationStation](../services/individual/emulationstation.md)** - Retro gaming frontend
## F
- **[Fail2Ban](../services/individual/fail2ban.md)** - Intrusion prevention system
- **[FileBrowser](../services/individual/filebrowser.md)** - Web-based file manager
- **[Firefly III](../services/individual/firefly.md)** - Personal finance manager
- **[Fluentd](../services/individual/fluentd.md)** - Data collection and log management
- **[FreshRSS](../services/individual/freshrss.md)** - RSS feed aggregator
## G
- **[Gitea](../services/individual/gitea.md)** - Self-hosted Git service
- **[GitLab](../services/individual/gitlab.md)** - Complete DevOps platform
- **[Glances](../services/individual/glances.md)** - Cross-platform system monitoring
- **[Gotify](../services/individual/gotify.md)** - Self-hosted push notification service
- **[Grafana](../services/individual/grafana.md)** - Observability and monitoring platform
- **[Graylog](../services/individual/graylog.md)** - Log management platform
## H
- **[HAProxy](../services/individual/haproxy.md)** - Load balancer and proxy server
- **[Headscale](../services/individual/headscale.md)** - Self-hosted Tailscale control server
- **[Hoarder](../services/individual/hoarder.md)** - Bookmark and content manager
- **[Home Assistant](../services/individual/home-assistant.md)** - Home automation platform
## I
- **[Immich](../services/individual/immich.md)** - Self-hosted photo and video management
- **[InfluxDB](../services/individual/influxdb.md)** - Time series database
- **[Invidious](../services/individual/invidious.md)** - Privacy-focused YouTube frontend
- **[Iperf3](../services/individual/iperf3.md)** - Network performance testing tool
- **[IT Tools](../services/individual/it-tools.md)** - Collection of developer utilities
## J
- **[JDownloader2](../services/individual/jdownloader2.md)** - Download management tool
- **[Jellyfin](../services/individual/jellyfin.md)** - Free media server software
- **[Jitsi Meet](../services/individual/jitsi.md)** - Video conferencing platform
- **[Joplin Server](../services/individual/joplin.md)** - Note-taking synchronization server
- **[Jupyter](../services/individual/jupyter.md)** - Interactive computing notebooks
## K
- **[Kanboard](../services/individual/kanboard.md)** - Project management software
- **[KeeWeb](../services/individual/keeweb.md)** - Web-based password manager
## L
- **[Left 4 Dead 2](../services/individual/l4d2.md)** - Game server
- **[LibreSpeed](../services/individual/librespeed.md)** - Speed test server
- **[Lidarr](../services/individual/lidarr.md)** - Music collection manager
- **[LlamaGPT](../services/individual/llamagpt.md)** - Self-hosted ChatGPT alternative
- **[Loki](../services/individual/loki.md)** - Log aggregation system
## M
- **[Mailcow](../services/individual/mailcow.md)** - Email server suite
- **[Mastodon](../services/individual/mastodon.md)** - Decentralized social network
- **[Matrix Synapse](../services/individual/matrix.md)** - Decentralized communication server
- **[Mattermost](../services/individual/mattermost.md)** - Team collaboration platform
- **[Minecraft](../services/individual/minecraft.md)** - Game server
- **[Miniflux](../services/individual/miniflux.md)** - Minimalist RSS reader
- **[MLflow](../services/individual/mlflow.md)** - Machine learning lifecycle management
- **[MySQL](../services/individual/mysql.md)** - Relational database management system
## N
- **[Netdata](../services/individual/netdata.md)** - Real-time performance monitoring
- **[Nextcloud](../services/individual/nextcloud.md)** - Personal cloud platform
- **[Nginx Proxy Manager](../services/individual/nginx-proxy-manager.md)** - Reverse proxy management
- **[Node Exporter](../services/individual/node-exporter.md)** - Hardware and OS metrics exporter
- **[NTFY](../services/individual/ntfy.md)** - Push notification service
## O
- **[Ollama](../services/individual/ollama.md)** - Local large language model hosting
- **[OnlyOffice](../services/individual/onlyoffice.md)** - Office suite and document collaboration
- **[OpenHands](../services/individual/openhands.md)** - AI-powered coding assistant
- **[OpenProject](../services/individual/openproject.md)** - Project management suite
- **[OpenVPN](../services/individual/openvpn.md)** - VPN server software
- **[ownCloud](../services/individual/owncloud.md)** - File sync and share platform
## P
- **[Paperless-ngx](../services/individual/paperless-ngx.md)** - Document management system
- **[Passbolt](../services/individual/passbolt.md)** - Team password manager
- **[Perplexica](../services/individual/perplexica.md)** - AI-powered search engine
- **[Pi-hole](../services/individual/pihole.md)** - Network-wide ad blocking
- **[Piped](../services/individual/piped.md)** - Privacy-friendly YouTube frontend
- **[Plane.so](../services/individual/plane.md)** - Modern project management platform
- **[Plex](../services/individual/plex.md)** - Media server and streaming platform
- **[Portainer](../services/individual/portainer.md)** - Container management platform
- **[PostgreSQL](../services/individual/postgresql.md)** - Advanced relational database
- **[Prometheus](../services/individual/prometheus.md)** - Monitoring and alerting toolkit
- **[Promtail](../services/individual/promtail.md)** - Log shipping agent for Loki
- **[Prowlarr](../services/individual/prowlarr.md)** - Indexer manager for *arr suite
- **[Proxitok](../services/individual/proxitok.md)** - Privacy-focused TikTok frontend
- **[Pterodactyl](../services/individual/pterodactyl.md)** - Game server management panel
- **[PufferPanel](../services/individual/pufferpanel.md)** - Game server management platform
## Q
- **[qBittorrent](../services/individual/qbittorrent.md)** - BitTorrent client
## R
- **[Radarr](../services/individual/radarr.md)** - Movie collection manager
- **[Rclone](../services/individual/rclone.md)** - Cloud storage synchronization
- **[Readarr](../services/individual/readarr.md)** - Book collection manager
- **[Redis](../services/individual/redis.md)** - In-memory data structure store
- **[Redlib](../services/individual/redlib.md)** - Privacy-focused Reddit frontend
- **[Resilio Sync](../services/individual/resilio.md)** - BitTorrent-based file sync
- **[Restic](../services/individual/restic.md)** - Fast, secure backup program
- **[RetroArch](../services/individual/retroarch.md)** - Multi-platform emulator
- **[Rocket.Chat](../services/individual/rocketchat.md)** - Team communication platform
- **[ROMM](../services/individual/romm.md)** - ROM management system
- **[Roundcube](../services/individual/roundcube.md)** - Web-based email client
## S
- **[SABnzbd](../services/individual/sabnzbd.md)** - Usenet binary downloader
- **[Samba](../services/individual/samba.md)** - SMB/CIFS file sharing
- **[Satisfactory](../services/individual/satisfactory.md)** - Dedicated game server
- **[Seafile](../services/individual/seafile.md)** - File hosting and collaboration platform
- **[SFTP](../services/individual/sftp.md)** - Secure file transfer protocol server
- **[Signal API](../services/individual/signal-api.md)** - Signal messaging bridge
- **[Smokeping](../services/individual/smokeping.md)** - Network latency monitoring
- **[SOGo](../services/individual/sogo.md)** - Groupware server
- **[Sonarr](../services/individual/sonarr.md)** - TV series collection manager
- **[Speedtest](../services/individual/speedtest.md)** - Network speed testing
- **[Stirling PDF](../services/individual/stirling-pdf.md)** - PDF manipulation toolkit
- **[Syncthing](../services/individual/syncthing.md)** - Continuous file synchronization
## T
- **[Tailscale](../services/individual/tailscale.md)** - Zero-config VPN mesh network
- **[Taiga](../services/individual/taiga.md)** - Agile project management platform
- **[Tdarr](../services/individual/tdarr.md)** - Media transcoding automation
- **[Telegraf](../services/individual/telegraf.md)** - Metrics collection agent
- **[TensorBoard](../services/individual/tensorboard.md)** - Machine learning visualization
- **[TiddlyWiki](../services/individual/tiddlywiki.md)** - Non-linear documentation tool
- **[Traefik](../services/individual/traefik.md)** - Modern reverse proxy and load balancer
## U
- **[Unbound](../services/individual/unbound.md)** - Recursive DNS resolver
- **[Uptime Kuma](../services/individual/uptime-kuma.md)** - Self-hosted uptime monitoring
## V
- **[Vaultwarden](../services/individual/vaultwarden.md)** - Bitwarden-compatible password manager
## W
- **[Wallabag](../services/individual/wallabag.md)** - Read-later application
- **[Watchtower](../services/individual/watchtower.md)** - Automated Docker container updates
- **[Weights & Biases](../services/individual/wandb.md)** - ML experiment tracking
- **[WireGuard](../services/individual/wireguard.md)** - Modern VPN protocol
## Y
- **[yt-dlp](../services/individual/yt-dlp.md)** - YouTube and media downloader
## Service Statistics
### By Category
- **Media Management**: 15 services
- **Development & DevOps**: 12 services
- **File Storage & Sync**: 8 services
- **Monitoring & Observability**: 14 services
- **Networking & Security**: 11 services
- **Communication & Collaboration**: 9 services
- **Productivity & Office**: 8 services
- **Gaming & Entertainment**: 7 services
- **Utilities & Tools**: 10 services
- **AI & Machine Learning**: 6 services
- **Finance & Personal**: 5 services
- **Social & Content**: 8 services
### By Server
- **Atlantis**: 40+ services (Primary media and storage)
- **Calypso**: 25+ services (Development and backup)
- **Concord NUC**: 15+ services (Edge and IoT)
- **Homelab VM**: 30+ services (Development and testing)
- **Raspberry Pi**: 5+ services (Monitoring and lightweight)
### Resource Requirements
- **Total Services**: 115+ active services
- **Total RAM Usage**: ~80GB across all servers
- **Total Storage**: 25TB+ with RAID redundancy
- **Network Bandwidth**: Gigabit with 10GbE backbone
## Quick Access by Function
### Most Used Services
- [Plex Media Server](http://atlantis.vish.local:32400) - Media streaming
- [Portainer](http://atlantis.vish.local:9000) - Container management
- [Grafana](http://atlantis.vish.local:3000) - Monitoring dashboards
- [Nextcloud](http://atlantis.vish.local:8080) - File storage
- [Gitea](http://calypso.vish.local:3000) - Git repository
### Administrative Services
- [Nginx Proxy Manager](http://calypso.vish.local:81) - Reverse proxy
- [Authentik](http://calypso.vish.local:9000) - Authentication
- [Uptime Kuma](http://raspberry-pi.vish.local:3001) - Uptime monitoring
- [AdGuard Home](http://concord.vish.local:3000) - DNS filtering
### Development Services
- [Code Server](http://homelab-vm.vish.local:8443) - Web IDE
- [Jupyter](http://homelab-vm.vish.local:8888) - Notebooks
- [OpenHands](http://homelab-vm.vish.local:3000) - AI coding assistant
- [Plane.so](http://guava.vish.local:3000) - Project management
## Related Documentation
- **[Service Categories](20-Service-Categories.md)** - Services organized by function
- **[Deployment Guide](30-Deployment-Guide.md)** - How to deploy services
- **[Common Issues](40-Common-Issues.md)** - Troubleshooting guide
- **[Individual Service Docs](../services/individual/)** - Detailed service documentation
---
*This service index provides quick access to all services in the homelab environment. Individual service documentation includes configuration details, deployment instructions, and troubleshooting information.*

743
docs/getting-started/30-Deployment-Guide.md Normal file
View File

@@ -0,0 +1,743 @@
# Deployment Guide
## Overview
This guide provides comprehensive instructions for deploying services in the homelab environment using GitOps principles with Docker Compose and Portainer. All deployments follow infrastructure-as-code practices with version control and automated workflows.
## Deployment Architecture
### GitOps Workflow
```
Developer ──▶ Git Repository ──▶ Portainer ──▶ Docker Compose ──▶ Running Services
│ │ │ │
│ │ │ └─▶ Health Checks
│ │ └─▶ Stack Management
│ └─▶ Configuration Validation
└─▶ Documentation Updates
```
### Repository Structure
```
homelab/
├── hosts/
│ ├── atlantis/ # Atlantis server configs
│ ├── calypso/ # Calypso server configs
│ ├── concord_nuc/ # Concord NUC configs
│ ├── homelab_vm/ # Homelab VM configs
│ └── raspberry-pi-5-vish/ # Raspberry Pi configs
├── common/ # Shared configurations
├── docs/ # Documentation
└── scripts/ # Automation scripts
```
## Prerequisites
### Required Access
- **Git Repository**: Read access to homelab repository
- **Portainer Access**: Admin credentials for container management
- **SSH Access**: Server administration capabilities
- **Network Access**: Internal network connectivity
### Required Tools
```bash
# Install required tools
sudo apt update && sudo apt install -y \
git \
docker.io \
docker-compose \
curl \
wget \
vim
# Verify installations
git --version
docker --version
docker-compose --version
```
### Environment Setup
```bash
# Clone repository
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
# Set up environment variables
export HOMELAB_ENV="production"
export DOCKER_HOST="tcp://atlantis.vish.local:2376"
export PORTAINER_URL="http://atlantis.vish.local:9000"
```
## Deployment Methods
### Method 1: Portainer Stack Deployment (Recommended)
#### Step 1: Access Portainer
1. Navigate to [Portainer](http://atlantis.vish.local:9000)
2. Login with admin credentials
3. Select the appropriate endpoint
#### Step 2: Create New Stack
1. Go to **Stacks** → **Add Stack**
2. Choose deployment method:
- **Git Repository** (recommended)
- **Upload** (for local files)
- **Web Editor** (for quick edits)
#### Step 3: Configure Git Repository
```yaml
Repository URL: https://git.vish.gg/Vish/homelab.git
Reference: refs/heads/main
Compose Path: hosts/atlantis/service-name.yml
```
#### Step 4: Set Environment Variables
```bash
# Common variables
PUID=1000
PGID=1000
TZ=America/New_York
DOMAIN=vish.local
# Service-specific variables
SERVICE_PORT=8080
SERVICE_DATA=/mnt/storage/service-name
```
#### Step 5: Deploy Stack
1. Click **Deploy the Stack**
2. Monitor deployment logs
3. Verify service health
### Method 2: Command Line Deployment
#### Direct Docker Compose
```bash
# Navigate to service directory
cd hosts/atlantis
# Deploy service
docker-compose -f service-name.yml up -d
# Check status
docker-compose -f service-name.yml ps
# View logs
docker-compose -f service-name.yml logs -f
```
#### Using Deployment Scripts
```bash
# Run deployment script
./scripts/deploy-service.sh atlantis service-name
# Bulk deployment
./scripts/deploy-all.sh atlantis
# Update existing service
./scripts/update-service.sh atlantis service-name
```
### Method 3: Ansible Automation
#### Playbook Deployment
```bash
# Deploy single service
ansible-playbook -i inventory.ini ansible/deploy-service.yml \
-e target_host=atlantis \
-e service_name=plex
# Deploy full stack
ansible-playbook -i inventory.ini ansible/deploy-full-stack.yml \
-e target_host=atlantis
# Update all services
ansible-playbook -i inventory.ini ansible/update-all.yml
```
## Service Configuration
### Docker Compose Template
```yaml
version: '3.8'
services:
service-name:
image: organization/service:latest
container_name: service-name
restart: unless-stopped
environment:
- PUID=${PUID:-1000}
- PGID=${PGID:-1000}
- TZ=${TZ:-UTC}
- SERVICE_CONFIG=${SERVICE_CONFIG}
volumes:
- ${DATA_PATH}/config:/config
- ${DATA_PATH}/data:/data
- /etc/localtime:/etc/localtime:ro
ports:
- "${SERVICE_PORT}:8080"
networks:
- homelab
labels:
- "traefik.enable=true"
- "traefik.http.routers.service.rule=Host(`service.${DOMAIN}`)"
- "traefik.http.services.service.loadbalancer.server.port=8080"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
networks:
homelab:
external: true
volumes:
service-config:
driver: local
service-data:
driver: local
```
### Environment Variables
```bash
# Create .env file
cat > .env << EOF
# User Configuration
PUID=1000
PGID=1000
TZ=America/New_York
# Network Configuration
DOMAIN=vish.local
SUBNET=192.168.10.0/24
# Storage Configuration
DATA_ROOT=/mnt/storage
CONFIG_ROOT=/mnt/config
BACKUP_ROOT=/mnt/backup
# Service Configuration
SERVICE_PORT=8080
SERVICE_NAME=example-service
SERVICE_VERSION=latest
# Security Configuration
SSL_CERT_PATH=/etc/ssl/certs
SSL_KEY_PATH=/etc/ssl/private
ADMIN_EMAIL=admin@vish.local
EOF
```
## Server-Specific Deployments
### Atlantis (Primary Server)
```bash
# Media services
./deploy-service.sh atlantis plex
./deploy-service.sh atlantis sonarr
./deploy-service.sh atlantis radarr
# Storage services
./deploy-service.sh atlantis nextcloud
./deploy-service.sh atlantis syncthing
# Monitoring services
./deploy-service.sh atlantis grafana
./deploy-service.sh atlantis prometheus
```
### Calypso (Secondary Server)
```bash
# Development services
./deploy-service.sh calypso gitea
./deploy-service.sh calypso portainer
# Authentication services
./deploy-service.sh calypso authentik
./deploy-service.sh calypso nginx-proxy-manager
# Game servers
./deploy-service.sh calypso minecraft
./deploy-service.sh calypso satisfactory
```
### Concord NUC (Edge Server)
```bash
# Network services
./deploy-service.sh concord adguard
./deploy-service.sh concord pihole
# IoT services
./deploy-service.sh concord homeassistant
./deploy-service.sh concord node-exporter
# Media streaming
./deploy-service.sh concord invidious
./deploy-service.sh concord piped
```
### Homelab VM (Development)
```bash
# AI/ML services
./deploy-service.sh homelab-vm ollama
./deploy-service.sh homelab-vm openhands
# Communication services
./deploy-service.sh homelab-vm mattermost
./deploy-service.sh homelab-vm signal-api
# Testing services
./deploy-service.sh homelab-vm test-environment
```
### Raspberry Pi (Monitoring)
```bash
# Monitoring services
./deploy-service.sh raspberry-pi uptime-kuma
./deploy-service.sh raspberry-pi glances
# Lightweight services
./deploy-service.sh raspberry-pi immich
./deploy-service.sh raspberry-pi syncthing
```
## Network Configuration
### Docker Networks
```bash
# Create homelab network
docker network create \
--driver bridge \
--subnet=172.20.0.0/16 \
--gateway=172.20.0.1 \
homelab
# Create monitoring network
docker network create \
--driver bridge \
--subnet=172.21.0.0/16 \
--gateway=172.21.0.1 \
monitoring
# List networks
docker network ls
```
### Reverse Proxy Configuration
```yaml
# Nginx Proxy Manager
version: '3.8'
services:
nginx-proxy-manager:
image: jc21/nginx-proxy-manager:latest
container_name: nginx-proxy-manager
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "81:81"
volumes:
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
environment:
DB_MYSQL_HOST: "db"
DB_MYSQL_PORT: 3306
DB_MYSQL_USER: "npm"
DB_MYSQL_PASSWORD: "npm"
DB_MYSQL_NAME: "npm"
```
## Storage Configuration
### Volume Mapping
```yaml
# Standard volume structure
volumes:
- ${DATA_ROOT}/service-name/config:/config
- ${DATA_ROOT}/service-name/data:/data
- ${MEDIA_ROOT}:/media:ro
- ${DOWNLOAD_ROOT}:/downloads
- /etc/localtime:/etc/localtime:ro
```
### Backup Integration
```yaml
# Backup-aware service
services:
service-name:
# ... service configuration ...
volumes:
- service-data:/data
- backup-volume:/backup
labels:
- "backup.enable=true"
- "backup.schedule=0 2 * * *"
- "backup.retention=30d"
volumes:
backup-volume:
driver: local
driver_opts:
type: nfs
o: addr=backup-server.local,rw
device: ":/mnt/backup/service-name"
```
## Security Configuration
### Container Security
```yaml
services:
secure-service:
# ... other configuration ...
# Security options
security_opt:
- no-new-privileges:true
# Read-only root filesystem
read_only: true
# Temporary filesystem for writable areas
tmpfs:
- /tmp
- /var/tmp
# User namespace
user: "${PUID}:${PGID}"
# Capabilities
cap_drop:
- ALL
cap_add:
- CHOWN
- SETUID
- SETGID
```
### Network Security
```yaml
# Isolated network configuration
networks:
frontend:
driver: bridge
internal: false
backend:
driver: bridge
internal: true
services:
web-service:
networks:
- frontend
- backend
database:
networks:
- backend
```
## Monitoring Integration
### Health Checks
```yaml
services:
monitored-service:
# ... service configuration ...
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
labels:
- "monitoring.enable=true"
- "monitoring.port=8080"
- "monitoring.path=/metrics"
```
### Logging Configuration
```yaml
services:
logged-service:
# ... service configuration ...
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
labels: "service,environment"
labels:
- "logging.enable=true"
- "logging.service=service-name"
```
## Deployment Validation
### Pre-deployment Checks
```bash
#!/bin/bash
# validate-deployment.sh
echo "Validating deployment configuration..."
# Check Docker Compose syntax
docker-compose -f $1 config > /dev/null
if [ $? -eq 0 ]; then
echo "✅ Docker Compose syntax valid"
else
echo "❌ Docker Compose syntax error"
exit 1
fi
# Check required environment variables
required_vars=("PUID" "PGID" "TZ" "DOMAIN")
for var in "${required_vars[@]}"; do
if [ -z "${!var}" ]; then
echo "❌ Missing required variable: $var"
exit 1
else
echo "✅ Variable $var is set"
fi
done
# Check storage paths
if [ ! -d "$DATA_ROOT" ]; then
echo "❌ Data root directory does not exist: $DATA_ROOT"
exit 1
else
echo "✅ Data root directory exists"
fi
echo "✅ All validation checks passed"
```
### Post-deployment Verification
```bash
#!/bin/bash
# verify-deployment.sh
SERVICE_NAME=$1
EXPECTED_PORT=$2
echo "Verifying deployment of $SERVICE_NAME..."
# Check container status
if docker ps | grep -q $SERVICE_NAME; then
echo "✅ Container is running"
else
echo "❌ Container is not running"
exit 1
fi
# Check port accessibility
if curl -f http://localhost:$EXPECTED_PORT/health > /dev/null 2>&1; then
echo "✅ Service is responding on port $EXPECTED_PORT"
else
echo "❌ Service is not responding on port $EXPECTED_PORT"
fi
# Check logs for errors
if docker logs $SERVICE_NAME 2>&1 | grep -i error; then
echo "⚠️ Errors found in logs"
else
echo "✅ No errors in logs"
fi
echo "✅ Deployment verification complete"
```
## Troubleshooting
### Common Issues
#### Container Won't Start
```bash
# Check container logs
docker logs container-name
# Check resource usage
docker stats
# Verify configuration
docker-compose config
# Check port conflicts
netstat -tulpn | grep :8080
```
#### Permission Issues
```bash
# Fix ownership
sudo chown -R $PUID:$PGID /mnt/storage/service-name
# Check permissions
ls -la /mnt/storage/service-name
# Verify user mapping
docker exec container-name id
```
#### Network Connectivity
```bash
# Test container networking
docker exec container-name ping google.com
# Check network configuration
docker network inspect homelab
# Verify DNS resolution
docker exec container-name nslookup service.local
```
#### Storage Issues
```bash
# Check disk space
df -h
# Verify mount points
mount | grep storage
# Check RAID status
cat /proc/mdstat
```
### Emergency Procedures
#### Service Recovery
```bash
# Stop problematic service
docker-compose -f service.yml down
# Remove containers and volumes
docker-compose -f service.yml down -v
# Restore from backup
./scripts/restore-service.sh service-name
# Redeploy service
docker-compose -f service.yml up -d
```
#### System Recovery
```bash
# Stop all services
docker stop $(docker ps -q)
# Clean up system
docker system prune -a
# Restart Docker daemon
sudo systemctl restart docker
# Redeploy critical services
./scripts/deploy-critical.sh
```
## Automation Scripts
### Deployment Automation
```bash
#!/bin/bash
# deploy-service.sh
HOST=$1
SERVICE=$2
COMPOSE_FILE="hosts/$HOST/$SERVICE.yml"
if [ ! -f "$COMPOSE_FILE" ]; then
echo "Error: Compose file not found: $COMPOSE_FILE"
exit 1
fi
echo "Deploying $SERVICE on $HOST..."
# Validate configuration
docker-compose -f $COMPOSE_FILE config > /dev/null
if [ $? -ne 0 ]; then
echo "Error: Invalid compose configuration"
exit 1
fi
# Deploy service
docker-compose -f $COMPOSE_FILE up -d
# Wait for service to be ready
sleep 30
# Verify deployment
./scripts/verify-deployment.sh $SERVICE
echo "Deployment complete: $SERVICE"
```
### Update Automation
```bash
#!/bin/bash
# update-service.sh
SERVICE=$1
echo "Updating $SERVICE..."
# Pull latest images
docker-compose -f hosts/*/$(SERVICE).yml pull
# Recreate containers
docker-compose -f hosts/*/$(SERVICE).yml up -d
# Clean up old images
docker image prune -f
echo "Update complete: $SERVICE"
```
## Best Practices
### Configuration Management
- Use environment variables for configuration
- Store secrets in Docker secrets or external vaults
- Version control all configuration files
- Document all custom configurations
### Resource Management
- Set appropriate resource limits
- Monitor resource usage
- Plan for capacity growth
- Implement resource quotas
### Security Practices
- Use non-root users in containers
- Implement network segmentation
- Regular security updates
- Monitor for vulnerabilities
### Backup Strategies
- Automate backup processes
- Test restore procedures
- Implement versioned backups
- Store backups offsite
## Related Documentation
- **[Service Categories](20-Service-Categories.md)** - Available services overview
- **[Common Issues](40-Common-Issues.md)** - Troubleshooting guide
- **[Ansible Automation](50-Ansible-Automation.md)** - Automated deployments
- **[GitOps Guide](../admin/gitops-deployment-guide.md)** - GitOps workflows
---
*This deployment guide provides comprehensive instructions for deploying and managing services in the homelab environment using modern DevOps practices and tools.*

806
docs/getting-started/40-Common-Issues.md Normal file
View File

@@ -0,0 +1,806 @@
# Common Issues & Troubleshooting
## Overview
This guide covers the most frequently encountered issues in the homelab environment and provides step-by-step solutions. Issues are organized by category with diagnostic steps and resolution procedures.
## Container & Docker Issues
### Container Won't Start
#### Symptoms
- Container exits immediately after starting
- "Container exited with code 1" errors
- Service unavailable after deployment
#### Diagnostic Steps
```bash
# Check container status
docker ps -a
# View container logs
docker logs container-name
# Inspect container configuration
docker inspect container-name
# Check resource usage
docker stats
```
#### Common Causes & Solutions
**Port Conflicts**
```bash
# Check port usage
netstat -tulpn | grep :8080
ss -tulpn | grep :8080
# Solution: Change port in docker-compose.yml
ports:
- "8081:8080" # Use different external port
```
**Permission Issues**
```bash
# Check file ownership
ls -la /mnt/storage/service-name
# Fix ownership
sudo chown -R 1000:1000 /mnt/storage/service-name
# Set proper permissions
sudo chmod -R 755 /mnt/storage/service-name
```
**Missing Environment Variables**
```bash
# Check environment variables
docker exec container-name env
# Add missing variables to .env file
echo "MISSING_VAR=value" >> .env
# Recreate container
docker-compose up -d --force-recreate
```
### Container Memory Issues
#### Symptoms
- Container killed by OOM (Out of Memory)
- Slow performance or timeouts
- System becomes unresponsive
#### Diagnostic Steps
```bash
# Check memory usage
free -h
docker stats
# Check system logs for OOM kills
dmesg | grep -i "killed process"
journalctl -u docker.service | grep -i oom
```
#### Solutions
```bash
# Add memory limits to docker-compose.yml
services:
service-name:
deploy:
resources:
limits:
memory: 2G
reservations:
memory: 1G
# Increase system swap
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
```
### Docker Daemon Issues
#### Symptoms
- "Cannot connect to Docker daemon" errors
- Docker commands hang or timeout
- Services become unresponsive
#### Diagnostic Steps
```bash
# Check Docker daemon status
systemctl status docker
# Check Docker daemon logs
journalctl -u docker.service -f
# Test Docker connectivity
docker version
docker info
```
#### Solutions
```bash
# Restart Docker daemon
sudo systemctl restart docker
# Clean up Docker system
docker system prune -a
# Reset Docker daemon (last resort)
sudo systemctl stop docker
sudo rm -rf /var/lib/docker
sudo systemctl start docker
```
## Network & Connectivity Issues
### Service Not Accessible
#### Symptoms
- Connection refused errors
- Timeouts when accessing services
- Services work internally but not externally
#### Diagnostic Steps
```bash
# Test local connectivity
curl -I http://localhost:8080
# Test network connectivity
curl -I http://server-ip:8080
# Check firewall rules
sudo ufw status
iptables -L
# Check port binding
netstat -tulpn | grep :8080
```
#### Solutions
```bash
# Open firewall ports
sudo ufw allow 8080/tcp
# Check Docker port binding
# Ensure ports are properly exposed in docker-compose.yml
ports:
- "0.0.0.0:8080:8080" # Bind to all interfaces
# Restart networking
sudo systemctl restart networking
```
### DNS Resolution Issues
#### Symptoms
- Cannot resolve service hostnames
- "Name or service not known" errors
- Services can't communicate with each other
#### Diagnostic Steps
```bash
# Test DNS resolution
nslookup service.local
dig service.local
# Check DNS configuration
cat /etc/resolv.conf
# Test container DNS
docker exec container-name nslookup google.com
```
#### Solutions
```bash
# Update DNS servers
echo "nameserver 8.8.8.8" | sudo tee -a /etc/resolv.conf
# Restart systemd-resolved
sudo systemctl restart systemd-resolved
# Configure Docker DNS
# Add to /etc/docker/daemon.json
{
"dns": ["8.8.8.8", "8.8.4.4"]
}
sudo systemctl restart docker
```
### Reverse Proxy Issues
#### Symptoms
- 502 Bad Gateway errors
- SSL certificate errors
- Services accessible directly but not through proxy
#### Diagnostic Steps
```bash
# Check proxy container logs
docker logs nginx-proxy-manager
# Test backend connectivity
curl -I http://backend-service:8080
# Check proxy configuration
docker exec nginx-proxy-manager cat /etc/nginx/nginx.conf
```
#### Solutions
```bash
# Verify backend service is running
docker ps | grep backend-service
# Check network connectivity between proxy and backend
docker exec nginx-proxy-manager ping backend-service
# Regenerate SSL certificates
# Through Nginx Proxy Manager UI or:
certbot renew --force-renewal
```
## Storage & File System Issues
### Disk Space Issues
#### Symptoms
- "No space left on device" errors
- Services failing to write data
- System performance degradation
#### Diagnostic Steps
```bash
# Check disk usage
df -h
du -sh /*
# Check Docker space usage
docker system df
# Find large files
find / -type f -size +1G 2>/dev/null
```
#### Solutions
```bash
# Clean Docker system
docker system prune -a
docker volume prune
# Clean log files
sudo journalctl --vacuum-time=7d
sudo find /var/log -name "*.log" -type f -mtime +30 -delete
# Move data to larger partition
sudo mv /var/lib/docker /mnt/storage/docker
sudo ln -s /mnt/storage/docker /var/lib/docker
```
### Permission Issues
#### Symptoms
- "Permission denied" errors
- Services can't read/write files
- Configuration files not loading
#### Diagnostic Steps
```bash
# Check file permissions
ls -la /mnt/storage/service-name
# Check user/group IDs
id username
docker exec container-name id
# Check mount points
mount | grep storage
```
#### Solutions
```bash
# Fix ownership recursively
sudo chown -R 1000:1000 /mnt/storage/service-name
# Set proper permissions
sudo chmod -R 755 /mnt/storage/service-name
# Add user to docker group
sudo usermod -aG docker $USER
# Set PUID/PGID in docker-compose.yml
environment:
- PUID=1000
- PGID=1000
```
### RAID Array Issues
#### Symptoms
- Degraded RAID arrays
- Disk failure notifications
- Slow storage performance
#### Diagnostic Steps
```bash
# Check RAID status
cat /proc/mdstat
sudo mdadm --detail /dev/md0
# Check disk health
sudo smartctl -a /dev/sda
# Check system logs
dmesg | grep -i raid
journalctl | grep -i mdadm
```
#### Solutions
```bash
# Replace failed disk
sudo mdadm --manage /dev/md0 --remove /dev/sdb
# Physically replace disk
sudo mdadm --manage /dev/md0 --add /dev/sdb
# Force array rebuild
sudo mdadm --manage /dev/md0 --re-add /dev/sdb
# Monitor rebuild progress
watch cat /proc/mdstat
```
## Service-Specific Issues
### Database Connection Issues
#### Symptoms
- "Connection refused" to database
- "Too many connections" errors
- Database corruption warnings
#### Diagnostic Steps
```bash
# Check database container status
docker logs postgres-container
# Test database connectivity
docker exec postgres-container psql -U user -d database -c "SELECT 1;"
# Check connection limits
docker exec postgres-container psql -U user -c "SHOW max_connections;"
```
#### Solutions
```bash
# Restart database container
docker-compose restart postgres
# Increase connection limits
# In postgresql.conf:
max_connections = 200
# Clean up idle connections
docker exec postgres-container psql -U user -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE state = 'idle';"
```
### Web Service Issues
#### Symptoms
- 500 Internal Server Error
- Slow response times
- Service timeouts
#### Diagnostic Steps
```bash
# Check service logs
docker logs web-service
# Test service health
curl -I http://localhost:8080/health
# Check resource usage
docker stats web-service
```
#### Solutions
```bash
# Restart service
docker-compose restart web-service
# Increase resource limits
deploy:
resources:
limits:
memory: 2G
cpus: '1.0'
# Check application configuration
docker exec web-service cat /config/app.conf
```
### Authentication Issues
#### Symptoms
- Login failures
- "Unauthorized" errors
- SSO integration problems
#### Diagnostic Steps
```bash
# Check authentication service logs
docker logs authentik-server
# Test authentication endpoint
curl -X POST http://auth.local/api/v3/auth/login
# Check user database
docker exec authentik-server ak list_users
```
#### Solutions
```bash
# Reset user password
docker exec authentik-server ak reset_password username
# Restart authentication service
docker-compose restart authentik
# Check LDAP connectivity (if applicable)
docker exec authentik-server ldapsearch -x -H ldap://server
```
## Monitoring & Alerting Issues
### Metrics Collection Issues
#### Symptoms
- Missing metrics in Grafana
- Prometheus targets down
- Exporters not responding
#### Diagnostic Steps
```bash
# Check Prometheus targets
curl http://prometheus:9090/api/v1/targets
# Test exporter endpoints
curl http://node-exporter:9100/metrics
# Check Prometheus configuration
docker exec prometheus cat /etc/prometheus/prometheus.yml
```
#### Solutions
```bash
# Restart monitoring stack
docker-compose -f monitoring.yml restart
# Reload Prometheus configuration
curl -X POST http://prometheus:9090/-/reload
# Check network connectivity
docker exec prometheus ping node-exporter
```
### Alert Manager Issues
#### Symptoms
- Alerts not firing
- Notifications not received
- Alert routing problems
#### Diagnostic Steps
```bash
# Check AlertManager status
curl http://alertmanager:9093/api/v1/status
# View active alerts
curl http://alertmanager:9093/api/v1/alerts
# Check routing configuration
docker exec alertmanager cat /etc/alertmanager/alertmanager.yml
```
#### Solutions
```bash
# Test notification channels
curl -X POST http://alertmanager:9093/api/v1/alerts \
-H "Content-Type: application/json" \
-d '[{"labels":{"alertname":"test"}}]'
# Restart AlertManager
docker-compose restart alertmanager
# Validate configuration
docker exec alertmanager amtool config check
```
## Performance Issues
### High CPU Usage
#### Symptoms
- System sluggishness
- High load averages
- Services timing out
#### Diagnostic Steps
```bash
# Check system load
uptime
htop
# Check container CPU usage
docker stats
# Identify CPU-intensive processes
top -o %CPU
```
#### Solutions
```bash
# Limit container CPU usage
deploy:
resources:
limits:
cpus: '0.5'
# Optimize service configuration
# Reduce worker processes, adjust cache settings
# Scale services horizontally
docker-compose up -d --scale web-service=3
```
### High Memory Usage
#### Symptoms
- System swapping
- OOM kills
- Slow performance
#### Diagnostic Steps
```bash
# Check memory usage
free -h
cat /proc/meminfo
# Check container memory usage
docker stats
# Check for memory leaks
ps aux --sort=-%mem | head
```
#### Solutions
```bash
# Add memory limits
deploy:
resources:
limits:
memory: 1G
# Increase system memory or swap
sudo fallocate -l 2G /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# Optimize application memory usage
# Adjust JVM heap size, database buffers, etc.
```
### Network Performance Issues
#### Symptoms
- Slow file transfers
- High network latency
- Connection timeouts
#### Diagnostic Steps
```bash
# Test network speed
iperf3 -c server-ip
# Check network interface statistics
ip -s link show
# Monitor network traffic
iftop
nethogs
```
#### Solutions
```bash
# Optimize network settings
echo 'net.core.rmem_max = 16777216' >> /etc/sysctl.conf
echo 'net.core.wmem_max = 16777216' >> /etc/sysctl.conf
sysctl -p
# Check for network congestion
# Upgrade network infrastructure if needed
# Optimize Docker networking
# Use host networking for performance-critical services
network_mode: host
```
## Security Issues
### SSL Certificate Issues
#### Symptoms
- Certificate expired warnings
- SSL handshake failures
- Browser security warnings
#### Diagnostic Steps
```bash
# Check certificate expiration
openssl x509 -in cert.pem -text -noout | grep "Not After"
# Test SSL connectivity
openssl s_client -connect domain.com:443
# Check certificate chain
curl -I https://domain.com
```
#### Solutions
```bash
# Renew Let's Encrypt certificates
certbot renew
# Generate new self-signed certificate
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
# Update certificate in services
# Copy new certificates to appropriate volumes
```
### Authentication Failures
#### Symptoms
- Repeated login failures
- Account lockouts
- Suspicious access attempts
#### Diagnostic Steps
```bash
# Check authentication logs
journalctl -u ssh.service | grep "Failed password"
docker logs authentik-server | grep "login failed"
# Check fail2ban status
sudo fail2ban-client status
sudo fail2ban-client status sshd
```
#### Solutions
```bash
# Unban IP addresses
sudo fail2ban-client set sshd unbanip IP_ADDRESS
# Strengthen authentication
# Enable 2FA, use SSH keys, implement rate limiting
# Monitor for brute force attacks
# Set up alerting for repeated failures
```
## Emergency Procedures
### Complete System Recovery
#### When to Use
- Multiple service failures
- System corruption
- Hardware failures
#### Recovery Steps
```bash
# 1. Stop all services
docker stop $(docker ps -q)
# 2. Check system integrity
fsck /dev/sda1
# 3. Restore from backup
./scripts/restore-system.sh
# 4. Restart critical services
./scripts/deploy-critical.sh
# 5. Verify system health
./scripts/health-check.sh
```
### Data Recovery
#### When to Use
- Data corruption
- Accidental deletion
- Storage failures
#### Recovery Steps
```bash
# 1. Stop affected services
docker-compose down
# 2. Mount backup storage
mount /dev/backup /mnt/restore
# 3. Restore data
rsync -av /mnt/restore/service-data/ /mnt/storage/service-data/
# 4. Fix permissions
chown -R 1000:1000 /mnt/storage/service-data
# 5. Restart services
docker-compose up -d
```
### Network Recovery
#### When to Use
- Network connectivity loss
- DNS failures
- Routing issues
#### Recovery Steps
```bash
# 1. Check physical connectivity
ip link show
# 2. Restart networking
systemctl restart networking
# 3. Reset network configuration
netplan apply
# 4. Flush DNS cache
systemctl restart systemd-resolved
# 5. Test connectivity
ping 8.8.8.8
```
## Prevention Strategies
### Monitoring & Alerting
- Set up comprehensive monitoring
- Configure proactive alerts
- Regular health checks
- Performance baselines
### Backup & Recovery
- Automated backup schedules
- Regular restore testing
- Offsite backup storage
- Documentation of procedures
### Maintenance
- Regular system updates
- Capacity planning
- Performance optimization
- Security hardening
### Documentation
- Incident response procedures
- Configuration documentation
- Change management processes
- Knowledge sharing
## Related Documentation
- **[Monitoring Setup](../admin/monitoring-setup.md)** - Monitoring configuration
- **[Security Guidelines](../security/README.md)** - Security best practices
- **[Backup Procedures](../admin/backup-procedures.md)** - Backup and recovery
- **[Emergency Contacts](../admin/README.md)** - Emergency procedures
---
*This troubleshooting guide provides comprehensive solutions for common issues encountered in the homelab environment. Keep this guide updated with new issues and solutions as they are discovered.*

266
docs/getting-started/BEGINNER_QUICKSTART.md Normal file
View File

@@ -0,0 +1,266 @@
# Beginner's Quick Start Guide
**New to homelabs?** This guide walks you through deploying your first service step-by-step.
## 🎯 What You'll Learn
By the end of this guide, you'll know how to:
- Access your homelab tools (Gitea, Portainer)
- Deploy a simple service
- Understand the basic workflow
- Troubleshoot common issues
## 📋 Before You Start
### What You Need
- [ ] **Computer with internet access**
- [ ] **Web browser** (Chrome, Firefox, Safari, etc.)
- [ ] **Text editor** (Notepad++, VS Code, or even basic Notepad)
- [ ] **Basic understanding** of copy/paste and file editing
### What You DON'T Need
- Advanced programming knowledge
- Command line experience (we'll show you the easy way)
- Docker expertise
## 🚀 Step-by-Step: Deploy Your First Service
### Step 1: Access Your Tools
#### Gitea (Your Code Repository)
1. Open your web browser
2. Go to `https://git.vish.gg` (or your Gitea URL)
3. Log in with your credentials
4. Navigate to the `homelab` repository
#### Portainer (Your Container Manager)
1. Open a new browser tab
2. Go to your Portainer URL (usually `https://portainer.yourdomain.com`)
3. Log in with your credentials
4. You should see the Portainer dashboard
### Step 2: Choose What to Deploy
**For your first deployment, let's use a simple service like:**
- **Uptime Kuma** - Website monitoring
- **IT Tools** - Handy web utilities
- **Stirling PDF** - PDF manipulation tools
**We'll use IT Tools as our example.**
### Step 3: Create Your Service File
#### Option A: Using Gitea Web Interface (Easiest)
1. **In Gitea**, navigate to your homelab repository
2. **Choose your server location**:
- Click on `hosts/` folder
- Click on your server type (e.g., `synology/`)
- Click on your server name (e.g., `atlantis/`)
3. **Create new file**:
- Click the "+" button or "New File"
- Name it: `it-tools.yml`
4. **Copy this configuration**:
```yaml
version: '3.8'
services:
it-tools:
image: corentinth/it-tools:latest
container_name: it-tools
restart: unless-stopped
ports:
- "8080:80" # Change 8080 if this port is already used
networks:
- homelab
networks:
homelab:
external: true
```
5. **Save the file**:
- Add a commit message: "Add IT Tools service"
- Click "Commit Changes"
#### Option B: Using Git (If You're Comfortable)
```bash
# Clone the repository
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
# Create the file
nano hosts/synology/atlantis/it-tools.yml
# (Copy the YAML content above)
# Save and commit
git add hosts/synology/atlantis/it-tools.yml
git commit -m "Add IT Tools service"
git push
```
### Step 4: Deploy via Portainer
1. **In Portainer**, go to "Stacks" (left sidebar)
2. **Click "Add stack"**
3. **Fill in the details**:
- **Name**: `it-tools`
- **Build method**: Select "Repository"
- **Repository URL**: `https://git.vish.gg/Vish/homelab`
- **Repository reference**: `refs/heads/main`
- **Compose path**: `hosts/synology/atlantis/it-tools.yml`
- **Automatic updates**: Check this box (optional)
4. **Click "Deploy the stack"**
5. **Wait for deployment** - You'll see logs showing the progress
### Step 5: Access Your Service
1. **Find your server's IP address** (e.g., 192.168.1.100)
2. **Open your browser** and go to: `http://192.168.1.100:8080`
3. **You should see IT Tools running!**
## 🎉 Congratulations!
You just deployed your first homelab service! Here's what happened:
1. **You created** a Docker Compose file describing your service
2. **Gitea stored** your configuration safely
3. **Portainer read** the configuration from Gitea
4. **Docker deployed** your service automatically
## 🔧 Understanding Your Setup
### The Files You Work With
```
homelab/
├── hosts/ # Server configurations
│ ├── synology/atlantis/ # Your main NAS
│ ├── synology/calypso/ # Your backup NAS
│ ├── vms/homelab-vm/ # Your virtual machines
│ └── physical/concord-nuc/ # Your physical servers
├── docs/ # Documentation (like this guide)
└── scripts/ # Helpful automation scripts
```
### The Tools Working Together
1. **Gitea** = Your filing cabinet (stores all configurations)
2. **Portainer** = Your deployment assistant (reads from Gitea and deploys)
3. **Docker** = Your service runner (actually runs the applications)
### The Workflow
```
You edit file → Gitea stores it → Portainer deploys it → Service runs
```
## 🛠️ Common Tasks
### Deploy Another Service
1. **Find an example** in the existing files
2. **Copy and modify** it for your needs
3. **Change the ports** to avoid conflicts
4. **Deploy via Portainer** using the same steps
### Update a Service
1. **Edit the YAML file** in Gitea
2. **Commit your changes**
3. **In Portainer**, go to your stack and click "Update"
4. **Portainer will redeploy** with your changes
### Remove a Service
1. **In Portainer**, go to "Stacks"
2. **Find your service** and click the trash icon
3. **Confirm deletion**
4. **Optionally delete** the YAML file from Gitea
## 🚨 Troubleshooting
### "Port already in use" Error
**Problem**: Another service is using the same port.
**Solution**: Change the port in your YAML file:
```yaml
ports:
- "8081:80" # Changed from 8080 to 8081
```
### "Cannot access service" Error
**Checklist**:
- [ ] Is the service running? (Check Portainer → Stacks)
- [ ] Are you using the right IP address?
- [ ] Are you using the right port number?
- [ ] Is your firewall blocking the port?
### "Deployment failed" Error
**Common causes**:
- **YAML syntax error** - Check indentation (use spaces, not tabs)
- **Invalid image name** - Verify the Docker image exists
- **Volume path doesn't exist** - Create the directory first
### Getting Help
1. **Check the logs** in Portainer (Stacks → Your Stack → Logs)
2. **Look at similar services** in the repository for examples
3. **Check the service documentation** on Docker Hub
## 📚 Next Steps
### Learn More About Docker Compose
**Key concepts to understand**:
- **Services** - The applications you run
- **Ports** - How to access services from outside
- **Volumes** - Where data is stored
- **Networks** - How services talk to each other
### Explore Advanced Features
- **Environment variables** for configuration
- **Multiple services** in one file
- **Service dependencies** and startup order
- **Resource limits** and health checks
### Popular Services to Try
**Media Management**:
- Plex/Jellyfin (media server)
- Sonarr/Radarr (media automation)
- Overseerr (media requests)
**Productivity**:
- Nextcloud (file sync)
- Bitwarden (password manager)
- Paperless-ngx (document management)
**Monitoring**:
- Uptime Kuma (uptime monitoring)
- Grafana (dashboards)
- Portainer Agent (container monitoring)
## 🔗 Useful Resources
### Documentation
- [Docker Compose Reference](https://docs.docker.com/compose/)
- [Portainer Documentation](https://docs.portainer.io/)
- [Your homelab's DEVELOPMENT.md](DEVELOPMENT.md)
### Finding Services
- [Docker Hub](https://hub.docker.com/) - Official Docker images
- [LinuxServer.io](https://docs.linuxserver.io/) - Well-maintained homelab images
- [Awesome-Selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted) - Huge list of self-hosted services
---
**Remember**: Start simple, learn as you go, and don't be afraid to experiment! Your homelab is a safe place to try new things.
*Happy homelabbing! 🏠🔬*

301
docs/getting-started/DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,301 @@
# 🛠️ Development Guide
*Development environment setup and contribution guidelines for the homelab project*
## 🎯 Overview
This guide covers setting up a development environment for contributing to the homelab infrastructure, including local testing, GitOps workflows, and best practices.
## 🚀 Quick Start
### Prerequisites
- Docker and Docker Compose
- Git with SSH key configured
- Text editor (VS Code recommended)
- Basic understanding of REDACTED_APP_PASSWORD
### Environment Setup
```bash
# Clone the repository
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
# Set up development environment
./scripts/setup-dev-environment.sh
# Validate compose files
./scripts/validate-compose.sh
```
## 🏗️ Development Workflow
### 1. Local Development
```bash
# Create feature branch
git checkout -b feature/new-service
# Make changes to compose files
# Test locally if possible
docker-compose -f hosts/vms/seattle/new-service.yml up -d
# Validate configuration
docker-compose -f hosts/vms/seattle/new-service.yml config
```
### 2. Testing Changes
- **Syntax Validation**: Use `validate-compose.sh` script
- **Local Testing**: Test compose files locally when possible
- **Documentation**: Update relevant documentation
- **Security Review**: Check for security implications
### 3. GitOps Deployment
```bash
# Commit changes
git add .
git commit -m "feat: add new service deployment"
# Push to repository
git push origin feature/new-service
# Create pull request for review
```
## 📁 Repository Structure
### Directory Organization
```
homelab/
├── hosts/ # Host-specific configurations
│ ├── synology/ # Synology NAS deployments
│ │ ├── atlantis/ # Primary NAS (DS1823xs+)
│ │ └── calypso/ # Secondary NAS (DS723+)
│ ├── vms/ # Virtual machine deployments
│ │ ├── seattle/ # Main VM services
│ │ └── homelab_vm/ # Secondary VM services
│ ├── physical/ # Physical server deployments
│ │ └── concord_nuc/ # Intel NUC services
│ └── edge/ # Edge device deployments
│ └── raspberry-pi-5-vish/
├── docs/ # Documentation
├── scripts/ # Automation scripts
├── grafana/ # Grafana configurations
├── prometheus/ # Prometheus configurations
└── deployments/ # Special deployments
```
### File Naming Conventions
- **Compose Files**: `service-name.yml` or `service-name.yaml`
- **Configuration**: `service-name.conf` or `config/`
- **Documentation**: `README.md` or `SERVICE_NAME.md`
- **Scripts**: `action-description.sh`
## 🐳 Docker Compose Guidelines
### Best Practices
```yaml
version: '3.8'
services:
service-name:
image: organization/image:tag # Always specify tags
container_name: service-name # Consistent naming
restart: unless-stopped # Restart policy
environment:
- PUID=1000 # User/group IDs
- PGID=1000
- TZ=America/Los_Angeles # Timezone
volumes:
- ./config:/config # Relative paths
- /data/service:/data # Absolute for data
ports:
- "8080:8080" # Explicit port mapping
networks:
- homelab # Custom networks
labels:
- "traefik.enable=true" # Reverse proxy labels
- "com.centurylinklabs.watchtower.enable=true"
networks:
homelab:
external: true
```
### Security Considerations
- **User IDs**: Always set PUID/PGID
- **Secrets**: Use Docker secrets or external files
- **Networks**: Use custom networks, avoid host networking
- **Volumes**: Minimize host volume mounts
- **Images**: Use official images when possible
## 🔧 Development Tools
### Recommended Extensions (VS Code)
- **Docker**: Container management
- **YAML**: Syntax highlighting and validation
- **GitLens**: Git integration and history
- **Markdown**: Documentation editing
- **Remote SSH**: Remote development
### Useful Scripts
```bash
# Validate all compose files
./scripts/validate-compose.sh
# Check service status
./scripts/verify-infrastructure-status.sh
# Test NTFY notifications
./scripts/test-ntfy-notifications.sh
# Generate service documentation
./scripts/generate_service_docs.py
```
## 📝 Documentation Standards
### Markdown Guidelines
- Use clear headings and structure
- Include code examples with syntax highlighting
- Add links to related documentation
- Keep content up-to-date with changes
### Service Documentation
Each service should include:
- **Purpose**: What the service does
- **Configuration**: Key configuration options
- **Access**: How to access the service
- **Troubleshooting**: Common issues and solutions
- **Dependencies**: Required services or configurations
## 🔄 GitOps Integration
### Portainer Configuration
- **Repository**: https://git.vish.gg/Vish/homelab.git
- **Branch**: main (production deployments)
- **Webhook**: Automatic deployment on push
- **Compose Path**: Relative paths from repository root
### Deployment Process
1. **Push to Repository**: Changes committed to main branch
2. **Webhook Trigger**: Portainer receives webhook notification
3. **Stack Update**: Affected stacks automatically redeploy
4. **Health Check**: Monitor deployment status
5. **Rollback**: Available through Git history
## 🧪 Testing Procedures
### Pre-Deployment Testing
```bash
# Syntax validation
docker-compose -f service.yml config
# Security scan
docker-compose -f service.yml config | docker run --rm -i hadolint/hadolint
# Local testing (if applicable)
docker-compose -f service.yml up -d
docker-compose -f service.yml logs
docker-compose -f service.yml down
```
### Post-Deployment Validation
- **Service Health**: Check container status in Portainer
- **Connectivity**: Verify service accessibility
- **Logs**: Review container logs for errors
- **Monitoring**: Check Grafana dashboards for metrics
## 🔐 Security Development
### Security Checklist
- [ ] No hardcoded secrets in compose files
- [ ] Proper user/group ID configuration
- [ ] Network isolation where appropriate
- [ ] Regular image updates via Watchtower
- [ ] SSL/TLS termination at reverse proxy
- [ ] Access control via Authentik SSO
### Vulnerability Management
- **Image Scanning**: Regular vulnerability scans
- **Update Policy**: Automated updates via Watchtower
- **Security Patches**: Prompt application of security updates
- **Access Review**: Regular review of service access
## 🚨 Troubleshooting
### Common Issues
1. **Port Conflicts**: Check for conflicting port assignments
2. **Volume Permissions**: Ensure proper file permissions
3. **Network Issues**: Verify network configuration
4. **Resource Limits**: Check CPU/memory constraints
5. **Image Availability**: Verify image exists and is accessible
### Debugging Tools
```bash
# Container inspection
docker inspect container-name
# Network debugging
docker network ls
docker network inspect network-name
# Volume inspection
docker volume ls
docker volume inspect volume-name
# Log analysis
docker logs container-name --tail 100 -f
```
## 📊 Monitoring Integration
### Metrics Collection
- **Node Exporter**: System metrics on all hosts
- **cAdvisor**: Container metrics
- **Custom Metrics**: Application-specific metrics
- **Health Checks**: Service availability monitoring
### Dashboard Development
- **Grafana**: Create dashboards for new services
- **Prometheus**: Define custom metrics and alerts
- **Documentation**: Document dashboard usage
## 🤝 Contributing
### Pull Request Process
1. **Fork Repository**: Create personal fork
2. **Feature Branch**: Create descriptive branch name
3. **Make Changes**: Follow development guidelines
4. **Test Thoroughly**: Validate all changes
5. **Update Documentation**: Keep docs current
6. **Submit PR**: Include detailed description
### Code Review
- **Security Review**: Check for security implications
- **Best Practices**: Ensure adherence to guidelines
- **Documentation**: Verify documentation updates
- **Testing**: Confirm adequate testing
## 📚 Additional Resources
### External Documentation
- [Docker Compose Reference](https://docs.docker.com/compose/)
- [Portainer Documentation](https://docs.portainer.io/)
- [Prometheus Configuration](https://prometheus.io/docs/prometheus/latest/configuration/)
- [Grafana Documentation](https://grafana.com/docs/)
### Internal Resources
- [GitOps Deployment Guide](../admin/gitops-deployment-guide.md)
- [Monitoring Setup](../admin/monitoring-setup.md)
- [Operational Status](../admin/operational-status.md)
- [Infrastructure Documentation](../infrastructure/INFRASTRUCTURE_OVERVIEW.md)
---
**Last Updated**: February 24, 2026
**Development Environment**: Docker-based with GitOps integration
**Status**: ✅ **ACTIVE** - Ready for contributions

504
docs/getting-started/QUICK_START.md Normal file
View File

@@ -0,0 +1,504 @@
# Quick Start Guide
## Overview
This guide will help you deploy your first service in the homelab environment within 15 minutes. We'll use Uptime Kuma as an example service since it's lightweight, useful, and demonstrates the core deployment workflow.
## Prerequisites Check
Before starting, ensure you have:
- [ ] SSH access to a homelab server
- [ ] Docker and Docker Compose installed
- [ ] Git repository access
- [ ] Basic understanding of Docker concepts
```bash
# Quick verification
ssh homelab@server-ip
docker --version
docker-compose --version
git --version
```
## Step 1: Choose Your Deployment Method
### Option A: Portainer (Recommended for Beginners)
- Web-based interface
- Visual stack management
- Built-in monitoring
- Easy rollbacks
### Option B: Command Line (Recommended for Advanced Users)
- Direct Docker Compose
- Faster deployment
- Scriptable automation
- Full control
## Step 2: Deploy Uptime Kuma (Portainer Method)
### Access Portainer
1. Navigate to [Portainer](http://atlantis.vish.local:9000)
2. Login with your credentials
3. Select the **local** endpoint
### Create New Stack
1. Go to **Stacks** → **Add Stack**
2. Name: `uptime-kuma-quickstart`
3. Choose **Web Editor**
### Paste Configuration
```yaml
version: '3.8'
services:
uptime-kuma:
image: louislam/uptime-kuma:1
container_name: uptime-kuma-quickstart
restart: unless-stopped
ports:
- "3001:3001"
volumes:
- uptime-kuma-data:/app/data
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PUID=1000
- PGID=1000
labels:
- "traefik.enable=true"
- "traefik.http.routers.uptime-kuma.rule=Host(`uptime.vish.local`)"
- "traefik.http.services.uptime-kuma.loadbalancer.server.port=3001"
volumes:
uptime-kuma-data:
driver: local
```
### Deploy Stack
1. Click **Deploy the Stack**
2. Wait for deployment to complete
3. Check **Containers** tab for running status
### Access Service
- Direct: http://server-ip:3001
- Domain: http://uptime.vish.local (if DNS configured)
## Step 3: Deploy Uptime Kuma (Command Line Method)
### Clone Repository
```bash
# Clone homelab repository
git clone https://git.vish.gg/Vish/homelab.git
cd homelab
# Navigate to appropriate server directory
cd hosts/raspberry-pi-5-vish # or your target server
```
### Create Service File
```bash
# Create uptime-kuma.yml
cat > uptime-kuma-quickstart.yml << 'EOF'
version: '3.8'
services:
uptime-kuma:
image: louislam/uptime-kuma:1
container_name: uptime-kuma-quickstart
restart: unless-stopped
ports:
- "3001:3001"
volumes:
- uptime-kuma-data:/app/data
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PUID=1000
- PGID=1000
volumes:
uptime-kuma-data:
driver: local
EOF
```
### Deploy Service
```bash
# Deploy with Docker Compose
docker-compose -f uptime-kuma-quickstart.yml up -d
# Check status
docker-compose -f uptime-kuma-quickstart.yml ps
# View logs
docker-compose -f uptime-kuma-quickstart.yml logs -f
```
## Step 4: Initial Configuration
### First-Time Setup
1. Access Uptime Kuma at http://server-ip:3001
2. Create admin account:
- Username: `admin`
- Password: "REDACTED_PASSWORD"
- Email: `admin@vish.local`
### Add Your First Monitor
1. Click **Add New Monitor**
2. Configure basic HTTP monitor:
- **Monitor Type**: HTTP(s)
- **Friendly Name**: `Homelab Wiki`
- **URL**: `https://git.vish.gg/Vish/homelab/wiki`
- **Heartbeat Interval**: `60 seconds`
- **Max Retries**: `3`
3. Click **Save**
### Configure Notifications (Optional)
1. Go to **Settings** → **Notifications**
2. Add notification method:
- **NTFY**: `http://homelab-vm.vish.local:80/homelab-alerts`
- **Email**: Configure SMTP settings
- **Discord**: Add webhook URL
## Step 5: Verification & Testing
### Health Check
```bash
# Check container health
docker ps | grep uptime-kuma
# Test HTTP endpoint
curl -I http://localhost:3001
# Check logs for errors
docker logs uptime-kuma-quickstart
```
### Monitor Verification
1. Wait 2-3 minutes for first heartbeat
2. Verify monitor shows **UP** status
3. Check response time graphs
4. Test notification (if configured)
### Resource Usage
```bash
# Check resource consumption
docker stats uptime-kuma-quickstart
# Expected usage:
# CPU: < 5%
# Memory: < 100MB
# Network: Minimal
```
## Step 6: Integration with Homelab
### Add to Monitoring Stack
```yaml
# Add to existing monitoring docker-compose.yml
uptime-kuma:
# ... existing configuration ...
networks:
- monitoring
labels:
- "monitoring.enable=true"
- "backup.enable=true"
networks:
monitoring:
external: true
```
### Configure Reverse Proxy
```yaml
# Nginx Proxy Manager configuration
# Host: uptime.vish.local
# Forward Hostname/IP: uptime-kuma-quickstart
# Forward Port: 3001
# SSL: Let's Encrypt or self-signed
```
### Add to Backup Schedule
```bash
# Add volume to backup script
echo "uptime-kuma-data" >> /etc/backup/volumes.list
# Test backup
./scripts/backup-volumes.sh uptime-kuma-data
```
## Common Quick Start Issues
### Port Already in Use
```bash
# Check what's using port 3001
netstat -tulpn | grep :3001
# Solution: Change external port
ports:
- "3002:3001" # Use port 3002 instead
```
### Permission Denied
```bash
# Fix volume permissions
sudo chown -R 1000:1000 /var/lib/docker/volumes/uptime-kuma-data
# Or use named volume (recommended)
volumes:
uptime-kuma-data:
driver: local
```
### Container Won't Start
```bash
# Check Docker daemon
systemctl status docker
# Check logs
docker logs uptime-kuma-quickstart
# Restart container
docker-compose restart uptime-kuma
```
### Can't Access Web Interface
```bash
# Check firewall
sudo ufw status
sudo ufw allow 3001/tcp
# Check container port binding
docker port uptime-kuma-quickstart
# Test local connectivity
curl http://localhost:3001
```
## Next Steps
### Expand Monitoring
1. **Add More Monitors**:
- Internal services (Plex, Nextcloud, etc.)
- External websites
- API endpoints
- Database connections
2. **Configure Status Pages**:
- Public status page for external services
- Internal dashboard for homelab services
- Custom branding and themes
3. **Set Up Alerting**:
- Email notifications for critical services
- NTFY push notifications
- Discord/Slack integration
- Escalation policies
### Deploy More Services
1. **[Grafana](../services/individual/grafana.md)** - Advanced monitoring dashboards
2. **[Nextcloud](../services/individual/nextcloud.md)** - Personal cloud storage
3. **[Plex](../services/individual/plex.md)** - Media server
4. **[Portainer](../services/individual/portainer.md)** - Container management
### Learn Advanced Concepts
1. **[GitOps Deployment](../admin/gitops-deployment-guide.md)** - Infrastructure as code
2. **[Service Categories](20-Service-Categories.md)** - Explore all available services
3. **[Architecture Overview](03-Architecture-Overview.md)** - Understand the infrastructure
4. **[Security Guidelines](../security/README.md)** - Harden your deployment
## Deployment Templates
### Basic Service Template
```yaml
version: '3.8'
services:
service-name:
image: organization/service:latest
container_name: service-name
restart: unless-stopped
ports:
- "8080:8080"
volumes:
- service-data:/data
- service-config:/config
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
volumes:
service-data:
service-config:
```
### Service with Database
```yaml
version: '3.8'
services:
app:
image: app:latest
container_name: app
restart: unless-stopped
depends_on:
- db
ports:
- "8080:8080"
environment:
- DB_HOST=db
- DB_USER=appuser
- DB_PASS="REDACTED_PASSWORD"
- DB_NAME=appdb
db:
image: postgres:15
container_name: app-db
restart: unless-stopped
environment:
- POSTGRES_USER=appuser
- POSTGRES_PASSWORD="REDACTED_PASSWORD"
- POSTGRES_DB=appdb
volumes:
- db-data:/var/lib/postgresql/data
volumes:
db-data:
```
### Service with Reverse Proxy
```yaml
version: '3.8'
services:
app:
image: app:latest
container_name: app
restart: unless-stopped
expose:
- "8080"
networks:
- proxy
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`app.vish.local`)"
- "traefik.http.services.app.loadbalancer.server.port=8080"
networks:
proxy:
external: true
```
## Automation Scripts
### Quick Deploy Script
```bash
#!/bin/bash
# quick-deploy.sh
SERVICE_NAME=$1
SERVER=$2
if [ -z "$SERVICE_NAME" ] || [ -z "$SERVER" ]; then
echo "Usage: $0 <service-name> <server>"
echo "Example: $0 uptime-kuma raspberry-pi"
exit 1
fi
echo "Deploying $SERVICE_NAME on $SERVER..."
# Navigate to server directory
cd "hosts/$SERVER" || exit 1
# Check if service file exists
if [ ! -f "$SERVICE_NAME.yml" ]; then
echo "Error: $SERVICE_NAME.yml not found in hosts/$SERVER/"
exit 1
fi
# Deploy service
docker-compose -f "$SERVICE_NAME.yml" up -d
# Wait for service to start
sleep 10
# Check status
docker-compose -f "$SERVICE_NAME.yml" ps
echo "Deployment complete!"
echo "Check logs with: docker-compose -f hosts/$SERVER/$SERVICE_NAME.yml logs -f"
```
### Health Check Script
```bash
#!/bin/bash
# health-check.sh
SERVICE_NAME=$1
EXPECTED_PORT=$2
if [ -z "$SERVICE_NAME" ] || [ -z "$EXPECTED_PORT" ]; then
echo "Usage: $0 <service-name> <port>"
exit 1
fi
echo "Checking health of $SERVICE_NAME on port $EXPECTED_PORT..."
# Check container status
if docker ps | grep -q "$SERVICE_NAME"; then
echo "✅ Container is running"
else
echo "❌ Container is not running"
exit 1
fi
# Check port accessibility
if curl -f "http://localhost:$EXPECTED_PORT" > /dev/null 2>&1; then
echo "✅ Service is responding"
else
echo "❌ Service is not responding"
exit 1
fi
echo "✅ Health check passed!"
```
## Support & Resources
### Documentation
- **[Full Documentation](../README.md)** - Complete homelab documentation
- **[Service Categories](20-Service-Categories.md)** - All available services
- **[Troubleshooting](40-Common-Issues.md)** - Common issues and solutions
### Community
- **[Homelab Subreddit](https://reddit.com/r/homelab)** - Community discussions
- **[Self-Hosted](https://reddit.com/r/selfhosted)** - Self-hosting community
- **[Docker Community](https://forums.docker.com/)** - Docker support
### Tools
- **[Portainer](http://atlantis.vish.local:9000)** - Container management
- **[Grafana](http://atlantis.vish.local:3000)** - Monitoring dashboards
- **[Uptime Kuma](http://raspberry-pi.vish.local:3001)** - Service monitoring
---
*This quick start guide gets you up and running with your first service deployment. Once comfortable with the basics, explore the comprehensive documentation for advanced configurations and additional services.*

332
docs/getting-started/architecture.md Normal file
View File

@@ -0,0 +1,332 @@
# 🏗️ Architecture Overview
**🟡 Intermediate Guide**
## 🎯 High-Level Architecture
This homelab follows a **distributed microservices architecture** using Docker containers across multiple physical and virtual hosts. Each service runs in isolation while being orchestrated through a combination of Docker Compose and Ansible automation.
```
┌─────────────────────────────────────────────────────────────┐
│ HOMELAB NETWORK │
│ (Tailscale VPN) │
├─────────────────┬─────────────────┬─────────────────────────┤
│ SYNOLOGY NAS │ COMPUTE NODES │ EDGE DEVICES │
│ │ │ │
│ ┌─────────────┐ │ ┌─────────────┐ │ ┌─────────────────────┐ │
│ │ Atlantis │ │ │ Homelab VM │ │ │ Concord NUC │ │
│ │ (55 svcs) │ │ │ (36 svcs) │ │ │ (9 svcs) │ │
│ └─────────────┘ │ └─────────────┘ │ └─────────────────────┘ │
│ │ │ │
│ ┌─────────────┐ │ ┌─────────────┐ │ ┌─────────────────────┐ │
│ │ Calypso │ │ │ Chicago VM │ │ │ Raspberry Pi │ │
│ │ (17 svcs) │ │ │ (8 svcs) │ │ │ (2 nodes) │ │
│ └─────────────┘ │ └─────────────┘ │ └─────────────────────┘ │
│ │ │ │
│ ┌─────────────┐ │ ┌─────────────┐ │ ┌─────────────────────┐ │
│ │ Setillo │ │ │Bulgaria VM │ │ │ Remote VMs │ │
│ │ (4 svcs) │ │ │ (12 svcs) │ │ │ (Contabo, etc.) │ │
│ └─────────────┘ │ └─────────────┘ │ └─────────────────────┘ │
└─────────────────┴─────────────────┴─────────────────────────┘
```
## 🏠 Host Categories
### 📦 **Synology NAS Cluster** (Primary Storage & Core Services)
**Purpose**: Centralized storage, media services, and always-on applications
| Host | Model | Services | Primary Role |
|------|-------|----------|--------------|
| **Atlantis** | Synology NAS | 55 services | Media hub, monitoring, core infrastructure |
| **Calypso** | Synology NAS | 17 services | Development, backup, secondary services |
| **Setillo** | Synology NAS | 4 services | Monitoring, network services |
**Key Characteristics**:
- **Always-on**: 24/7 operation with UPS backup
- **High storage capacity**: Multiple TB of redundant storage
- **Low power consumption**: Efficient ARM/x86 processors
- **Built-in RAID**: Data protection and redundancy
### 💻 **Compute Nodes** (Processing & Workloads)
**Purpose**: CPU/RAM intensive applications, isolated workloads, testing
| Host | Type | Services | Primary Role |
|------|------|----------|--------------|
| **Homelab VM** | Proxmox VM | 36 services | General purpose, experimentation |
| **Chicago VM** | Proxmox VM | 8 services | Gaming servers, entertainment |
| **Bulgaria VM** | Proxmox VM | 12 services | Communication, productivity |
| **Anubis** | Physical | 8 services | High-performance computing |
| **Guava** | Physical | 6 services | AI/ML workloads, development |
**Key Characteristics**:
- **Scalable resources**: Can allocate CPU/RAM as needed
- **Isolation**: VMs provide security boundaries
- **Flexibility**: Easy to create/destroy for testing
- **Performance**: Dedicated resources for demanding applications
### 🌐 **Edge Devices** (IoT, Networking, Remote Access)
**Purpose**: Network services, IoT hub, remote connectivity
| Host | Type | Services | Primary Role |
|------|------|----------|--------------|
| **Concord NUC** | Intel NUC | 9 services | Home automation, edge computing |
| **Pi-5** | Raspberry Pi 5 | 1 service | Lightweight services, sensors |
| **Pi-5-Kevin** | Raspberry Pi 5 | 1 service | Secondary Pi node |
| **Contabo VM** | Remote VPS | 1 service | External services, backup |
**Key Characteristics**:
- **Low power**: Efficient ARM processors
- **Always accessible**: External connectivity
- **IoT integration**: GPIO pins, sensors, automation
- **Redundancy**: Multiple edge nodes for reliability
## 🌐 Network Architecture
### 🔗 **Connectivity Layer**
```
Internet
│
├── Tailscale VPN (Overlay Network)
│ ├── 100.x.x.x addresses for all nodes
│ └── Secure mesh networking
│
└── Local Network (10.0.0.0/24)
├── Core Infrastructure
├── IoT Devices
└── User Devices
```
**Key Features**:
- **Tailscale VPN**: Secure mesh network connecting all nodes
- **Zero-trust networking**: Each connection is authenticated
- **Remote access**: Access homelab from anywhere securely
- **Automatic failover**: Multiple connection paths
### 🚦 **Service Discovery & Load Balancing**
```
External Request
│
├── Nginx Proxy Manager (Atlantis)
│ ├── SSL Termination
│ ├── Domain routing
│ └── Access control
│
└── Internal Services
├── Docker networks
├── Service mesh
└── Health checks
```
## 🐳 Container Architecture
### 📦 **Docker Compose Patterns**
Each service follows consistent patterns:
```yaml
version: '3.9'
services:
service-name:
image: official/image:tag
container_name: Service-Name
hostname: service-hostname
# Security
security_opt:
- no-new-privileges:true
user: 1026:100 # Synology user mapping
# Health & Reliability
healthcheck:
test: ["CMD", "health-check-command"]
interval: 30s
timeout: 10s
retries: 3
restart: on-failure:5
# Resources
deploy:
resources:
limits:
memory: 2G
cpus: '1.0'
# Networking
networks:
- service-network
ports:
- "8080:80"
# Storage
volumes:
- /volume1/docker/service:/data:rw
- /etc/localtime:/etc/localtime:ro
# Configuration
environment:
- TZ=America/Los_Angeles
- CUSTOM_VAR=value
env_file:
- .env
networks:
service-network:
name: service-network
ipam:
config:
- subnet: 192.168.x.0/24
```
### 🔧 **Common Patterns**
1. **Security Hardening**:
- Non-root users where possible
- Read-only containers for stateless services
- No new privileges flag
- Minimal base images
2. **Resource Management**:
- Memory and CPU limits
- Health checks for reliability
- Restart policies for resilience
3. **Data Management**:
- Persistent volumes for data
- Backup-friendly mount points
- Timezone synchronization
4. **Networking**:
- Custom networks for isolation
- Consistent port mapping
- Service discovery via hostnames
## 📊 Data Flow Architecture
### 🔄 **Monitoring & Observability**
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Prometheus │◄───│ Node Exporters │◄───│ Services │
│ (Metrics) │ │ (Collectors) │ │ (Health Data) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Grafana │◄───│ AlertManager │◄───│ Uptime │
│ (Dashboards) │ │ (Notifications)│ │ Kuma │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
### 💾 **Data Storage Strategy**
```
┌─────────────────┐
│ Application │
│ Data │
├─────────────────┤
│ /volume1/docker │ ◄── Primary storage (Synology)
│ /volume2/backup │ ◄── Backup storage (Synology)
│ /mnt/external │ ◄── External backup (USB/Cloud)
└─────────────────┘
```
**Storage Tiers**:
1. **Hot Storage**: Frequently accessed data on SSDs
2. **Warm Storage**: Regular data on fast HDDs
3. **Cold Storage**: Backups on slower HDDs
4. **Archive Storage**: Long-term backups off-site
## 🔐 Security Architecture
### 🛡️ **Defense in Depth**
```
Internet
│
├── Firewall (Router level)
│ └── Port restrictions, DDoS protection
│
├── VPN (Tailscale)
│ └── Encrypted mesh network
│
├── Reverse Proxy (Nginx)
│ └── SSL termination, access control
│
├── Container Security
│ └── User namespaces, capabilities
│
└── Application Security
└── Authentication, authorization
```
### 🔑 **Authentication & Authorization**
- **Single Sign-On**: Where possible, integrated auth
- **Strong passwords**: Generated and stored in Vaultwarden
- **2FA**: Multi-factor authentication for critical services
- **API keys**: Secure service-to-service communication
- **Certificate management**: Automated SSL/TLS certificates
## 🚀 Deployment Architecture
### 🤖 **Infrastructure as Code**
```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Git Repository│───►│ Ansible Control│───►│ Target Hosts │
│ (This repo) │ │ Node │ │ (All systems) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
**Ansible Automation**:
- **Inventory management**: All hosts and their roles
- **Playbook execution**: Automated deployment
- **Configuration management**: Consistent settings
- **Health monitoring**: Automated checks
### 📈 **Scaling Strategy**
1. **Horizontal Scaling**: Add more hosts as needed
2. **Vertical Scaling**: Upgrade existing hardware
3. **Service Distribution**: Spread load across hosts
4. **Resource Optimization**: Monitor and adjust allocations
## 🔄 Backup & Recovery Architecture
### 💾 **Backup Strategy**
```
Production Data
│
├── Local Snapshots (Hourly)
│ └── Synology snapshot replication
│
├── Cross-site Backup (Daily)
│ └── Synology to Synology replication
│
└── Off-site Backup (Weekly)
└── Cloud storage (encrypted)
```
**Recovery Objectives**:
- **RTO** (Recovery Time): < 4 hours for critical services
- **RPO** (Recovery Point): < 1 hour data loss maximum
- **Testing**: Monthly recovery drills
## 📋 Next Steps
Now that you understand the architecture:
1. **[Prerequisites](prerequisites.md)**: What you need to get started
2. **[Quick Start Guide](quick-start.md)**: Deploy your first service
3. **[Service Categories](../services/categories.md)**: Explore available services
4. **[Infrastructure Details](../infrastructure/hosts.md)**: REDACTED_APP_PASSWORD host
---
*This architecture has evolved over time and continues to grow. Start simple and expand based on your needs!*

510
docs/getting-started/beginner-homelab-guide.md Normal file
View File

@@ -0,0 +1,510 @@
# 🏠 Complete Beginner's Guide to Building Your Own Homelab
**🟢 Beginner Guide - No Prior Experience Required**
This guide is designed for colleagues, friends, or anyone who wants to build their own homelab but has little to no experience with servers, networking, or self-hosting. We'll start from the absolute basics and build up to a fully functional homelab.
## 🤔 What is a Homelab?
### **Simple Definition**
A homelab is like having your own personal cloud at home. Instead of relying on Google Drive, Netflix, or other online services, you run your own versions on your own hardware that you control completely.
### **Real-World Examples**
- **Instead of Google Photos**: Run Immich to store and organize your photos
- **Instead of Netflix**: Run Plex to stream your own movie collection
- **Instead of Google Drive**: Run Nextcloud for file storage and sharing
- **Instead of LastPass**: Run Vaultwarden for password management
- **Instead of Gmail**: Run your own email server (advanced)
### **Why Build a Homelab?**
- **Privacy**: Your data stays on your hardware
- **Learning**: Gain valuable IT skills
- **Cost**: No monthly subscription fees
- **Control**: Customize everything to your needs
- **Fun**: It's genuinely enjoyable to build and maintain
---
## 💰 Budget Planning
### **Starter Budget: $500-800**
Perfect for beginners who want to try homelabbing:
- **NAS**: Synology DS220+ (~$300)
- **Drives**: 2x 4TB WD Red (~$200)
- **Network**: Basic router upgrade (~$100-200)
- **Accessories**: Cables, UPS (~$100)
### **Intermediate Budget: $1,500-2,500**
For those ready to commit to a serious homelab:
- **NAS**: Synology DS723+ or DS920+ (~$500-600)
- **Drives**: 4x 8TB drives (~$800)
- **Network**: 10GbE switch and cards (~$300)
- **Compute**: Intel NUC or mini PC (~$400-600)
- **UPS**: Proper backup power (~$200)
### **Advanced Budget: $3,000-5,000+**
For enthusiasts who want enterprise-level features:
- **NAS**: Synology DS1823xs+ (~$1,200)
- **Drives**: 8x 16TB enterprise drives (~$2,000)
- **Network**: Full 10GbE infrastructure (~$500)
- **Compute**: Multiple servers/workstations (~$1,000+)
- **Rack**: Server rack and professional equipment (~$500)
---
## 🛒 Shopping List for Beginners
### **Essential Hardware (Start Here)**
#### **1. Network Attached Storage (NAS)**
**Recommended**: Synology DS220+ or DS723+
**Why Synology?**
- Beginner-friendly web interface
- Excellent documentation and community
- Reliable hardware with good warranty
- Easy software installation (Package Center)
**What to Look For:**
- At least 2 drive bays (for redundancy)
- Gigabit Ethernet (minimum)
- ARM or x86 processor (x86 preferred for Docker)
- Expandable RAM (if possible)
#### **2. Hard Drives**
**Recommended**: WD Red or Seagate IronWolf (NAS-specific drives)
**Beginner Setup:**
- 2x 4TB drives in RAID 1 (mirrored) = 4TB usable space
- 2x 8TB drives in RAID 1 (mirrored) = 8TB usable space
**Why NAS Drives?**
- Designed for 24/7 operation
- Better vibration resistance
- Longer warranty (3-5 years)
- Optimized for RAID configurations
#### **3. Network Equipment**
**Router**: TP-Link Archer AX73 or similar WiFi 6 router
**Switch**: TP-Link TL-SG108 (8-port Gigabit switch)
**Cables**: Cat6 Ethernet cables
#### **4. Power Protection**
**UPS**: APC Back-UPS 1500VA
- Protects against power outages
- Allows graceful shutdown
- Prevents data corruption
### **Optional but Recommended**
#### **5. Mini PC for Additional Services**
**Options:**
- Intel NUC (compact, efficient)
- Beelink Mini PC (budget-friendly)
- Raspberry Pi 4 (very budget-friendly)
**Use Cases:**
- Home Assistant (smart home automation)
- Pi-hole (network-wide ad blocking)
- Additional Docker containers
#### **6. Cables and Accessories**
- HDMI cable (for initial setup)
- USB drive (for OS installation)
- Cable management (velcro ties, cable clips)
- Labels (for organization)
---
## 📋 Step-by-Step Setup Guide
### **Phase 1: Planning and Preparation (Day 1)**
#### **Step 1: Choose Your Location**
```bash
# Ideal homelab location checklist:
☐ Good ventilation (equipment generates heat)
☐ Stable power supply
☐ Ethernet connection to router
☐ Away from high-traffic areas (noise)
☐ Accessible for maintenance
☐ Secure from pets/children
```
#### **Step 2: Network Planning**
```bash
# Basic network setup:
Internet → Modem → Router → Switch → NAS/Devices
# IP Address planning:
Router: 192.168.1.1
NAS: 192.168.1.100
Mini PC: 192.168.1.101
Your computer: 192.168.1.50 (example)
```
#### **Step 3: Create Accounts**
```bash
# Create these accounts before starting:
☐ Synology account (for NAS setup)
☐ Docker Hub account (for containers)
☐ Tailscale account (for VPN access)
☐ Domain registrar account (optional, for external access)
```
### **Phase 2: Hardware Setup (Day 1-2)**
#### **Step 1: NAS Assembly**
```bash
# Synology NAS setup:
1. Unbox NAS and drives carefully
2. Install drives in drive bays (follow manual)
3. Connect power adapter (don't power on yet)
4. Connect Ethernet cable to router
5. Power on NAS (listen for beep sequence)
```
#### **Step 2: Initial Network Setup**
```bash
# Find your NAS on the network:
1. Download Synology Assistant from synology.com
2. Run Synology Assistant to find your NAS
3. Or browse to http://find.synology.com
4. Click on your NAS to begin setup
```
#### **Step 3: DSM Installation**
```bash
# DiskStation Manager (DSM) setup:
1. Download latest DSM for your model
2. Upload .pat file during setup wizard
3. Create admin account (use strong password!)
4. Set up storage pool (choose SHR for beginners)
5. Create Volume 1 with Btrfs file system
```
### **Phase 3: Basic Configuration (Day 2-3)**
#### **Step 1: Essential Settings**
```bash
# Control Panel configurations:
1. Network → Set static IP address
2. Regional Options → Set timezone
3. Notification → Configure email alerts
4. Security → Enable 2FA, disable default accounts
5. File Services → Enable SMB, disable unnecessary services
```
#### **Step 2: Create Shared Folders**
```bash
# Essential shared folders:
☐ homes (user directories)
☐ media (movies, TV shows, music)
☐ documents (important files)
☐ photos (photo library)
☐ backups (backup storage)
☐ docker (container data)
```
#### **Step 3: User Management**
```bash
# Create users:
1. Control Panel → User & Group
2. Create user accounts for family members
3. Set appropriate permissions for shared folders
4. Enable home folders for each user
```
### **Phase 4: First Applications (Day 3-4)**
#### **Step 1: Install Docker**
```bash
# Docker installation:
1. Package Center → Search "Docker"
2. Install Docker package
3. Open Docker app
4. Familiarize yourself with the interface
```
#### **Step 2: Install Plex Media Server**
```bash
# Plex setup (easiest first application):
1. Package Center → Search "Plex Media Server"
2. Install Plex Media Server
3. Open Plex and create account
4. Add media libraries:
- Movies: /volume1/media/movies
- TV Shows: /volume1/media/tv
- Music: /volume1/media/music
5. Upload some media files to test
```
#### **Step 3: Install File Station and Photo Station**
```bash
# Built-in Synology apps:
1. File Station: Web-based file manager
2. Moments or Synology Photos: Photo management
3. Audio Station: Music streaming
4. Video Station: Video streaming (alternative to Plex)
```
### **Phase 5: Advanced Services (Day 4-7)**
#### **Step 1: Password Manager (Vaultwarden)**
```bash
# Deploy Vaultwarden via Docker:
1. Docker → Registry → Search "vaultwarden/server"
2. Download image
3. Create container with these settings:
- Port: 8080:80
- Volume: /volume1/docker/vaultwarden:/data
- Environment: DOMAIN=http://your-nas-ip:8080
4. Access via http://your-nas-ip:8080
5. Create first user account
```
#### **Step 2: Ad Blocking (Pi-hole)**
```bash
# Pi-hole setup:
1. Docker → Registry → Search "pihole/pihole"
2. Create container with network mode: host
3. Set environment variables:
- WEBPASSWORD="REDACTED_PASSWORD"
- TZ=your-timezone
4. Configure router to use NAS IP as DNS server
5. Access web interface at http://your-nas-ip/admin
```
#### **Step 3: Monitoring (Uptime Kuma)**
```bash
# Service monitoring:
1. Docker → Registry → Search "louislam/uptime-kuma"
2. Create container:
- Port: 3001:3001
- Volume: /volume1/docker/uptime-kuma:/app/data
3. Access via http://your-nas-ip:3001
4. Add monitors for your services
5. Configure notifications (email, Discord, etc.)
```
---
## 🔧 Common Beginner Mistakes (And How to Avoid Them)
### **1. Not Planning Storage Properly**
**Mistake**: Buying drives that are too small or not NAS-rated
**Solution**: Always buy NAS-specific drives (WD Red, Seagate IronWolf)
**Tip**: Start with 2x drives in RAID 1, expand later
### **2. Ignoring Backups**
**Mistake**: Thinking RAID is a backup (it's not!)
**Solution**: Follow 3-2-1 backup rule:
- 3 copies of important data
- 2 different storage types
- 1 offsite backup
### **3. Poor Network Planning**
**Mistake**: Not setting static IP addresses
**Solution**: Use DHCP reservations or static IPs for all servers
### **4. Security Oversights**
**Mistake**: Using default passwords, no 2FA
**Solution**:
- Change ALL default passwords
- Enable 2FA everywhere possible
- Use a password manager
- Keep software updated
### **5. REDACTED_APP_PASSWORD Initially**
**Mistake**: Trying to set up everything at once
**Solution**: Start simple, add services gradually
---
## 📚 Learning Resources for Beginners
### **YouTube Channels**
- **SpaceInvaderOne**: Excellent Docker tutorials
- **TechnoTim**: Homelab and self-hosting guides
- **NetworkChuck**: Networking and IT fundamentals
- **Lawrence Systems**: Business IT and homelab content
- **Craft Computing**: Hardware reviews and tutorials
### **Websites and Forums**
- **r/homelab**: Reddit community with great advice
- **r/synology**: Synology-specific help and tips
- **Synology Community**: Official forums
- **Self-Hosted Podcast**: Great for staying current
- **Awesome-Selfhosted**: Comprehensive list of applications
### **Documentation**
- **Synology Knowledge Base**: Official documentation
- **Docker Documentation**: Learn REDACTED_APP_PASSWORD
- **LinuxServer.io**: Pre-built Docker images with great docs
---
## 🛡️ Security Best Practices for Beginners
### **Network Security**
```bash
# Essential security steps:
☐ Change default router password
☐ Disable WPS on router
☐ Enable WPA3 WiFi security
☐ Disable unnecessary router services
☐ Set up guest WiFi network
☐ Enable router firewall
```
### **NAS Security**
```bash
# Synology security checklist:
☐ Change admin password (use password manager)
☐ Enable 2FA for admin account
☐ Disable default accounts (guest, etc.)
☐ Enable auto-block for failed logins
☐ Keep DSM updated
☐ Enable firewall
☐ Disable unnecessary services
☐ Use HTTPS for web access
```
### **Application Security**
```bash
# Docker container security:
☐ Only use trusted images
☐ Keep containers updated
☐ Use non-root users when possible
☐ Limit container permissions
☐ Use secrets for passwords
☐ Enable container logging
```
---
## 🔄 Maintenance Schedule for Beginners
### **Daily (Automated)**
- System health checks
- Backup verification
- Security updates
- Service monitoring
### **Weekly (5 minutes)**
```bash
☐ Check system notifications
☐ Review service status
☐ Check available storage space
☐ Review security logs
```
### **Monthly (30 minutes)**
```bash
☐ Update all applications
☐ Review backup integrity
☐ Check hardware health (SMART status)
☐ Clean up old files and logs
☐ Review user access permissions
```
### **Quarterly (2 hours)**
```bash
☐ Full system backup
☐ Test disaster recovery procedures
☐ Review and update documentation
☐ Plan capacity upgrades
☐ Security audit and password changes
```
---
## 🚀 Next Steps After Basic Setup
### **Intermediate Projects**
1. **Home Automation**: Set up Home Assistant
2. **Media Automation**: Configure Sonarr/Radarr for automatic downloads
3. **Remote Access**: Set up VPN (Tailscale or WireGuard)
4. **Monitoring**: Advanced monitoring with Grafana/Prometheus
5. **Backup Automation**: Automated offsite backups
### **Advanced Projects**
1. **Kubernetes**: Container orchestration
2. **CI/CD Pipeline**: GitLab or Jenkins
3. **Network Segmentation**: VLANs and advanced networking
4. **High Availability**: Clustering and failover
5. **Custom Applications**: Develop your own services
---
## 💡 Tips for Success
### **Start Small**
- Begin with one or two services
- Master the basics before adding complexity
- Document everything you do
### **Join the Community**
- Ask questions on Reddit r/homelab
- Share your setup and get feedback
- Help others when you can
### **Be Patient**
- Learning takes time
- Expect things to break (it's part of learning)
- Keep backups of working configurations
### **Have Fun**
- Experiment with new services
- Customize to your needs
- Enjoy the learning process
---
## 🆘 Getting Help
### **When Things Go Wrong**
1. **Check the logs**: Most problems are logged somewhere
2. **Search the error**: Google/Reddit search exact error messages
3. **Ask for help**: Include relevant details (hardware, software versions, error messages)
4. **Document the solution**: Help others and your future self
### **Where to Get Help**
- **Reddit**: r/homelab, r/synology, r/selfhosted
- **Discord**: Various homelab Discord servers
- **Forums**: Synology Community, LinuxServer.io forums
- **Documentation**: Always check official docs first
---
## 🎯 Your First 30 Days
### **Week 1: Foundation**
- Set up NAS and basic storage
- Install first application (Plex)
- Configure basic security
- Create backup strategy
### **Week 2: Core Services**
- Add password manager (Vaultwarden)
- Set up network-wide ad blocking (Pi-hole)
- Configure monitoring (Uptime Kuma)
- Implement proper backup routine
### **Week 3: Expansion**
- Add file synchronization (Syncthing)
- Set up photo management (Immich)
- Configure remote access (Tailscale)
- Optimize performance
### **Week 4: Mastery**
- Document your setup
- Plan next additions
- Help others in community
- Celebrate your success!
---
**🎉 Congratulations!** You're now ready to start your homelab journey. Remember, everyone started as a beginner, and the homelab community is incredibly helpful and welcoming. Don't be afraid to ask questions, make mistakes, and most importantly, have fun learning!
**📞 Need Help?** Feel free to reach out to the homelab community or reference the comprehensive documentation in this repository for more advanced configurations and troubleshooting.

991
docs/getting-started/complete-rebuild-guide.md Normal file
View File

@@ -0,0 +1,991 @@
# 🏗️ Complete Homelab Rebuild Guide - From Hardware to Services
**🔴 Advanced Guide - Complete Infrastructure Rebuild**
This guide provides step-by-step instructions for rebuilding the entire homelab infrastructure from scratch, including hardware setup, network configuration, and service deployment. Use this guide for complete disaster recovery or when setting up a new homelab.
## 📋 Prerequisites & Planning
### **Required Hardware Inventory**
Before starting, ensure you have all hardware components:
#### **Primary Infrastructure**
- [ ] **Synology DS1823xs+** (8-bay NAS)
- [ ] **8x Seagate IronWolf Pro 16TB** (ST16000NT001)
- [ ] **2x Crucial P310 1TB NVMe** (CT1000P310SSD801)
- [ ] **1x Synology SNV5420-400G NVMe**
- [ ] **Synology E10M20-T1** (10GbE + M.2 adapter)
- [ ] **TP-Link TL-SX1008** (10GbE switch)
- [ ] **TP-Link Archer BE800** (Wi-Fi 7 router)
#### **Compute Infrastructure**
- [ ] **Intel NUC6i3SYB** (Concord NUC)
- [ ] **Raspberry Pi 5 16GB** (with PiRonMan case)
- [ ] **Raspberry Pi 5 8GB** (Kevin)
- [ ] **NVIDIA Shield TV Pro** (travel device)
- [ ] **MSI Prestige 13 AI Plus** (travel laptop)
#### **Network & Power**
- [ ] **UPS system** (1500VA minimum)
- [ ] **Ethernet cables** (Cat6/Cat6a for 10GbE)
- [ ] **Power cables and adapters**
- [ ] **HDMI cables** (for initial setup)
### **Required Software & Accounts**
- [ ] **Synology DSM** (latest version)
- [ ] **Docker** and **Docker Compose**
- [ ] **Tailscale account** (for VPN mesh)
- [ ] **Domain registration** (for external access)
- [ ] **Email account** (for SMTP notifications)
- [ ] **Cloud storage** (for offsite backups)
---
## 🌐 Phase 1: Network Infrastructure Setup (Day 1)
### **Step 1: Router Configuration**
#### **TP-Link Archer BE800 Setup**
```bash
# 1. Physical connections
# - Connect modem to WAN port
# - Connect computer to LAN port 1
# - Power on router and wait 2-3 minutes
# 2. Initial access
# Open browser: http://192.168.0.1 or http://tplinkwifi.net
# Default login: admin/admin
# 3. Basic configuration
# - Set admin password (store in password manager)
# - Configure internet connection (DHCP/Static/PPPoE)
# - Set WiFi SSID: "Vish-Homelab-5G" and "Vish-Homelab-2.4G"
# - Set WiFi password (WPA3, strong password)
# 4. Network settings
# - Change LAN subnet to 192.168.1.0/24
# - Set DHCP range: 192.168.1.100-192.168.1.200
# - Set DNS servers: 1.1.1.1, 8.8.8.8
# - Enable UPnP (for media services)
# - Disable WPS (security)
```
#### **Static IP Reservations**
```bash
# Configure DHCP reservations for all devices
# Router > Advanced > Network > DHCP Server > Address Reservation
# Primary Infrastructure
atlantis.vish.local → 192.168.1.100 # DS1823xs+
calypso.vish.local → 192.168.1.101 # DS723+ (if present)
setillo.vish.local → 192.168.1.108 # Monitoring NAS
# Compute Hosts
concord-nuc.vish.local → 192.168.1.102 # Intel NUC
homelab-vm.vish.local → 192.168.1.103 # Proxmox VM
chicago-vm.vish.local → 192.168.1.104 # Gaming VM
bulgaria-vm.vish.local → 192.168.1.105 # Communication VM
# Physical Hosts
anubis.vish.local → 192.168.1.106 # Mac Mini
guava.vish.local → 192.168.1.107 # AMD Workstation
shinku-ryuu.vish.local → 192.168.1.120 # Main Desktop
# Edge Devices
rpi-vish.vish.local → 192.168.1.109 # Raspberry Pi 5 (16GB)
rpi-kevin.vish.local → 192.168.1.110 # Raspberry Pi 5 (8GB)
nvidia-shield.vish.local → 192.168.1.111 # NVIDIA Shield TV Pro
# Travel Devices
msi-laptop.vish.local → 192.168.1.115 # MSI Prestige 13 AI Plus
```
### **Step 2: 10 Gigabit Network Setup**
#### **TP-Link TL-SX1008 Configuration**
```bash
# 1. Physical setup
# - Connect TL-SX1008 to router LAN port via 1GbE
# - Power on switch
# - No configuration needed (unmanaged switch)
# 2. Device connections (as devices come online)
# Port 1: Atlantis (via E10M20-T1 card)
# Port 2: Calypso (via PCIe 10GbE card)
# Port 3: Shinku-Ryuu (via PCIe 10GbE card)
# Port 4: Guava (via PCIe 10GbE card)
# Ports 5-8: Available for future expansion
```
### **Step 3: DNS and Domain Setup**
#### **Dynamic DNS Configuration**
```bash
# 1. Choose DDNS provider (Synology, No-IP, DuckDNS)
# 2. Register domain: vishinator.synology.me (or custom domain)
# 3. Configure in router:
# - Advanced > Dynamic DNS
# - Provider: Synology
# - Hostname: vishinator.synology.me
# - Username/Password: "REDACTED_PASSWORD" account credentials
# 4. Test DDNS
# Wait 10 minutes, then test:
nslookup vishinator.synology.me
# Should return your external IP address
```
---
## 🏛️ Phase 2: Primary NAS Setup (Day 1-2)
### **Step 1: Synology DS1823xs+ Hardware Assembly**
#### **Drive Installation**
```bash
# 1. Unpack DS1823xs+ and drives
# 2. Install drives in order (for RAID consistency):
# Bay 1: Seagate IronWolf Pro 16TB #1
# Bay 2: Seagate IronWolf Pro 16TB #2
# Bay 3: Seagate IronWolf Pro 16TB #3
# Bay 4: Seagate IronWolf Pro 16TB #4
# Bay 5: Seagate IronWolf Pro 16TB #5
# Bay 6: Seagate IronWolf Pro 16TB #6
# Bay 7: Seagate IronWolf Pro 16TB #7
# Bay 8: Seagate IronWolf Pro 16TB #8
# 3. Install M.2 drives:
# Slot 1: Crucial P310 1TB #1
# Slot 2: Crucial P310 1TB #2
# 4. Install expansion card:
# PCIe Slot 1: Synology E10M20-T1
# E10M20-T1 M.2 Slot: Synology SNV5420-400G
# 5. Install RAM upgrade:
# - Remove existing 4GB module
# - Install 32GB DDR4 ECC module
```
#### **Network Connections**
```bash
# 1. Primary connections:
# - LAN 1: Connect to router (1GbE management)
# - LAN 2: Available for bonding/backup
# - 10GbE: Connect to TL-SX1008 switch
# 2. Power connection:
# - Connect 180W power adapter
# - Connect to UPS if available
```
### **Step 2: DSM Installation and Initial Setup**
#### **DSM Installation**
```bash
# 1. Power on DS1823xs+
# 2. Wait for boot (2-3 minutes, listen for beep)
# 3. Find NAS on network:
# - Use Synology Assistant (download from synology.com)
# - Or browse to http://find.synology.com
# - Or direct IP: http://192.168.1.100
# 4. DSM Installation:
# - Download latest DSM for DS1823xs+
# - Upload .pat file during setup
# - Follow installation wizard
# - Create admin account (store credentials securely)
```
#### **Basic DSM Configuration**
```bash
# 1. Network settings:
# - Control Panel > Network > Network Interface
# - Set static IP: 192.168.1.100
# - Subnet: 255.255.255.0
# - Gateway: 192.168.1.1
# - DNS: 1.1.1.1, 8.8.8.8
# 2. Time and region:
# - Control Panel > Regional Options
# - Time zone: America/Los_Angeles
# - NTP server: pool.ntp.org
# 3. Notifications:
# - Control Panel > Notification > Email
# - SMTP server: smtp.gmail.com:587
# - Configure email notifications for critical events
```
### **Step 3: Storage Configuration**
#### **RAID Array Setup**
```bash
# 1. Storage Manager > Storage > Create
# 2. Choose RAID type:
# - RAID 6: Best balance of capacity and redundancy
# - Can survive 2 drive failures
# - Usable capacity: ~96TB (6 drives worth)
# 3. Volume creation:
# - Create Volume 1 on RAID array
# - File system: Btrfs (for snapshots and data integrity)
# - Enable data checksum
# - Enable compression (if desired)
```
#### **M.2 Storage Configuration**
```bash
# CRITICAL: Install 007revad scripts FIRST
# SSH to NAS as admin user
# 1. Download and install scripts:
cd /volume1
git clone https://github.com/007revad/Synology_HDD_db.git
git clone https://github.com/007revad/Synology_M2_volume.git
git clone https://github.com/007revad/Synology_enable_M2_volume.git
# 2. Run HDD database script:
cd Synology_HDD_db
sudo ./syno_hdd_db.sh
# This adds IronWolf Pro drives to compatibility database
# 3. Enable M.2 volume support:
cd ../Synology_enable_M2_volume
sudo ./syno_enable_m2_volume.sh
# 4. Create M.2 volumes:
cd ../Synology_M2_volume
sudo ./syno_m2_volume.sh
# 5. Configure M.2 storage:
# Storage Manager > Storage > Create
# - Volume 2: Crucial P310 drives in RAID 1 (high-performance storage)
# - Volume 3: Synology SNV5420 (cache and metadata)
```
### **Step 4: Essential Services Setup**
#### **Docker Installation**
```bash
# 1. Package Center > Search "Docker"
# 2. Install Docker package
# 3. Enable SSH (Control Panel > Terminal & SNMP > Enable SSH)
# 4. SSH to NAS and verify Docker:
ssh admin@192.168.1.100
docker --version
docker-compose --version
```
#### **File Sharing Setup**
```bash
# 1. Create shared folders:
# Control Panel > Shared Folder > Create
# Essential folders:
# - docker (for container data)
# - media (for Plex library)
# - documents (for Paperless-NGX)
# - backups (for system backups)
# - homes (for user directories)
# 2. Set permissions:
# - admin: Read/Write access to all folders
# - Create service accounts as needed
```
---
## 🔧 Phase 3: Core Services Deployment (Day 2-3)
### **Step 1: Infrastructure Services**
#### **Portainer (Container Management)**
```bash
# 1. Create Portainer directory:
mkdir -p /volume1/docker/portainer
# 2. Deploy Portainer:
docker run -d \
--name portainer \
--restart always \
-p 9000:9000 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /volume1/docker/portainer:/data \
portainer/portainer-ce:latest
# 3. Access: http://192.168.1.100:9000
# 4. Create admin account
# 5. Connect to local Docker environment
```
#### **Watchtower (Auto-Updates)**
```bash
# Deploy Watchtower for automatic container updates:
docker run -d \
--name watchtower \
--restart always \
-v /var/run/docker.sock:/var/run/docker.sock \
containrrr/watchtower \
--schedule "0 0 4 * * *" \
--cleanup
```
### **Step 2: Security Services**
#### **Vaultwarden (Password Manager)**
```bash
# 1. Create directory structure:
mkdir -p /volume2/metadata/docker/vaultwarden/{data,db}
# 2. Deploy using the commented configuration:
# Copy /workspace/project/homelab/Atlantis/vaultwarden.yaml
# Update passwords and tokens
# Deploy: docker-compose -f vaultwarden.yaml up -d
# 3. Initial setup:
# - Access http://192.168.1.100:4080
# - Create first user account
# - Configure admin panel with admin token
```
#### **Pi-hole (DNS Filtering)**
```bash
# 1. Create Pi-hole directory:
mkdir -p /volume1/docker/pihole/{etc,dnsmasq}
# 2. Deploy Pi-hole:
docker run -d \
--name pihole \
--restart always \
-p 53:53/tcp -p 53:53/udp \
-p 8080:80 \
-e TZ=America/Los_Angeles \
-e WEBPASSWORD="REDACTED_PASSWORD" \
-v /volume1/docker/pihole/etc:/etc/pihole \
-v /volume1/docker/pihole/dnsmasq:/etc/dnsmasq.d \
pihole/pihole:latest
# 3. Configure router to use Pi-hole:
# Router DNS: 192.168.1.100
```
### **Step 3: Monitoring Stack**
#### **Grafana and Prometheus**
```bash
# 1. Create monitoring directories:
mkdir -p /volume1/docker/{grafana,prometheus}
# 2. Deploy monitoring stack:
# Copy monitoring-stack.yaml from homelab repo
# Update configurations
# Deploy: docker-compose -f monitoring-stack.yaml up -d
# 3. Configure dashboards:
# - Import Synology dashboard
# - Configure data sources
# - Set up alerting
```
#### **Uptime Kuma (Service Monitoring)**
```bash
# 1. Deploy Uptime Kuma:
docker run -d \
--name uptime-kuma \
--restart always \
-p 3001:3001 \
-v /volume1/docker/uptime-kuma:/app/data \
louislam/uptime-kuma:1
# 2. Configure monitoring:
# - Add all critical services
# - Set up notifications
# - Configure status page
```
---
## 📺 Phase 4: Media Services (Day 3-4)
### **Step 1: Plex Media Server**
```bash
# 1. Create Plex directories:
mkdir -p /volume1/docker/plex
mkdir -p /volume1/data/media/{movies,tv,music,photos}
# 2. Deploy Plex using commented configuration:
# Copy plex.yaml from homelab repo
# Update PUID/PGID and timezone
# Deploy: docker-compose -f plex.yaml up -d
# 3. Initial setup:
# - Access http://192.168.1.100:32400/web
# - Claim server with Plex account
# - Add media libraries
# - Configure hardware transcoding
```
### **Step 2: Media Management (Arr Suite)**
```bash
# 1. Deploy Arr suite services:
# - Sonarr (TV shows)
# - Radarr (Movies)
# - Prowlarr (Indexer management)
# - SABnzbd (Download client)
# 2. Configure each service:
# - Set up indexers in Prowlarr
# - Configure download clients
# - Set up media folders
# - Configure quality profiles
```
### **Step 3: Photo Management**
```bash
# 1. Deploy Immich (if using):
# Copy immich configuration
# Set up database and Redis
# Configure storage paths
# 2. Alternative: PhotoPrism
# Deploy PhotoPrism container
# Configure photo directories
# Set up face recognition
```
---
## 🌐 Phase 5: Network Services (Day 4-5)
### **Step 1: VPN Setup**
#### **Tailscale Mesh VPN**
```bash
# 1. Install Tailscale on NAS:
# Download Tailscale package for Synology
# Install via Package Center or manual installation
# 2. Configure Tailscale:
sudo tailscale up --advertise-routes=192.168.1.0/24
# Approve subnet routes in Tailscale admin console
# 3. Install on all devices:
# - Concord NUC
# - Raspberry Pi nodes
# - NVIDIA Shield
# - Travel devices
```
#### **WireGuard (Alternative/Backup VPN)**
```bash
# 1. Deploy WireGuard container:
docker run -d \
--name wireguard \
--restart always \
--cap-add=NET_ADMIN \
--cap-add=SYS_MODULE \
-e PUID=1029 \
-e PGID=65536 \
-e TZ=America/Los_Angeles \
-p 51820:51820/udp \
-v /volume1/docker/wireguard:/config \
-v /lib/modules:/lib/modules \
linuxserver/wireguard
# 2. Configure port forwarding:
# Router: External 51820/UDP → 192.168.1.100:51820
```
### **Step 2: Reverse Proxy**
#### **Nginx Proxy Manager**
```bash
# 1. Deploy Nginx Proxy Manager:
docker run -d \
--name nginx-proxy-manager \
--restart always \
-p 8341:80 \
-p 8766:443 \
-p 8181:81 \
-v /volume1/docker/nginx-proxy-manager:/data \
-v /volume1/docker/nginx-proxy-manager/letsencrypt:/etc/letsencrypt \
jc21/nginx-proxy-manager:latest
# 2. Configure SSL certificates:
# - Set up Let's Encrypt
# - Configure proxy hosts
# - Set up access lists
```
---
## 🖥️ Phase 6: Compute Nodes Setup (Day 5-6)
### **Step 1: Intel NUC (Concord)**
#### **Operating System Installation**
```bash
# 1. Create Ubuntu 22.04 LTS installation media
# 2. Boot from USB and install Ubuntu
# 3. Configure network:
sudo netplan apply
# Set static IP: 192.168.1.102
# 4. Install Docker:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
# 5. Install Docker Compose:
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
```
#### **Home Assistant Setup**
```bash
# 1. Create Home Assistant directory:
mkdir -p ~/docker/homeassistant
# 2. Deploy Home Assistant:
docker run -d \
--name homeassistant \
--restart always \
--privileged \
--net=host \
-e TZ=America/Los_Angeles \
-v ~/docker/homeassistant:/config \
ghcr.io/home-assistant/home-assistant:stable
# 3. Access: http://192.168.1.102:8123
```
### **Step 2: Raspberry Pi Cluster**
#### **Pi-5 (Vish) Setup**
```bash
# 1. Flash Raspberry Pi OS Lite (64-bit)
# 2. Enable SSH and configure WiFi
# 3. Boot and configure:
sudo raspi-config
# - Enable SSH
# - Set timezone
# - Expand filesystem
# 4. Install Docker:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker pi
# 5. Install Tailscale:
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
```
#### **Pi-5-Kevin Setup**
```bash
# Follow same process as Pi-5 (Vish)
# Configure as secondary node
# Set static IP: 192.168.1.110
```
---
## 📱 Phase 7: Edge and Travel Devices (Day 6-7)
### **Step 1: NVIDIA Shield TV Pro**
#### **Initial Setup**
```bash
# 1. Connect to TV and complete Android TV setup
# 2. Enable Developer Options:
# Settings > Device Preferences > About
# Click "Build" 7 times
# 3. Enable USB Debugging:
# Settings > Device Preferences > Developer Options
# Enable "USB Debugging"
# 4. Install Tailscale:
# - Download Tailscale APK
# - Install via file manager or ADB
# - Configure with homelab tailnet
```
#### **Media Apps Configuration**
```bash
# 1. Install Plex app from Play Store
# 2. Configure Plex server connection:
# Server: atlantis.vish.local:32400
# Or Tailscale IP: 100.83.230.112:32400
# 3. Install additional apps:
# - VLC Media Player
# - Chrome Browser
# - Termux (for SSH access)
```
### **Step 2: MSI Prestige 13 AI Plus**
#### **Tailscale Setup**
```bash
# 1. Download and install Tailscale for Windows
# 2. Sign in with homelab account
# 3. Configure as exit node (optional):
# Tailscale > Settings > Use as exit node
# 4. Test connectivity:
ping atlantis.vish.local
ping 100.83.230.112
```
#### **Development Environment**
```bash
# 1. Install WSL2:
wsl --install Ubuntu-22.04
# 2. Configure WSL2:
# - Install Docker Desktop
# - Enable WSL2 integration
# - Install development tools
# 3. SSH key setup:
ssh-keygen -t ed25519 -C "msi-laptop@homelab"
# Copy public key to homelab hosts
```
---
## 🔄 Phase 8: Backup and Monitoring (Day 7)
### **Step 1: Backup Configuration**
#### **Local Backups**
```bash
# 1. Configure Synology backup tasks:
# Control Panel > Task Scheduler > Create > Backup
# 2. Critical backup jobs:
# - Docker configurations (daily)
# - Database backups (daily)
# - System configurations (weekly)
# - Media metadata (weekly)
# 3. Backup verification:
# - Test restore procedures
# - Verify backup integrity
# - Document recovery procedures
```
#### **Offsite Backups**
```bash
# 1. Configure cloud backup:
# - Synology C2 Backup
# - Or AWS S3/Glacier
# - Or Google Drive/OneDrive
# 2. Encrypt sensitive backups:
# - Use Synology encryption
# - Or GPG encryption for scripts
# - Store encryption keys securely
```
### **Step 2: Monitoring Setup**
#### **Service Monitoring**
```bash
# 1. Configure Uptime Kuma monitors:
# - All critical services
# - Network connectivity
# - Certificate expiration
# - Disk space usage
# 2. Set up notifications:
# - Email alerts
# - Discord/Slack webhooks
# - SMS for critical alerts
```
#### **Performance Monitoring**
```bash
# 1. Configure Grafana dashboards:
# - System performance
# - Network utilization
# - Service health
# - Storage usage
# 2. Set up alerting rules:
# - High CPU/memory usage
# - Disk space warnings
# - Service failures
# - Network issues
```
---
## 🧪 Phase 9: Testing and Validation (Day 8)
### **Step 1: Service Testing**
#### **Connectivity Tests**
```bash
# 1. Internal network tests:
ping atlantis.vish.local
ping concord-nuc.vish.local
ping rpi-vish.vish.local
# 2. Service accessibility tests:
curl -I http://atlantis.vish.local:32400 # Plex
curl -I http://atlantis.vish.local:9000 # Portainer
curl -I http://atlantis.vish.local:4080 # Vaultwarden
# 3. External access tests:
# Test from mobile device or external network
# Verify VPN connectivity
# Test domain resolution
```
#### **Performance Tests**
```bash
# 1. Network performance:
iperf3 -s # On server
iperf3 -c atlantis.vish.local # From client
# 2. Storage performance:
dd if=/dev/zero of=/volume1/test bs=1M count=1000
rm /volume1/test
# 3. Media streaming tests:
# Test Plex transcoding
# Verify hardware acceleration
# Test multiple concurrent streams
```
### **Step 2: Disaster Recovery Testing**
#### **Backup Restoration Tests**
```bash
# 1. Test configuration restore:
# - Stop a service
# - Restore from backup
# - Verify functionality
# 2. Test database restore:
# - Create test database backup
# - Restore to different location
# - Verify data integrity
# 3. Test complete service rebuild:
# - Remove service completely
# - Rebuild from documentation
# - Restore data from backup
```
#### **Failover Tests**
```bash
# 1. Network failover:
# - Disconnect primary network
# - Test Tailscale connectivity
# - Verify service accessibility
# 2. Power failure simulation:
# - Graceful shutdown test
# - UPS functionality test
# - Startup sequence verification
# 3. Drive failure simulation:
# - Remove one drive from RAID
# - Verify RAID degraded mode
# - Test rebuild process
```
---
## 📚 Phase 10: Documentation and Maintenance (Ongoing)
### **Step 1: Documentation Updates**
#### **Configuration Documentation**
```bash
# 1. Update network documentation:
# - IP address assignments
# - Port forwarding rules
# - DNS configurations
# - VPN settings
# 2. Update service documentation:
# - Container configurations
# - Database schemas
# - API endpoints
# - Access credentials
# 3. Update hardware documentation:
# - Serial numbers
# - Warranty information
# - Replacement procedures
# - Performance baselines
```
#### **Procedure Documentation**
```bash
# 1. Create runbooks:
# - Service restart procedures
# - Backup and restore procedures
# - Troubleshooting guides
# - Emergency contacts
# 2. Update disaster recovery plans:
# - Recovery time objectives
# - Recovery point objectives
# - Escalation procedures
# - Communication plans
```
### **Step 2: Maintenance Schedules**
#### **Daily Tasks**
```bash
# Automated:
# - Service health checks
# - Backup verification
# - Security updates
# - Log rotation
# Manual:
# - Review monitoring alerts
# - Check service status
# - Verify backup completion
```
#### **Weekly Tasks**
```bash
# - Review system performance
# - Check disk usage
# - Update documentation
# - Test backup restores
# - Review security logs
```
#### **Monthly Tasks**
```bash
# - Full system backup
# - Hardware health check
# - Security audit
# - Performance optimization
# - Documentation review
```
#### **Quarterly Tasks**
```bash
# - Disaster recovery drill
# - Hardware warranty review
# - Software license review
# - Capacity planning
# - Security assessment
```
---
## 🚨 Emergency Procedures
### **Critical Service Failures**
```bash
# 1. Vaultwarden failure:
# - Use offline password backup
# - Restore from latest backup
# - Verify database integrity
# - Test all password access
# 2. Network failure:
# - Check physical connections
# - Verify router configuration
# - Test internet connectivity
# - Activate backup internet (mobile hotspot)
# 3. Storage failure:
# - Check RAID status
# - Replace failed drives
# - Monitor rebuild progress
# - Verify data integrity
```
### **Complete Infrastructure Failure**
```bash
# 1. Assess damage:
# - Check power systems
# - Verify network connectivity
# - Test individual components
# - Document failures
# 2. Prioritize recovery:
# - Network infrastructure first
# - Critical services (Vaultwarden, DNS)
# - Media and productivity services
# - Development and testing services
# 3. Execute recovery plan:
# - Follow this rebuild guide
# - Restore from backups
# - Verify service functionality
# - Update documentation
```
---
## 📋 Final Checklist
### **Infrastructure Validation**
```bash
☐ All hardware installed and functional
☐ Network connectivity verified (1GbE and 10GbE)
☐ Static IP assignments configured
☐ DNS resolution working
☐ VPN access functional (Tailscale and WireGuard)
☐ External domain access working
☐ SSL certificates installed and valid
```
### **Service Validation**
```bash
☐ Vaultwarden accessible and functional
☐ Plex streaming working with hardware transcoding
☐ Pi-hole DNS filtering active
☐ Monitoring stack operational (Grafana, Prometheus)
☐ Backup systems configured and tested
☐ All Docker services running and healthy
☐ Mobile and travel device access verified
```
### **Security Validation**
```bash
☐ All default passwords changed
☐ SSH keys configured for key-based authentication
☐ Firewall rules configured
☐ SSL/TLS encryption enabled for all web services
☐ 2FA enabled for critical accounts
☐ Backup encryption verified
☐ Access logs reviewed
```
### **Documentation Validation**
```bash
☐ Network configuration documented
☐ Service configurations documented
☐ Backup and restore procedures tested
☐ Emergency contact information updated
☐ Hardware warranty information recorded
☐ Disaster recovery procedures validated
```
---
**🎉 Congratulations!** You have successfully rebuilt your complete homelab infrastructure. This process typically takes 7-8 days for a complete rebuild, but the result is a fully documented, monitored, and maintainable homelab environment.
**🔄 Next Steps:**
1. Monitor system performance for the first week
2. Fine-tune configurations based on usage patterns
3. Schedule regular maintenance tasks
4. Plan for future expansions and upgrades
5. Share your experience with the homelab community
**💡 Pro Tip:** Keep this guide updated as you make changes to your infrastructure. A well-documented homelab is much easier to maintain and troubleshoot.

510
docs/getting-started/docker-compose-guide.md Normal file
View File

@@ -0,0 +1,510 @@
# 🐳 Docker Compose Guide
*Comprehensive guide for Docker Compose usage in the homelab environment*
## 📋 Overview
This guide covers Docker Compose best practices, patterns, and configurations used throughout the homelab infrastructure for consistent and maintainable container deployments.
## 🏗️ Standard Compose Structure
### Basic Template
```yaml
version: '3.8'
services:
service-name:
image: organization/image:latest
container_name: service-name
restart: unless-stopped
environment:
- PUID=1000
- PGID=1000
- TZ=America/Los_Angeles
volumes:
- ./config:/config
- /data/service:/data
ports:
- "8080:8080"
networks:
- homelab
labels:
- "traefik.enable=true"
- "traefik.http.routers.service.rule=Host(`service.vish.gg`)"
- "com.centurylinklabs.watchtower.enable=true"
networks:
homelab:
external: true
```
## 🔧 Configuration Patterns
### Environment Variables
```yaml
environment:
# User/Group IDs (required for file permissions)
- PUID=1000
- PGID=1000
# Timezone (consistent across all services)
- TZ=America/Los_Angeles
# Service-specific configuration
- DATABASE_URL=postgresql://user:REDACTED_PASSWORD@db:5432/dbname
- REDIS_URL=redis://redis:6379
# Security settings
- SECURE_SSL_REDIRECT=true
- SESSION_COOKIE_SECURE=true
```
### Volume Mapping
```yaml
volumes:
# Configuration (relative to compose file)
- ./config:/config
- ./data:/data
# Shared storage (absolute paths)
- /mnt/storage/media:/media:ro
- /mnt/storage/downloads:/downloads
# System integration
- /var/run/docker.sock:/var/run/docker.sock:ro
- /etc/localtime:/etc/localtime:ro
```
### Network Configuration
```yaml
networks:
# External network (created separately)
homelab:
external: true
# Internal network (service-specific)
internal:
driver: bridge
internal: true
```
## 🏷️ Labeling Standards
### Traefik Integration
```yaml
labels:
# Enable Traefik
- "traefik.enable=true"
# HTTP Router
- "traefik.http.routers.service.rule=Host(`service.vish.gg`)"
- "traefik.http.routers.service.entrypoints=websecure"
- "traefik.http.routers.service.tls.certresolver=letsencrypt"
# Service configuration
- "traefik.http.services.service.loadbalancer.server.port=8080"
# Middleware
- "traefik.http.routers.service.middlewares=auth@file"
```
### Watchtower Configuration
```yaml
labels:
# Enable automatic updates
- "com.centurylinklabs.watchtower.enable=true"
# Update schedule (optional)
- "com.centurylinklabs.watchtower.schedule=0 0 4 * * *"
# Notification settings
- "com.centurylinklabs.watchtower.notification-url=ntfy://ntfy.vish.gg/watchtower"
```
### Monitoring Labels
```yaml
labels:
# Prometheus monitoring
- "prometheus.io/scrape=true"
- "prometheus.io/port=9090"
- "prometheus.io/path=/metrics"
# Service metadata
- "homelab.service.category=media"
- "homelab.service.tier=production"
- "homelab.service.owner=vish"
```
## 🔐 Security Best Practices
### User and Permissions
```yaml
# Always specify user/group IDs
environment:
- PUID=1000
- PGID=1000
# Or use user directive
user: "1000:1000"
# For root-required services, minimize privileges
security_opt:
- no-new-privileges:true
```
### Secrets Management
```yaml
# Use Docker secrets
secrets:
db_password:
"REDACTED_PASSWORD" ./secrets/db_password.txt
services:
app:
secrets:
- db_password
environment:
- DB_PASSWORD_FILE=/run/secrets/db_password
```
### Network Security
```yaml
# Avoid host networking
network_mode: host # ❌ Avoid this
# Use custom networks instead
networks:
- internal # ✅ Preferred approach
# Limit exposed ports
ports:
- "127.0.0.1:8080:8080" # ✅ Bind to localhost only
```
## 📊 Resource Management
### Resource Limits
```yaml
services:
service-name:
deploy:
resources:
limits:
cpus: '2.0'
memory: 2G
reservations:
cpus: '0.5'
memory: 512M
```
### Health Checks
```yaml
services:
service-name:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
```
### Restart Policies
```yaml
# Standard restart policy
restart: unless-stopped
# Alternative policies
restart: "no" # Never restart
restart: always # Always restart
restart: on-failure # Restart on failure only
```
## 🗂️ Multi-Service Patterns
### Database Integration
```yaml
version: '3.8'
services:
app:
image: myapp:latest
depends_on:
- database
environment:
- DATABASE_URL=postgresql://user:REDACTED_PASSWORD@database:5432/myapp
networks:
- internal
database:
image: postgres:15
environment:
- POSTGRES_DB=myapp
- POSTGRES_USER=user
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
volumes:
- db_data:/var/lib/postgresql/data
networks:
- internal
secrets:
- db_password
volumes:
db_data:
networks:
internal:
driver: bridge
secrets:
db_password:
"REDACTED_PASSWORD" ./secrets/db_password.txt
```
### Reverse Proxy Integration
```yaml
services:
app:
image: myapp:latest
networks:
- homelab
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`app.vish.gg`)"
- "traefik.http.routers.app.entrypoints=websecure"
- "traefik.http.routers.app.tls.certresolver=letsencrypt"
networks:
homelab:
external: true
```
## 🔄 Development vs Production
### Development Override
```yaml
# docker-compose.override.yml
version: '3.8'
services:
app:
build: .
volumes:
- .:/app
environment:
- DEBUG=true
ports:
- "8080:8080"
```
### Production Configuration
```yaml
# docker-compose.prod.yml
version: '3.8'
services:
app:
image: myapp:v1.2.3
restart: unless-stopped
deploy:
resources:
limits:
memory: 1G
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
```
## 📝 Documentation Standards
### Service Documentation
```yaml
# At the top of each compose file
# Service: Application Name
# Purpose: Brief description of what this service does
# Access: How to access the service (URL, port, etc.)
# Dependencies: Other services this depends on
# Volumes: Important volume mappings
# Configuration: Key environment variables
```
### Inline Comments
```yaml
services:
app:
image: myapp:latest
container_name: myapp
restart: unless-stopped
environment:
# Required: User/group for file permissions
- PUID=1000
- PGID=1000
# Optional: Custom configuration
- CUSTOM_SETTING=value
volumes:
# Configuration directory
- ./config:/config
# Data storage (persistent)
- app_data:/data
ports:
# Web interface
- "8080:8080"
```
## 🚀 Deployment Strategies
### GitOps Deployment
```yaml
# Compose files are deployed via Portainer GitOps
# Repository: https://git.vish.gg/Vish/homelab.git
# Branch: main
# Automatic deployment on git push
```
### Manual Deployment
```bash
# Deploy stack
docker-compose up -d
# Update stack
docker-compose pull
docker-compose up -d
# Remove stack
docker-compose down
```
### Stack Management
```bash
# View running services
docker-compose ps
# View logs
docker-compose logs -f service-name
# Execute commands
docker-compose exec service-name bash
# Scale services
docker-compose up -d --scale worker=3
```
## 🔍 Troubleshooting
### Common Issues
```bash
# Check service status
docker-compose ps
# View logs
docker-compose logs service-name
# Validate configuration
docker-compose config
# Check resource usage
docker stats
```
### Debug Commands
```bash
# Inspect container
docker inspect container-name
# Check networks
docker network ls
docker network inspect network-name
# Volume inspection
docker volume ls
docker volume inspect volume-name
```
## 📊 Monitoring Integration
### Prometheus Metrics
```yaml
services:
app:
labels:
- "prometheus.io/scrape=true"
- "prometheus.io/port=9090"
- "prometheus.io/path=/metrics"
```
### Log Management
```yaml
services:
app:
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
labels: "service,environment"
```
## 🔧 Advanced Patterns
### Init Containers
```yaml
services:
app:
image: myapp:latest
depends_on:
init:
condition: service_completed_successfully
init:
image: busybox
command: ["sh", "-c", "echo 'Initialization complete'"]
```
### Sidecar Containers
```yaml
services:
app:
image: myapp:latest
volumes:
- shared_data:/data
sidecar:
image: nginx:alpine
volumes:
- shared_data:/usr/share/nginx/html:ro
ports:
- "80:80"
volumes:
shared_data:
```
## 📚 Additional Resources
### External Documentation
- [Docker Compose Reference](https://docs.docker.com/compose/compose-file/)
- [Docker Best Practices](https://docs.docker.com/develop/best-practices/)
- [Traefik Docker Integration](https://doc.traefik.io/traefik/providers/docker/)
### Internal Resources
- [Development Guide](DEVELOPMENT.md)
- [GitOps Deployment Guide](../admin/gitops-deployment-guide.md)
- [Security Guidelines](../security/SECURITY_GUIDELINES.md)
---
**Last Updated**: February 24, 2026
**Docker Compose Version**: 3.8+ recommended
**Status**: ✅ **PRODUCTION** - Used across all homelab services

37
docs/getting-started/faq.md Normal file
View File

@@ -0,0 +1,37 @@
# Frequently Asked Questions
## General
| Question | Answer |
|----------|--------|
| **What is the Homelab?** | A collection of self‑hosted services managed via GitOps on a cluster of NAS, VM, and edge devices. |
| **How do I get started?** | See the [Getting‑Started](getting-started/BEGINNER_QUICKSTART.md) guide. |
| **Which hosts are available?** | See the [Host Inventory](docs/architecture/host-inventory.md). |
## Deployment
| Question | Answer |
|----------|--------|
| **Can I deploy services manually?** | Yes – use Portainer UI, but recommended is GitOps for consistency. |
| **How to rollback a service?** | Use `git rollback` tag or refer to the *GitOps rollback* page. |
| **What if a stack fails?** | Check Portainer stack events and the corresponding logs in Loki. |
## Troubleshooting
| Question | Answer |
|----------|--------|
| **Container exits unexpectedly** | Look at `docker logs <container>`, verify health‑check, and check resource limits. |
| **Service not reachable** | Ensure firewall allows port, confirm DNS, and verify internal IP. |
| **Database connection fails** | Check credentials, network policy, and service health. |
## Security
| Question | Answer |
|----------|--------|
| **How to rotate SSH keys?** | Use the `ssh-keyrotator.sh` script in `scripts/`. |
| **Where are secrets stored?** | Hashicorp Vault – credentials never in repo. |
| **How to enable MFA?** | Enable on Authentik → Users → MFA. |
---
> **Got an issue not covered here?** Create an issue with the *documentation* label in the repo or ping me on Fluxer chat.

420
docs/getting-started/prerequisites.md Normal file
View File

@@ -0,0 +1,420 @@
# 📋 Prerequisites & Requirements
**🟢 Beginner-Friendly Guide**
Before diving into homelab services, let's make sure you have everything you need. This guide covers both the technical requirements and the knowledge you'll need to be successful.
## 🎯 Skill Level Assessment
### 🟢 **Absolute Beginner** ("What is a computer?")
**What you need to know:**
- How to use a computer and web browser
- Basic understanding that computers can run programs
- Willingness to follow step-by-step instructions
- Patience when things don't work the first time
**Recommended starting point:**
- Start with [What is a Homelab?](what-is-homelab.md)
- Follow the [Quick Start Guide](quick-start.md)
- Begin with 1-2 simple services
### 🟡 **Intermediate** (Some technical experience)
**What you should know:**
- Basic command line usage (cd, ls, mkdir)
- Understanding of files, directories, and permissions
- Familiarity with web browsers and URLs
- Basic networking concepts (IP addresses, ports)
**Recommended starting point:**
- Review [Architecture Overview](architecture.md)
- Explore [Service Categories](../services/categories.md)
- Try deploying 5-10 services
### 🔴 **Advanced** (IT professional/enthusiast)
**What you should know:**
- Docker and REDACTED_APP_PASSWORD concepts
- Linux system administration
- Networking and security principles
- Infrastructure as Code concepts
**Recommended starting point:**
- Dive into [Deployment Guide](../admin/deployment.md)
- Explore [Advanced Topics](../advanced/ansible.md)
- Consider the full infrastructure
---
## 💻 Hardware Requirements
### 🏠 **Minimum Setup** ($100-500)
Perfect for beginners and basic services.
**Option 1: Raspberry Pi**
- **Device**: Raspberry Pi 4 (4GB RAM minimum)
- **Storage**: 64GB microSD + 1TB USB SSD
- **Network**: Ethernet connection
- **Power**: Official Pi power supply
- **Services**: 5-10 lightweight services
**Option 2: Old Laptop/Desktop**
- **CPU**: Any dual-core processor from last 10 years
- **RAM**: 4GB minimum, 8GB recommended
- **Storage**: 100GB available space
- **Network**: Ethernet or stable WiFi
- **OS**: Ubuntu, Debian, or similar Linux
### 🏢 **Intermediate Setup** ($500-2000)
Good for most homelab enthusiasts.
**Option 1: Mini PC (Intel NUC, etc.)**
- **CPU**: Intel i5 or AMD Ryzen 5
- **RAM**: 16GB DDR4
- **Storage**: 512GB NVMe SSD + 2TB HDD
- **Network**: Gigabit Ethernet
- **Services**: 20-50 services
**Option 2: Synology NAS**
- **Model**: DS920+ or similar 4-bay
- **RAM**: 8GB (upgraded from 4GB)
- **Storage**: 4x 4TB WD Red drives
- **Network**: Gigabit Ethernet
- **Services**: 30-60 services
### 🏭 **Advanced Setup** ($2000+)
For serious homelabbers and learning environments.
**Server Hardware**
- **CPU**: Intel Xeon or AMD EPYC
- **RAM**: 32GB+ ECC memory
- **Storage**: Multiple SSDs + HDDs in RAID
- **Network**: 10 Gigabit Ethernet
- **Redundancy**: UPS, multiple hosts
- **Services**: 100+ services
---
## 🌐 Network Requirements
### 🔌 **Basic Networking**
**Essential:**
- **Internet connection**: Stable broadband (25+ Mbps)
- **Router**: Any modern router with Ethernet ports
- **Ethernet cable**: Cat5e or Cat6 for server connection
- **Local network**: 192.168.x.x or 10.x.x.x range
**Recommended:**
- **Gigabit network**: For better performance
- **Static IP**: For your server (or DHCP reservation)
- **Port forwarding**: If you want external access
### 🛡️ **Security Considerations**
**Firewall:**
- Router firewall enabled
- Only open necessary ports
- Use VPN for remote access
**Network Segmentation:**
- Separate IoT devices if possible
- Consider VLAN setup for advanced users
- Monitor network traffic
---
## 🛠️ Software Requirements
### 🐧 **Operating System**
**🟢 Recommended for Beginners:**
- **Ubuntu Server 22.04 LTS**: Most compatible, lots of documentation
- **Debian 12**: Stable, lightweight, similar to Ubuntu
- **Raspberry Pi OS**: If using Raspberry Pi
**🟡 Intermediate Options:**
- **CentOS Stream/Rocky Linux**: Enterprise-focused
- **Proxmox VE**: For virtualization
- **Synology DSM**: If using Synology NAS
**🔴 Advanced Options:**
- **Arch Linux**: Cutting-edge, requires expertise
- **FreeBSD**: Different approach, advanced users only
- **Kubernetes**: Container orchestration platform
### 🐳 **Docker & Container Runtime**
**Required Software:**
```bash
# Docker Engine (container runtime)
sudo apt install docker.io
# Docker Compose (multi-container applications)
sudo apt install docker-compose
# Optional: Portainer (web-based Docker management)
# We'll install this as a service
```
**Version Requirements:**
- **Docker**: 20.10+ (latest stable recommended)
- **Docker Compose**: 2.0+ (v2 syntax)
- **Python**: 3.8+ (for Ansible automation)
### 📝 **Text Editor**
You'll need to edit configuration files:
**🟢 Beginner-friendly:**
- **nano**: Simple, built into most Linux systems
- **VS Code**: If you prefer graphical editors
**🟡 Intermediate:**
- **vim**: Powerful but has learning curve
- **emacs**: Alternative to vim
---
## 🧠 Knowledge Requirements
### 📚 **Essential Concepts**
**🟢 Must Know:**
- **Files and directories**: How to navigate file systems
- **Text editing**: How to edit configuration files
- **Copy and paste**: How to copy commands and configurations
- **Web browsers**: How to access web interfaces
**🟡 Should Know:**
- **Command line basics**: cd, ls, mkdir, cp, mv, rm
- **File permissions**: chmod, chown concepts
- **Process management**: How to start/stop services
- **Basic networking**: IP addresses, ports, DNS
**🔴 Advanced:**
- **Docker concepts**: Images, containers, volumes, networks
- **Linux administration**: Users, groups, systemd, logs
- **Networking**: VLANs, firewalls, reverse proxies
- **Security**: SSL/TLS, authentication, authorization
### 🔧 **Command Line Comfort Level**
**🟢 Beginner Commands:**
```bash
# Navigate directories
cd /home/user
ls -la
pwd
# Create and edit files
mkdir my-folder
nano config.txt
cp file1.txt file2.txt
# View file contents
cat file.txt
less file.txt
```
**🟡 Intermediate Commands:**
```bash
# File permissions
chmod 755 script.sh
chown user:group file.txt
# Process management
ps aux | grep docker
sudo systemctl status docker
# Network troubleshooting
ping google.com
curl http://localhost:8080
```
**🔴 Advanced Commands:**
```bash
# Docker management
docker ps -a
docker logs container-name
docker exec -it container /bin/bash
# System monitoring
htop
df -h
netstat -tulpn
```
---
## 🔐 Security Knowledge
### 🛡️ **Basic Security Concepts**
**🟢 Essential:**
- **Strong passwords**: Use unique, complex passwords
- **Software updates**: Keep systems updated
- **Firewall basics**: Block unnecessary ports
- **Backup importance**: Regular data backups
**🟡 Intermediate:**
- **SSH keys**: Public/private key authentication
- **SSL/TLS**: HTTPS and certificate management
- **VPN usage**: Secure remote access
- **Network segmentation**: Isolate services
**🔴 Advanced:**
- **Container security**: User namespaces, capabilities
- **Network security**: IDS/IPS, monitoring
- **Compliance**: GDPR, data protection
- **Incident response**: Handling security breaches
---
## 🛠️ Tools You'll Need
### 💻 **On Your Computer**
**🟢 Essential:**
- **SSH client**: PuTTY (Windows) or built-in (Mac/Linux)
- **Text editor**: VS Code, Notepad++, or similar
- **Web browser**: Chrome, Firefox, or similar
- **File transfer**: SCP, SFTP, or WinSCP
**🟡 Helpful:**
- **Git client**: For version control
- **Network scanner**: Nmap, Advanced IP Scanner
- **Terminal emulator**: Windows Terminal, iTerm2
- **Documentation**: Obsidian, Notion, or similar
### 📱 **Mobile Apps** (Optional)
- **SSH client**: Termius, JuiceSSH
- **Network scanner**: Fing, Network Analyzer
- **Password manager**: Bitwarden, 1Password
- **Monitoring**: Uptime Kuma mobile, Grafana mobile
---
## 💰 Cost Breakdown
### 💵 **Initial Investment**
**🟢 Budget Setup ($100-300):**
- Raspberry Pi 4 (8GB): $75
- MicroSD card (64GB): $15
- USB SSD (1TB): $80
- Power supply & case: $30
- **Total**: ~$200
**🟡 Intermediate Setup ($500-1500):**
- Mini PC (Intel NUC): $400-800
- RAM upgrade (16GB): $100
- Storage (SSD + HDD): $200-400
- Network equipment: $100-200
- **Total**: ~$800-1500
**🔴 Advanced Setup ($2000+):**
- Server hardware: $1000-3000
- Storage array: $500-2000
- Network equipment: $300-1000
- UPS and accessories: $200-500
- **Total**: $2000-6500+
### 💡 **Ongoing Costs**
- **Electricity**: $50-200/year depending on hardware
- **Internet**: Existing broadband (no additional cost)
- **Domain names**: $10-50/year (optional)
- **Cloud backup**: $5-50/month (optional)
- **Hardware replacement**: Budget 10-20% annually
---
## ⏰ Time Investment
### 🕐 **Learning Phase** (1-3 months)
- **Week 1-2**: Basic concepts, first service deployment
- **Week 3-4**: Understanding Docker, networking basics
- **Month 2**: Deploy 5-10 services, learn troubleshooting
- **Month 3**: Advanced services, automation basics
### 🕑 **Ongoing Maintenance**
- **Daily**: 5-10 minutes checking alerts/status
- **Weekly**: 30-60 minutes reviewing logs, updates
- **Monthly**: 2-4 hours major updates, new services
- **Quarterly**: Half day for major maintenance
### 🕒 **Project Time Estimates**
- **First service**: 2-4 hours
- **Basic monitoring setup**: 4-8 hours
- **Media server (Plex/Jellyfin)**: 6-12 hours
- **Full homelab (20+ services)**: 40-80 hours
- **Advanced automation**: 20-40 hours
---
## ✅ Readiness Checklist
### 🎯 **Before Starting**
- [ ] Hardware selected and purchased
- [ ] Operating system installed and updated
- [ ] Network connectivity verified
- [ ] Basic command line comfort achieved
- [ ] Docker and Docker Compose installed
- [ ] SSH access configured
- [ ] Backup strategy planned
- [ ] Time allocated for learning
### 🎯 **Knowledge Check**
- [ ] Can navigate command line (cd, ls, mkdir)
- [ ] Can edit text files (nano, vim, or GUI editor)
- [ ] Understand basic networking (IP, ports, DNS)
- [ ] Know how to copy/paste commands
- [ ] Comfortable with web browsers and URLs
- [ ] Understand importance of backups
- [ ] Have patience for troubleshooting
### 🎯 **Environment Check**
- [ ] Server/computer ready and accessible
- [ ] Network connection stable
- [ ] Firewall configured appropriately
- [ ] Storage space available (100GB+ recommended)
- [ ] Power supply reliable (UPS recommended)
- [ ] Documentation method chosen (notes, wiki, etc.)
---
## 🚀 Next Steps
### 🎯 **If You're Ready**
1. **[Quick Start Guide](quick-start.md)**: Deploy your first service
2. **[Architecture Overview](architecture.md)**: Understand the big picture
3. **[Service Categories](../services/categories.md)**: Explore available services
### 🎯 **If You Need More Preparation**
1. **[What is a Homelab?](what-is-homelab.md)**: Understand the concepts
2. **Linux tutorials**: Learn command line basics
3. **Docker tutorials**: Understand REDACTED_APP_PASSWORD
4. **Networking basics**: Learn about IP addresses and ports
### 📚 **Recommended Learning Resources**
- **Linux**: "Linux Command Line for Beginners"
- **Docker**: Official Docker documentation and tutorials
- **Networking**: "Networking for Dummies" or similar
- **YouTube**: Channels like TechnoTim, NetworkChuck, Craft Computing
---
## 💡 Final Tips
### ✅ **Success Strategies**
- **Start small**: Begin with 1-2 simple services
- **Document everything**: Keep notes on what you do
- **Join communities**: Reddit r/homelab, Discord servers
- **Be patient**: Learning takes time, mistakes are normal
- **Have fun**: This should be enjoyable, not stressful
### ⚠️ **Common Pitfalls to Avoid**
- **Over-engineering**: Don't build enterprise solutions for home use
- **Scope creep**: Resist adding "just one more service"
- **Neglecting backups**: Always have a backup strategy
- **Ignoring security**: Don't expose services without proper security
- **Perfectionism**: Good enough is often good enough
---
*Remember: Everyone starts somewhere. The most important prerequisite is curiosity and willingness to learn. You don't need to know everything before you start - you'll learn as you go!*

379
docs/getting-started/quick-start.md Normal file
View File

@@ -0,0 +1,379 @@
# 🚀 Quick Start Guide
**🟢 Beginner-Friendly**
Get up and running with your first homelab service in under 30 minutes! This guide will walk you through deploying a simple service using the established patterns from this homelab.
## 🎯 What We'll Build
We'll deploy **Uptime Kuma** - a simple, beginner-friendly monitoring tool that will:
- Monitor your other services
- Send you alerts when things go down
- Provide a beautiful dashboard
- Teach you the basic deployment patterns
## 📋 Prerequisites
### ✅ **What You Need**
- A computer running Linux (Ubuntu, Debian, or similar)
- Docker and Docker Compose installed
- Basic command line knowledge
- 30 minutes of time
### 🔧 **Install Docker (if needed)**
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Add your user to docker group
sudo usermod -aG docker $USER
# Install Docker Compose
sudo apt install docker-compose -y
# Verify installation
docker --version
docker-compose --version
```
## 📁 Step 1: Create Project Structure
```bash
# Create project directory
mkdir -p ~/homelab/monitoring
cd ~/homelab/monitoring
# Create the directory structure
mkdir -p uptime-kuma/data
```
## 📝 Step 2: Create Docker Compose File
Create the main configuration file:
```bash
cat > uptime-kuma/docker-compose.yml << 'EOF'
version: '3.9'
services:
uptime-kuma:
image: louislam/uptime-kuma:latest
container_name: Uptime-Kuma
hostname: uptime-kuma
# Security settings
security_opt:
- no-new-privileges:true
user: 1000:1000 # Adjust for your system
# Health check
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3001/api/status-page/heartbeat/default"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
# Restart policy
restart: on-failure:5
# Resource limits
deploy:
resources:
limits:
memory: 512M
cpus: '0.5'
# Port mapping
ports:
- "3001:3001"
# Data persistence
volumes:
- ./data:/app/data:rw
- /etc/localtime:/etc/localtime:ro
# Environment variables
environment:
- TZ=America/Los_Angeles # Change to your timezone
# Custom network
networks:
- monitoring-network
networks:
monitoring-network:
name: monitoring-network
ipam:
config:
- subnet: 192.168.100.0/24
EOF
```
## 🔧 Step 3: Configure Environment
Create an environment file for easy customization:
```bash
cat > uptime-kuma/.env << 'EOF'
# Timezone (change to your location)
TZ=America/Los_Angeles
# User ID and Group ID (run 'id' command to find yours)
PUID=1000
PGID=1000
# Port (change if 3001 is already in use)
PORT=3001
EOF
```
## 🚀 Step 4: Deploy the Service
```bash
# Navigate to the service directory
cd uptime-kuma
# Start the service
docker-compose up -d
# Check if it's running
docker-compose ps
# View logs
docker-compose logs -f
```
You should see output like:
```
uptime-kuma_1 | Welcome to Uptime Kuma
uptime-kuma_1 | Server is running on port 3001
```
## 🌐 Step 5: Access Your Service
1. **Open your web browser**
2. **Navigate to**: `http://your-server-ip:3001`
3. **Create admin account** on first visit
4. **Start monitoring services!**
## 🎯 Step 6: Add Your First Monitor
1. **Click "Add New Monitor"**
2. **Configure a basic HTTP monitor**:
- **Monitor Type**: HTTP(s)
- **Friendly Name**: Google
- **URL**: https://google.com
- **Heartbeat Interval**: 60 seconds
3. **Click "Save"**
Congratulations! You've deployed your first homelab service! 🎉
## 🔍 Understanding What We Built
### 📦 **Docker Compose Structure**
```yaml
# This tells Docker what version of compose syntax we're using
version: '3.9'
# Services section defines our containers
services:
uptime-kuma: # Service name
image: louislam/uptime-kuma # Docker image to use
container_name: Uptime-Kuma # Custom container name
ports: # Port mapping (host:container)
- "3001:3001"
volumes: # Data persistence
- ./data:/app/data:rw # Maps local ./data to container /app/data
environment: # Environment variables
- TZ=America/Los_Angeles
```
### 🔐 **Security Features**
- **no-new-privileges**: Prevents privilege escalation
- **User mapping**: Runs as non-root user
- **Resource limits**: Prevents resource exhaustion
- **Health checks**: Monitors service health
### 📊 **Monitoring Features**
- **Health checks**: Docker monitors the container
- **Restart policy**: Automatically restarts on failure
- **Logging**: All output captured by Docker
## 🎓 Next Steps - Expand Your Homelab
### 🟢 **Beginner Services** (Try Next)
1. **Pi-hole** - Block ads network-wide
```bash
# Copy the uptime-kuma pattern and adapt for Pi-hole
mkdir ~/homelab/pihole
# Use the Pi-hole configuration from Atlantis/pihole.yml
```
2. **Portainer** - Manage Docker containers with a web UI
```bash
mkdir ~/homelab/portainer
# Adapt the pattern for Portainer
```
3. **Nginx Proxy Manager** - Manage reverse proxy with SSL
```bash
mkdir ~/homelab/proxy
# Use the pattern from Atlantis/nginxproxymanager/
```
### 🟡 **Intermediate Services** (When Ready)
1. **Plex or Jellyfin** - Media streaming
2. **Vaultwarden** - Password manager
3. **Grafana + Prometheus** - Advanced monitoring
### 🔴 **Advanced Services** (For Later)
1. **GitLab** - Complete DevOps platform
2. **Home Assistant** - Smart home automation
3. **Matrix Synapse** - Decentralized chat
## 🛠️ Common Customizations
### 🔧 **Change the Port**
If port 3001 is already in use:
```yaml
ports:
- "3002:3001" # Use port 3002 instead
```
### 🔧 **Different Data Location**
To store data elsewhere:
```yaml
volumes:
- /home/user/uptime-data:/app/data:rw
```
### 🔧 **Add Resource Limits**
For a more powerful server:
```yaml
deploy:
resources:
limits:
memory: 1G
cpus: '1.0'
```
## 🚨 Troubleshooting
### ❌ **Service Won't Start**
```bash
# Check logs for errors
docker-compose logs
# Check if port is already in use
sudo netstat -tulpn | grep :3001
# Check file permissions
ls -la data/
```
### ❌ **Can't Access Web Interface**
```bash
# Check if container is running
docker ps
# Test internal connectivity
docker exec Uptime-Kuma curl http://localhost:3001
# Check firewall
sudo ufw status
sudo ufw allow 3001
```
### ❌ **Data Not Persisting**
```bash
# Check volume mount
docker inspect Uptime-Kuma | grep -A 10 Mounts
# Fix permissions
sudo chown -R 1000:1000 ./data
```
## 📚 Learning Resources
### 📖 **Understanding Docker**
- **Images**: Pre-built software packages
- **Containers**: Running instances of images
- **Volumes**: Persistent data storage
- **Networks**: How containers communicate
### 🔗 **Useful Commands**
```bash
# View running containers
docker ps
# View all containers (including stopped)
docker ps -a
# View container logs
docker logs container-name
# Execute command in container
docker exec -it container-name /bin/bash
# Stop and remove everything
docker-compose down
# Update and restart
docker-compose pull && docker-compose up -d
```
## 🎯 What You've Learned
✅ **Docker Compose basics**
✅ **Service deployment patterns**
✅ **Data persistence with volumes**
✅ **Network configuration**
✅ **Security best practices**
✅ **Health monitoring**
✅ **Troubleshooting basics**
## 🌐 External Access (Optional)
Once you have services running locally, you might want to access them from outside your home network:
### 🌟 **Option 1: Tailscale (Recommended)**
**Zero-config mesh VPN with split-brain DNS**
```bash
# Install Tailscale on your server and devices
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
# Access services using local hostnames from anywhere:
# http://your-server.vish.local:3001
```
📖 **[Complete Tailscale Setup Guide](../infrastructure/tailscale-setup-guide.md)**
### 🔧 **Option 2: Port Forwarding (Traditional)**
**Forward specific ports on your router**
- Forward port 3001 to your server's IP
- Access via: `http://your-external-ip:3001`
- ⚠️ **Security risk**: Exposes services directly to internet
📖 **[Port Forwarding Guide](../infrastructure/port-forwarding-guide.md)**
### 🛡️ **Security Recommendation**
**Use Tailscale for secure, easy access without exposing services to the internet!**
## 📋 Next Reading
- **[🌟 Tailscale Setup Guide](../infrastructure/tailscale-setup-guide.md)**: **RECOMMENDED** - Secure external access
- **[Architecture Overview](architecture.md)**: Understand how everything fits together
- **[Service Categories](../services/categories.md)**: Explore what services are available
- **[Deployment Guide](../admin/deployment.md)**: Learn advanced deployment patterns
- **[Common Issues](../troubleshooting/common-issues.md)**: Troubleshoot problems
---
**🎉 Congratulations!** You've successfully deployed your first homelab service using the same patterns used across all 176 services in this infrastructure. You're now ready to explore more complex services and build your own homelab empire!
*Remember: Every expert was once a beginner. Start small, learn continuously, and don't be afraid to break things - that's how you learn!*

520
docs/getting-started/shopping-guide.md Normal file
View File

@@ -0,0 +1,520 @@
# 🛒 Complete Homelab Shopping Guide
**💰 Budget-Conscious Recommendations with Real Prices**
This guide provides specific product recommendations with current pricing to help you build a homelab that matches your budget and needs. All prices are approximate and may vary by retailer and region.
## 💵 Budget Categories
### **🟢 Starter Budget: $500-800**
Perfect for beginners who want to dip their toes into homelabbing
### **🟡 Intermediate Budget: $1,500-2,500**
For enthusiasts ready to build a serious homelab
### **🔴 Advanced Budget: $3,000-5,000+**
For those who want enterprise-level features and performance
---
## 🟢 STARTER HOMELAB ($500-800)
### **Core Components**
#### **NAS: Synology DS220+ - $299**
```
✅ Perfect for beginners
✅ 2-bay design with room to grow
✅ Intel Celeron J4025 processor
✅ 2GB RAM (upgradeable to 6GB)
✅ Excellent software ecosystem
✅ 3-year warranty
Where to buy:
- Amazon: ~$299
- B&H Photo: ~$289
- Newegg: ~$309
- Best Buy: ~$299
```
#### **Storage: 2x WD Red 4TB - $89 each ($178 total)**
```
✅ NAS-optimized drives
✅ 3-year warranty
✅ 5,400 RPM (quiet operation)
✅ CMR technology
✅ RAID 1 = 4TB usable space
Alternative: Seagate IronWolf 4TB - $94 each
```
#### **Network: TP-Link Archer AX73 Router - $149**
```
✅ WiFi 6 support
✅ Gigabit Ethernet ports
✅ Good range and performance
✅ Easy setup and management
Budget alternative: TP-Link Archer A7 - $79
```
#### **Power Protection: APC Back-UPS 600VA - $79**
```
✅ 600VA/360W capacity
✅ 7 outlets (3 battery backup)
✅ USB connectivity for auto-shutdown
✅ 3-year warranty
Upgrade option: APC Back-UPS 1000VA - $129
```
#### **Accessories - $95**
```
- Cat6 Ethernet cables (5-pack): $25
- 8-port Gigabit switch: $35
- Cable management kit: $20
- USB drive for setup: $15
```
**STARTER TOTAL: $800**
### **Starter Setup Services**
```bash
# What you can run with this setup:
✅ Plex Media Server (movies, TV, music)
✅ File sharing and backup
✅ Synology Photos (Google Photos replacement)
✅ Basic Docker containers
✅ VPN server for remote access
✅ Download station for torrents
✅ Basic monitoring and alerts
```
---
## 🟡 INTERMEDIATE HOMELAB ($1,500-2,500)
### **Core Components**
#### **NAS: Synology DS723+ - $549**
```
✅ 2-bay plus design with expansion
✅ AMD Ryzen R1600 processor
✅ 2GB RAM (upgradeable to 32GB)
✅ 2x M.2 NVMe slots for cache
✅ PCIe expansion slot
✅ 10GbE ready
Alternative: DS920+ (4-bay) - $599
```
#### **Storage: 2x Seagate IronWolf Pro 8TB - $219 each ($438 total)**
```
✅ Pro series with 5-year warranty
✅ 7,200 RPM performance
✅ 256MB cache
✅ CMR technology
✅ RAID 1 = 8TB usable space
Future expansion: Add 2 more drives later
```
#### **SSD Cache: 2x WD Black SN750 SE 500GB - $45 each ($90 total)**
```
✅ NVMe M.2 2280 form factor
✅ PCIe Gen3 interface
✅ Read/write cache configuration
✅ Significant performance boost
Alternative: Samsung 980 500GB - $55 each
```
#### **RAM Upgrade: 16GB DDR4 SO-DIMM - $89**
```
✅ Crucial or Kingston compatible
✅ Brings total to 18GB RAM
✅ Better Docker performance
✅ More concurrent services
Max upgrade: 32GB kit - $179
```
#### **Network: 10GbE Upgrade - $299**
```
- TP-Link TL-SX1008 10GbE switch: $199
- 10GbE PCIe card for NAS: $100
Benefits:
✅ 10x faster file transfers
✅ Better multi-user performance
✅ Future-proof networking
```
#### **Compute: Intel NUC 11 - $449**
```
✅ Intel Core i5-1135G7
✅ 8GB RAM, 256GB SSD
✅ Compact form factor
✅ Low power consumption
✅ Perfect for additional services
Use cases:
- Home Assistant
- Pi-hole
- Additional Docker containers
- Development environment
```
#### **Power: APC Back-UPS 1500VA - $199**
```
✅ 1500VA/865W capacity
✅ 10 outlets (5 battery backup)
✅ LCD display
✅ Network management
✅ 3-year warranty
```
#### **Accessories - $150**
```
- Cat6a cables for 10GbE: $50
- Cable management: $30
- Labels and organization: $20
- Additional USB drives: $25
- Rack shelf (optional): $25
```
**INTERMEDIATE TOTAL: $2,263**
### **Intermediate Setup Services**
```bash
# What you can run with this setup:
✅ Everything from starter setup, plus:
✅ Home Assistant (smart home automation)
✅ Grafana + Prometheus (advanced monitoring)
✅ Nextcloud (complete Google Workspace replacement)
✅ Vaultwarden (password manager)
✅ Sonarr/Radarr (media automation)
✅ Multiple game servers
✅ Development environments
✅ Advanced networking (VLANs, VPN)
✅ 10GbE performance for large file transfers
```
---
## 🔴 ADVANCED HOMELAB ($3,000-5,000+)
### **Core Components**
#### **NAS: Synology DS1823xs+ - $1,199**
```
✅ 8-bay enterprise design
✅ AMD Ryzen V1780B processor
✅ 8GB RAM (upgradeable to 32GB)
✅ 2x M.2 NVMe slots
✅ 2x PCIe expansion slots
✅ Dual 10GbE ports built-in
✅ 5-year warranty
This is the same model as "Atlantis" in our setup
```
#### **Storage: 8x Seagate IronWolf Pro 16TB - $329 each ($2,632 total)**
```
✅ Enterprise-grade drives
✅ 7,200 RPM performance
✅ 256MB cache per drive
✅ 5-year warranty with rescue service
✅ SHR-2 = ~96TB usable space (2-drive fault tolerance)
Budget option: 8x 12TB drives = $1,999 total
```
#### **SSD Cache: 2x Samsung 980 PRO 1TB - $99 each ($198 total)**
```
✅ PCIe Gen4 NVMe
✅ Exceptional performance
✅ 1TB cache pool
✅ 5-year warranty
Alternative: Crucial P5 Plus 1TB - $89 each
```
#### **RAM: 32GB DDR4 ECC SO-DIMM Kit - $299**
```
✅ ECC memory for data integrity
✅ Maximum supported capacity
✅ Enterprise reliability
✅ Better for heavy workloads
```
#### **Network: Enterprise 10GbE Setup - $599**
```
- Ubiquiti UniFi Dream Machine Pro: $379
- 10GbE SFP+ switch: $220
Features:
✅ Professional network management
✅ Advanced security features
✅ VLAN support
✅ Enterprise monitoring
✅ Scalable architecture
```
#### **Compute: Dell OptiPlex 7090 Micro - $699**
```
✅ Intel Core i7-11700T
✅ 16GB RAM, 512GB SSD
✅ Ultra-compact design
✅ Enterprise reliability
✅ 3-year warranty
Use cases:
- Kubernetes cluster node
- CI/CD pipeline
- Development workstation
- High-performance services
```
#### **Power: APC Smart-UPS 2200VA - $449**
```
✅ 2200VA/1980W capacity
✅ Pure sine wave output
✅ Network management card
✅ Hot-swappable batteries
✅ Enterprise features
```
#### **Rack Infrastructure - $399**
```
- 12U wall-mount rack: $199
- Rack shelves and cable management: $100
- Professional patch panel: $100
Benefits:
✅ Professional appearance
✅ Better organization
✅ Improved cooling
✅ Easier maintenance
```
**ADVANCED TOTAL: $6,474**
### **Advanced Setup Services**
```bash
# What you can run with this setup:
✅ Everything from previous tiers, plus:
✅ Kubernetes cluster
✅ GitLab with CI/CD
✅ Multiple isolated environments
✅ High-availability services
✅ Enterprise monitoring stack
✅ Multiple game servers simultaneously
✅ Video transcoding farm
✅ Machine learning workloads
✅ Complete enterprise simulation
✅ Multi-tenant hosting
✅ Advanced security lab
```
---
## 🛍️ Where to Buy
### **Best Overall Prices**
```bash
# Electronics and Components:
1. Amazon - Best selection, fast shipping
2. Newegg - Good for computer components
3. B&H Photo - Professional equipment, no tax in most states
4. Micro Center - In-store pickup, excellent prices
5. Best Buy - Price matching, local pickup
# Hard Drives (Best Prices):
1. Amazon - Frequent sales
2. Newegg - Shell Shocker deals
3. B&H Photo - Professional pricing
4. ServerPartDeals - Refurbished enterprise drives
```
### **Synology Authorized Dealers**
```bash
# Recommended for warranty support:
- Amazon (sold by Amazon)
- B&H Photo
- CDW
- Insight
- Synology Direct
⚠️ Avoid: eBay, unknown sellers (warranty issues)
```
---
## 📅 When to Buy (Best Deals)
### **Annual Sale Events**
```bash
# Best times for homelab shopping:
- Black Friday/Cyber Monday (November)
- Amazon Prime Day (July)
- Back-to-School sales (August)
- End of fiscal year (March/September)
- CES aftermath (February)
```
### **Seasonal Patterns**
```bash
# Hard Drives: Best prices in Q1 (January-March)
# NAS Units: Best deals during Black Friday
# Network Equipment: Good deals year-round
# UPS Systems: Best prices in summer (low demand)
```
---
## 💡 Money-Saving Tips
### **Buy Smart**
```bash
✅ Start with 2-bay NAS, expand later
✅ Buy drives in pairs for RAID
✅ Consider refurbished enterprise equipment
✅ Join r/buildapcsales for deal alerts
✅ Use price tracking tools (CamelCamelCamel)
✅ Consider previous-generation hardware
```
### **Avoid These Mistakes**
```bash
❌ Buying consumer drives for NAS use
❌ Skipping UPS (data corruption risk)
❌ Undersizing power requirements
❌ Buying too small initially (expensive to upgrade)
❌ Ignoring warranty terms
❌ Mixing drive brands/models in RAID
```
---
## 🔧 DIY vs Pre-Built
### **DIY Advantages**
```bash
✅ Lower cost
✅ Custom configuration
✅ Learning experience
✅ Upgrade flexibility
✅ Component choice
```
### **Pre-Built Advantages**
```bash
✅ Warranty coverage
✅ Professional support
✅ Tested compatibility
✅ Time savings
✅ Reliability guarantee
```
### **Recommendation**
For beginners: **Start with pre-built NAS** (Synology/QNAP)
For experienced users: **DIY for compute, pre-built for storage**
---
## 📊 Cost Comparison
### **5-Year Total Cost of Ownership**
#### **Starter Setup**
```bash
Initial cost: $800
Power consumption: ~$50/year
Upgrades: ~$200 over 5 years
Total 5-year cost: $1,300
```
#### **Intermediate Setup**
```bash
Initial cost: $2,263
Power consumption: ~$120/year
Upgrades: ~$500 over 5 years
Total 5-year cost: $3,363
```
#### **Advanced Setup**
```bash
Initial cost: $6,474
Power consumption: ~$200/year
Upgrades: ~$1,000 over 5 years
Total 5-year cost: $8,474
```
### **Cloud Service Comparison**
```bash
# Equivalent cloud services (5 years):
Google Workspace Business: $3,600
Dropbox Business: $3,000
Netflix + Spotify + iCloud: $1,800
Total cloud equivalent: $8,400
# Advanced homelab provides MORE features for similar cost!
```
---
## 🎯 Recommended Upgrade Path
### **Year 1: Foundation**
- Start with Starter or Intermediate setup
- Focus on learning and basic services
- Establish backup routines
### **Year 2: Expansion**
- Add more storage drives
- Upgrade network to 10GbE
- Add compute nodes
### **Year 3: Optimization**
- Implement monitoring and automation
- Add redundancy and high availability
- Optimize power and cooling
### **Year 4-5: Advanced Features**
- Kubernetes and container orchestration
- Advanced networking (VLANs, segmentation)
- Enterprise-level services
---
## 📞 Support and Warranty
### **Warranty Recommendations**
```bash
# Essential warranty coverage:
✅ NAS: Minimum 2-year, prefer 3-5 year
✅ Hard drives: Minimum 3-year, prefer 5-year
✅ Network equipment: 1-year minimum
✅ UPS: 2-3 year coverage
✅ Compute: 1-3 year depending on use
```
### **Extended Warranty Worth It?**
```bash
# Generally worth it for:
✅ NAS units (complex electronics)
✅ Enterprise hard drives (high usage)
✅ UPS systems (battery replacement)
# Usually not worth it for:
❌ Network switches (reliable)
❌ Cables and accessories
❌ Consumer SSDs (reliable, cheap to replace)
```
---
**🛒 Happy Shopping!** Remember to start with your budget and needs, then grow your homelab over time. The homelab journey is a marathon, not a sprint. Buy quality components that will serve you well for years to come.
**💰 Pro Tip**: Set up price alerts for your wishlist items and be patient. Good deals come to those who wait, and you can often save 20-30% by timing your purchases right.

163
docs/getting-started/what-is-homelab.md Normal file
View File

@@ -0,0 +1,163 @@
# 🏠 What is a Homelab?
**🟢 Beginner-Friendly Guide**
## 🤔 The Simple Answer
A **homelab** is like having your own personal internet inside your home. Instead of relying on big companies like Google, Netflix, or Dropbox for all your digital needs, you run your own versions of these services on computers you own and control.
Think of it as building your own digital ecosystem where you are the boss!
## 🌟 Why Would Someone Want a Homelab?
### 🔒 **Privacy & Control**
- **Your data stays home**: Photos, documents, and personal information never leave your house
- **No monthly subscriptions**: Stop paying for cloud storage, streaming services, or productivity apps
- **You decide the rules**: No terms of service changes, no sudden price increases
### 🎓 **Learning & Skills**
- **Hands-on experience**: Learn about networking, servers, and system administration
- **Career development**: Gain valuable IT skills that employers love
- **Problem-solving**: Become more tech-savvy and self-reliant
### 🚀 **Cool Features**
- **Custom solutions**: Build exactly what you need, not what companies think you need
- **Integration**: Make all your services work together seamlessly
- **Performance**: Often faster than cloud services since everything is local
## 🏗️ What's Actually in a Homelab?
### 🖥️ **The Hardware (Physical Stuff)**
Think of these as the "buildings" where your digital services live:
- **Computers/Servers**: Can be anything from a Raspberry Pi ($35) to enterprise servers
- **Network Equipment**: Routers, switches, and cables to connect everything
- **Storage**: Hard drives and SSDs to store your data
### 🐳 **The Software (Digital Stuff)**
These are the "tenants" living in your buildings:
- **Operating System**: Usually Linux (like Ubuntu) - the foundation everything runs on
- **Docker**: Think of it as apartment buildings for software - keeps everything organized
- **Services**: The actual applications (like your own Netflix, Google Drive, etc.)
## 🏠 What's in Vish's Homelab?
This particular homelab is quite extensive with **176 different services** running across **13 different computers**! Here are some examples:
### 🎬 **Media & Entertainment**
- **Plex/Jellyfin**: Your own Netflix with your movie/TV collection
- **Immich**: Your own Google Photos for storing and organizing pictures
- **Navidrome**: Your own Spotify for your music collection
### 💼 **Productivity**
- **Paperless-NGX**: Scan and organize all your documents digitally
- **Firefly III**: Track your finances and budgets
- **Joplin**: Take notes and organize your thoughts
### 🔧 **Development & Tech**
- **GitLab**: Store and manage code projects
- **Grafana**: Create beautiful dashboards to monitor everything
- **Portainer**: Easy web interface to manage all your services
### 🛡️ **Security & Privacy**
- **Vaultwarden**: Your own password manager (like LastPass)
- **Pi-hole**: Block ads and trackers across your entire network
- **Wireguard**: Secure VPN to access your homelab from anywhere
## 🤷 "But I'm Not Technical..."
**That's okay!** Everyone starts somewhere. Here's what you actually need to know:
### 🎯 **Absolute Minimum Knowledge**
- How to use a computer and web browser
- Basic understanding that computers can run programs
- Willingness to follow step-by-step instructions
- Patience when things don't work the first time
### 📚 **You'll Learn Along the Way**
- Basic command line usage (typing commands instead of clicking)
- How networks work (how computers talk to each other)
- What Docker is and why it's useful
- How to read error messages and troubleshoot problems
## 💰 How Much Does This Cost?
### 🏠 **Starter Homelab ($100-500)**
- Raspberry Pi 4 or old laptop
- Basic network setup
- A few essential services
### 🏢 **Intermediate Homelab ($500-2000)**
- Dedicated mini PC or NAS
- Better networking equipment
- More storage and services
### 🏭 **Advanced Homelab ($2000+)**
- Multiple servers
- Enterprise networking
- Redundancy and high availability
**💡 Tip**: Start small! You can begin with a $35 Raspberry Pi and grow from there.
## 🚦 Getting Started Path
### 1️⃣ **Learn the Basics** (1-2 weeks)
- Read through this documentation
- Watch some YouTube videos about homelabs
- Understand what Docker containers are
### 2️⃣ **Start Simple** (1-2 weeks)
- Get a Raspberry Pi or use an old computer
- Install a basic Linux operating system
- Run your first Docker container
### 3️⃣ **Add Services Gradually** (ongoing)
- Start with one service you actually need
- Get it working properly before adding more
- Learn from each service you deploy
### 4️⃣ **Expand and Improve** (ongoing)
- Add more hardware as needed
- Implement monitoring and backups
- Automate common tasks
## ⚠️ Important Warnings
### 🔥 **This Can Be Addictive**
- You might find yourself constantly wanting to add "just one more service"
- Your electricity bill might increase
- You might start looking at server racks on eBay
### 🕰️ **Time Investment**
- Initial setup takes time and patience
- Things will break and need fixing
- Learning curve can be steep at first
### 💸 **Cost Creep**
- "I just need one more hard drive..."
- "This server would be perfect for..."
- "Maybe I should upgrade the network..."
## 🎉 The Payoff
Despite the challenges, running a homelab is incredibly rewarding:
- **Independence**: You control your own digital life
- **Skills**: You become significantly more tech-savvy
- **Satisfaction**: There's nothing quite like building something yourself
- **Community**: The homelab community is welcoming and helpful
## 🔗 Next Steps
Ready to dive deeper? Check out:
1. [Prerequisites](prerequisites.md) - What you need before starting
2. [Architecture Overview](architecture.md) - How this homelab is organized
3. [Quick Start Guide](quick-start.md) - Your first steps
Remember: **Everyone's homelab journey is different**. Don't feel pressured to replicate everything you see here. Start with what interests you and build from there!
---
*"The best time to start a homelab was 10 years ago. The second best time is now."*