# ☸️ Kubernetes Cluster Setup Guide **πŸ”΄ Advanced Guide** This guide covers deploying and managing a production-ready Kubernetes cluster in your homelab, including high availability, storage, networking, and service deployment. ## 🎯 Kubernetes Architecture for Homelab ### **Cluster Design** ```bash # Recommended cluster topology: # Control Plane Nodes (3 nodes for HA) k8s-master-01: 192.168.10.201 (Concord-NUC) k8s-master-02: 192.168.10.202 (Homelab-VM) k8s-master-03: 192.168.10.203 (Chicago-VM) # Worker Nodes (3+ nodes) k8s-worker-01: 192.168.10.211 (Bulgaria-VM) k8s-worker-02: 192.168.10.212 (Guava) k8s-worker-03: 192.168.10.213 (Setillo) # Storage Nodes (Ceph/Longhorn) k8s-storage-01: 192.168.10.221 (Atlantis) k8s-storage-02: 192.168.10.222 (Calypso) k8s-storage-03: 192.168.10.223 (Anubis) ``` ### **Resource Requirements** ```bash # Control Plane Nodes (minimum) CPU: 2 cores RAM: 4 GB Storage: 50 GB SSD Network: 1 Gbps # Worker Nodes (minimum) CPU: 4 cores RAM: 8 GB Storage: 100 GB SSD Network: 1 Gbps # Storage Nodes (recommended) CPU: 4 cores RAM: 16 GB Storage: 500 GB+ SSD + additional storage Network: 10 Gbps (if available) ``` --- ## πŸš€ Cluster Installation ### **Method 1: kubeadm (Recommended for Learning)** #### **Prerequisites on All Nodes** ```bash # Update system sudo apt update && sudo apt upgrade -y # Install required packages sudo apt install -y apt-transport-https ca-certificates curl gpg # Disable swap sudo swapoff -a sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab # Load kernel modules cat < homelab-app/values.yaml replicaCount: 1 image: repository: nginx tag: latest pullPolicy: IfNotPresent service: type: ClusterIP port: 80 ingress: enabled: true className: nginx annotations: cert-manager.io/cluster-issuer: letsencrypt-prod hosts: - host: app.k8s.vish.local paths: - path: / pathType: Prefix tls: - secretName: app-tls hosts: - app.k8s.vish.local persistence: enabled: true storageClass: longhorn-fast size: 10Gi resources: limits: cpu: 500m memory: 512Mi requests: cpu: 100m memory: 128Mi EOF # Install chart helm install my-app ./homelab-app ``` --- ## πŸ”’ Security Configuration ### **Pod Security Standards** ```bash # Create Pod Security Policy cat < /usr/local/bin/etcd-backup.sh #!/bin/bash ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-snapshot-\$(date +%Y%m%d-%H%M%S).db \ --endpoints=https://127.0.0.1:2379 \ --cacert=/etc/kubernetes/pki/etcd/ca.crt \ --cert=/etc/kubernetes/pki/etcd/server.crt \ --key=/etc/kubernetes/pki/etcd/server.key # Keep only last 7 days of backups find /backup -name "etcd-snapshot-*.db" -mtime +7 -delete EOF chmod +x /usr/local/bin/etcd-backup.sh # Schedule daily backups echo "0 2 * * * /usr/local/bin/etcd-backup.sh" | crontab - ``` #### **Velero for Application Backup** ```bash # Install Velero CLI wget https://github.com/vmware-tanzu/velero/releases/latest/download/velero-linux-amd64.tar.gz tar -xzf velero-linux-amd64.tar.gz sudo mv velero-*/velero /usr/local/bin/ # Install Velero server (using MinIO for storage) velero install \ --provider aws \ --plugins velero/velero-plugin-for-aws:v1.8.0 \ --bucket velero-backups \ --secret-file ./credentials-velero \ --use-volume-snapshots=false \ --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.vish.local:9000 # Create backup schedule velero schedule create daily-backup --schedule="0 1 * * *" ``` ### **Cluster Upgrades** ```bash # Upgrade control plane nodes (one at a time) # 1. Drain node kubectl drain k8s-master-01 --ignore-daemonsets --delete-emptydir-data # 2. Upgrade kubeadm sudo apt update sudo apt-mark unhold kubeadm sudo apt install kubeadm=1.29.x-00 sudo apt-mark hold kubeadm # 3. Upgrade cluster sudo kubeadm upgrade plan sudo kubeadm upgrade apply v1.29.x # 4. Upgrade kubelet and kubectl sudo apt-mark unhold kubelet kubectl sudo apt install kubelet=1.29.x-00 kubectl=1.29.x-00 sudo apt-mark hold kubelet kubectl sudo systemctl daemon-reload sudo systemctl restart kubelet # 5. Uncordon node kubectl uncordon k8s-master-01 # Repeat for other control plane nodes and workers ``` ### **Troubleshooting** ```bash # Common troubleshooting commands kubectl get nodes -o wide kubectl get pods --all-namespaces kubectl describe node NODE_NAME kubectl logs -n kube-system POD_NAME # Check cluster health kubectl get componentstatuses kubectl cluster-info kubectl get events --sort-by=.metadata.creationTimestamp # Debug networking kubectl run debug --image=nicolaka/netshoot -it --rm -- /bin/bash ``` --- ## πŸ“‹ Migration Strategy ### **Phase 1: Cluster Setup** ```bash ☐ Plan cluster architecture and resource allocation ☐ Install Kubernetes on all nodes ☐ Configure networking and storage ☐ Install monitoring and logging ☐ Set up backup and disaster recovery ☐ Configure security policies ☐ Test cluster functionality ``` ### **Phase 2: Service Migration** ```bash ☐ Identify services suitable for Kubernetes ☐ Convert Docker Compose to Kubernetes manifests ☐ Create Helm charts for complex applications ☐ Set up ingress and SSL certificates ☐ Configure persistent storage ☐ Test service functionality ☐ Update DNS and load balancing ``` ### **Phase 3: Production Cutover** ```bash ☐ Migrate non-critical services first ☐ Update monitoring and alerting ☐ Test disaster recovery procedures ☐ Migrate critical services during maintenance window ☐ Update documentation and runbooks ☐ Train team on Kubernetes operations ☐ Decommission old Docker Compose services ``` --- ## πŸ”— Related Documentation - [Network Architecture](networking.md) - Network design and VLANs for Kubernetes - [Ubiquiti Enterprise Setup](ubiquiti-enterprise-setup.md) - Enterprise networking for cluster infrastructure - [Laptop Travel Setup](laptop-travel-setup.md) - Remote access to Kubernetes cluster - [Tailscale Setup Guide](tailscale-setup-guide.md) - VPN access to cluster services - [Disaster Recovery Guide](../troubleshooting/disaster-recovery.md) - Cluster backup and recovery - [Security Model](security.md) - Security architecture and policies --- **πŸ’‘ Pro Tip**: Start with a small, non-critical service migration to Kubernetes. Learn the platform gradually before moving mission-critical services. Kubernetes has a steep learning curve, but the benefits of container orchestration, scaling, and management are worth the investment for a growing homelab!