Sanitized mirror from private repository - 2026-04-19 08:46:29 UTC

OPERATIONAL_STATUS.md

@@ -0,0 +1,167 @@
+# 📊 Operational Status Report
+
+*Current status of all homelab services and infrastructure*
+
+## 🎯 Executive Summary
+
+**Infrastructure Health**: ✅ **OPERATIONAL**  
+**Total Services**: 50+ containers across 5 hosts  
+**GitOps Status**: ✅ **ACTIVE** - 2 managed stacks  
+**Monitoring**: ✅ **ONLINE** - Full observability stack  
+**Last Updated**: February 24, 2026
+
+## 🖥️ Host Status
+
+### Primary Infrastructure
+| Host | Status | Services | CPU | Memory | Storage |
+|------|--------|----------|-----|--------|---------|
+| **Atlantis** (DS1823xs+) | 🟢 Online | 50+ | 8 cores | 31.3 GB | Primary NAS |
+| **Calypso** (DS723+) | 🟢 Online | 46 | 4 cores | 31.3 GB | Secondary NAS |
+| **Concord NUC** | 🟢 Online | 17 | 4 cores | 15.5 GB | Edge Computing |
+| **Homelab VM** | 🟢 Online | 23 | 4 cores | 28.7 GB | Cloud Services |
+| **Raspberry Pi 5** | 🟢 Online | 4 | 4 cores | 15.8 GB | IoT/Edge |
+
+### Gaming Infrastructure
+| Service | Status | Location | Players | Uptime |
+|---------|--------|----------|---------|--------|
+| **Minecraft Server** | 🟢 Online | Port 25565 | Active | 99.9% |
+| **Garry's Mod** | 🟢 Online | Port 27015 | Active | 99.5% |
+| **PufferPanel** | 🟢 Online | Port 8080 | Management | 100% |
+| **Stoat Chat** | 🟢 Online | st.vish.gg | Community | 99.8% |
+
+## 🚀 GitOps Deployment Status
+
+### Active Stacks
+- **Stack Count**: 2 active GitOps deployments
+- **Repository**: https://git.vish.gg/Vish/homelab.git
+- **Sync Status**: ✅ Synchronized
+- **Last Deployment**: Automatic sync enabled
+
+### Deployment Health
+- **Success Rate**: 100% successful deployments
+- **Average Deploy Time**: < 2 minutes
+- **Rollback Capability**: ✅ Available
+- **Webhook Integration**: ✅ Configured
+
+## 📊 Service Categories
+
+### Media & Entertainment
+- **Plex Media Server** - ✅ Online - Primary streaming
+- **Jellyfin** - ✅ Online - Alternative media server
+- **Sonarr/Radarr/Lidarr** - ✅ Online - Media automation
+- **Jellyseerr** - ✅ Online - Request management
+- **Tautulli** - ✅ Online - Plex analytics
+
+### Development & DevOps
+- **Gitea** - ✅ Online - Git repositories
+- **Portainer** - ✅ Online - Container management
+- **Grafana** - ✅ Online - Metrics visualization
+- **Prometheus** - ✅ Online - Metrics collection
+- **Watchtower** - ✅ Online - Auto-updates
+
+### Productivity & Storage
+- **Immich** - ✅ Online - Photo management
+- **PaperlessNGX** - ✅ Online - Document management
+- **Syncthing** - ✅ Online - File synchronization
+- **Nextcloud** - ✅ Online - Cloud storage
+
+### Network & Infrastructure
+- **AdGuard Home** - ✅ Online - DNS filtering
+- **Nginx Proxy Manager** - ✅ Online - Reverse proxy
+- **Authentik** - ✅ Online - SSO provider
+- **Tailscale** - ✅ Online - Mesh VPN
+
+## 🔍 Monitoring & Observability
+
+### Monitoring Stack
+- **Grafana Dashboard**: https://gf.vish.gg
+- **Prometheus Metrics**: ✅ Collecting
+- **Alert Manager**: ✅ Configured
+- **SNMP Monitoring**: ✅ Synology devices
+- **Container Health**: ✅ All services monitored
+
+### Key Metrics
+- **System Uptime**: 99.9% average
+- **Response Time**: < 100ms average
+- **Storage Usage**: Monitored across all hosts
+- **Network Performance**: Optimal
+
+## 🔐 Security Status
+
+### Access Control
+- **SSH Security**: ✅ Key-based authentication
+- **Firewall**: ✅ UFW configured with rate limiting
+- **VPN Access**: ✅ Tailscale mesh network
+- **SSL/TLS**: ✅ Let's Encrypt certificates
+- **SSO Integration**: ✅ Authentik for service auth
+
+### Security Monitoring
+- **Fail2ban**: ✅ Active intrusion prevention
+- **Log Monitoring**: ✅ Centralized logging
+- **Vulnerability Scanning**: ✅ Regular updates
+- **Backup Verification**: ✅ Automated testing
+
+## 🎮 Gaming Services
+
+### Game Servers
+- **Minecraft**: Java Edition, latest version, custom modpack
+- **Garry's Mod**: Sandbox/DarkRP modes, custom addons
+- **Management**: PufferPanel web interface for both servers
+
+### Communication
+- **Stoat Chat**: Self-hosted Revolt instance with voice/video
+- **Features**: Custom branding, LiveKit integration
+- **Community**: Active user base with gaming coordination
+
+## 🔄 Backup & Recovery
+
+### Backup Status
+- **Schedule**: Daily incremental, weekly full backups
+- **Storage**: Multiple locations (local + cloud)
+- **Verification**: ✅ Automated backup testing
+- **Retention**: 30 days incremental, 12 months full
+
+### Disaster Recovery
+- **RTO**: < 4 hours for critical services
+- **RPO**: < 24 hours maximum data loss
+- **Testing**: Monthly DR drills performed
+- **Documentation**: Complete recovery procedures
+
+## 📈 Performance Metrics
+
+### Resource Utilization
+- **CPU Usage**: 15-30% average across hosts
+- **Memory Usage**: 60-80% average utilization
+- **Storage**: Adequate capacity with monitoring
+- **Network**: Optimal performance on gigabit
+
+### Service Response Times
+- **Web Services**: < 200ms average response
+- **API Endpoints**: < 100ms average response
+- **Database Queries**: < 50ms average
+- **File Access**: < 10ms local network
+
+## 🚨 Recent Issues & Resolutions
+
+### Resolved Issues
+- **Watchtower Deployment**: ✅ Fixed notification system
+- **Monitoring Dashboards**: ✅ Fixed template variables
+- **GitOps Sync**: ✅ Improved webhook reliability
+
+### Ongoing Maintenance
+- **Security Updates**: Regular patching schedule
+- **Performance Optimization**: Continuous monitoring
+- **Capacity Planning**: Proactive resource management
+
+## 📞 Support & Contact
+
+- **Repository**: [git.vish.gg/Vish/homelab](https://git.vish.gg/Vish/homelab)
+- **Issues**: Repository issue tracker
+- **Chat**: Stoat chat community (st.vish.gg)
+- **Emergency**: SSH access available for critical issues
+
+---
+
+**Report Generated**: February 24, 2026  
+**Next Review**: March 1, 2026  
+**Overall Status**: ✅ **HEALTHY** - All systems operational