Sanitized mirror from private repository - 2026-04-19 08:15:48 UTC

2026-04-19 08:15:48 +00:00
commit 57b1fe47f2
1437 changed files with 363051 additions and 0 deletions
--- a/docs/services/individual/grafana.md
+++ b/docs/services/individual/grafana.md
@@ -0,0 +1,153 @@
+# Grafana
+
+**Monitoring Service**
+
+## Service Overview
+
+| Property | Value |
+|----------|-------|
+| **Service Name** | grafana |
+| **Host** | homelab-vm (192.168.0.210) |
+| **Port** | 3300 |
+| **URL** | `https://gf.vish.gg` (Authentik SSO) |
+| **Category** | Monitoring |
+| **Docker Image** | `grafana/grafana-oss:12.4.0` |
+| **Compose File** | `hosts/vms/homelab-vm/monitoring.yaml` |
+| **Stack** | `monitoring-stack` (Portainer stack ID 687, endpoint 443399) |
+| **Deployment** | GitOps via Portainer |
+
+## Purpose
+
+Grafana is the metrics visualization and dashboarding layer for the homelab monitoring stack. It connects to Prometheus as its datasource and provides dashboards for infrastructure health, NAS metrics, and node-level detail.
+
+## Access
+
+| Method | URL |
+|--------|-----|
+| **External (SSO)** | `https://gf.vish.gg` |
+| **Internal** | `http://192.168.0.210:3300` |
+| **Local (on VM)** | `http://localhost:3300` |
+
+Authentication is via **Authentik SSO** (`sso.vish.gg`). The local `admin` account is also available for API/CLI use.
+
+## Dashboards
+
+| Dashboard | UID | Source |
+|-----------|-----|--------|
+| Node Details - Full Metrics *(default home)* | `node-details-v2` | DB (imported) |
+| Infrastructure Overview - All Devices | `infrastructure-overview-v2` | Provisioned (monitoring.yaml) |
+| Synology NAS Monitoring | `synology-dashboard-v2` | Provisioned (monitoring.yaml) |
+| Node Exporter Full | `rYdddlPWk` | DB (imported from grafana.com) |
+
+> **Note**: `node-details-v2` and `Node Exporter Full` exist only in the `grafana-data` volume (DB). If the volume is deleted, they must be re-imported. The provisioned dashboards (Infrastructure Overview, Synology NAS) are embedded in `monitoring.yaml` and survive volume deletion.
+
+The default home dashboard (`node-details-v2`) is set via the Grafana org preferences API and persists in the DB across container restarts.
+
+## Configuration
+
+### Key Environment Variables
+
+| Variable | Value | Description |
+|----------|-------|-------------|
+| `GF_SECURITY_ADMIN_USER` | `admin` | Local admin username |
+| `GF_SECURITY_ADMIN_PASSWORD` | `admin2024` | Local admin password (first-run only; does not override DB after initial setup) |
+| `GF_FEATURE_TOGGLES_DISABLE` | `kubernetesDashboards` | Disables Grafana 12 unified storage feature toggle (prevents log spam, restores stable behavior) |
+| `GF_SERVER_ROOT_URL` | `https://gf.vish.gg` | Public URL for redirect/SSO |
+| `GF_AUTH_GENERIC_OAUTH_ENABLED` | `true` | Authentik SSO enabled |
+
+### Ports
+
+| Host Port | Container Port | Purpose |
+|-----------|----------------|---------|
+| 3300 | 3000 | Web interface |
+
+### Volumes
+
+| Volume | Container Path | Purpose |
+|--------|----------------|---------|
+| `monitoring-stack_grafana-data` | `/var/lib/grafana` | Persistent data (DB, plugins, sessions) |
+
+### Provisioned Configs (Docker configs, not bind mounts)
+
+| Config | Target | Purpose |
+|--------|--------|---------|
+| `grafana_datasources` | `/etc/grafana/provisioning/datasources/datasources.yaml` | Prometheus datasource |
+| `grafana_dashboards_config` | `/etc/grafana/provisioning/dashboards/dashboards.yaml` | Dashboard provider config |
+| `dashboard_infrastructure` | `/etc/grafana/provisioning/dashboards/json/infrastructure-overview.json` | Infrastructure Overview dashboard |
+| `dashboard_synology` | `/etc/grafana/provisioning/dashboards/json/synology-monitoring.json` | Synology NAS dashboard |
+
+## Authentik SSO
+
+Grafana OAuth2 is configured to use Authentik at `sso.vish.gg`. Role mapping:
+
+| Authentik Group | Grafana Role |
+|-----------------|-------------|
+| `Grafana Admins` | Admin |
+| `Grafana Editors` | Editor |
+| *(everyone else)* | Viewer |
+
+See `docs/services/individual/grafana-oauth.md` for setup details.
+
+## Useful Commands
+
+```bash
+# Check container status
+docker ps --filter name=grafana
+
+# View logs
+docker logs grafana -f
+
+# Reset admin password (if locked out)
+docker exec grafana grafana cli --homepath /usr/share/grafana admin reset-admin-password <newpassword>
+
+# Set org home dashboard via API
+curl -X PUT http://admin:<password>@localhost:3300/api/org/preferences \
+  -H "Content-Type: application/json" \
+  -d '{"REDACTED_APP_PASSWORD": "node-details-v2"}'
+
+# Check current home dashboard
+curl -s http://admin:<password>@localhost:3300/api/org/preferences
+```
+
+## Troubleshooting
+
+### Admin password not working after redeploy
+`GF_SECURITY_ADMIN_PASSWORD` only applies on the very first run (empty DB). Subsequent redeployments do not reset it. Use the CLI reset:
+```bash
+docker exec grafana grafana cli --homepath /usr/share/grafana admin reset-admin-password <newpassword>
+```
+
+### Home dashboard reverts to Grafana welcome page
+The home dashboard is stored in the `preferences` table in `grafana.db`. It survives container restarts as long as the `grafana-data` volume is not deleted. If lost, re-set it via:
+```bash
+curl -X PUT http://admin:<password>@localhost:3300/api/org/preferences \
+  -H "Content-Type: application/json" \
+  -d '{"REDACTED_APP_PASSWORD": "node-details-v2"}'
+```
+
+### "No last resource version found" log spam
+This is caused by the `kubernetesDashboards` feature toggle being on by default in Grafana 12. It is disabled via `GF_FEATURE_TOGGLES_DISABLE=kubernetesDashboards` in `monitoring.yaml`.
+
+### Dashboards missing after volume wipe
+Re-import `Node Details - Full Metrics` and `Node Exporter Full` from grafana.com (IDs: search grafana.com/grafana/dashboards). The provisioned dashboards (Infrastructure Overview, Synology NAS) will auto-restore from `monitoring.yaml` configs.
+
+## Related Services
+
+- **Prometheus** — metrics datasource (`http://prometheus:9090`)
+- **Node Exporter** — host metrics (port 9100)
+- **SNMP Exporter** — Synology NAS metrics (port 9116)
+- **Authentik** — SSO provider (`sso.vish.gg`)
+- **Nginx Proxy Manager** — reverse proxy for `gf.vish.gg`
+
+## Related Documentation
+
+- `docs/admin/monitoring-setup.md` — monitoring stack quick reference
+- `docs/admin/monitoring.md` — full monitoring & observability guide
+- `docs/services/individual/grafana-oauth.md` — Authentik SSO setup
+- `docs/infrastructure/monitoring/README.md` — monitoring stack architecture
+- `hosts/vms/homelab-vm/monitoring.yaml` — compose file (source of truth)
+
+---
+
+**Last Updated**: 2026-03-08
+**Configuration Source**: `hosts/vms/homelab-vm/monitoring.yaml`