# Grafana

**Monitoring Service**

## Service Overview

| Property | Value |
|----------|-------|
| **Service Name** | grafana |
| **Host** | homelab-vm (192.168.0.210) |
| **Port** | 3300 |
| **URL** | `https://gf.vish.gg` (Authentik SSO) |
| **Category** | Monitoring |
| **Docker Image** | `grafana/grafana-oss:12.4.0` |
| **Compose File** | `hosts/vms/homelab-vm/monitoring.yaml` |
| **Stack** | `monitoring-stack` (Portainer stack ID 687, endpoint 443399) |
| **Deployment** | GitOps via Portainer |

## Purpose

Grafana is the metrics visualization and dashboarding layer for the homelab monitoring stack. It connects to Prometheus as its datasource and provides dashboards for infrastructure health, NAS metrics, and node-level detail.

## Access

| Method | URL |
|--------|-----|
| **External (SSO)** | `https://gf.vish.gg` |
| **Internal** | `http://192.168.0.210:3300` |
| **Local (on VM)** | `http://localhost:3300` |

Authentication is via **Authentik SSO** (`sso.vish.gg`). The local `admin` account is also available for API/CLI use.

## Dashboards

| Dashboard | UID | Source |
|-----------|-----|--------|
| Node Details - Full Metrics *(default home)* | `node-details-v2` | DB (imported) |
| Infrastructure Overview - All Devices | `infrastructure-overview-v2` | Provisioned (monitoring.yaml) |
| Synology NAS Monitoring | `synology-dashboard-v2` | Provisioned (monitoring.yaml) |
| Node Exporter Full | `rYdddlPWk` | DB (imported from grafana.com) |

> **Note**: `node-details-v2` and `Node Exporter Full` exist only in the `grafana-data` volume (DB). If the volume is deleted, they must be re-imported. The provisioned dashboards (Infrastructure Overview, Synology NAS) are embedded in `monitoring.yaml` and survive volume deletion.

The default home dashboard (`node-details-v2`) is set via the Grafana org preferences API and persists in the DB across container restarts.

## Configuration

### Key Environment Variables

| Variable | Value | Description |
|----------|-------|-------------|
| `GF_SECURITY_ADMIN_USER` | `admin` | Local admin username |
| `GF_SECURITY_ADMIN_PASSWORD` | `admin2024` | Local admin password (first-run only; does not override DB after initial setup) |
| `GF_FEATURE_TOGGLES_DISABLE` | `kubernetesDashboards` | Disables Grafana 12 unified storage feature toggle (prevents log spam, restores stable behavior) |
| `GF_SERVER_ROOT_URL` | `https://gf.vish.gg` | Public URL for redirect/SSO |
| `GF_AUTH_GENERIC_OAUTH_ENABLED` | `true` | Authentik SSO enabled |

### Ports

| Host Port | Container Port | Purpose |
|-----------|----------------|---------|
| 3300 | 3000 | Web interface |

### Volumes

| Volume | Container Path | Purpose |
|--------|----------------|---------|
| `monitoring-stack_grafana-data` | `/var/lib/grafana` | Persistent data (DB, plugins, sessions) |

### Provisioned Configs (Docker configs, not bind mounts)

| Config | Target | Purpose |
|--------|--------|---------|
| `grafana_datasources` | `/etc/grafana/provisioning/datasources/datasources.yaml` | Prometheus datasource |
| `grafana_dashboards_config` | `/etc/grafana/provisioning/dashboards/dashboards.yaml` | Dashboard provider config |
| `dashboard_infrastructure` | `/etc/grafana/provisioning/dashboards/json/infrastructure-overview.json` | Infrastructure Overview dashboard |
| `dashboard_synology` | `/etc/grafana/provisioning/dashboards/json/synology-monitoring.json` | Synology NAS dashboard |

## Authentik SSO

Grafana OAuth2 is configured to use Authentik at `sso.vish.gg`. Role mapping:

| Authentik Group | Grafana Role |
|-----------------|-------------|
| `Grafana Admins` | Admin |
| `Grafana Editors` | Editor |
| *(everyone else)* | Viewer |

See `docs/services/individual/grafana-oauth.md` for setup details.

## Useful Commands

```bash
# Check container status
docker ps --filter name=grafana

# View logs
docker logs grafana -f

# Reset admin password (if locked out)
docker exec grafana grafana cli --homepath /usr/share/grafana admin reset-admin-password <newpassword>

# Set org home dashboard via API
curl -X PUT http://admin:<password>@localhost:3300/api/org/preferences \
  -H "Content-Type: application/json" \
  -d '{"REDACTED_APP_PASSWORD": "node-details-v2"}'

# Check current home dashboard
curl -s http://admin:<password>@localhost:3300/api/org/preferences
```

## Troubleshooting

### Admin password not working after redeploy
`GF_SECURITY_ADMIN_PASSWORD` only applies on the very first run (empty DB). Subsequent redeployments do not reset it. Use the CLI reset:
```bash
docker exec grafana grafana cli --homepath /usr/share/grafana admin reset-admin-password <newpassword>
```

### Home dashboard reverts to Grafana welcome page
The home dashboard is stored in the `preferences` table in `grafana.db`. It survives container restarts as long as the `grafana-data` volume is not deleted. If lost, re-set it via:
```bash
curl -X PUT http://admin:<password>@localhost:3300/api/org/preferences \
  -H "Content-Type: application/json" \
  -d '{"REDACTED_APP_PASSWORD": "node-details-v2"}'
```

### "No last resource version found" log spam
This is caused by the `kubernetesDashboards` feature toggle being on by default in Grafana 12. It is disabled via `GF_FEATURE_TOGGLES_DISABLE=kubernetesDashboards` in `monitoring.yaml`.

### Dashboards missing after volume wipe
Re-import `Node Details - Full Metrics` and `Node Exporter Full` from grafana.com (IDs: search grafana.com/grafana/dashboards). The provisioned dashboards (Infrastructure Overview, Synology NAS) will auto-restore from `monitoring.yaml` configs.

## Related Services

- **Prometheus** — metrics datasource (`http://prometheus:9090`)
- **Node Exporter** — host metrics (port 9100)
- **SNMP Exporter** — Synology NAS metrics (port 9116)
- **Authentik** — SSO provider (`sso.vish.gg`)
- **Nginx Proxy Manager** — reverse proxy for `gf.vish.gg`

## Related Documentation

- `docs/admin/monitoring-setup.md` — monitoring stack quick reference
- `docs/admin/monitoring.md` — full monitoring & observability guide
- `docs/services/individual/grafana-oauth.md` — Authentik SSO setup
- `docs/infrastructure/monitoring/README.md` — monitoring stack architecture
- `hosts/vms/homelab-vm/monitoring.yaml` — compose file (source of truth)

---

**Last Updated**: 2026-03-08
**Configuration Source**: `hosts/vms/homelab-vm/monitoring.yaml`
