Sanitized mirror from private repository - 2026-04-20 01:24:42 UTC
This commit is contained in:
Gitea Mirror Bot
300
docs/troubleshooting/internet-outage-access.md
Normal file
300
docs/troubleshooting/internet-outage-access.md
Normal file
@@ -0,0 +1,300 @@
|
||||
# Accessing the Homelab During an Internet Outage
|
||||
|
||||
**When your internet goes down, the homelab keeps running.** This guide covers exactly how to reach each service via LAN or Tailscale (which uses peer-to-peer WireGuard — it continues working between nodes that already have keys exchanged, even without the coordination server).
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference — What Still Works
|
||||
|
||||
| Category | Services | Access Method |
|
||||
|----------|----------|---------------|
|
||||
| **Streaming** | Plex, Jellyfin, Audiobookshelf | LAN IP or Tailscale IP |
|
||||
| **Media mgmt** | Sonarr, Radarr, SABnzbd, Prowlarr | LAN IP or Tailscale IP |
|
||||
| **Photos** | Immich (Atlantis + Calypso) | LAN IP or Tailscale IP |
|
||||
| **Documents** | Paperless-NGX | LAN IP or Tailscale IP |
|
||||
| **Passwords** | Vaultwarden | LAN IP or Tailscale IP |
|
||||
| **Files** | Seafile, Syncthing | LAN IP or Tailscale IP |
|
||||
| **Notes** | Joplin, BookStack | LAN IP or Tailscale IP |
|
||||
| **Git/CI** | Gitea, Portainer | LAN IP or Tailscale IP |
|
||||
| **Monitoring** | Grafana, Prometheus, Uptime Kuma | LAN IP or Tailscale IP |
|
||||
| **Home Auto** | Home Assistant | LAN IP or Tailscale IP |
|
||||
| **Dashboard** | Homarr | LAN IP or Tailscale IP |
|
||||
| **Finance** | Actual Budget | LAN IP or Tailscale IP |
|
||||
| **Comms** | Mattermost, Matrix (local rooms) | LAN IP or Tailscale IP |
|
||||
| **Auth** | Authentik SSO | LAN IP or Tailscale IP (fully local) |
|
||||
|
||||
**What does NOT work without internet:**
|
||||
- New downloads (Sonarr/Radarr can't search indexers, SABnzbd can't download)
|
||||
- Invidious, Piped, Redlib (they ARE the internet)
|
||||
- YourSpotify, ProtonMail Bridge
|
||||
- External access via `*.vish.gg` domains (Cloudflare proxy down)
|
||||
- iOS push notifications via ntfy (ntfy.sh upstream unavailable)
|
||||
- AI tagging in Hoarder (OpenAI API)
|
||||
|
||||
---
|
||||
|
||||
## Access Methods
|
||||
|
||||
### Method 1 — LAN (same network as Atlantis/Calypso)
|
||||
|
||||
You must be physically connected to the home network (Ethernet or WiFi).
|
||||
|
||||
| Host | LAN IP | Notes |
|
||||
|------|--------|-------|
|
||||
| Atlantis | `192.168.0.200` | Primary NAS — most services |
|
||||
| Calypso | `192.168.0.250` | Secondary NAS — Gitea, Authentik, Paperless, Immich |
|
||||
| Homelab VM | `192.168.0.X` | Check router DHCP — runs monitoring, Mattermost |
|
||||
| Concord NUC | `192.168.0.X` | Check router DHCP |
|
||||
| Pi-5 | `192.168.0.66` | Uptime Kuma, Glances |
|
||||
| Guava (TrueNAS) | `192.168.0.100` | NAS shares |
|
||||
| Home Assistant | `192.168.12.202` (behind MT3000) | HA Green |
|
||||
|
||||
### Method 2 — Tailscale / Headscale (any network, any location)
|
||||
|
||||
Tailscale uses WireGuard peer-to-peer. **Once nodes have exchanged keys, they communicate directly without needing the coordination server (headscale on Calypso).** An internet outage does not break existing Tailscale sessions.
|
||||
|
||||
| Host | Tailscale IP | SSH Alias |
|
||||
|------|-------------|-----------|
|
||||
| Atlantis | `100.83.230.112` | `atlantis` |
|
||||
| Calypso | `100.103.48.78` | `calypso` |
|
||||
| Homelab VM | `100.67.40.126` | `homelab-vm` |
|
||||
| Concord NUC | `100.72.55.21` | `nuc` |
|
||||
| Pi-5 | `100.77.151.40` | `pi-5` |
|
||||
| Guava | `100.75.252.64` | `guava` |
|
||||
| Moon | `100.64.0.6` | `moon` |
|
||||
| Setillo | `100.125.0.20` | `setillo` |
|
||||
| Seattle VPS | `100.82.197.124` | `seattle-tailscale` |
|
||||
|
||||
**MagicDNS** also works on Tailscale: `atlantis.tail.vish.gg`, `calypso.tail.vish.gg`, etc.
|
||||
|
||||
> **Note:** If headscale itself needs to restart during an outage, it will now start fine (fixed 2026-03-16 — `only_start_if_oidc_is_available: false`). Existing node sessions survive a headscale restart indefinitely.
|
||||
|
||||
---
|
||||
|
||||
## Service Access Cheatsheet
|
||||
|
||||
### Portainer (container management)
|
||||
```
|
||||
LAN: http://192.168.0.200:10000
|
||||
Tailscale: http://100.83.230.112:10000
|
||||
Public: https://pt.vish.gg ← requires internet
|
||||
```
|
||||
|
||||
### Gitea (code repos, CI/CD)
|
||||
```
|
||||
LAN: http://192.168.0.250:3052
|
||||
Tailscale: http://100.103.48.78:3052 or http://calypso.tail.vish.gg:3052
|
||||
Public: https://git.vish.gg ← requires internet (Cloudflare proxy)
|
||||
```
|
||||
> GitOps still works during outage — Portainer pulls from `git.vish.gg` which resolves to Calypso on LAN.
|
||||
|
||||
### Plex
|
||||
```
|
||||
LAN: http://192.168.0.200:32400/web
|
||||
Tailscale: http://100.83.230.112:32400/web
|
||||
Note: Plex account login may fail (plex.tv unreachable) — use local account
|
||||
```
|
||||
|
||||
### Jellyfin
|
||||
```
|
||||
LAN: http://192.168.0.200:8096
|
||||
Tailscale: http://100.83.230.112:8096
|
||||
```
|
||||
|
||||
### Immich (Atlantis)
|
||||
```
|
||||
LAN: http://192.168.0.200:8212
|
||||
Tailscale: http://atlantis.tail.vish.gg:8212
|
||||
```
|
||||
|
||||
### Immich (Calypso)
|
||||
```
|
||||
LAN: http://192.168.0.250:8212
|
||||
Tailscale: http://calypso.tail.vish.gg:8212
|
||||
```
|
||||
|
||||
### Paperless-NGX
|
||||
```
|
||||
LAN: http://192.168.0.250:8777
|
||||
Tailscale: http://100.103.48.78:8777
|
||||
Public: https://docs.vish.gg ← requires internet
|
||||
SSO: Still works (Authentik is local)
|
||||
```
|
||||
|
||||
### Vaultwarden
|
||||
```
|
||||
LAN: http://192.168.0.200:4080
|
||||
Tailscale: http://100.83.230.112:4080
|
||||
Public: https://pw.vish.gg ← requires internet
|
||||
Note: Use local login (password + security key) — SSO still works too
|
||||
```
|
||||
|
||||
### Homarr (dashboard)
|
||||
```
|
||||
LAN: http://192.168.0.200:7575
|
||||
Tailscale: http://100.83.230.112:7575
|
||||
Note: Use credentials login if SSO is unavailable
|
||||
```
|
||||
|
||||
### Actual Budget
|
||||
```
|
||||
LAN: http://192.168.0.250:8304
|
||||
Tailscale: http://100.103.48.78:8304
|
||||
Public: https://actual.vish.gg ← requires internet
|
||||
Note: Password login available (OIDC also works since Authentik is local)
|
||||
```
|
||||
|
||||
### Hoarder
|
||||
```
|
||||
Tailscale: http://100.67.40.126:3000 (homelab-vm)
|
||||
Public: https://hoarder.thevish.io ← requires internet
|
||||
```
|
||||
|
||||
### Grafana
|
||||
```
|
||||
LAN: http://192.168.0.200:3300
|
||||
Tailscale: http://100.83.230.112:3300
|
||||
Public: https://gf.vish.gg ← requires internet
|
||||
```
|
||||
|
||||
### Authentik SSO
|
||||
```
|
||||
LAN: http://192.168.0.250:9000
|
||||
Tailscale: http://100.103.48.78:9000
|
||||
Public: https://sso.vish.gg ← requires internet
|
||||
Note: Fully functional locally — all OIDC flows work without internet
|
||||
```
|
||||
|
||||
### Home Assistant
|
||||
```
|
||||
LAN: http://192.168.12.202:8123 (behind GL-MT3600BE / Beryl 7)
|
||||
Tailscale: http://homeassistant.tail.vish.gg (via Tailscale)
|
||||
Note: Automations and local devices work; cloud integrations may fail
|
||||
```
|
||||
|
||||
### Guava SMB shares (Windows)
|
||||
```
|
||||
LAN: \\192.168.0.100\<sharename>
|
||||
Note: Credentials stored in Windows Credential Manager
|
||||
User: vish (see Vaultwarden if password needed)
|
||||
```
|
||||
|
||||
### Uptime Kuma
|
||||
```
|
||||
LAN: http://192.168.0.66:3001 (Pi-5)
|
||||
Tailscale: http://100.77.151.40:3001
|
||||
```
|
||||
|
||||
### Sonarr / Radarr / Arr suite
|
||||
```
|
||||
LAN: http://192.168.0.200:<port>
|
||||
Sonarr: 8989 Radarr: 7878
|
||||
Lidarr: 8686 Prowlarr: 9696
|
||||
Bazarr: 6767 SABnzbd: 8880
|
||||
Tailscale: http://100.83.230.112:<port>
|
||||
Note: Can still manage library, mark as watched, etc.
|
||||
New downloads fail (no indexer access without internet)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SSH Access During Outage
|
||||
|
||||
All hosts have SSH key-based auth. From any machine on LAN or Tailscale:
|
||||
|
||||
```bash
|
||||
# Atlantis (Synology DSM)
|
||||
ssh -p 60000 vish@192.168.0.200 # LAN
|
||||
ssh atlantis # Tailscale (uses ~/.ssh/config)
|
||||
|
||||
# Calypso (Synology DSM)
|
||||
ssh -p 62000 Vish@192.168.0.250 # LAN (capital V)
|
||||
ssh calypso # Tailscale
|
||||
|
||||
# Homelab VM
|
||||
ssh homelab@100.67.40.126 # Tailscale only (no LAN port forward)
|
||||
|
||||
# Concord NUC
|
||||
ssh nuc # Tailscale
|
||||
|
||||
# Pi-5
|
||||
ssh pi-5 # Tailscale (vish@100.77.151.40)
|
||||
|
||||
# Guava (TrueNAS)
|
||||
ssh vish@192.168.0.100 # LAN
|
||||
ssh guava # Tailscale
|
||||
|
||||
# Moon (remote)
|
||||
ssh moon # Tailscale only (100.64.0.6)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## NPM / Reverse Proxy
|
||||
|
||||
NPM runs on Calypso (`192.168.0.250`, port 81 admin UI). During an internet outage, NPM itself keeps running and continues to proxy internal traffic. SSL certs remain valid for up to 90 days — cert renewal requires internet (Let's Encrypt + Cloudflare DNS).
|
||||
|
||||
For LAN access you don't go through NPM at all — use the direct host:port addresses above.
|
||||
|
||||
---
|
||||
|
||||
## Tailscale Not Working?
|
||||
|
||||
If Tailscale connectivity is lost during an outage:
|
||||
|
||||
1. **Check if headscale is up on Calypso:**
|
||||
```bash
|
||||
ssh -p 62000 Vish@192.168.0.250 "sudo /usr/local/bin/docker ps | grep headscale"
|
||||
```
|
||||
|
||||
2. **Restart headscale if needed** (it will start even without internet now):
|
||||
```bash
|
||||
ssh -p 62000 Vish@192.168.0.250 "sudo /usr/local/bin/docker restart headscale"
|
||||
```
|
||||
|
||||
3. **Force re-auth on a node:**
|
||||
```bash
|
||||
sudo tailscale up --login-server=https://headscale.vish.gg:8443
|
||||
# headscale.vish.gg resolves via LAN since it's unproxied (direct home IP)
|
||||
```
|
||||
|
||||
4. **If headscale.vish.gg DNS fails** (DDNS not updated yet), use the direct IP:
|
||||
```bash
|
||||
sudo tailscale up --login-server=http://192.168.0.250:8080
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## DDNS / External Access Recovery
|
||||
|
||||
When internet comes back after an outage, DDNS updaters on Atlantis automatically update Cloudflare within ~5 minutes. No manual action needed.
|
||||
|
||||
If your external IP changed during the outage and you need to update manually:
|
||||
```bash
|
||||
# Check current external IP
|
||||
curl https://ipv4.icanhazip.com
|
||||
|
||||
# Check what Cloudflare has for a domain
|
||||
dig +short headscale.vish.gg A
|
||||
|
||||
# If they differ, restart the DDNS updater on Atlantis to force immediate update
|
||||
ssh atlantis "sudo /var/packages/REDACTED_APP_PASSWORD/usr/bin/docker restart \
|
||||
dyndns-updater-stack-ddns-vish-unproxied-1 \
|
||||
dyndns-updater-stack-ddns-vish-proxied-1 \
|
||||
dyndns-updater-stack-ddns-thevish-proxied-1 \
|
||||
dyndns-updater-stack-ddns-thevish-unproxied-1"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Docs
|
||||
|
||||
- [Common Issues](common-issues.md) — Tailscale routing, SMB problems
|
||||
- [Guava SMB Incident](guava-smb-incident-2026-03-14.md) — Tailscale subnet route issues
|
||||
- [Offline Password Access](offline-password-access.md) — If Vaultwarden itself is down
|
||||
- [Disaster Recovery](disaster-recovery.md) — Full hardware failure scenarios
|
||||
- [SSO/OIDC Status](../admin/sso-oidc-status.md) — Which services have local login fallback
|
||||
|
||||
---
|
||||
|
||||
**Last updated:** 2026-03-16
|
||||
Reference in New Issue
Block a user