Vish/homelab-optimized
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sanitized mirror from private repository - 2026-03-21 08:52:36 UTC
Some checks failed
Documentation / Deploy to GitHub Pages (push) Has been cancelled
Details
Documentation / Build Docusaurus (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Gitea Mirror Bot
2026-03-21 08:52:36 +00:00
commit 51e6e509dd
1235 changed files with 306153 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

856
docs/diagrams/service-architecture.md Normal file
View File

@@ -0,0 +1,856 @@
# 🏗️ Service Architecture
## Overview
This document shows how the 157+ Docker services (plus Olares K8s) interact, their dependencies, and the data flows between them.
---
## 🎬 Media Stack Architecture (Mermaid)
```mermaid
graph TB
subgraph Internet["☁️ Internet Sources"]
USENET["Usenet<br/>Providers"]
TORRENT["Torrent<br/>Trackers"]
INDEXERS["Indexers<br/>(NZB/Torrent)"]
end
subgraph Acquisition["📥 Content Acquisition (Atlantis)"]
PROWLARR["Prowlarr<br/>Indexer Manager"]
SONARR["Sonarr<br/>TV Shows"]
RADARR["Radarr<br/>Movies"]
LIDARR["Lidarr<br/>Music"]
READARR["Readarr<br/>Books"]
WHISPARR["Whisparr<br/>Adult"]
BAZARR["Bazarr<br/>Subtitles"]
SAB["SABnzbd<br/>Usenet Client"]
DELUGE["Deluge<br/>Torrent Client<br/>(via Gluetun VPN)"]
end
subgraph Storage["💾 Storage (Atlantis NAS)"]
MEDIA_TV["/volume1/media/tv"]
MEDIA_MOV["/volume1/media/movies"]
MEDIA_MUS["/volume1/media/music"]
MEDIA_BOOK["/volume1/media/books"]
end
subgraph Streaming["📺 Media Streaming"]
PLEX["Plex<br/>Media Server"]
JELLYFIN["Jellyfin<br/>Media Server"]
TAUTULLI["Tautulli<br/>Plex Analytics"]
end
subgraph Clients["📱 Client Devices"]
TV["Smart TVs"]
PHONE["Phones/Tablets"]
WEB["Web Browsers"]
APPS["Desktop Apps"]
end
%% Acquisition flow
INDEXERS --> PROWLARR
PROWLARR --> SONARR & RADARR & LIDARR & READARR & WHISPARR
SONARR --> SAB & DELUGE
RADARR --> SAB & DELUGE
LIDARR --> SAB & DELUGE
READARR --> SAB & DELUGE
WHISPARR --> SAB & DELUGE
USENET --> SAB
TORRENT --> DELUGE
%% Storage flow
SAB --> MEDIA_TV & MEDIA_MOV & MEDIA_MUS & MEDIA_BOOK
DELUGE --> MEDIA_TV & MEDIA_MOV & MEDIA_MUS & MEDIA_BOOK
BAZARR --> MEDIA_TV & MEDIA_MOV
%% Streaming flow
MEDIA_TV & MEDIA_MOV --> PLEX & JELLYFIN
PLEX --> TAUTULLI
%% Client access
PLEX & JELLYFIN --> TV & PHONE & WEB & APPS
classDef acquisition fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
classDef storage fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
classDef streaming fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff
classDef client fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff
class PROWLARR,SONARR,RADARR,LIDARR,READARR,WHISPARR,BAZARR,SAB,DELUGE acquisition
class MEDIA_TV,MEDIA_MOV,MEDIA_MUS,MEDIA_BOOK storage
class PLEX,JELLYFIN,TAUTULLI streaming
class TV,PHONE,WEB,APPS client
```
---
## 📊 Monitoring Stack Architecture
```mermaid
graph TB
subgraph Targets["🎯 Monitored Targets"]
subgraph Synology["Synology NAS"]
ATL_SNMP["Atlantis<br/>SNMP"]
CAL_SNMP["Calypso<br/>SNMP"]
SET_SNMP["Setillo<br/>SNMP"]
end
subgraph Hosts["Linux Hosts"]
NODE1["Homelab VM<br/>node_exporter"]
NODE2["Guava<br/>node_exporter"]
NODE3["Anubis<br/>node_exporter"]
end
subgraph Containers["Containers"]
CADV["cAdvisor<br/>Container Metrics"]
end
subgraph Network["Network"]
BLACK["Blackbox Exporter<br/>HTTP/ICMP Probes"]
end
end
subgraph Collection["📥 Metric Collection (Homelab VM)"]
PROM["Prometheus<br/>Time Series DB"]
SNMP_EXP["SNMP Exporter"]
end
subgraph Visualization["📈 Visualization"]
GRAFANA["Grafana<br/>Dashboards"]
end
subgraph Alerting["🚨 Alerting"]
ALERTMGR["Alertmanager"]
NTFY["ntfy<br/>Push Notifications"]
UPTIME["Uptime Kuma<br/>Status Page"]
end
%% Collection
ATL_SNMP & CAL_SNMP & SET_SNMP --> SNMP_EXP
SNMP_EXP --> PROM
NODE1 & NODE2 & NODE3 --> PROM
CADV --> PROM
BLACK --> PROM
%% Visualization
PROM --> GRAFANA
PROM --> ALERTMGR
ALERTMGR --> NTFY
%% Uptime Kuma separate
BLACK -.-> UPTIME
classDef target fill:#e67e22,stroke:#333,stroke-width:2px,color:#fff
classDef collection fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
classDef viz fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff
classDef alert fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
class ATL_SNMP,CAL_SNMP,SET_SNMP,NODE1,NODE2,NODE3,CADV,BLACK target
class PROM,SNMP_EXP collection
class GRAFANA viz
class ALERTMGR,NTFY,UPTIME alert
```
---
## 🔐 Authentication & Security Stack
### Complete Authentication Architecture
```mermaid
graph TB
subgraph External["🌐 External Access"]
USERS["👤 Users"]
CLOUDFLARE["☁️ Cloudflare<br/>DNS/WAF/DDoS"]
end
subgraph Gateway["🚪 Gateway Layer (matrix-ubuntu)"]
NPM["🔀 Nginx Proxy Manager<br/>matrix-ubuntu :81/:443<br/>Reverse Proxy + SSL"]
CFT["🚇 Cloudflare Tunnel<br/>Zero Trust Access"]
end
subgraph AuthLayer["🔐 Authentication Layer (Calypso)"]
AUTH_SRV["🔐 Authentik Server<br/>:9000"]
AUTH_PROXY["🛡️ Authentik Outpost<br/>:9444<br/>Forward Auth Proxy"]
AUTH_WRK["⚙️ Authentik Worker"]
AUTH_DB["🐘 PostgreSQL"]
AUTH_RED["🔴 Redis"]
end
subgraph VPN["🔒 VPN Layer"]
WIREGUARD["🔒 Wireguard<br/>Atlantis :51820"]
TAILSCALE["🔷 Tailscale<br/>100.x.x.x"]
HEADSCALE["🌐 Headscale<br/>Calypso :8080"]
end
subgraph DNS["🌐 DNS & Ad Blocking"]
ADGUARD1["🛡️ AdGuard<br/>Calypso :53"]
ADGUARD2["🛡️ AdGuard<br/>Atlantis :53"]
ADGUARD3["🛡️ AdGuard<br/>NUC :53"]
end
subgraph SecVault["🔑 Secrets Management"]
VAULT["🔑 Vaultwarden<br/>vault.vish.gg"]
end
subgraph ProtectedServices["🛡️ Protected Services"]
GRAFANA["📊 Grafana"]
PAPERLESS["📄 Paperless"]
IMMICH["📸 Immich"]
ACTUAL["💰 Actual Budget"]
GITEA["🔧 Gitea"]
NETBOX["🔌 NetBox"]
HOMARR["🏠 Homarr"]
RXRESUME["📝 Reactive Resume"]
HEADPLANE["🌐 Headplane"]
end
subgraph PublicServices["🌍 Public/Self-Auth Services"]
PLEX["📺 Plex"]
SEAFILE["☁️ Seafile"]
OST["🚀 OpenSpeedTest"]
NTFY["📣 ntfy"]
end
%% External flow
USERS --> CLOUDFLARE
CLOUDFLARE --> NPM
CLOUDFLARE --> CFT
USERS --> TAILSCALE
%% NPM to Auth
NPM -->|"Forward Auth<br/>Header Check"| AUTH_PROXY
AUTH_PROXY -->|"Validate Session"| AUTH_SRV
%% Auth internal
AUTH_SRV --> AUTH_DB
AUTH_SRV --> AUTH_RED
AUTH_WRK --> AUTH_DB
AUTH_WRK --> AUTH_RED
%% Protected services via NPM + Auth
NPM -->|"✓ Authenticated"| ProtectedServices
%% Public services direct
NPM --> PublicServices
%% VPN access
TAILSCALE --> HEADSCALE
WIREGUARD --> ProtectedServices
TAILSCALE --> ProtectedServices
%% DNS
ADGUARD1 -.-> ProtectedServices
ADGUARD2 -.-> PublicServices
classDef external fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
classDef gateway fill:#f39c12,stroke:#333,stroke-width:2px,color:#fff
classDef auth fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff
classDef dns fill:#1abc9c,stroke:#333,stroke-width:2px,color:#fff
classDef protected fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
classDef public fill:#27ae60,stroke:#333,stroke-width:2px,color:#fff
class USERS,CLOUDFLARE external
class NPM,CFT gateway
class AUTH_SRV,AUTH_PROXY,AUTH_WRK,AUTH_DB,AUTH_RED,VAULT auth
class ADGUARD1,ADGUARD2,ADGUARD3 dns
class GRAFANA,PAPERLESS,IMMICH,ACTUAL,GITEA,NETBOX,HOMARR,RXRESUME,HEADPLANE protected
class PLEX,SEAFILE,OST,NTFY public
```
---
### Authentik SSO Flow (Detailed)
```mermaid
sequenceDiagram
autonumber
participant U as 👤 User
participant CF as ☁️ Cloudflare
participant NPM as 🔀 NPM (matrix-ubuntu)
participant OUT as 🛡️ Outpost (Calypso)
participant AUTH as 🔐 Authentik (Calypso)
participant APP as 📱 Application
U->>CF: Request app.vish.gg
CF->>NPM: Forward (HTTPS)
NPM->>OUT: Forward Auth Request<br/>(/outpost.goauthentik.io/auth/nginx)
alt No Valid Session
OUT->>AUTH: Check Session
AUTH-->>OUT: No Session
OUT-->>NPM: 401 Unauthorized
NPM-->>U: Redirect to sso.vish.gg/flows/default-authentication/
U->>AUTH: Login Page
U->>AUTH: Submit Credentials + 2FA
AUTH->>AUTH: Validate
AUTH-->>U: Set Cookie + Redirect to app
U->>NPM: Retry with Session Cookie
NPM->>OUT: Forward Auth (with cookie)
end
OUT->>AUTH: Validate Session
AUTH-->>OUT: Valid ✓
OUT-->>NPM: 200 OK + Headers<br/>(X-authentik-username, X-authentik-email)
NPM->>APP: Proxy Request (with auth headers)
APP-->>U: Response
```
---
### NPM Proxy Host Configuration
```mermaid
graph TB
subgraph NPM["🔀 Nginx Proxy Manager (matrix-ubuntu :81)"]
subgraph ProxyHosts["Proxy Hosts"]
PH1["sso.vish.gg → Calypso:9000"]
PH2["git.vish.gg → Calypso:3052"]
PH3["gf.vish.gg → homelab-vm:3300"]
PH4["nb.vish.gg → homelab-vm:8443"]
PH5["ntfy.vish.gg → homelab-vm:8081"]
PH6["dash.vish.gg → Atlantis:7575"]
PH7["paperless.vish.gg → Calypso:8777"]
PH8["rx.vish.gg → Calypso:4550"]
PH9["actual.vish.gg → Calypso:8304"]
PH10["kuma.vish.gg → RPi5:3001"]
end
subgraph SSL["SSL Certificates"]
WILD["*.vish.gg<br/>Cloudflare DNS Challenge"]
end
subgraph AccessControl["Access Control"]
AUTH_LOC["Authentik Forward Auth<br/>Location: /outpost.goauthentik.io"]
end
end
subgraph Services["Backend Services"]
direction LR
S1["Authentik"]
S2["Gitea"]
S3["Grafana"]
S4["NetBox"]
S5["ntfy"]
S6["Homarr"]
S7["Paperless"]
S8["Reactive Resume"]
S9["Actual"]
S10["Uptime Kuma"]
end
PH1 --> S1
PH2 --> S2
PH3 --> S3
PH4 --> S4
PH5 --> S5
PH6 --> S6
PH7 --> S7
PH8 --> S8
PH9 --> S9
PH10 --> S10
```
---
### Services Protected by Authentik
| Domain | Service | Host | Auth Type | Notes |
|--------|---------|------|-----------|-------|
| `sso.vish.gg` | Authentik | Calypso | - | Identity Provider |
| `git.vish.gg` | Gitea | Calypso | OAuth2/OIDC | Source Control |
| `gf.vish.gg` | Grafana | Homelab VM | OAuth2/OIDC | Monitoring |
| `nb.vish.gg` | NetBox | Homelab VM | OAuth2/OIDC | DCIM/IPAM |
| `dash.vish.gg` | Homarr | Atlantis | OAuth2/OIDC | Dashboard |
| `rx.vish.gg` | Reactive Resume | Calypso | OAuth2/OIDC | Resume Builder |
| `immich` | Immich | Calypso | OAuth2/OIDC | Photos |
| `headscale.vish.gg/admin` | Headplane | Calypso | OAuth2/OIDC | VPN Admin |
| `paperless.vish.gg` | Paperless-NGX | Calypso | Forward Auth | Documents |
| `actual.vish.gg` | Actual Budget | Calypso | Forward Auth | Finance |
### Services NOT Protected (Public/Self-Auth)
| Domain | Service | Host | Reason |
|--------|---------|------|--------|
| `plex.vish.gg` | Plex | Atlantis | Has Plex Auth |
| `sf.vish.gg` | Seafile | Calypso | Has built-in auth + share links |
| `ntfy.vish.gg` | ntfy | Homelab | Has built-in auth + public topics |
| `ost.vish.gg` | OpenSpeedTest | Calypso | Public utility |
---
### Authentik Forward Auth Setup (NPM)
To protect a service with Authentik Forward Auth in NPM:
1. **Create Provider in Authentik**:
- Type: Proxy Provider
- External Host: `https://app.vish.gg`
- Mode: Forward auth (single application)
2. **Create Application in Authentik**:
- Link to the provider
- Set policies for access control
3. **Create Outpost in Authentik**:
- Type: Proxy
- Include the application
4. **Configure NPM Proxy Host**:
```nginx
# Custom Nginx Configuration (Advanced tab)
# Authentik Forward Auth
location /outpost.goauthentik.io {
proxy_pass http://calypso.vish.local:9444/outpost.goauthentik.io;
proxy_set_header Host $host;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
add_header Set-Cookie $auth_cookie;
auth_request_set $auth_cookie $upstream_http_set_cookie;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
}
location / {
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# Forward auth headers to application
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-email $authentik_email;
proxy_pass http://backend;
}
location @goauthentik_proxy_signin {
internal;
add_header Set-Cookie $auth_cookie;
return 302 /outpost.goauthentik.io/start?rd=$request_uri;
}
```
---
## 📝 ASCII Service Distribution by Host
```
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ SERVICE DISTRIBUTION BY HOST ║
╚══════════════════════════════════════════════════════════════════════════════════════════╝
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🏛️ ATLANTIS (51 Containers) - Media & Communication Hub │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 📺 Media 🔐 Security 🛠️ Infrastructure │
│ ───────────── ───────────── ───────────────── │
│ • Plex • Vaultwarden • Portainer │
│ • Jellyfin • Wireguard • DokuWiki │
│ • Immich • Dozzle │
│ • Tautulli • Watchtower │
│ • Homarr (dash) • IT-Tools │
│ • AdGuard Home (backup DNS) │
│ │
│ 💬 Communication 📝 Productivity 🎮 Other │
│ ───────────── ───────────── ───────────── │
│ • Matrix Synapse • Documenso • Stirling PDF │
│ • Mastodon • Joplin Server • YouTube DL │
│ • Mattermost │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🏢 CALYPSO (54 Containers) - Auth, Proxy, Arr Suite & Development │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 🔐 Auth 📥 Arr Suite 💻 Development 📦 Infrastructure │
│ ───────────── ───────────── ───────────── ───────────── │
│ • Authentik • Sonarr • Gitea • Headscale │
│ • Authentik Outpost • Radarr • Reactive Resume • AdGuard Home │
│ • Lidarr • Seafile • Portainer Agent │
│ • Readarr • Wireguard │
│ 💰 Finance • Prowlarr 📝 Productivity │
│ ───────────── • SABnzbd ───────────── │
│ • Actual Budget • Deluge (Gluetun) • Paperless-NGX │
│ • Bazarr • Rustdesk │
│ • Whisparr │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 💻 HOMELAB VM (30 Containers) - Monitoring, Tools & Privacy │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 📊 Monitoring 🔔 Notifications 🔌 DCIM 🔧 Utilities │
│ ───────────── ───────────── ───────────── ───────────── │
│ • Grafana • ntfy • NetBox • Archivebox │
│ • Prometheus • Signal-API • Hoarder │
│ • Alertmanager 🔒 Privacy • Perplexica │
│ • SNMP Exporter 🤖 AI/Dev ───────────── • OpenHands │
│ • node_exporter ───────────── • Redlib │
│ • OpenHands • Binternet │
│ • Perplexica • ProxiTok │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🌐 CONCORD NUC (19 Containers) - Home Automation & Edge │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 🏠 Home Automation 📺 Media 🎵 Music 🔧 Network │
│ ───────────── ───────────── ───────────── ───────────── │
│ • Home Assistant • Plex • Your-Spotify • AdGuard Home │
│ • Matter Server • Invidious • Wireguard │
│ • Whisper (STT) │
│ • Piper (TTS) │
│ • OpenWakeWord │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🍓 RPi 5 (3 Containers) - Monitoring │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 📊 Monitoring │
│ ───────────── │
│ • Uptime Kuma │
│ • Glances │
│ • Portainer Agent │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🤖 OLARES - K8s Node (Core Ultra 9 275HX, RTX 5090, 96GB) │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 🧠 AI/ML (Kubernetes, not Docker) │
│ ───────────────────────────────── │
│ • Ollama (LLM serving) │
│ • vLLM (high-throughput inference) │
│ • OpenClaw (robotics foundation model) │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ 🌵 SETILLO (4 Services) - Tucson Remote │
│ ═══════════════════════════════════════════════════════════════════════════════════════│
│ │
│ 📊 Monitoring 🌐 DNS │
│ ───────────── ───────────── │
│ • Prometheus • AdGuard Home │
│ • SNMP Exporter • Syncthing │
│ │
└─────────────────────────────────────────────────────────────────────────────────────────┘
╔════════════════════════════════════════════════════════════════════════════════════════╗
║ SERVICE COUNT SUMMARY ║
║ ═════════════════════ ║
║ Atlantis: 59 containers │ Calypso: 61 containers ║
║ Homelab VM: 38 containers │ Concord NUC: 19 containers ║
║ RPi 5: 6 containers │ matrix-ubuntu: 12+ containers (NPM, Matrix) ║
║ Olares: K8s (~60 pods, not Portainer) ║
║ ──────────────────────────────────────────────────────────────────────────────────────║
║ TOTAL: ~195 containers across 5 Portainer endpoints + matrix-ubuntu + Olares ║
╚════════════════════════════════════════════════════════════════════════════════════════╝
```
---
## 🔗 Related Diagrams
- [Network Topology](network-topology.md) - How hosts connect
- [Storage Topology](storage-topology.md) - Where data lives
- [Tailscale Mesh](tailscale-mesh.md) - Cross-location access
---
## 💬 Communication Stack Architecture
```mermaid
graph TB
subgraph Internet["☁️ Internet / Federation"]
FEDI["Fediverse<br/>(ActivityPub)"]
MATRIX_FED["Matrix<br/>Federation"]
WEBRTC["WebRTC<br/>Voice/Video"]
end
subgraph Cloudflare["🛡️ Cloudflare"]
CF_PROXY["Cloudflare<br/>Proxy/WAF"]
CF_TUNNEL["Cloudflare<br/>Tunnel"]
end
subgraph MatrixUbuntuVM["🐧 Matrix-Ubuntu VM (Atlantis)"]
subgraph Mastodon["🐘 Mastodon Stack"]
MASTO_WEB["Mastodon Web<br/>:3000"]
MASTO_STREAM["Mastodon Streaming<br/>:4000"]
MASTO_SIDEKIQ["Sidekiq<br/>Background Jobs"]
end
subgraph Matrix["🔐 Matrix Stack"]
SYNAPSE["Synapse<br/>:8008 / :8018"]
ELEMENT["Element Web<br/>Client"]
COTURN["Coturn<br/>TURN Server<br/>:3478"]
end
subgraph Mattermost["💬 Mattermost"]
MM_APP["Mattermost<br/>:8065"]
end
subgraph SharedDB["🗄️ Shared Services"]
POSTGRES["PostgreSQL<br/>:5432"]
REDIS["Redis<br/>:6379"]
end
NPM_VM["NPM<br/>Reverse Proxy<br/>(host nginx disabled)"]
end
subgraph Atlantis["🏛️ Atlantis NAS"]
subgraph JitsiStack["📹 Jitsi Meet"]
JITSI_WEB["Jitsi Web"]
JITSI_JVB["Jitsi Video Bridge"]
JITSI_PROSODY["Prosody XMPP"]
end
subgraph Vaultwarden["🔑 Vaultwarden"]
VW["Vaultwarden<br/>Password Manager"]
end
subgraph Joplin["📝 Joplin"]
JOPLIN_SRV["Joplin Server"]
end
end
subgraph Clients["📱 Clients"]
BROWSER["Web Browsers"]
MOBILE["Mobile Apps"]
DESKTOP["Desktop Apps"]
end
%% External connections
FEDI <--> CF_PROXY
MATRIX_FED <--> CF_PROXY
WEBRTC <--> COTURN
%% Cloudflare to services
CF_PROXY --> NPM_VM
CF_TUNNEL --> NPM_VM
%% NPM routing (host nginx disabled, NPM handles all)
NPM_VM --> MASTO_WEB & MASTO_STREAM
NPM_VM --> SYNAPSE & ELEMENT
NPM_VM --> MM_APP
%% Database connections
MASTO_WEB & MASTO_SIDEKIQ --> POSTGRES & REDIS
SYNAPSE --> POSTGRES
MM_APP --> POSTGRES
%% Client access
BROWSER & MOBILE & DESKTOP --> CF_PROXY
BROWSER & MOBILE & DESKTOP --> JITSI_WEB
BROWSER & MOBILE & DESKTOP --> VW
BROWSER & MOBILE & DESKTOP --> JOPLIN_SRV
classDef mastodon fill:#6364FF,stroke:#333,stroke-width:2px,color:#fff
classDef matrix fill:#0DBD8B,stroke:#333,stroke-width:2px,color:#fff
classDef mattermost fill:#0058CC,stroke:#333,stroke-width:2px,color:#fff
classDef infra fill:#e67e22,stroke:#333,stroke-width:2px,color:#fff
class MASTO_WEB,MASTO_STREAM,MASTO_SIDEKIQ mastodon
class SYNAPSE,ELEMENT,COTURN matrix
class MM_APP mattermost
class POSTGRES,REDIS,NPM_VM infra
```
### Communication Services Summary
| Service | Domain | Protocol | Purpose |
|---------|--------|----------|---------|
| **Mastodon** | mastodon.vish.gg | ActivityPub | Fediverse microblogging |
| **Matrix (Primary)** | mx.vish.gg | Matrix | Federated chat |
| **Matrix (Legacy)** | matrix.thevish.io | Matrix | Legacy homeserver |
| **Mattermost** | mm.crista.love | Proprietary | Team collaboration |
| **Jitsi Meet** | meet.vish.gg | WebRTC | Video conferencing |
| **Joplin** | joplin.vish.gg | Joplin Sync | Note synchronization |
| **Vaultwarden** | vault.vish.gg | Bitwarden | Password management |
### Deployment Scripts
| Script | Location | Description |
|--------|----------|-------------|
| Mastodon Install | [mastodon-production/](../mastodon-production/) | Bare metal & Docker deployment |
| Matrix Install | [matrix-element/](../matrix-element/) | Synapse + Element + TURN |
| Mattermost Install | [mattermost-production/](../mattermost-production/) | Docker deployment |
| VM Config | [matrix-ubuntu-vm/](../matrix-ubuntu-vm/) | Complete VM configuration |
---
## 🔄 CI/CD Pipeline Architecture
### Git Repository Mirroring
The homelab repository uses Gitea Actions for automated CI/CD, including sanitized public mirroring.
```mermaid
graph LR
subgraph Development["💻 Development"]
DEV["Developer<br/>Pushes Code"]
end
subgraph Gitea["🔧 Gitea (Calypso)"]
PRIVATE["🔒 Private Repo<br/>homelab"]
PUBLIC["🌐 Public Repo<br/>homelab-optimized"]
RUNNER["🏃 Gitea Runners<br/>(homelab, calypso, pi5)"]
end
subgraph Workflow["⚙️ CI/CD Workflow"]
CHECKOUT["📥 Checkout Code"]
SANITIZE["🧹 Sanitize<br/>Remove Secrets"]
PUSH["📤 Force Push<br/>Fresh History"]
end
subgraph Deployment["🚀 Deployment"]
ANSIBLE["📋 Ansible<br/>Multi-host"]
PORTAINER["🐳 Portainer<br/>5 Endpoints"]
end
DEV -->|"git push"| PRIVATE
PRIVATE -->|"Triggers"| RUNNER
RUNNER --> CHECKOUT
CHECKOUT --> SANITIZE
SANITIZE --> PUSH
PUSH --> PUBLIC
PRIVATE --> ANSIBLE
ANSIBLE --> PORTAINER
```
### Sanitization Process
The sanitization script removes sensitive data before public mirroring:
| Removed | Pattern | Example |
|---------|---------|---------|
| Passwords | `password:`, `PASS=` | `password: "REDACTED_PASSWORD" |
| API Keys | `api_key:`, `API_KEY=` | `api_key: REDACTED_API_KEY` |
| Tokens | `token:`, `TOKEN=` | `token: REDACTED_TOKEN` |
| Secrets | `secret:`, `SECRET=` | `secret: REDACTED_SECRET` |
| Private Keys | `-----BEGIN.*KEY-----` | File removed |
| SSH Keys | `id_rsa`, `id_ed25519` | File removed |
| Personal Emails | `*@gmail.com`, `*@*.com` | `REDACTED_EMAIL@example.com` |
| JWT Secrets | `JWT_SECRET=` | `JWT_SECRET=REDACTED` |
### Gitea Runner Setup
```mermaid
graph TB
subgraph Calypso["🌊 Calypso (DS723+)"]
GITEA["🔧 Gitea Server<br/>:3052"]
RUNNER_CAL["🏃 Runner (calypso)"]
end
subgraph HomelabVM["💻 Homelab VM"]
RUNNER_HLB["🏃 Runner (homelab)"]
end
subgraph Pi5["🍓 RPi 5"]
RUNNER_PI["🏃 Runner (pi5)"]
end
GITEA -->|"Workflow Dispatch"| RUNNER_CAL
GITEA -->|"Workflow Dispatch"| RUNNER_HLB
GITEA -->|"Workflow Dispatch"| RUNNER_PI
```
**Runner Configuration:**
- Runner binary: `act_runner` v0.2.6, systemd service (not Docker container)
- Labels: `ubuntu-latest`, `linux`, `python` (all 3 runners)
- Runners: homelab (VM), calypso, pi5
- Trigger: Push to main branch
### Ansible Automation
```mermaid
graph TB
subgraph Control["📋 Ansible Control"]
SITE["site.yml<br/>Master Playbook"]
INV["inventory.yml<br/>13 Hosts"]
ROLES["Roles<br/>docker_stack, directory_setup"]
end
subgraph Hosts["🖥️ Target Hosts"]
SYN["Synology<br/>Atlantis, Calypso, Setillo"]
VMS["VMs<br/>Homelab, matrix-ubuntu"]
PHYS["Physical<br/>Guava, NUC, Shinku-Ryuu"]
EDGE["Edge<br/>RPi5, Jellyfish"]
CLOUD["Cloud<br/>Seattle VPS"]
end
SITE --> INV
INV --> SYN
INV --> VMS
INV --> PHYS
INV --> EDGE
INV --> CLOUD
```
**Ansible Commands:**
```bash
# Deploy everything
ansible-playbook site.yml
# Deploy to specific host
ansible-playbook site.yml --limit atlantis
# Deploy by category
ansible-playbook site.yml --tags synology
# Check status
ansible-playbook playbooks/common/status.yml
```
---
## 🧠 AI/ML Stack Architecture
```mermaid
graph TB
subgraph Olares["🤖 Olares K8s Node (Core Ultra 9 275HX, RTX 5090, 96GB)"]
OLLAMA["🦙 Ollama<br/>LLM Serving<br/>Local Models"]
VLLM["⚡ vLLM<br/>High-Throughput<br/>Inference Engine"]
OPENCLAW["🤖 OpenClaw<br/>Robotics Foundation<br/>Model"]
end
subgraph Clients["📱 AI Consumers"]
ANYTHINGLLM["💬 AnythingLLM<br/>RAG Chat"]
OPENWEBUI["🌐 Open WebUI"]
API_CLIENTS["🔧 API Clients"]
end
OLLAMA -->|"OpenAI-compatible API"| Clients
VLLM -->|"OpenAI-compatible API"| Clients
classDef ai fill:#8e44ad,stroke:#333,stroke-width:2px,color:#fff
classDef client fill:#2980b9,stroke:#333,stroke-width:2px,color:#fff
class OLLAMA,VLLM,OPENCLAW ai
class ANYTHINGLLM,OPENWEBUI,API_CLIENTS client
```
### AI/ML Services Summary
| Service | Host | Type | Purpose |
|---------|------|------|---------|
| **Ollama** | Olares (K8s) | LLM Server | Local model serving (Llama, Mistral, etc.) |
| **vLLM** | Olares (K8s) | Inference Engine | High-throughput batched inference |
| **OpenClaw** | Olares (K8s) | Foundation Model | Robotics/manipulation research |
| **AnythingLLM** | Homelab VM | RAG Client | Document Q&A with local LLMs |
---
## 🔗 Related Diagrams
- [Network Topology](network-topology.md) - How hosts connect
- [Storage Topology](storage-topology.md) - Where data lives
- [Tailscale Mesh](tailscale-mesh.md) - Cross-location access