# 🏗️ Service Architecture

## Overview

This document shows how the 176+ Docker services interact, their dependencies, and the data flows between them.

---

## 🎬 Media Stack Architecture (Mermaid)

```mermaid
graph TB
    subgraph Internet["☁️ Internet Sources"]
        USENET["Usenet<br/>Providers"]
        TORRENT["Torrent<br/>Trackers"]
        INDEXERS["Indexers<br/>(NZB/Torrent)"]
    end

    subgraph Acquisition["📥 Content Acquisition (Atlantis)"]
        PROWLARR["Prowlarr<br/>Indexer Manager"]
        SONARR["Sonarr<br/>TV Shows"]
        RADARR["Radarr<br/>Movies"]
        LIDARR["Lidarr<br/>Music"]
        READARR["Readarr<br/>Books"]
        BAZARR["Bazarr<br/>Subtitles"]
        
        SAB["SABnzbd<br/>Usenet Client"]
        QBIT["qBittorrent<br/>Torrent Client"]
    end

    subgraph Storage["💾 Storage (Atlantis NAS)"]
        MEDIA_TV["/volume1/media/tv"]
        MEDIA_MOV["/volume1/media/movies"]
        MEDIA_MUS["/volume1/media/music"]
        MEDIA_BOOK["/volume1/media/books"]
    end

    subgraph Streaming["📺 Media Streaming"]
        PLEX["Plex<br/>Media Server"]
        JELLYFIN["Jellyfin<br/>Media Server"]
        NAVIDROME["Navidrome<br/>Music Server"]
        TAUTULLI["Tautulli<br/>Plex Analytics"]
    end

    subgraph Clients["📱 Client Devices"]
        TV["Smart TVs"]
        PHONE["Phones/Tablets"]
        WEB["Web Browsers"]
        APPS["Desktop Apps"]
    end

    %% Acquisition flow
    INDEXERS --> PROWLARR
    PROWLARR --> SONARR & RADARR & LIDARR & READARR
    
    SONARR --> SAB & QBIT
    RADARR --> SAB & QBIT
    LIDARR --> SAB & QBIT
    READARR --> SAB & QBIT
    
    USENET --> SAB
    TORRENT --> QBIT
    
    %% Storage flow
    SAB --> MEDIA_TV & MEDIA_MOV & MEDIA_MUS & MEDIA_BOOK
    QBIT --> MEDIA_TV & MEDIA_MOV & MEDIA_MUS & MEDIA_BOOK
    
    BAZARR --> MEDIA_TV & MEDIA_MOV
    
    %% Streaming flow
    MEDIA_TV & MEDIA_MOV --> PLEX & JELLYFIN
    MEDIA_MUS --> NAVIDROME
    
    PLEX --> TAUTULLI
    
    %% Client access
    PLEX & JELLYFIN & NAVIDROME --> TV & PHONE & WEB & APPS

    classDef acquisition fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
    classDef storage fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
    classDef streaming fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff
    classDef client fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff

    class PROWLARR,SONARR,RADARR,LIDARR,READARR,BAZARR,SAB,QBIT acquisition
    class MEDIA_TV,MEDIA_MOV,MEDIA_MUS,MEDIA_BOOK storage
    class PLEX,JELLYFIN,NAVIDROME,TAUTULLI streaming
    class TV,PHONE,WEB,APPS client
```

---

## 📊 Monitoring Stack Architecture

```mermaid
graph TB
    subgraph Targets["🎯 Monitored Targets"]
        subgraph Synology["Synology NAS"]
            ATL_SNMP["Atlantis<br/>SNMP"]
            CAL_SNMP["Calypso<br/>SNMP"]
            SET_SNMP["Setillo<br/>SNMP"]
        end
        
        subgraph Hosts["Linux Hosts"]
            NODE1["Homelab VM<br/>node_exporter"]
            NODE2["Guava<br/>node_exporter"]
            NODE3["Anubis<br/>node_exporter"]
        end
        
        subgraph Containers["Containers"]
            CADV["cAdvisor<br/>Container Metrics"]
        end
        
        subgraph Network["Network"]
            BLACK["Blackbox Exporter<br/>HTTP/ICMP Probes"]
        end
    end

    subgraph Collection["📥 Metric Collection (Homelab VM)"]
        PROM["Prometheus<br/>Time Series DB"]
        SNMP_EXP["SNMP Exporter"]
    end

    subgraph Visualization["📈 Visualization"]
        GRAFANA["Grafana<br/>Dashboards"]
    end

    subgraph Alerting["🚨 Alerting"]
        ALERTMGR["Alertmanager"]
        NTFY["ntfy<br/>Push Notifications"]
        UPTIME["Uptime Kuma<br/>Status Page"]
    end

    %% Collection
    ATL_SNMP & CAL_SNMP & SET_SNMP --> SNMP_EXP
    SNMP_EXP --> PROM
    NODE1 & NODE2 & NODE3 --> PROM
    CADV --> PROM
    BLACK --> PROM

    %% Visualization
    PROM --> GRAFANA
    PROM --> ALERTMGR
    ALERTMGR --> NTFY
    
    %% Uptime Kuma separate
    BLACK -.-> UPTIME

    classDef target fill:#e67e22,stroke:#333,stroke-width:2px,color:#fff
    classDef collection fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
    classDef viz fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff
    classDef alert fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff

    class ATL_SNMP,CAL_SNMP,SET_SNMP,NODE1,NODE2,NODE3,CADV,BLACK target
    class PROM,SNMP_EXP collection
    class GRAFANA viz
    class ALERTMGR,NTFY,UPTIME alert
```

---

## 🔐 Authentication & Security Stack

### Complete Authentication Architecture

```mermaid
graph TB
    subgraph External["🌐 External Access"]
        USERS["👤 Users"]
        CLOUDFLARE["☁️ Cloudflare<br/>DNS/WAF/DDoS"]
    end

    subgraph Gateway["🚪 Gateway Layer (Atlantis)"]
        NPM["🔀 Nginx Proxy Manager<br/>:81/:443<br/>Reverse Proxy + SSL"]
        CFT["🚇 Cloudflare Tunnel<br/>Zero Trust Access"]
    end

    subgraph AuthLayer["🔐 Authentication Layer (Calypso)"]
        AUTH_SRV["🔐 Authentik Server<br/>:9000"]
        AUTH_PROXY["🛡️ Authentik Outpost<br/>:9444<br/>Forward Auth Proxy"]
        AUTH_WRK["⚙️ Authentik Worker"]
        AUTH_DB["🐘 PostgreSQL"]
        AUTH_RED["🔴 Redis"]
    end

    subgraph VPN["🔒 VPN Layer"]
        WIREGUARD["🔒 Wireguard<br/>Atlantis :51820"]
        TAILSCALE["🔷 Tailscale<br/>100.x.x.x"]
        HEADSCALE["🌐 Headscale<br/>Calypso :8080"]
    end

    subgraph DNS["🌐 DNS & Ad Blocking"]
        PIHOLE["🕳️ Pi-hole<br/>Atlantis :53"]
        ADGUARD1["🛡️ AdGuard<br/>Calypso :53"]
        ADGUARD2["🛡️ AdGuard<br/>Setillo :53"]
    end

    subgraph SecVault["🔑 Secrets Management"]
        VAULT["🔑 Vaultwarden<br/>vault.vish.gg"]
    end

    subgraph ProtectedServices["🛡️ Protected Services"]
        GRAFANA["📊 Grafana"]
        PAPERLESS["📄 Paperless"]
        IMMICH["📸 Immich"]
        FIREFLY["🔥 Firefly III"]
        ACTUAL["💰 Actual Budget"]
        GITEA["🔧 Gitea"]
    end

    subgraph PublicServices["🌍 Public/Self-Auth Services"]
        PLEX["📺 Plex"]
        SEAFILE["☁️ Seafile"]
        OST["🚀 OpenSpeedTest"]
        NTFY["📣 ntfy"]
    end

    %% External flow
    USERS --> CLOUDFLARE
    CLOUDFLARE --> NPM
    CLOUDFLARE --> CFT
    USERS --> TAILSCALE
    
    %% NPM to Auth
    NPM -->|"Forward Auth<br/>Header Check"| AUTH_PROXY
    AUTH_PROXY -->|"Validate Session"| AUTH_SRV
    
    %% Auth internal
    AUTH_SRV --> AUTH_DB
    AUTH_SRV --> AUTH_RED
    AUTH_WRK --> AUTH_DB
    AUTH_WRK --> AUTH_RED
    
    %% Protected services via NPM + Auth
    NPM -->|"✓ Authenticated"| ProtectedServices
    
    %% Public services direct
    NPM --> PublicServices
    
    %% VPN access
    TAILSCALE --> HEADSCALE
    WIREGUARD --> ProtectedServices
    TAILSCALE --> ProtectedServices
    
    %% DNS
    ADGUARD1 -.-> ProtectedServices
    PIHOLE -.-> PublicServices

    classDef external fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
    classDef gateway fill:#f39c12,stroke:#333,stroke-width:2px,color:#fff
    classDef auth fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff
    classDef dns fill:#1abc9c,stroke:#333,stroke-width:2px,color:#fff
    classDef protected fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
    classDef public fill:#27ae60,stroke:#333,stroke-width:2px,color:#fff

    class USERS,CLOUDFLARE external
    class NPM,CFT gateway
    class AUTH_SRV,AUTH_PROXY,AUTH_WRK,AUTH_DB,AUTH_RED,VAULT auth
    class PIHOLE,ADGUARD1,ADGUARD2 dns
    class GRAFANA,PAPERLESS,IMMICH,FIREFLY,ACTUAL,GITEA protected
    class PLEX,SEAFILE,OST,NTFY public
```

---

### Authentik SSO Flow (Detailed)

```mermaid
sequenceDiagram
    autonumber
    participant U as 👤 User
    participant CF as ☁️ Cloudflare
    participant NPM as 🔀 NPM (Atlantis)
    participant OUT as 🛡️ Outpost (Calypso)
    participant AUTH as 🔐 Authentik (Calypso)
    participant APP as 📱 Application

    U->>CF: Request app.vish.gg
    CF->>NPM: Forward (HTTPS)
    NPM->>OUT: Forward Auth Request<br/>(/outpost.goauthentik.io/auth/nginx)
    
    alt No Valid Session
        OUT->>AUTH: Check Session
        AUTH-->>OUT: No Session
        OUT-->>NPM: 401 Unauthorized
        NPM-->>U: Redirect to sso.vish.gg/flows/default-authentication/
        U->>AUTH: Login Page
        U->>AUTH: Submit Credentials + 2FA
        AUTH->>AUTH: Validate
        AUTH-->>U: Set Cookie + Redirect to app
        U->>NPM: Retry with Session Cookie
        NPM->>OUT: Forward Auth (with cookie)
    end
    
    OUT->>AUTH: Validate Session
    AUTH-->>OUT: Valid ✓
    OUT-->>NPM: 200 OK + Headers<br/>(X-authentik-username, X-authentik-email)
    NPM->>APP: Proxy Request (with auth headers)
    APP-->>U: Response
```

---

### NPM Proxy Host Configuration

```mermaid
graph TB
    subgraph NPM["🔀 Nginx Proxy Manager (Atlantis :81)"]
        subgraph ProxyHosts["Proxy Hosts"]
            PH1["sso.vish.gg → Calypso:9000"]
            PH2["git.vish.gg → Calypso:3000"]
            PH3["docs.vish.gg → Atlantis:8088"]
            PH4["photos.vish.gg → Calypso:2283"]
            PH5["gf.vish.gg → Atlantis:3000"]
            PH6["actual.vish.gg → Calypso:5006"]
            PH7["ff.vish.gg → Calypso:8888"]
            PH8["plex.vish.gg → Atlantis:32400"]
            PH9["sf.vish.gg → Calypso:8092"]
            PH10["ntfy.vish.gg → Homelab:7080"]
            PH11["rackula.vish.gg → Calypso:4999"]
        end
        
        subgraph SSL["SSL Certificates"]
            WILD["*.vish.gg<br/>Cloudflare DNS Challenge"]
        end
        
        subgraph AccessControl["Access Control"]
            AUTH_LOC["Authentik Forward Auth<br/>Location: /outpost.goauthentik.io"]
        end
    end
    
    subgraph Services["Backend Services"]
        direction LR
        S1["Authentik"]
        S2["Gitea"]
        S3["Paperless"]
        S4["Immich"]
        S5["Grafana"]
        S6["Actual"]
        S7["Firefly"]
        S8["Plex"]
        S9["Seafile"]
        S10["ntfy"]
        S11["Rackula"]
    end
    
    PH1 --> S1
    PH2 --> S2
    PH3 --> S3
    PH4 --> S4
    PH5 --> S5
    PH6 --> S6
    PH7 --> S7
    PH8 --> S8
    PH9 --> S9
    PH10 --> S10
    PH11 --> S11
```

---

### Services Protected by Authentik

| Domain | Service | Host | Auth Type | Notes |
|--------|---------|------|-----------|-------|
| `sso.vish.gg` | Authentik | Calypso | - | Identity Provider |
| `git.vish.gg` | Gitea | Calypso | OAuth2/OIDC | Source Control |
| `gf.vish.gg` | Grafana | Atlantis | OAuth2/OIDC | Monitoring |
| `docs.vish.gg` | Paperless-NGX | Atlantis | Forward Auth | Documents |
| `photos.vish.gg` | Immich | Calypso | Forward Auth | Photos |
| `actual.vish.gg` | Actual Budget | Calypso | Forward Auth | Finance |
| `ff.vish.gg` | Firefly III | Calypso | Forward Auth | Finance |
| `rackula.vish.gg` | Rackula | Calypso | Forward Auth | Rack Diagram |

### Services NOT Protected (Public/Self-Auth)

| Domain | Service | Host | Reason |
|--------|---------|------|--------|
| `plex.vish.gg` | Plex | Atlantis | Has Plex Auth |
| `sf.vish.gg` | Seafile | Calypso | Has built-in auth + share links |
| `ntfy.vish.gg` | ntfy | Homelab | Has built-in auth + public topics |
| `ost.vish.gg` | OpenSpeedTest | Calypso | Public utility |

---

### Authentik Forward Auth Setup (NPM)

To protect a service with Authentik Forward Auth in NPM:

1. **Create Provider in Authentik**:
   - Type: Proxy Provider
   - External Host: `https://app.vish.gg`
   - Mode: Forward auth (single application)

2. **Create Application in Authentik**:
   - Link to the provider
   - Set policies for access control

3. **Create Outpost in Authentik**:
   - Type: Proxy
   - Include the application

4. **Configure NPM Proxy Host**:
   ```nginx
   # Custom Nginx Configuration (Advanced tab)
   
   # Authentik Forward Auth
   location /outpost.goauthentik.io {
       proxy_pass              http://calypso.vish.local:9444/outpost.goauthentik.io;
       proxy_set_header        Host $host;
       proxy_set_header        X-Original-URL $scheme://$http_host$request_uri;
       add_header              Set-Cookie $auth_cookie;
       auth_request_set        $auth_cookie $upstream_http_set_cookie;
       proxy_pass_request_body off;
       proxy_set_header        Content-Length "";
   }
   
   location / {
       auth_request        /outpost.goauthentik.io/auth/nginx;
       error_page          401 = @goauthentik_proxy_signin;
       auth_request_set    $auth_cookie $upstream_http_set_cookie;
       add_header          Set-Cookie $auth_cookie;
       
       # Forward auth headers to application
       auth_request_set    $authentik_username $upstream_http_x_authentik_username;
       auth_request_set    $authentik_email $upstream_http_x_authentik_email;
       proxy_set_header    X-authentik-username $authentik_username;
       proxy_set_header    X-authentik-email $authentik_email;
       
       proxy_pass          http://backend;
   }
   
   location @goauthentik_proxy_signin {
       internal;
       add_header Set-Cookie $auth_cookie;
       return 302 /outpost.goauthentik.io/start?rd=$request_uri;
   }
   ```

---

## 📝 ASCII Service Distribution by Host

```
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║                        SERVICE DISTRIBUTION BY HOST                                       ║
╚══════════════════════════════════════════════════════════════════════════════════════════╝

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  🏛️ ATLANTIS (55 Services) - Primary Hub                                                │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  📺 Media              📊 Monitoring         🔐 Security          🛠️ Infrastructure     │
│  ─────────────         ─────────────         ─────────────        ─────────────────     │
│  • Plex                • Grafana             • Vaultwarden        • Portainer           │
│  • Jellyfin            • Prometheus          • Wireguard          • Nginx Proxy Mgr     │
│  • Immich              • Uptime Kuma         • Pi-hole            • DokuWiki            │
│  • Tautulli            • SNMP Exporter                            • Dozzle              │
│  • Arr Suite (7)       • Blackbox Exp                             • Watchtower          │
│  • Navidrome                                                                             │
│                                                                                          │
│  💬 Communication      📝 Productivity       🎮 Other                                    │
│  ─────────────         ─────────────         ─────────────                              │
│  • Matrix Synapse      • Paperless-NGX       • IT-Tools                                 │
│  • Mastodon            • Firefly III         • Stirling PDF                             │
│  • Joplin Server       • Documenso           • YouTube DL                               │
│                        • Netbox                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  🏢 CALYPSO (17 Services) - Development & Backup                                         │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  💻 Development        💰 Finance            📦 Infrastructure    📸 Media              │
│  ─────────────         ─────────────         ─────────────        ─────────────         │
│  • Gitea               • Firefly III         • APT-Cacher-NG      • Immich (backup)     │
│  • Reactive Resume     • Actual Budget       • Prometheus         • Seafile             │
│  • Rustdesk                                  • Wireguard                                │
│                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  💻 HOMELAB VM (36 Services) - Experimentation                                           │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  🔗 URL Services       🎮 Gaming             📊 Monitoring        🔧 Utilities          │
│  ─────────────         ─────────────         ─────────────        ─────────────         │
│  • Shlink              • Satisfactory        • Prometheus Hub     • Archivebox          │
│  • ntfy                • Minecraft           • node_exporter      • WebCheck            │
│  • Hoarder             • L4D2                                     • Redlib              │
│                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  🌐 CONCORD NUC (9 Services) - Edge/IoT                                                  │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  🏠 Home Automation    📺 Media              🎵 Music             🔧 Network            │
│  ─────────────         ─────────────         ─────────────        ─────────────         │
│  • Home Assistant      • Plex                • YourSpotify        • AdGuard Home        │
│  • Matter Server       • Invidious           • Piped              • Wireguard           │
│                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  🤖 ANUBIS (8 Services) - AI/HPC (Mac Mini Ubuntu)                                       │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  🧠 AI/ML              📸 Photo              🔧 Development                              │
│  ─────────────         ─────────────         ─────────────                              │
│  • Ollama              • PhotoPrism          • Draw.io                                  │
│  • ChatGPT UI          • Archivebox          • Element Web                              │
│                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────────────┐
│  🌵 SETILLO (4 Services) - Tucson Remote                                                 │
│  ═══════════════════════════════════════════════════════════════════════════════════════│
│                                                                                          │
│  📊 Monitoring         🌐 DNS                                                            │
│  ─────────────         ─────────────                                                    │
│  • Prometheus          • AdGuard Home                                                   │
│  • SNMP Exporter       • Syncthing                                                      │
│                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

╔════════════════════════════════════════════════════════════════════════════════════════╗
║  SERVICE COUNT SUMMARY                                                                  ║
║  ═════════════════════                                                                  ║
║  Atlantis:     55 services    │  Calypso:      17 services                             ║
║  Homelab VM:   36 services    │  Chicago VM:    8 services                             ║
║  Bulgaria VM:  12 services    │  Concord NUC:   9 services                             ║
║  Anubis:        8 services    │  Guava:         6 services                             ║
║  Setillo:       4 services    │  RPi nodes:     2 services                             ║
║  Contabo:       1 service     │                                                         ║
║  ──────────────────────────────────────────────────────────────────────────────────────║
║  TOTAL:       176+ services across 13 hosts                                            ║
╚════════════════════════════════════════════════════════════════════════════════════════╝
```

---

## 🔗 Related Diagrams
- [Network Topology](network-topology.md) - How hosts connect
- [Storage Topology](storage-topology.md) - Where data lives
- [Tailscale Mesh](tailscale-mesh.md) - Cross-location access

---

## 💬 Communication Stack Architecture

```mermaid
graph TB
    subgraph Internet["☁️ Internet / Federation"]
        FEDI["Fediverse<br/>(ActivityPub)"]
        MATRIX_FED["Matrix<br/>Federation"]
        WEBRTC["WebRTC<br/>Voice/Video"]
    end

    subgraph Cloudflare["🛡️ Cloudflare"]
        CF_PROXY["Cloudflare<br/>Proxy/WAF"]
        CF_TUNNEL["Cloudflare<br/>Tunnel"]
    end

    subgraph MatrixUbuntuVM["🐧 Matrix-Ubuntu VM (Atlantis)"]
        subgraph Mastodon["🐘 Mastodon Stack"]
            MASTO_WEB["Mastodon Web<br/>:3000"]
            MASTO_STREAM["Mastodon Streaming<br/>:4000"]
            MASTO_SIDEKIQ["Sidekiq<br/>Background Jobs"]
        end

        subgraph Matrix["🔐 Matrix Stack"]
            SYNAPSE["Synapse<br/>:8008 / :8018"]
            ELEMENT["Element Web<br/>Client"]
            COTURN["Coturn<br/>TURN Server<br/>:3478"]
        end

        subgraph Mattermost["💬 Mattermost"]
            MM_APP["Mattermost<br/>:8065"]
        end

        subgraph SharedDB["🗄️ Shared Services"]
            POSTGRES["PostgreSQL<br/>:5432"]
            REDIS["Redis<br/>:6379"]
        end

        NGINX_VM["Nginx<br/>Reverse Proxy"]
    end

    subgraph Atlantis["🏛️ Atlantis NAS"]
        subgraph JitsiStack["📹 Jitsi Meet"]
            JITSI_WEB["Jitsi Web"]
            JITSI_JVB["Jitsi Video Bridge"]
            JITSI_PROSODY["Prosody XMPP"]
        end

        subgraph Vaultwarden["🔑 Vaultwarden"]
            VW["Vaultwarden<br/>Password Manager"]
        end

        subgraph Joplin["📝 Joplin"]
            JOPLIN_SRV["Joplin Server"]
        end
    end

    subgraph Clients["📱 Clients"]
        BROWSER["Web Browsers"]
        MOBILE["Mobile Apps"]
        DESKTOP["Desktop Apps"]
    end

    %% External connections
    FEDI <--> CF_PROXY
    MATRIX_FED <--> CF_PROXY
    WEBRTC <--> COTURN

    %% Cloudflare to services
    CF_PROXY --> NGINX_VM
    CF_TUNNEL --> NGINX_VM

    %% Nginx routing
    NGINX_VM --> MASTO_WEB & MASTO_STREAM
    NGINX_VM --> SYNAPSE & ELEMENT
    NGINX_VM --> MM_APP

    %% Database connections
    MASTO_WEB & MASTO_SIDEKIQ --> POSTGRES & REDIS
    SYNAPSE --> POSTGRES
    MM_APP --> POSTGRES

    %% Client access
    BROWSER & MOBILE & DESKTOP --> CF_PROXY
    BROWSER & MOBILE & DESKTOP --> JITSI_WEB
    BROWSER & MOBILE & DESKTOP --> VW
    BROWSER & MOBILE & DESKTOP --> JOPLIN_SRV

    classDef mastodon fill:#6364FF,stroke:#333,stroke-width:2px,color:#fff
    classDef matrix fill:#0DBD8B,stroke:#333,stroke-width:2px,color:#fff
    classDef mattermost fill:#0058CC,stroke:#333,stroke-width:2px,color:#fff
    classDef infra fill:#e67e22,stroke:#333,stroke-width:2px,color:#fff

    class MASTO_WEB,MASTO_STREAM,MASTO_SIDEKIQ mastodon
    class SYNAPSE,ELEMENT,COTURN matrix
    class MM_APP mattermost
    class POSTGRES,REDIS,NGINX_VM infra
```

### Communication Services Summary

| Service | Domain | Protocol | Purpose |
|---------|--------|----------|---------|
| **Mastodon** | mastodon.vish.gg | ActivityPub | Fediverse microblogging |
| **Matrix (Primary)** | mx.vish.gg | Matrix | Federated chat |
| **Matrix (Legacy)** | matrix.thevish.io | Matrix | Legacy homeserver |
| **Mattermost** | mm.crista.love | Proprietary | Team collaboration |
| **Jitsi Meet** | meet.vish.gg | WebRTC | Video conferencing |
| **Joplin** | joplin.vish.gg | Joplin Sync | Note synchronization |
| **Vaultwarden** | vault.vish.gg | Bitwarden | Password management |

### Deployment Scripts

| Script | Location | Description |
|--------|----------|-------------|
| Mastodon Install | [mastodon-production/](../mastodon-production/) | Bare metal & Docker deployment |
| Matrix Install | [matrix-element/](../matrix-element/) | Synapse + Element + TURN |
| Mattermost Install | [mattermost-production/](../mattermost-production/) | Docker deployment |
| VM Config | [matrix-ubuntu-vm/](../matrix-ubuntu-vm/) | Complete VM configuration |

---

## 🔄 CI/CD Pipeline Architecture

### Git Repository Mirroring

The homelab repository uses Gitea Actions for automated CI/CD, including sanitized public mirroring.

```mermaid
graph LR
    subgraph Development["💻 Development"]
        DEV["Developer<br/>Pushes Code"]
    end
    
    subgraph Gitea["🔧 Gitea (Calypso)"]
        PRIVATE["🔒 Private Repo<br/>homelab"]
        PUBLIC["🌐 Public Repo<br/>homelab-optimized"]
        RUNNER["🏃 Gitea Runner<br/>(Calypso)"]
    end
    
    subgraph Workflow["⚙️ CI/CD Workflow"]
        CHECKOUT["📥 Checkout Code"]
        SANITIZE["🧹 Sanitize<br/>Remove Secrets"]
        PUSH["📤 Force Push<br/>Fresh History"]
    end
    
    subgraph Deployment["🚀 Deployment"]
        ANSIBLE["📋 Ansible<br/>164 Services"]
        PORTAINER["🐳 Portainer<br/>5 Endpoints"]
    end
    
    DEV -->|"git push"| PRIVATE
    PRIVATE -->|"Triggers"| RUNNER
    RUNNER --> CHECKOUT
    CHECKOUT --> SANITIZE
    SANITIZE --> PUSH
    PUSH --> PUBLIC
    
    PRIVATE --> ANSIBLE
    ANSIBLE --> PORTAINER
```

### Sanitization Process

The sanitization script removes sensitive data before public mirroring:

| Removed | Pattern | Example |
|---------|---------|---------|
| Passwords | `password:`, `PASS=` | `password: "REDACTED_PASSWORD" |
| API Keys | `api_key:`, `API_KEY=` | `api_key: REDACTED_API_KEY` |
| Tokens | `token:`, `TOKEN=` | `token: REDACTED_TOKEN` |
| Secrets | `secret:`, `SECRET=` | `secret: REDACTED_SECRET` |
| Private Keys | `-----BEGIN.*KEY-----` | File removed |
| SSH Keys | `id_rsa`, `id_ed25519` | File removed |
| Personal Emails | `*@gmail.com`, `*@*.com` | `REDACTED_EMAIL@example.com` |
| JWT Secrets | `JWT_SECRET=` | `JWT_SECRET=REDACTED` |

### Gitea Runner Setup

```mermaid
graph TB
    subgraph Calypso["🌊 Calypso (DS920+)"]
        GITEA["🔧 Gitea Server<br/>:3000"]
        RUNNER["🏃 Gitea Runner<br/>act_runner:latest"]
        DOCKER["🐳 Docker Socket"]
    end
    
    GITEA -->|"Workflow Dispatch"| RUNNER
    RUNNER -->|"docker.sock"| DOCKER
    DOCKER -->|"Spawn Containers"| RUNNER
```

**Runner Configuration:**
- Image: `gitea/act_runner:latest`
- Labels: `ubuntu-latest`, `ubuntu-22.04`, `python`
- Location: Portainer stack on Calypso
- Trigger: Push to main branch

### Ansible Automation

```mermaid
graph TB
    subgraph Control["📋 Ansible Control"]
        SITE["site.yml<br/>Master Playbook"]
        INV["inventory.yml<br/>13 Hosts"]
        ROLES["Roles<br/>docker_stack, directory_setup"]
    end
    
    subgraph Hosts["🖥️ Target Hosts"]
        SYN["Synology<br/>Atlantis, Calypso, Setillo"]
        VMS["VMs<br/>Homelab, Chicago, Bulgaria"]
        PHYS["Physical<br/>Guava, NUC, Anubis"]
        EDGE["Edge<br/>RPi5"]
    end
    
    SITE --> INV
    INV --> SYN
    INV --> VMS
    INV --> PHYS
    INV --> EDGE
```

**Ansible Commands:**
```bash
# Deploy everything
ansible-playbook site.yml

# Deploy to specific host
ansible-playbook site.yml --limit atlantis

# Deploy by category
ansible-playbook site.yml --tags synology

# Check status
ansible-playbook playbooks/common/status.yml
```

---

## 🔗 Related Diagrams
- [Network Topology](network-topology.md) - How hosts connect
- [Storage Topology](storage-topology.md) - Where data lives
- [Tailscale Mesh](tailscale-mesh.md) - Cross-location access