Files
homelab-optimized/docs/CHANGELOG.md
Gitea Mirror Bot d90cf1f849
Some checks failed
Documentation / Deploy to GitHub Pages (push) Has been cancelled
Documentation / Build Docusaurus (push) Has been cancelled
Sanitized mirror from private repository - 2026-04-19 09:52:01 UTC
2026-04-19 09:52:01 +00:00

431 lines
28 KiB
Markdown

## 2026-04-16
**Infrastructure**
- Deploy GL-MT3600BE (Beryl 7) as primary gateway replacing GL-MT3000 — Headscale node ID:28, exit node enabled, subnet route 192.168.12.0/24, watchdog cron, SSH key auth
- Jellyfish now on Beryl 7 LAN (192.168.12.181), moon confirmed at 192.168.12.223
**Features**
- Enhanced HTML email template for all automated emails — color-coded headers per script, status indicators, keyword highlighting (ERROR/WARNING/OK), monospace formatting for plain-text reports
- All automated emails (digest, backup, disk, drift, stack, receipt, subscription) now file directly into Proton Bridge "Digests" IMAP folder via APPEND instead of SMTP (no more Inbox clutter or mis-categorization by email organizer)
- Email digest now uses lib/notify.py shared email infrastructure with bar chart visualizations per category
- Refactored email-digest.py to use shared lib/notify.py instead of its own SMTP/IMAP code
**Fixes**
- Fix NFS stale mount on atlantis_archive caused by Tailscale table 52 routing (LAN traffic routed through WireGuard tunnel instead of physical NIC)
- Use Folders/Digests IMAP path for Proton Bridge compatibility (top-level folders silently ignored)
- Add Date header to emails for Proton Bridge IMAP APPEND RFC 5322 compliance
---
## 2026-04-13
**Features**
- Switch to qwen3:32b-fast (thinking disabled) for faster response times (13s vs 206s)
- Migrate all qwen3-coder references to qwen3:32b across services and documentation
- Switch all Ollama usage from qwen3-coder (30B MoE) to qwen3:32b (dense) for better reasoning
- Add CLAUDE.md with deployment, config, networking, LLM, and Olares guidelines
**Fixes**
- Remove watchtower from guava tdarr-node, switch hoarder to ollama
- Standardize AnythingLLM and Perplexica to qwen3-coder:latest to avoid VRAM swap
- Revert standardization to qwen3-coder:latest to avoid VRAM swap cycles
- Switch ollama lib to /api/chat endpoint with 4000 token budget for qwen3:32b thinking mode
- Revert back to qwen3:32b (with thinking) as -fast variant breaks tool calling
- Rename client.spotify.vish.gg to spotify-client.vish.gg for wildcard SSL routing
- Add dedup check (date+vendor+amount) and skip $0 entries in receipt-tracker
**Documentation**
- Document qwen3-coder to qwen3:32b migration with verification results
- Update fluxer deployment docs to reflect current unified server architecture
- Add YourSpotify DNS migration documentation
- Session changelog including YourSpotify migration, Portainer auth issue, Tailscale/arr fixes
---
## 2026-04-06
**Features**
- **Dashboard**
- Added temperature widget and enriched Plex Now Playing
- Added Tdarr cluster widget with live worker progress, fps, node status, and stats
- Added color legend to automations page, fixed timeline schedule thresholds
- Added health score, Kuma monitors, disk usage, recently added media, Cloudflare/Authentik/Gitea, quick actions, automation timeline
- Added Kuma monitors, health score, disk usage, Cloudflare, Authentik, Gitea, media history, quick actions, automation timeline APIs
- Added color-coded text throughout - hosts, categories, statuses, amounts, server names
- Overhauled media page with Plex integration, automations redesign, larger fonts
- Added Baikal calendar card with upcoming events
- Added AI chat with live Headscale/Jellyfin/AdGuard data, keyword aliases, repo doc search (max 2K tokens), smart chat with live homelab context, quick prompts
- Richer activity feed with more event types and better formatting
- Port mockup styling - colored stats, glowing dots, gradient bg, ring gauge
- Glassmorphism redesign inspired by dashdot
- Added network/logs pages, chat widget, toasts, keyboard shortcuts, sparklines, responsive design
- Added Prowlarr, Bazarr, Audiobookshelf, Deluge to media page
- Added 8 themes — midnight, light, cyberpunk, steampunk, portland, racing, ocean, aurora
- Added Exo 2 custom font for the entire dashboard
- Added loading skeletons, empty states, favicon, global search (Cmd+K), click-to-copy
- Visual flair effects (sparkles, card glow, gradient text) + 4 new themes (crimson, trinidad, samurai, supra)
- Layout overhaul + more glass transparency + 4 new themes (sakura, emerald, sunset, arctic)
- True frosted glass effect with visible background gradients bleeding through semi-transparent cards
- Complete Next.js frontend with all 5 pages and components
- Implemented all backend API routes
- Project scaffolding with FastAPI + Docker Compose
- **Tdarr**
- Deployed Tdarr node on Olares with RTX 5090 GPU transcoding
- Updated all 4 instances to v2.67.01 (same digest)
- Pinned all tdarr images to v2.66.01 digest, disabled auto-updates
- **Automations**
- Added 11 Ollama-powered homelab automations
- Added AI-powered PR reviewer using Ollama
- Added daily email organizer digest
- Enhanced Gmail organizer with 10 improvements
**Fixes**
- **Dashboard**
- Accurate disk usage — filter NFS/CIFS, aggregate ZFS pools, deduplicate
- Container logs modal uses correct API URL format (query param not path)
- Timeline interval lookup order - specific keys before generic (digest before email)
- Timeline falls back to file mtime when log has no timestamps
- Health score only penalizes crashed containers, not cleanly stopped ones
- Replace all escaped unicode across entire codebase with plain text symbols
- Replace escaped unicode symbols with plain text in command search and nav
- Headscale protobuf timestamp conversion, expenses defaults to current month
- Headscale uses snake_case field names (ip_addresses, given_name, last_seen)
- AuthentikStats type to match API (created not timestamp, optional users)
- TypeScript error in authentik user rendering
- Headscale uses sudo docker, authentik shows users + filters noise events
- Make ollama chat opaque so text is readable over glass background
- Replace broken unicode emoji icons with clean text badges in quick actions
- Remove card inner glow pseudo-elements
- Remove choppy border-glow animation, use smooth transition-only hover
- Solid opaque theme dropdown, polished nav with gradient accent line
- Force white text in dark mode with CSS specificity overrides
- Restore .dark CSS defaults so text is visible before JS hydration
- Solid dark card backgrounds for all themes - no more invisible cards
- Remove CSS defaults that override theme vars, fix nav-bg, add missing shadcn vars
- Major contrast boost - semi-opaque dark cards, brighter text across all themes
- Boost card opacity, text brightness, nav contrast across all themes
- Align network/logs pages with actual API response shapes
- Align frontend with API, enhance UI visuals
- Align frontend types with actual API field names
- Compact nav tabs to prevent scrollbar overflow
- Jellyfin API auth via query param (nested SSH quoting fix)
- Wrap MCP tool functions with @_safe to prevent server crashes
- **Tdarr**
- Remove read-only flag on media mount — Tdarr needs write access to replace transcoded files
**Infrastructure**
- **MCP**
- Optimized MCP server + added Jellyfin/Olares/Ollama tools
- Rate-limit Ollama calls and cap receipt-tracker to prevent overload
- **Notifications**
- Switched all notifications from ntfy to email (admin@thevish.io)
**Documentation**
- **Dashboard**
- Comprehensive session documentation — dashboard, Tdarr Olares, automations, MCP enhancements
- Add Tdarr Olares node documentation with GPU transcoding details
- Add Tdarr version sync documentation
- Update dashboard docs with expanded Cloudflare DNS record table
- Add Fenrus font customization notes for later
- Comprehensive dashboard documentation with all endpoints, themes, and setup instructions
- Add homelab dashboard implementation plan
- Add homelab dashboard design spec
- **Tdarr**
- Add Tdarr Olares node documentation with GPU transcoding details
- **Automations**
- Add comprehensive README for all automation scripts
- Add Jellyfin on Olares, Plex chart, update Olares docs
- Add AdGuard DNS mesh rollout, switch Headscale to Tailscale IPs
- Add iperf3 benchmarks for all hosts against Calypso
- Add staggered speedtest results for all 10 nodes
- Add GL.iNet router fixes, speedtest results, iperf3 benchmarks
- Document Calypso 5-minute Tailscale disconnect root cause and fix
- Update LAN routing fix for all hosts, add Tailscale mesh test
- Add DERP connectivity diagnosis and fix script
- Update NetBox with MAC addresses for all reachable nodes
- **Miscellaneous**
- Tighten backup-validator LLM prompt to stop hallucinating concerns
---
# Changelog
## 2026-03-27
### Security
* **crowdsec**: Deployed CrowdSec intrusion detection + prevention on matrix-ubuntu, co-located with NPM. Engine parses all 36 NPM proxy host logs + host syslog. Firewall bouncer (nftables) blocks banned IPs at the network layer — avoids nginx `auth_request` conflicts with Authentik SSO. Kuma monitor added (ID 121, `/health` endpoint). Prometheus metrics on `:6060`.
### Monitoring
* **grafana dashboards**: Complete overhaul — 6 dashboards auto-provisioned from bind-mounted JSON files (`/home/homelab/docker/grafana-dashboards/`). Removed 900+ lines of embedded dashboard JSON from monitoring.yaml. Pinned Prometheus datasource UID (`cfbskvs8upds0b`).
* **grafana new dashboards**: Added Synology NAS Monitoring (SNMP disk temps/status, CPU, memory, volumes, network for Atlantis + Calypso), TrueNAS Guava Monitoring (CPU, RAM, ZFS pools, disk I/O), Tailscale Bandwidth (per-host TX/RX rates).
* **grafana fixes**: Fixed Infrastructure Overview + old Synology dashboard empty datasource UIDs. Fixed `$job` variable `allValue` (was empty string, now `.*`). Cleaned up duplicate provisioned `synology-dashboard-v2` ghost dashboard (required Grafana volume wipe). Setillo (DS223j) now showing in Synology dashboard after restarting stopped exporters.
* **kuma**: Added Setillo Node Exporter (ID 122) and SNMP Exporter (ID 123) monitors under Setillo group.
* **frigate**: Tested Frigate NVR on Seattle with Tapo camera (192.168.68.67) via Tailscale subnet routing. CPU detection working, go2rtc restreaming confirmed. Removed after validation — docs saved for future permanent deployment.
* **tailscale**: Enabled `--accept-routes=true` on Seattle to allow access to NUC's `192.168.68.0/22` subnet. NUC route was already advertised and approved in Headscale.
* **tdarr**: Synced all nodes to v2.66.01 (server was 2.65.01, Calypso node was 2.64.02). Redeployed arr-stack on Atlantis, tdarr-node on Calypso, Guava, PVE LXC. Expanded PVE LXC disk 16GB→32GB (was 100% full), pruned 2.86GB old images.
### Fixes
* **immich (calypso)**: Fixed Immich-SERVER crash (`getaddrinfo ENOTFOUND database`). Portainer git deploy does not load `env_file` references — all env vars (DB_HOSTNAME, DB_PASSWORD, etc.) added as Portainer stack environment overrides via API.
* **kuma**: Fixed broken monitor list caused by malformed `accepted_statuscodes_json` field (`[200-299]``["200-299"]`) in CrowdSec monitor entry. Fixed CrowdSec health check URL from `/v1/heartbeat` (requires auth, returns 401) to `/health` (unauthenticated, returns 200).
### Infrastructure
* **setillo**: Configured `vish` user for docker access — added to `wheel` group (NOPASSWD sudo), added `/usr/local/bin` to PATH via `.profile`. Docker (Synology ContainerManager) now accessible without full path or root login.
* **matrix-ubuntu**: VM resized — 16GB RAM (was ~8GB), 1TB disk (was smaller). LV extended online from 97GB to 1005GB via `growpart` + `pvresize` + `lvextend -r`. Now 893GB free (8% used).
* **mcp**: Added `seattle` as SSH host alias in homelab MCP server (alongside existing `seattle-tailscale`).
* **photoprism (jellyfish)**: Started PhotoPrism container on jellyfish (`/srv/nas/ametrine/Docker/photoprism/`, port 2342).
### Container Inventory (2026-03-27)
| Host | Running | Stopped | Total |
|------|---------|---------|-------|
| Atlantis | 59 | 0 | 59 |
| Calypso | 62 | 0 | 62 |
| Homelab-VM | 37 | 1 | 38 |
| Concord NUC | 22 | 0 | 22 |
| Matrix-Ubuntu | 12 | 0 | 12 |
| Guava | 28 | 6 | 34 |
| Seattle | 19 | 1 | 20 |
| RPi5 | 7 | 0 | 7 |
| Jellyfish | 1 | 1 | 2 |
| **Total** | **247** | **9** | **256** |
## 2026-03-25
### Infrastructure
* **portainer**: Updated server 2.39.0 → 2.39.1 LTS on atlantis. Updated edge agents to 2.39.1 on all 4 endpoints (homelab-vm, calypso, nuc, rpi5).
* **portainer stacks**: Fixed stale git credentials across atlantis and calypso. Cleaned up orphan Docker Compose projects (containers created outside Portainer with mismatched project labels) on atlantis, calypso, and homelab-vm.
* **netbox**: Migrated from standalone `docker compose` to Portainer GitOps stack (ID 738) on homelab-vm.
* **semaphore**: Removed — replaced by CLI + cron + MCP workflow. Compose archived.
### Features
* **AGENTS.md**: Overhauled Vesper agent identity — structured priorities, multi-host task guidance, failure handling, context budget, known footguns, tailscale mesh runbook.
* **MCP tools**: Added 5 Authentik SSO tools — `create_proxy_provider`, `create_application`, `list_sessions`, `delete_session`, `get_events`. Service onboarding is now 2 MCP calls.
* **email backup**: Daily incremental backup of 3 email accounts (dvish92, lzbellina92, admin@thevish.io) to atlantis NFS mount at `/volume1/archive/old_emails/`. IMAP auto-reconnect on Gmail throttling. Cron at 3 AM.
### Fixes
* **NFS mount**: Fixed atlantis `/volume1/archive` NFS export — removed krb5i (no Kerberos configured), added LAN routing rule to bypass Tailscale for 192.168.0.0/24.
* **ansible inventory**: Commented out offline hosts (pi-5-kevin, moon) to prevent exit code 4 on every playbook run.
* **image update docs**: Added step-by-step walkthrough, orphan container gotcha, and git auth troubleshooting.
* **moon jellyfish mount**: Added `noserverino` to CIFS mount — fixed "folder contents cannot be displayed" error in GUI file manager.
* **moon guava backup**: NFS mount from atlantis (`100.83.230.112:/volume1/archive/guava_full_backup``/home/moon/guava_backup_atlantis`), read-only over Tailscale. Added `100.64.0.6` to atlantis NFS export, persisted in fstab.
* **olares investigation**: Documented Olares internal Headscale/Tailscale architecture — runs its own coordination server inside k3s for reverse proxy tunneling. Cannot be replaced with external Headscale without breaking `*.olares.com` remote access.
### Stable Diffusion Forge (shinku-ryuu)
* **Forge WebUI**: Installed Stable Diffusion WebUI Forge on shinku-ryuu (RTX 4080, 16GB VRAM, i7-14700K, 96GB RAM). Conda env with Python 3.10, SDXL Base 1.0 model. Access at `http://100.98.93.15:7860` or `http://localhost:7860`. Launcher: `C:\stable-diffusion-webui-forge\run-forge.bat`.
* **Guava Gitea**: Increased avatar max file size from 1MB to 10MB in `/etc/gitea/app.ini`.
### Git Migration
* **playgrounds → Guava Gitea**: Migrated 35 git repos from moon (`~/Documents/playgrounds/`) to Guava Gitea (`http://guava.crista.home:30008`) under the `lulupearl` user. Sources: 8 bitbucket, 26 gitlab, 1 lulupearl_gitea. All repos private, commit history preserved. Cloned all 34 repos to homelab-vm at `/home/homelab/organized/repos/`.
### Tailscale Mesh Verification
* Verified full 30-path mesh across 6 SSH-accessible hosts. All direct connections. Setillo uses DERP initially but hole-punches to direct (~55ms WAN latency). Documented Synology-specific tailscale CLI paths and `ping` limitations.
## [Unreleased] (2026-02-27)
### Bug Fixes
* **credentials**: Restored all credentials broken by sanitization commit `037d766a`
- Affected stacks: authentik-sso, paperless, wireguard (calypso+nuc), monitoring,
dyndns (atlantis+nuc), watchtower, yourspotify, paperless-ai, alerting
- Root cause: sanitization commit replaced real values with `REDACTED_PASSWORD`
placeholders across 14+ compose files; containers redeployed with broken env vars
- Fix: recovered original values from git history (`037d766a^`) and pushed as
commits `50d8eca8` and `4e5607b7`; all 11 affected stacks redeployed via API
* **portainer**: Updated `portainer-homelab` saved Git credential with new Gitea token
- Previously expired token caused all 43 stacks using `credId:1` to fail git pulls
- Fixed via `PUT /api/users/1/gitcredentials/1`
* **portainer-api-guide**: Corrected authentication docs — `ptr_*` tokens require
`X-API-Key` header, not `Authorization: Bearer`; updated version 2.33.7 → 2.39.0
## [Unreleased] (2025-02-12)
### Features
* **arr-suite**: Implement Trash Guides language configuration for Radarr and Sonarr
- Added 4 custom formats: Language Not English (-10000), Anime Dual Audio (+500), Multi (+500), Language Not Original (0)
- Updated quality profiles to prioritize English content while allowing foreign films in original language
- Enhanced anime support with dual audio preference
- Enables proper handling of foreign films like "Cold War" in Polish
- Documentation: `docs/services/arr-suite-language-configuration.md`
## [0.10.3](https://github.com/stoatchat/stoatchat/compare/v0.10.2...v0.10.3) (2026-02-07)
### Bug Fixes
* update `Revolt` -> `Stoat` in email titles/desc. ([#508](https://github.com/stoatchat/stoatchat/issues/508)) ([84483ce](https://github.com/stoatchat/stoatchat/commit/84483cee7af3e5dfa16f7fe13e334c4d9f5abd60))
## [0.10.2](https://github.com/stoatchat/stoatchat/compare/v0.10.1...v0.10.2) (2026-01-25)
### Bug Fixes
* thREDACTED_APP_PASSWORD requires rgb8/rgba8 ([#505](https://github.com/stoatchat/stoatchat/issues/505)) ([413aa04](https://github.com/stoatchat/stoatchat/commit/413aa04dcaf8bff3935ed1e5f31432e11a03ce6f))
## [0.10.1](https://github.com/stoatchat/stoatchat/compare/v0.10.0...v0.10.1) (2026-01-25)
### Bug Fixes
* use Rust 1.92.0 for Docker build ([#503](https://github.com/stoatchat/stoatchat/issues/503)) ([98da8a2](https://github.com/stoatchat/stoatchat/commit/98da8a28a0aa2fee4e8eee1d86bd7c49e3187477))
## [0.10.0](https://github.com/stoatchat/stoatchat/compare/v0.9.4...v0.10.0) (2026-01-25)
### Features
* allow kicking members from voice channels ([#495](https://github.com/stoatchat/stoatchat/issues/495)) ([0dc5442](https://github.com/stoatchat/stoatchat/commit/0dc544249825a49c793309edee5ec1838458a6da))
* repository architecture for files crate w. added tests ([#498](https://github.com/stoatchat/stoatchat/issues/498)) ([01ded20](https://github.com/stoatchat/stoatchat/commit/01ded209c62208fc906d6aab9b08c04e860e10ef))
### Bug Fixes
* expose ratelimit headers via cors ([#496](https://github.com/stoatchat/stoatchat/issues/496)) ([a1a2125](https://github.com/stoatchat/stoatchat/commit/a1a21252d0ad58937e41f16e5fb86f96bebd2a51))
## [0.9.4](https://github.com/stoatchat/stoatchat/compare/v0.9.3...v0.9.4) (2026-01-10)
### Bug Fixes
* checkout repo. before bumping lock ([#490](https://github.com/stoatchat/stoatchat/issues/490)) ([b2da2a8](https://github.com/stoatchat/stoatchat/commit/b2da2a858787853be43136fd526a0bd72baf78ef))
* persist credentials for git repo ([#492](https://github.com/stoatchat/stoatchat/issues/492)) ([c674a9f](https://github.com/stoatchat/stoatchat/commit/c674a9fd4e0abbd51569870e4b38074d4a1de03c))
## [0.9.3](https://github.com/stoatchat/stoatchat/compare/v0.9.2...v0.9.3) (2026-01-10)
### Bug Fixes
* pipeline fixes ([#487](https://github.com/stoatchat/stoatchat/issues/487)) ([aeeafeb](https://github.com/stoatchat/stoatchat/commit/aeeafebefc36a43a656cf797c9251ca50292733c))
## [0.9.2](https://github.com/stoatchat/stoatchat/compare/v0.9.1...v0.9.2) (2026-01-10)
### Bug Fixes
* disable publish for services ([#485](https://github.com/stoatchat/stoatchat/issues/485)) ([d13609c](https://github.com/stoatchat/stoatchat/commit/d13609c37279d6a40445dcd99564e5c3dd03bac1))
## [0.9.1](https://github.com/stoatchat/stoatchat/compare/v0.9.0...v0.9.1) (2026-01-10)
### Bug Fixes
* **ci:** pipeline fixes (marked as fix to force release) ([#483](https://github.com/stoatchat/stoatchat/issues/483)) ([303e52b](https://github.com/stoatchat/stoatchat/commit/303e52b476585eea81c33837f1b01506ce387684))
## [0.9.0](https://github.com/stoatchat/stoatchat/compare/v0.8.8...v0.9.0) (2026-01-10)
### Features
* add id field to role ([#470](https://github.com/stoatchat/stoatchat/issues/470)) ([2afea56](https://github.com/stoatchat/stoatchat/commit/2afea56e56017f02de98e67316b4457568ad5b26))
* add ratelimits to gifbox ([1542047](https://github.com/stoatchat/stoatchat/commit/154204742d21cbeff6e2577b00f50b495ea44631))
* include groups and dms in fetch mutuals ([caa8607](https://github.com/stoatchat/stoatchat/commit/caa86074680d46223cebc20f41e9c91c41ec825d))
* include member payload in REDACTED_APP_PASSWORD event ([480f210](https://github.com/stoatchat/stoatchat/commit/480f210ce85271e13d1dac58a5dae08de108579d))
* initial work on tenor gif searching ([b0c977b](https://github.com/stoatchat/stoatchat/commit/b0c977b324b8144c1152589546eb8fec5954c3e7))
* make message lexer use unowned string ([1561481](https://github.com/stoatchat/stoatchat/commit/1561481eb4cdc0f385fbf0a81e4950408050e11f))
* ready payload field customisation ([db57706](https://github.com/stoatchat/stoatchat/commit/db577067948f13e830b5fb773034e9713a1abaff))
* require auth for search ([b5cd5e3](https://github.com/stoatchat/stoatchat/commit/b5cd5e30ef7d5e56e8964fb7c543965fa6bf5a4a))
* trending and categories routes ([5885e06](https://github.com/stoatchat/stoatchat/commit/5885e067a627b8fff1c8ce2bf9e852ff8cf3f07a))
* voice chats v2 ([#414](https://github.com/stoatchat/stoatchat/issues/414)) ([d567155](https://github.com/stoatchat/stoatchat/commit/d567155f124e4da74115b1a8f810062f7c6559d9))
### Bug Fixes
* add license to revolt-parser ([5335124](https://github.com/stoatchat/stoatchat/commit/53351243064cac8d499dd74284be73928fa78a43))
* allow for disabling default features ([65fbd36](https://github.com/stoatchat/stoatchat/commit/65fbd3662462aed1333b79e59155fa6377e83fcc))
* apple music to use original url instead of metadata url ([bfe4018](https://github.com/stoatchat/stoatchat/commit/bfe4018e436a4075bae780dd4d35a9b58315e12f))
* apply uname fix to january and autumn ([8f9015a](https://github.com/stoatchat/stoatchat/commit/8f9015a6ff181d208d9269ab8691bd417d39811a))
* **ci:** publish images under stoatchat and remove docker hub ([d65c1a1](https://github.com/stoatchat/stoatchat/commit/d65c1a1ab3bdc7e5684b03f280af77d881661a3d))
* correct miniz_oxide in lockfile ([#478](https://github.com/stoatchat/stoatchat/issues/478)) ([5d27a91](https://github.com/stoatchat/stoatchat/commit/5d27a91e901dd2ea3e860aeaed8468db6c5f3214))
* correct shebang for try-tag-and-release ([050ba16](https://github.com/stoatchat/stoatchat/commit/050ba16d4adad5d0fb247867aa3e94e3d42bd12d))
* correct string_cache in lockfile ([#479](https://github.com/stoatchat/stoatchat/issues/479)) ([0b178fc](https://github.com/stoatchat/stoatchat/commit/0b178fc791583064bf9ca94b1d39b42d021e1d79))
* don't remove timeouts when a member leaves a server ([#409](https://github.com/stoatchat/stoatchat/issues/409)) ([e635bc2](https://github.com/stoatchat/stoatchat/commit/e635bc23ec857d648d5705e1a3875d7bc3402b0d))
* don't update the same field while trying to remove it ([f4ee35f](https://github.com/stoatchat/stoatchat/commit/f4ee35fb093ca49f0a64ff4b17fd61587df28145)), closes [#392](https://github.com/stoatchat/stoatchat/issues/392)
* github webhook incorrect payload and formatting ([#468](https://github.com/stoatchat/stoatchat/issues/468)) ([dc9c82a](https://github.com/stoatchat/stoatchat/commit/dc9c82aa4e9667ea6639256c65ac8de37a24d1f7))
* implement Serialize to ClientMessage ([dea0f67](https://github.com/stoatchat/stoatchat/commit/dea0f675dde7a63c7a59b38d469f878b7a8a3af4))
* newly created roles should be ranked the lowest ([947eb15](https://github.com/stoatchat/stoatchat/commit/947eb15771ed6785b3dcd16c354c03ded5e4cbe0))
* permit empty `remove` array in edit requests ([6ad3da5](https://github.com/stoatchat/stoatchat/commit/6ad3da5f35f989a2e7d8e29718b98374248e76af))
* preserve order of replies in message ([#447](https://github.com/stoatchat/stoatchat/issues/447)) ([657a3f0](https://github.com/stoatchat/stoatchat/commit/657a3f08e5d652814bbf0647e089ed9ebb139bbf))
* prevent timing out members which have TimeoutMembers permission ([e36fc97](https://github.com/stoatchat/stoatchat/commit/e36fc9738bac0de4f3fcbccba521f1e3754f7ae7))
* relax settings name regex ([3a34159](https://github.com/stoatchat/stoatchat/commit/3a3415915f0d0fdce1499d47a2b7fa097f5946ea))
* remove authentication tag bytes from attachment download ([32e6600](https://github.com/stoatchat/stoatchat/commit/32e6600272b885c595c094f0bc69459250220dcb))
* rename openapi operation ids ([6048587](https://github.com/stoatchat/stoatchat/commit/6048587d348fbca0dc3a9b47690c56df8fece576)), closes [#406](https://github.com/stoatchat/stoatchat/issues/406)
* respond with 201 if no body in requests ([#465](https://github.com/stoatchat/stoatchat/issues/465)) ([24fedf8](https://github.com/stoatchat/stoatchat/commit/24fedf8c4d9cd3160bdec97aa451520f8beaa739))
* swap to using reqwest for query building ([38dd4d1](https://github.com/stoatchat/stoatchat/commit/38dd4d10797b3e6e397fc219e818f379bdff19f2))
* use `trust_cloudflare` config value instead of env var ([cc7a796](https://github.com/stoatchat/stoatchat/commit/cc7a7962a882e1627fcd0bc75858a017415e8cfc))
* use our own result types instead of tenors types ([a92152d](https://github.com/stoatchat/stoatchat/commit/a92152d86da136997817e797c7af8e38731cdde8))
## 2026-04-07 — Session: Infrastructure Fixes
### YourSpotify DNS Migration
- Renamed `client.spotify.vish.gg``spotify-client.vish.gg` (wildcard SSL cert compatibility)
- Created NPM proxy hosts: `spotify-client.vish.gg` → NUC:4000, `spotify.vish.gg` → NUC:15000
- Updated Cloudflare DNS to proxied through matrix-ubuntu
- Removed from NUC DYNDNS updater
- Updated compose env vars and Spotify Developer Dashboard redirect URI
### Portainer Git Auth Issue
- 71 of 94 GitOps stacks lost git credentials (likely from Portainer upgrade)
- Stacks continue running but cannot pull/redeploy until credentials are re-entered
- Credentials: Username=`vish`, Token=Gitea service account token
- Fix: Re-enter via Portainer UI → Stack → Editor → Repository → Authentication
- 22 stacks already have working auth
### Tailscale Fixes
- NUC (vish-concord-nuc): Tailscale daemon was stale, restarted with `sudo systemctl restart tailscaled`
- Now active with direct connection to all home nodes
### Arr Suite Recovery
- 6 containers stuck in "Created" state after Portainer pull-and-redeploy on Atlantis
- Affected: audiobookshelf, deluge, prowlarr, radarr, sonarr, tdarr
- Fixed by restarting each container via Portainer API
- Plex also restarted (had exited during arr-suite redeploy)
### Kuma Monitor Fix
- Plex monitor (ID 60) under Setillo group was pointing to wrong IP (Atlantis instead of Setillo)
- Updated to correct IP: 100.125.0.20:32400
### Jitsi Network Conflict
- Orphaned Docker network `jitsi-stack_meet.jitsi` (172.30.0.0/16) conflicted with new `turn_net` (172.30.0.0/24)
- Removed orphaned network, redeploy requires git auth re-entry
## 2026-04-07 — LLM Model Migration: qwen3-coder → qwen3:32b
### Why
- qwen3-coder (30B MoE, 3.3B active params) caused OpenCode to drift, plan instead of act, and stall after context compaction
- qwen3:32b (dense 32B, all params active every token) provides dramatically better instruction following and reasoning
### What changed
- All 13 config/compose files updated from qwen3-coder to qwen3:32b
- All documentation updated (AGENTS.md, CLAUDE.md, 8 service docs, scripts/README.md)
- OpenCode config: new model + "always respond with results" instruction + step limits reduced (50→20)
- OpenCode config: full host inventory, SSH aliases, service URLs added to instructions
- Perplexica Docker volume config updated
- AnythingLLM and Reactive Resume stacks redeployed on Portainer
- VRAM usage: 22.9/24.5 GB (similar to qwen3-coder)
### Verified working
- Ollama direct, Dashboard AI chat, Perplexica, AnythingLLM, Reactive Resume, Gmail organizers (3 accounts), MCP server