Sanitized mirror from private repository - 2026-04-05 12:03:50 UTC

scripts/README.md

@@ -0,0 +1,129 @@
+# Homelab Automation Scripts
+
+All scripts use the local Ollama LLM (qwen3-coder) on Olares at `http://192.168.0.145:31434`.
+
+## Shared Library (`lib/`)
+
+Reusable modules extracted from existing scripts:
+
+| Module | Purpose |
+|--------|---------|
+| `lib/ollama.py` | Ollama client with retry, backoff, think-tag stripping |
+| `lib/notify.py` | ntfy push notifications + Proton Bridge SMTP email |
+| `lib/portainer.py` | Portainer API client (all 5 endpoints) |
+| `lib/prometheus.py` | PromQL instant and range queries |
+| `lib/gitea.py` | Gitea API client (commits, releases) |
+
+## Automated Scripts (Cron)
+
+### Email Organizers
+| Script | Account | Schedule | Description |
+|--------|---------|----------|-------------|
+| `gmail-organizer/gmail_organizer.py` | lzbellina92@gmail.com | Every 30min | LLM email classification + Gmail labels |
+| `gmail-organizer-dvish/gmail_organizer.py` | your-email@example.com | Every 30min | LLM email classification + Gmail labels |
+| `proton-organizer/proton_organizer.py` | admin@thevish.io | Every 30min | LLM email classification via Proton Bridge |
+
+Control all 3 with: `scripts/gmail-organizer-ctl.sh [start|stop|status]`
+
+### Monitoring
+| Script | Schedule | Description |
+|--------|----------|-------------|
+| `backup-validator.py` | Daily 4am | Validates `/tmp/gmail-backup-daily.log`, LLM summarizes, emails report |
+| `disk-predictor.py` | Sunday 9am | Prometheus disk trends, linear regression, predicts when volumes fill |
+| `stack-restart.py` | Every 5min | Detects unhealthy containers, LLM assesses restart safety, auto-restarts if safe |
+
+### Data Mining
+| Script | Schedule | Description |
+|--------|----------|-------------|
+| `config-drift.py` | Daily 7am | Compares git compose files vs running containers via Portainer |
+| `receipt-tracker.py` | Daily 10am | Extracts vendor/amount from receipt emails → `data/expenses.csv` |
+| `subscription-auditor.py` | 1st of month 9am | Audits email subscriptions, flags dormant ones |
+
+### Developer
+| Script | Schedule | Description |
+|--------|----------|-------------|
+| `changelog-generator.py` | Monday 8am | Summarizes git commits into `docs/CHANGELOG.md` |
+| `email-digest.py` | Daily 8am | Aggregates all 3 organizer stats, emails digest to admin@thevish.io |
+| `pr-reviewer.py` | On PR (Gitea CI) | AI code review posted as PR comment |
+
+## Interactive Tools (CLI)
+
+```bash
+# Generate SSH commands from plain English
+python3 scripts/ssh-planner.py "restart all media services on atlantis"
+python3 scripts/ssh-planner.py "check disk space on NAS boxes" --execute
+
+# Generate Ansible playbooks from descriptions
+python3 scripts/ansible-generator.py "update all hosts and reboot if kernel changed"
+python3 scripts/ansible-generator.py "install tailscale on seattle" --name tailscale-seattle
+
+# Generate troubleshooting runbooks
+python3 scripts/runbook-generator.py --service jellyfin-olares
+python3 scripts/runbook-generator.py --all          # all services (burst: ~30 LLM calls)
+python3 scripts/runbook-generator.py --all --force   # regenerate even if unchanged
+```
+
+## Cron Setup
+
+Add these to crontab (`crontab -e`). Email organizer crons are already installed.
+
+```bash
+# Monitoring
+*/5 * * * * cd /home/homelab/organized/repos/homelab && python3 scripts/stack-restart.py >> /tmp/stack-restart.log 2>&1
+0 4 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/backup-validator.py >> /tmp/backup-validator.log 2>&1
+0 9 * * 0 cd /home/homelab/organized/repos/homelab && python3 scripts/disk-predictor.py >> /tmp/disk-predictor.log 2>&1
+
+# Data mining
+0 7 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/config-drift.py >> /tmp/config-drift.log 2>&1
+0 10 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/receipt-tracker.py >> /tmp/receipt-tracker.log 2>&1
+0 9 1 * * cd /home/homelab/organized/repos/homelab && python3 scripts/subscription-auditor.py >> /tmp/subscription-auditor.log 2>&1
+
+# Developer
+0 8 * * 1 cd /home/homelab/organized/repos/homelab && python3 scripts/changelog-generator.py >> /tmp/changelog-generator.log 2>&1
+```
+
+## Stack Restart Safety
+
+`stack-restart.py` has multiple safety layers:
+- **Blacklist**: portainer, headscale, adguard, postgres, mariadb, mongodb, redis are never auto-restarted
+- **LLM gate**: Ollama must analyze logs and say "SAFE" before any restart
+- **No LLM = no restart**: if Ollama is down, it only sends alerts
+- **Rate limit**: max 2 restarts per container per hour
+- **Grace period**: container must be unhealthy for 5+ minutes before action
+- **Dry-run**: test with `python3 scripts/stack-restart.py --dry-run`
+
+## Ollama Usage
+
+| Category | Calls/Day | Notes |
+|----------|-----------|-------|
+| Email organizers (3) | 0-15 | Sender cache reduces calls over time |
+| Monitoring (3) | 1-3 | stack-restart only fires on unhealthy |
+| Data mining (3) | 2-6 | receipt-tracker scales with purchases |
+| Developer (2) | 0-1 | changelog weekly, PR reviews on demand |
+| Interactive (3) | 0-5 | only when you use them |
+| **Total** | **~5-15** | **Negligible for RTX 5090** |
+
+## Graceful Degradation
+
+All automated scripts check `ollama_available()` before making LLM calls:
+- **Monitoring scripts**: send alerts with raw data (no LLM summary)
+- **Data mining scripts**: skip run, retry next schedule
+- **Interactive tools**: print error and exit
+
+## MCP Server Integration
+
+The homelab MCP server (`homelab-mcp/server.py`) includes an `ollama_query` tool that lets Claude Code query the local LLM directly. This saves API tokens for homelab-specific questions.
+
+## Logs
+
+All cron scripts log to `/tmp/`:
+```bash
+tail -f /tmp/stack-restart.log
+tail -f /tmp/backup-validator.log
+tail -f /tmp/disk-predictor.log
+tail -f /tmp/config-drift.log
+tail -f /tmp/receipt-tracker.log
+tail -f /tmp/subscription-auditor.log
+tail -f /tmp/changelog-generator.log
+tail -f /tmp/email-digest.log
+```