homelab-optimized/scripts/README.md

Homelab Automation Scripts

Dashboard UI: These scripts are visualized and controlled through the Homelab Dashboard at http://homelab.tail.vish.gg:3100. The dashboard API (dashboard/api/) reuses scripts/lib/ modules and reads automation databases/logs to display real-time status. Quick actions in the dashboard can pause/resume email organizers and trigger backups.

All scripts use the local Ollama LLM (qwen3-coder) on Olares at http://192.168.0.145:31434.

Shared Library (lib/)

Reusable modules extracted from existing scripts:

Module	Purpose
lib/ollama.py	Ollama client with retry, backoff, think-tag stripping
lib/notify.py	ntfy push notifications + Proton Bridge SMTP email
lib/portainer.py	Portainer API client (all 5 endpoints)
lib/prometheus.py	PromQL instant and range queries
lib/gitea.py	Gitea API client (commits, releases)

Automated Scripts (Cron)

Email Organizers

Script	Account	Schedule	Description
gmail-organizer/gmail_organizer.py	lzbellina92@gmail.com	Every 30min	LLM email classification + Gmail labels
gmail-organizer-dvish/gmail_organizer.py	your-email@example.com	Every 30min	LLM email classification + Gmail labels
proton-organizer/proton_organizer.py	admin@thevish.io	Every 30min	LLM email classification via Proton Bridge

Control all 3 with: scripts/gmail-organizer-ctl.sh [start|stop|status]

Monitoring

Script	Schedule	Description
backup-validator.py	Daily 4am	Validates /tmp/gmail-backup-daily.log, LLM summarizes, emails report
disk-predictor.py	Sunday 9am	Prometheus disk trends, linear regression, predicts when volumes fill
stack-restart.py	Every 5min	Detects unhealthy containers, LLM assesses restart safety, auto-restarts if safe

Data Mining

Script	Schedule	Description
config-drift.py	Daily 7am	Compares git compose files vs running containers via Portainer
receipt-tracker.py	Daily 10am	Extracts vendor/amount from receipt emails → data/expenses.csv
subscription-auditor.py	1st of month 9am	Audits email subscriptions, flags dormant ones

Developer

Script	Schedule	Description
changelog-generator.py	Monday 8am	Summarizes git commits into docs/CHANGELOG.md
email-digest.py	Daily 8am	Aggregates all 3 organizer stats, emails digest to admin@thevish.io
pr-reviewer.py	On PR (Gitea CI)	AI code review posted as PR comment

Interactive Tools (CLI)

# Generate SSH commands from plain English
python3 scripts/ssh-planner.py "restart all media services on atlantis"
python3 scripts/ssh-planner.py "check disk space on NAS boxes" --execute

# Generate Ansible playbooks from descriptions
python3 scripts/ansible-generator.py "update all hosts and reboot if kernel changed"
python3 scripts/ansible-generator.py "install tailscale on seattle" --name tailscale-seattle

# Generate troubleshooting runbooks
python3 scripts/runbook-generator.py --service jellyfin-olares
python3 scripts/runbook-generator.py --all          # all services (burst: ~30 LLM calls)
python3 scripts/runbook-generator.py --all --force   # regenerate even if unchanged

Cron Setup

Add these to crontab (crontab -e). Email organizer crons are already installed.

# Monitoring
*/5 * * * * cd /home/homelab/organized/repos/homelab && python3 scripts/stack-restart.py >> /tmp/stack-restart.log 2>&1
0 4 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/backup-validator.py >> /tmp/backup-validator.log 2>&1
0 9 * * 0 cd /home/homelab/organized/repos/homelab && python3 scripts/disk-predictor.py >> /tmp/disk-predictor.log 2>&1

# Data mining
0 7 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/config-drift.py >> /tmp/config-drift.log 2>&1
0 10 * * * cd /home/homelab/organized/repos/homelab && python3 scripts/receipt-tracker.py >> /tmp/receipt-tracker.log 2>&1
0 9 1 * * cd /home/homelab/organized/repos/homelab && python3 scripts/subscription-auditor.py >> /tmp/subscription-auditor.log 2>&1

# Developer
0 8 * * 1 cd /home/homelab/organized/repos/homelab && python3 scripts/changelog-generator.py >> /tmp/changelog-generator.log 2>&1

Stack Restart Safety

stack-restart.py has multiple safety layers:

• Blacklist: portainer, headscale, adguard, postgres, mariadb, mongodb, redis are never auto-restarted
• LLM gate: Ollama must analyze logs and say "SAFE" before any restart
• No LLM = no restart: if Ollama is down, it only sends alerts
• Rate limit: max 2 restarts per container per hour
• Grace period: container must be unhealthy for 5+ minutes before action
• Dry-run: test with python3 scripts/stack-restart.py --dry-run

Ollama Usage

Category	Calls/Day	Notes
Email organizers (3)	0-15	Sender cache reduces calls over time
Monitoring (3)	1-3	stack-restart only fires on unhealthy
Data mining (3)	2-6	receipt-tracker scales with purchases
Developer (2)	0-1	changelog weekly, PR reviews on demand
Interactive (3)	0-5	only when you use them
Total	~5-15	Negligible for RTX 5090

Graceful Degradation

All automated scripts check ollama_available() before making LLM calls:

• Monitoring scripts: send alerts with raw data (no LLM summary)
• Data mining scripts: skip run, retry next schedule
• Interactive tools: print error and exit

MCP Server Integration

The homelab MCP server (homelab-mcp/server.py) includes an ollama_query tool that lets Claude Code query the local LLM directly. This saves API tokens for homelab-specific questions.

Logs

All cron scripts log to /tmp/:

tail -f /tmp/stack-restart.log
tail -f /tmp/backup-validator.log
tail -f /tmp/disk-predictor.log
tail -f /tmp/config-drift.log
tail -f /tmp/receipt-tracker.log
tail -f /tmp/subscription-auditor.log
tail -f /tmp/changelog-generator.log
tail -f /tmp/email-digest.log