# Testing Procedures

*Testing guidelines for the homelab infrastructure*

---

## Overview

This document outlines testing procedures for deploying new services, making infrastructure changes, and validating functionality.

---

## Pre-Deployment Testing

### New Service Checklist

- [ ] Review Docker image (official, stars, updates)
- [ ] Check for security vulnerabilities
- [ ] Verify resource requirements
- [ ] Test locally first
- [ ] Verify compose syntax
- [ ] Check port availability
- [ ] Test volume paths

### Compose Validation

```bash
# Validate syntax
docker-compose config --quiet

# Check for errors
docker-compose up --dry-run

# Pull images
docker-compose pull
```

---

## Local Testing

### Docker Desktop / Mini Setup

1. Create test compose file
2. Run on local machine
3. Verify all features work
4. Document any issues

### Test Environment

If available, use staging:
- Staging host: `seattle` VM
- Test domain: `*.test.vish.local`
- Shared internally only

---

## Integration Testing

### Authentik SSO

```bash
# Test login flow
1. Open service
2. Click "Login with Authentik"
3. Verify redirect to Authentik
4. Enter credentials
5. Verify return to service
6. Check user profile
```

### Nginx Proxy Manager

```bash
# Test proxy host
curl -H "Host: service.vish.local" http://localhost

# Test SSL
curl -k https://service.vish.gg

# Check headers
curl -I https://service.vish.gg
```

### Database Connections

```bash
# PostgreSQL
docker exec <container> psql -U user -c "SELECT 1"

# Test from application
docker exec <app> nc -zv db 5432
```

---

## Monitoring Validation

### Prometheus Targets

1. Open Prometheus UI
2. Go to Status → Targets
3. Verify all targets are UP
4. Check for scrape errors

### Alert Testing

```bash
# Trigger test alert
curl -X POST http://alertmanager:9093/api/v1/alerts \
  -H "Content-Type: application/json" \
  -d '[{
    "labels": {
      "alertname": "TestAlert",
      "severity": "critical"
    },
    "annotations": {
      "summary": "Test alert"
    }
  }]'
```

### Grafana Dashboards

- [ ] All panels load
- [ ] Data populates
- [ ] No errors in console
- [ ] Alerts configured

---

## Backup Testing

### Full Backup Test

```bash
# Run backup
ansible-playbook ansible/automation/playbooks/backup_configs.yml
ansible-playbook ansible/automation/playbooks/backup_databases.yml

# Verify backup files exist
ls -la /backup/

# Test restore to test environment
# (do NOT overwrite production!)
```

### Restore Procedure Test

1. Stop service
2. Restore data from backup
3. Start service
4. Verify functionality
5. Check logs for errors

---

## Performance Testing

### Load Testing

```bash
# Using hey or ab
hey -n 1000 -c 10 https://service.vish.gg

# Check response times
curl -w "@curl-format.txt" -o /dev/null -s https://service.vish.gg

# curl-format.txt:
# time_namelookup:  %{time_namelookup}\n
# time_connect:     %{time_connect}\n
# time_appconnect:  %{time_appconnect}\n
# time_redirect:    %{time_redirect}\n
# time_pretransfer: %{time_pretransfer}\n
# time_starttransfer: %{time_starttransfer}\n
# time_total:       %{time_total}\n
```

### Resource Testing

```bash
# Monitor during load
docker stats --no-stream

# Check for OOM kills
dmesg | grep -i "out of memory"

# Monitor disk I/O
iostat -x 1
```

---

## Security Testing

### Vulnerability Scanning

```bash
# Trivy scan
trivy image --severity HIGH,CRITICAL <image>

# Check for secrets
trivy fs --security-checks secrets /path/to/compose

# Docker scan
docker scan <image>
```

### SSL/TLS Testing

```bash
# SSL Labs
# Visit: https://www.ssllabs.com/ssltest/

# CLI check
openssl s_client -connect service.vish.gg:443

# Check certificates
certinfo service.vish.gg
```

---

## Network Testing

### Connectivity

```bash
# Port scan
nmap -p 1-1000 192.168.0.x

# DNS check
dig service.vish.local
nslookup service.vish.local

# traceroute
traceroute service.vish.gg
```

### Firewall Testing

```bash
# Check open ports
ss -tulpn

# Test from outside
# Use online port scanner

# Test blocked access
curl -I http://internal-service:port
# Should fail without VPN
```

---

## Regression Testing

### After Updates

1. Check service starts
2. Verify all features
3. Test SSO if enabled
4. Check monitoring
5. Verify backups

### Critical Path Tests

| Path | Steps |
|------|-------|
| External access | VPN → NPM → Service |
| SSO login | Service → Auth → Dashboard |
| Media playback | Request → Download → Play |
| Backup restore | Stop → Restore → Verify → Start |

---

## Acceptance Criteria

### New Service

- [ ] Starts without errors
- [ ] UI accessible
- [ ] Basic function works
- [ ] SSO configured (if supported)
- [ ] Monitoring enabled
- [ ] Backup configured
- [ ] Documentation created

### Infrastructure Change

- [ ] All services running
- [ ] No new alerts
- [ ] Monitoring healthy
- [ ] Backups completed
- [ ] Users notified (if needed)

---

## Links

- [Monitoring Architecture](../infrastructure/MONITORING_ARCHITECTURE.md)
- [Backup Procedures](../BACKUP_PROCEDURES.md)
- [Disaster Recovery](../troubleshooting/disaster-recovery.md)