Sanitized mirror from private repository - 2026-03-16 10:55:35 UTC
This commit is contained in:
380
docs/admin/OPERATIONAL_STATUS.md
Normal file
380
docs/admin/OPERATIONAL_STATUS.md
Normal file
@@ -0,0 +1,380 @@
|
||||
# Stoatchat Operational Status & Testing Documentation
|
||||
|
||||
## 🎯 Instance Overview
|
||||
- **Domain**: st.vish.gg
|
||||
- **Status**: ✅ **FULLY OPERATIONAL**
|
||||
- **Deployment Date**: February 2026
|
||||
- **Last Tested**: February 11, 2026
|
||||
- **Platform**: Self-hosted Revolt chat server
|
||||
|
||||
## 🌐 Service Architecture
|
||||
|
||||
### Domain Structure
|
||||
| Service | URL | Port | Status |
|
||||
|---------|-----|------|--------|
|
||||
| **Frontend** | https://st.vish.gg/ | 14702 | ✅ Active |
|
||||
| **API** | https://api.st.vish.gg/ | 14702 | ✅ Active |
|
||||
| **Events (WebSocket)** | wss://events.st.vish.gg/ | 14703 | ✅ Active |
|
||||
| **Files** | https://files.st.vish.gg/ | 14704 | ✅ Active |
|
||||
| **Proxy** | https://proxy.st.vish.gg/ | 14705 | ✅ Active |
|
||||
| **Voice** | wss://voice.st.vish.gg/ | 7880 | ✅ Active |
|
||||
|
||||
### Infrastructure Components
|
||||
- **Reverse Proxy**: Nginx with SSL termination
|
||||
- **SSL Certificates**: Let's Encrypt (auto-renewal configured)
|
||||
- **Database**: Redis (port 6380)
|
||||
- **Voice/Video**: LiveKit integration
|
||||
- **Email**: Gmail SMTP (your-email@example.com)
|
||||
|
||||
## 🧪 Comprehensive Testing Results
|
||||
|
||||
### Test Suite Summary
|
||||
**Total Tests**: 6 categories
|
||||
**Passed**: 6/6 (100%)
|
||||
**Status**: ✅ **ALL TESTS PASSED**
|
||||
|
||||
### 1. Account Creation Test ✅
|
||||
- **Method**: API POST to `/auth/account/create`
|
||||
- **Test Email**: admin@example.com
|
||||
- **Password**: REDACTED_PASSWORD
|
||||
- **Result**: HTTP 204 (Success)
|
||||
- **Account ID**: 01KH5RZXBHDX7W29XXFN6FB35F
|
||||
- **Verification Token**: 2Kd_mgmImSvfNw2Mc8L1vi-oN0U0O5qL
|
||||
|
||||
### 2. Email Verification Test ✅
|
||||
- **SMTP Server**: Gmail (smtp.gmail.com:587)
|
||||
- **Sender**: your-email@example.com
|
||||
- **Recipient**: admin@example.com
|
||||
- **Delivery**: ✅ Successful
|
||||
- **Verification**: ✅ Completed manually
|
||||
- **Email System**: Fully functional
|
||||
|
||||
### 3. Authentication Test ✅
|
||||
- **Login Method**: API POST to `/auth/session/login`
|
||||
- **Credentials**: admin@example.com / REDACTED_PASSWORD
|
||||
- **Result**: HTTP 200 (Success)
|
||||
- **Session Token**: W_NfvzjWiukjVQEi30zNTmvPo4xo7pPJTKCZRvRP7TDQplfOjwgoad3AcuF9LEPI
|
||||
- **Session ID**: 01KH5S1TG66V7BPZS8CFKHGSCR
|
||||
- **User ID**: 01KH5RZXBHDX7W29XXFN6FB35F
|
||||
|
||||
### 4. Web Interface Test ✅
|
||||
- **Frontend URL**: https://st.vish.gg/
|
||||
- **Accessibility**: ✅ Fully accessible
|
||||
- **Login Process**: ✅ Successful via web interface
|
||||
- **UI Responsiveness**: ✅ Working correctly
|
||||
- **SSL Certificate**: ✅ Valid and trusted
|
||||
|
||||
### 5. Real-time Messaging Test ✅
|
||||
- **Test Channel**: Nerds channel
|
||||
- **Message Sending**: ✅ Successful
|
||||
- **Real-time Delivery**: ✅ Instant delivery
|
||||
- **Channel Participation**: ✅ Full functionality
|
||||
- **WebSocket Connection**: ✅ Stable
|
||||
|
||||
### 6. Infrastructure Health Test ✅
|
||||
- **All Services**: ✅ Running and responsive
|
||||
- **SSL Certificates**: ✅ Valid for all domains
|
||||
- **DNS Resolution**: ✅ All subdomains resolving
|
||||
- **Database Connection**: ✅ Redis connected
|
||||
- **File Upload Service**: ✅ Operational
|
||||
- **Voice/Video Service**: ✅ LiveKit integrated
|
||||
|
||||
## 📊 Performance Metrics
|
||||
|
||||
### Response Times
|
||||
- **API Calls**: < 200ms average
|
||||
- **Message Delivery**: < 1 second (real-time)
|
||||
- **File Uploads**: Dependent on file size
|
||||
- **Page Load**: < 2 seconds
|
||||
|
||||
### Uptime & Reliability
|
||||
- **Target Uptime**: 99.9%
|
||||
- **Current Status**: All services operational
|
||||
- **Last Downtime**: None recorded
|
||||
- **Monitoring**: Manual checks performed
|
||||
|
||||
## 🔐 Security Configuration
|
||||
|
||||
### SSL/TLS
|
||||
- **Certificate Authority**: Let's Encrypt
|
||||
- **Encryption**: TLS 1.2/1.3
|
||||
- **HSTS**: Enabled
|
||||
- **Certificate Renewal**: Automated
|
||||
|
||||
### Authentication
|
||||
- **Method**: Session-based authentication
|
||||
- **Password Requirements**: Enforced
|
||||
- **Email Verification**: Required
|
||||
- **Session Management**: Secure token-based
|
||||
|
||||
### Email Security
|
||||
- **SMTP Authentication**: App-specific password
|
||||
- **TLS Encryption**: Enabled
|
||||
- **Authorized Recipients**: Limited to specific domains
|
||||
|
||||
## 📧 Email Configuration
|
||||
|
||||
### SMTP Settings
|
||||
```toml
|
||||
[api.smtp]
|
||||
host = "smtp.gmail.com"
|
||||
port = 587
|
||||
username = "your-email@example.com"
|
||||
password = "REDACTED_PASSWORD"
|
||||
from_address = "your-email@example.com"
|
||||
use_tls = true
|
||||
```
|
||||
|
||||
### Authorized Email Recipients
|
||||
- your-email@example.com
|
||||
- admin@example.com
|
||||
- user@example.com
|
||||
|
||||
## 🛠️ Service Management
|
||||
|
||||
### Starting Services
|
||||
```bash
|
||||
cd /root/stoatchat
|
||||
./manage-services.sh start
|
||||
```
|
||||
|
||||
### Checking Status
|
||||
```bash
|
||||
./manage-services.sh status
|
||||
```
|
||||
|
||||
### Viewing Logs
|
||||
```bash
|
||||
# API logs
|
||||
tail -f api.log
|
||||
|
||||
# Events logs
|
||||
tail -f events.log
|
||||
|
||||
# Files logs
|
||||
tail -f files.log
|
||||
|
||||
# Proxy logs
|
||||
tail -f proxy.log
|
||||
```
|
||||
|
||||
### Service Restart
|
||||
```bash
|
||||
./manage-services.sh restart
|
||||
```
|
||||
|
||||
## 🔍 Monitoring & Maintenance
|
||||
|
||||
### Daily Checks
|
||||
- [ ] Service status verification
|
||||
- [ ] Log file review
|
||||
- [ ] SSL certificate validity
|
||||
- [ ] Disk space monitoring
|
||||
|
||||
### Weekly Checks
|
||||
- [ ] Performance metrics review
|
||||
- [ ] Security updates check
|
||||
- [ ] Backup verification
|
||||
- [ ] User activity monitoring
|
||||
|
||||
### Monthly Checks
|
||||
- [ ] SSL certificate renewal
|
||||
- [ ] System updates
|
||||
- [ ] Configuration backup
|
||||
- [ ] Performance optimization
|
||||
|
||||
## 🚨 Troubleshooting Guide
|
||||
|
||||
### Common Issues & Solutions
|
||||
|
||||
#### Services Not Starting
|
||||
```bash
|
||||
# Check logs for errors
|
||||
tail -50 api.log
|
||||
|
||||
# Verify port availability
|
||||
netstat -tulpn | grep :14702
|
||||
|
||||
# Restart specific service
|
||||
./manage-services.sh restart
|
||||
```
|
||||
|
||||
#### SSL Certificate Issues
|
||||
```bash
|
||||
# Check certificate status
|
||||
openssl s_client -connect st.vish.gg:443 -servername st.vish.gg
|
||||
|
||||
# Renew certificates
|
||||
sudo certbot renew
|
||||
|
||||
# Reload nginx
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
#### Email Not Sending
|
||||
1. Verify Gmail app password is valid
|
||||
2. Check SMTP configuration in `Revolt.overrides.toml`
|
||||
3. Test SMTP connection manually
|
||||
4. Review API logs for email errors
|
||||
|
||||
#### Database Connection Issues
|
||||
```bash
|
||||
# Test Redis connection
|
||||
redis-cli -p 6380 ping
|
||||
|
||||
# Check Redis status
|
||||
sudo systemctl status redis-server
|
||||
|
||||
# Restart Redis if needed
|
||||
sudo systemctl restart redis-server
|
||||
```
|
||||
|
||||
## 📈 Usage Statistics
|
||||
|
||||
### Test Account Details
|
||||
- **Email**: admin@example.com
|
||||
- **Account ID**: 01KH5RZXBHDX7W29XXFN6FB35F
|
||||
- **Status**: Verified and active
|
||||
- **Last Login**: February 11, 2026
|
||||
- **Test Messages**: Successfully sent in Nerds channel
|
||||
|
||||
### System Resources
|
||||
- **CPU Usage**: Normal operation levels
|
||||
- **Memory Usage**: Within expected parameters
|
||||
- **Disk Space**: Adequate for current usage
|
||||
- **Network**: All connections stable
|
||||
|
||||
## 🎯 Operational Readiness
|
||||
|
||||
### Production Readiness Checklist
|
||||
- [x] All services deployed and running
|
||||
- [x] SSL certificates installed and valid
|
||||
- [x] Email system configured and tested
|
||||
- [x] User registration working
|
||||
- [x] Authentication system functional
|
||||
- [x] Real-time messaging operational
|
||||
- [x] File upload/download working
|
||||
- [x] Voice/video calling available
|
||||
- [x] Web interface accessible
|
||||
- [x] API endpoints responding
|
||||
- [x] Database connections stable
|
||||
- [x] Monitoring procedures established
|
||||
|
||||
### Deployment Verification
|
||||
- [x] Account creation tested
|
||||
- [x] Email verification tested
|
||||
- [x] Login process tested
|
||||
- [x] Message sending tested
|
||||
- [x] Channel functionality tested
|
||||
- [x] Real-time features tested
|
||||
- [x] SSL security verified
|
||||
- [x] All domains accessible
|
||||
|
||||
## 📞 Support Information
|
||||
|
||||
### Technical Contacts
|
||||
- **System Administrator**: your-email@example.com
|
||||
- **Domain Owner**: vish.gg
|
||||
- **Technical Support**: admin@example.com
|
||||
|
||||
### Emergency Procedures
|
||||
1. **Service Outage**: Check service status and restart if needed
|
||||
2. **SSL Issues**: Verify certificate validity and renew if necessary
|
||||
3. **Database Problems**: Check Redis connection and restart service
|
||||
4. **Email Issues**: Verify SMTP configuration and Gmail app password
|
||||
|
||||
### Escalation Path
|
||||
1. Check service logs for error messages
|
||||
2. Attempt service restart
|
||||
3. Review configuration files
|
||||
4. Contact system administrator if issues persist
|
||||
|
||||
## 🔄 Watchtower Auto-Update System
|
||||
|
||||
### System Overview
|
||||
**Status**: ✅ **FULLY OPERATIONAL ACROSS ALL HOSTS**
|
||||
**Last Updated**: February 13, 2026
|
||||
**Configuration**: Scheduled updates with HTTP API monitoring
|
||||
|
||||
### Deployment Status by Host
|
||||
|
||||
| Host | Status | Schedule | Port | Network | Container ID |
|
||||
|------|--------|----------|------|---------|--------------|
|
||||
| **Homelab VM** | ✅ Running | 04:00 PST | 8083 | bridge | Active |
|
||||
| **Calypso** | ✅ Running | 04:00 PST | 8080 | bridge | Active |
|
||||
| **Atlantis** | ✅ Running | 02:00 PST | 8082 | prometheus-net | 51d8472bd7a4 |
|
||||
|
||||
### Configuration Features
|
||||
- **Scheduled Updates**: Daily automatic container updates
|
||||
- **Staggered Timing**: Prevents simultaneous updates across hosts
|
||||
- **HTTP API**: Monitoring and metrics endpoints enabled
|
||||
- **Prometheus Integration**: Metrics collection for monitoring
|
||||
- **Dependency Management**: Rolling restart disabled where needed
|
||||
|
||||
### Monitoring Endpoints
|
||||
```bash
|
||||
# Homelab VM
|
||||
curl -H "Authorization: Bearer REDACTED_WATCHTOWER_TOKEN" http://homelab-vm.local:8083/v1/update
|
||||
|
||||
# Calypso
|
||||
curl -H "Authorization: Bearer REDACTED_WATCHTOWER_TOKEN" http://calypso.local:8080/v1/update
|
||||
|
||||
# Atlantis
|
||||
curl -H "Authorization: Bearer REDACTED_WATCHTOWER_TOKEN" http://atlantis.local:8082/v1/update
|
||||
```
|
||||
|
||||
### Recent Fixes Applied
|
||||
- **Port Conflicts**: Resolved by using unique ports per host
|
||||
- **Dependency Issues**: Fixed rolling restart conflicts on Atlantis
|
||||
- **Configuration Conflicts**: Removed polling/schedule conflicts on Calypso
|
||||
- **Network Issues**: Created dedicated networks where needed
|
||||
|
||||
## 📝 Change Log
|
||||
|
||||
### February 13, 2026
|
||||
- ✅ **Watchtower System Fully Operational**
|
||||
- ✅ Fixed Atlantis dependency conflicts and port mapping
|
||||
- ✅ Resolved Homelab VM port conflicts and notification URLs
|
||||
- ✅ Fixed Calypso configuration conflicts
|
||||
- ✅ All hosts now have scheduled auto-updates working
|
||||
- ✅ HTTP API endpoints accessible for monitoring
|
||||
- ✅ Comprehensive documentation created
|
||||
|
||||
### February 11, 2026
|
||||
- ✅ Complete deployment testing performed
|
||||
- ✅ All functionality verified operational
|
||||
- ✅ Test account created and verified
|
||||
- ✅ Real-time messaging confirmed working
|
||||
- ✅ Documentation updated with test results
|
||||
|
||||
### Previous Changes
|
||||
- Initial deployment completed
|
||||
- SSL certificates configured
|
||||
- Email system integrated
|
||||
- All services deployed and configured
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Final Status
|
||||
|
||||
**STOATCHAT INSTANCE STATUS: FULLY OPERATIONAL** ✅
|
||||
|
||||
The Stoatchat instance at st.vish.gg is completely functional and ready for production use. All core features have been tested and verified working, including:
|
||||
|
||||
- ✅ User registration and verification
|
||||
- ✅ Authentication and session management
|
||||
- ✅ Real-time messaging and channels
|
||||
- ✅ File sharing capabilities
|
||||
- ✅ Voice/video calling integration
|
||||
- ✅ Web interface accessibility
|
||||
- ✅ API functionality
|
||||
- ✅ Email notifications
|
||||
- ✅ SSL security
|
||||
|
||||
**The deployment is complete and the service is ready for end users.**
|
||||
|
||||
---
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Last Updated**: February 11, 2026
|
||||
**Next Review**: February 18, 2026
|
||||
Reference in New Issue
Block a user