301 lines
8.8 KiB
Markdown
301 lines
8.8 KiB
Markdown
# 🛠️ Development Guide
|
|
|
|
*Development environment setup and contribution guidelines for the homelab project*
|
|
|
|
## 🎯 Overview
|
|
|
|
This guide covers setting up a development environment for contributing to the homelab infrastructure, including local testing, GitOps workflows, and best practices.
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### Prerequisites
|
|
- Docker and Docker Compose
|
|
- Git with SSH key configured
|
|
- Text editor (VS Code recommended)
|
|
- Basic understanding of REDACTED_APP_PASSWORD
|
|
|
|
### Environment Setup
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://git.vish.gg/Vish/homelab.git
|
|
cd homelab
|
|
|
|
# Set up development environment
|
|
./scripts/setup-dev-environment.sh
|
|
|
|
# Validate compose files
|
|
./scripts/validate-compose.sh
|
|
```
|
|
|
|
## 🏗️ Development Workflow
|
|
|
|
### 1. Local Development
|
|
```bash
|
|
# Create feature branch
|
|
git checkout -b feature/new-service
|
|
|
|
# Make changes to compose files
|
|
# Test locally if possible
|
|
docker-compose -f hosts/vms/seattle/new-service.yml up -d
|
|
|
|
# Validate configuration
|
|
docker-compose -f hosts/vms/seattle/new-service.yml config
|
|
```
|
|
|
|
### 2. Testing Changes
|
|
- **Syntax Validation**: Use `validate-compose.sh` script
|
|
- **Local Testing**: Test compose files locally when possible
|
|
- **Documentation**: Update relevant documentation
|
|
- **Security Review**: Check for security implications
|
|
|
|
### 3. GitOps Deployment
|
|
```bash
|
|
# Commit changes
|
|
git add .
|
|
git commit -m "feat: add new service deployment"
|
|
|
|
# Push to repository
|
|
git push origin feature/new-service
|
|
|
|
# Create pull request for review
|
|
```
|
|
|
|
## 📁 Repository Structure
|
|
|
|
### Directory Organization
|
|
```
|
|
homelab/
|
|
├── hosts/ # Host-specific configurations
|
|
│ ├── synology/ # Synology NAS deployments
|
|
│ │ ├── atlantis/ # Primary NAS (DS1823xs+)
|
|
│ │ └── calypso/ # Secondary NAS (DS723+)
|
|
│ ├── vms/ # Virtual machine deployments
|
|
│ │ ├── seattle/ # Main VM services
|
|
│ │ └── homelab_vm/ # Secondary VM services
|
|
│ ├── physical/ # Physical server deployments
|
|
│ │ └── concord_nuc/ # Intel NUC services
|
|
│ └── edge/ # Edge device deployments
|
|
│ └── raspberry-pi-5-vish/
|
|
├── docs/ # Documentation
|
|
├── scripts/ # Automation scripts
|
|
├── grafana/ # Grafana configurations
|
|
├── prometheus/ # Prometheus configurations
|
|
└── deployments/ # Special deployments
|
|
```
|
|
|
|
### File Naming Conventions
|
|
- **Compose Files**: `service-name.yml` or `service-name.yaml`
|
|
- **Configuration**: `service-name.conf` or `config/`
|
|
- **Documentation**: `README.md` or `SERVICE_NAME.md`
|
|
- **Scripts**: `action-description.sh`
|
|
|
|
## 🐳 Docker Compose Guidelines
|
|
|
|
### Best Practices
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
service-name:
|
|
image: organization/image:tag # Always specify tags
|
|
container_name: service-name # Consistent naming
|
|
restart: unless-stopped # Restart policy
|
|
|
|
environment:
|
|
- PUID=1000 # User/group IDs
|
|
- PGID=1000
|
|
- TZ=America/Los_Angeles # Timezone
|
|
|
|
volumes:
|
|
- ./config:/config # Relative paths
|
|
- /data/service:/data # Absolute for data
|
|
|
|
ports:
|
|
- "8080:8080" # Explicit port mapping
|
|
|
|
networks:
|
|
- homelab # Custom networks
|
|
|
|
labels:
|
|
- "traefik.enable=true" # Reverse proxy labels
|
|
- "com.centurylinklabs.watchtower.enable=true"
|
|
|
|
networks:
|
|
homelab:
|
|
external: true
|
|
```
|
|
|
|
### Security Considerations
|
|
- **User IDs**: Always set PUID/PGID
|
|
- **Secrets**: Use Docker secrets or external files
|
|
- **Networks**: Use custom networks, avoid host networking
|
|
- **Volumes**: Minimize host volume mounts
|
|
- **Images**: Use official images when possible
|
|
|
|
## 🔧 Development Tools
|
|
|
|
### Recommended Extensions (VS Code)
|
|
- **Docker**: Container management
|
|
- **YAML**: Syntax highlighting and validation
|
|
- **GitLens**: Git integration and history
|
|
- **Markdown**: Documentation editing
|
|
- **Remote SSH**: Remote development
|
|
|
|
### Useful Scripts
|
|
```bash
|
|
# Validate all compose files
|
|
./scripts/validate-compose.sh
|
|
|
|
# Check service status
|
|
./scripts/verify-infrastructure-status.sh
|
|
|
|
# Test NTFY notifications
|
|
./scripts/test-ntfy-notifications.sh
|
|
|
|
# Generate service documentation
|
|
./scripts/generate_service_docs.py
|
|
```
|
|
|
|
## 📝 Documentation Standards
|
|
|
|
### Markdown Guidelines
|
|
- Use clear headings and structure
|
|
- Include code examples with syntax highlighting
|
|
- Add links to related documentation
|
|
- Keep content up-to-date with changes
|
|
|
|
### Service Documentation
|
|
Each service should include:
|
|
- **Purpose**: What the service does
|
|
- **Configuration**: Key configuration options
|
|
- **Access**: How to access the service
|
|
- **Troubleshooting**: Common issues and solutions
|
|
- **Dependencies**: Required services or configurations
|
|
|
|
## 🔄 GitOps Integration
|
|
|
|
### Portainer Configuration
|
|
- **Repository**: https://git.vish.gg/Vish/homelab.git
|
|
- **Branch**: main (production deployments)
|
|
- **Webhook**: Automatic deployment on push
|
|
- **Compose Path**: Relative paths from repository root
|
|
|
|
### Deployment Process
|
|
1. **Push to Repository**: Changes committed to main branch
|
|
2. **Webhook Trigger**: Portainer receives webhook notification
|
|
3. **Stack Update**: Affected stacks automatically redeploy
|
|
4. **Health Check**: Monitor deployment status
|
|
5. **Rollback**: Available through Git history
|
|
|
|
## 🧪 Testing Procedures
|
|
|
|
### Pre-Deployment Testing
|
|
```bash
|
|
# Syntax validation
|
|
docker-compose -f service.yml config
|
|
|
|
# Security scan
|
|
docker-compose -f service.yml config | docker run --rm -i hadolint/hadolint
|
|
|
|
# Local testing (if applicable)
|
|
docker-compose -f service.yml up -d
|
|
docker-compose -f service.yml logs
|
|
docker-compose -f service.yml down
|
|
```
|
|
|
|
### Post-Deployment Validation
|
|
- **Service Health**: Check container status in Portainer
|
|
- **Connectivity**: Verify service accessibility
|
|
- **Logs**: Review container logs for errors
|
|
- **Monitoring**: Check Grafana dashboards for metrics
|
|
|
|
## 🔐 Security Development
|
|
|
|
### Security Checklist
|
|
- [ ] No hardcoded secrets in compose files
|
|
- [ ] Proper user/group ID configuration
|
|
- [ ] Network isolation where appropriate
|
|
- [ ] Regular image updates via Watchtower
|
|
- [ ] SSL/TLS termination at reverse proxy
|
|
- [ ] Access control via Authentik SSO
|
|
|
|
### Vulnerability Management
|
|
- **Image Scanning**: Regular vulnerability scans
|
|
- **Update Policy**: Automated updates via Watchtower
|
|
- **Security Patches**: Prompt application of security updates
|
|
- **Access Review**: Regular review of service access
|
|
|
|
## 🚨 Troubleshooting
|
|
|
|
### Common Issues
|
|
1. **Port Conflicts**: Check for conflicting port assignments
|
|
2. **Volume Permissions**: Ensure proper file permissions
|
|
3. **Network Issues**: Verify network configuration
|
|
4. **Resource Limits**: Check CPU/memory constraints
|
|
5. **Image Availability**: Verify image exists and is accessible
|
|
|
|
### Debugging Tools
|
|
```bash
|
|
# Container inspection
|
|
docker inspect container-name
|
|
|
|
# Network debugging
|
|
docker network ls
|
|
docker network inspect network-name
|
|
|
|
# Volume inspection
|
|
docker volume ls
|
|
docker volume inspect volume-name
|
|
|
|
# Log analysis
|
|
docker logs container-name --tail 100 -f
|
|
```
|
|
|
|
## 📊 Monitoring Integration
|
|
|
|
### Metrics Collection
|
|
- **Node Exporter**: System metrics on all hosts
|
|
- **cAdvisor**: Container metrics
|
|
- **Custom Metrics**: Application-specific metrics
|
|
- **Health Checks**: Service availability monitoring
|
|
|
|
### Dashboard Development
|
|
- **Grafana**: Create dashboards for new services
|
|
- **Prometheus**: Define custom metrics and alerts
|
|
- **Documentation**: Document dashboard usage
|
|
|
|
## 🤝 Contributing
|
|
|
|
### Pull Request Process
|
|
1. **Fork Repository**: Create personal fork
|
|
2. **Feature Branch**: Create descriptive branch name
|
|
3. **Make Changes**: Follow development guidelines
|
|
4. **Test Thoroughly**: Validate all changes
|
|
5. **Update Documentation**: Keep docs current
|
|
6. **Submit PR**: Include detailed description
|
|
|
|
### Code Review
|
|
- **Security Review**: Check for security implications
|
|
- **Best Practices**: Ensure adherence to guidelines
|
|
- **Documentation**: Verify documentation updates
|
|
- **Testing**: Confirm adequate testing
|
|
|
|
## 📚 Additional Resources
|
|
|
|
### External Documentation
|
|
- [Docker Compose Reference](https://docs.docker.com/compose/)
|
|
- [Portainer Documentation](https://docs.portainer.io/)
|
|
- [Prometheus Configuration](https://prometheus.io/docs/prometheus/latest/configuration/)
|
|
- [Grafana Documentation](https://grafana.com/docs/)
|
|
|
|
### Internal Resources
|
|
- [GitOps Deployment Guide](../GITOPS_DEPLOYMENT_GUIDE.md)
|
|
- [Monitoring Setup](../admin/monitoring-setup.md)
|
|
- [Operational Status](../OPERATIONAL_STATUS.md)
|
|
- [Infrastructure Documentation](../infrastructure/INFRASTRUCTURE_OVERVIEW.md)
|
|
|
|
---
|
|
|
|
**Last Updated**: February 24, 2026
|
|
**Development Environment**: Docker-based with GitOps integration
|
|
**Status**: ✅ **ACTIVE** - Ready for contributions |