207 lines
4.9 KiB
Markdown
207 lines
4.9 KiB
Markdown
# Ollama
|
|
|
|
**🟢 Ai Service**
|
|
|
|
## 📋 Service Overview
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| **Service Name** | ollama |
|
|
| **Host** | guava |
|
|
| **Category** | Ai |
|
|
| **Difficulty** | 🟢 |
|
|
| **Docker Image** | `ollama/ollama:latest` |
|
|
| **Compose File** | `guava/portainer_yaml/llama_gpt.yaml` |
|
|
| **Directory** | `guava/portainer_yaml` |
|
|
|
|
## 🎯 Purpose
|
|
|
|
Ollama is a tool for running large language models locally.
|
|
|
|
## Deployed Instances
|
|
|
|
### Guava (Primary — CPU-only)
|
|
|
|
- **URL**: `http://192.168.0.100:11434`
|
|
- **Tailscale**: `http://100.75.252.64:11434`
|
|
- **Hardware**: AMD Ryzen 5 8600G (6C/12T), 32GB RAM, Radeon 760M iGPU (not used for inference)
|
|
- **Storage**: `/mnt/data/llama`
|
|
- **Mode**: CPU-only inference
|
|
|
|
#### Installed Models (March 2026)
|
|
|
|
| Model | Size | Use Case |
|
|
|-------|------|----------|
|
|
| **qwen3:8b** | 5.2 GB | Recommended for AI assistant tasks |
|
|
| qwen2.5-coder:7b-instruct | 4.7 GB | Code generation |
|
|
| deepseek-coder-v2:lite | 8.9 GB | Code generation (larger) |
|
|
| llama3.1:8b | 4.9 GB | General purpose |
|
|
| phi3.5:3.8b-mini-instruct | 2.2 GB | Fast, lightweight tasks |
|
|
| nomic-embed-text | 274 MB | Text embeddings |
|
|
|
|
### Olares (GPU-accelerated)
|
|
|
|
Ollama also runs on the Olares Kubernetes appliance with RTX 5090 Max-Q GPU acceleration. See `docs/services/individual/olares.md` for details.
|
|
|
|
- **Qwen3.5 27B Q4_K_M**: `https://37e62186.vishinator.olares.com/v1` (OpenAI-compatible)
|
|
- **Note**: Olares Ollama models lack tool calling templates — use vLLM for coding agents
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### Prerequisites
|
|
- Docker and Docker Compose installed
|
|
- Basic understanding of REDACTED_APP_PASSWORD
|
|
- Access to the host system (guava)
|
|
|
|
### Deployment
|
|
```bash
|
|
# Navigate to service directory
|
|
cd guava/portainer_yaml
|
|
|
|
# Start the service
|
|
docker-compose up -d
|
|
|
|
# Check service status
|
|
docker-compose ps
|
|
|
|
# View logs
|
|
docker-compose logs -f ollama
|
|
```
|
|
|
|
## 🔧 Configuration
|
|
|
|
### Docker Compose Configuration
|
|
```yaml
|
|
container_name: ollama
|
|
environment:
|
|
- OLLAMA_KEEP_ALIVE=10m
|
|
image: ollama/ollama:latest
|
|
ports:
|
|
- 11434:11434
|
|
restart: unless-stopped
|
|
volumes:
|
|
- /mnt/data/llama:/root/.ollama
|
|
|
|
```
|
|
|
|
### Environment Variables
|
|
| Variable | Value | Description |
|
|
|----------|-------|-------------|
|
|
| `OLLAMA_KEEP_ALIVE` | `10m` | Configuration variable |
|
|
|
|
|
|
### Port Mappings
|
|
| Host Port | Container Port | Protocol | Purpose |
|
|
|-----------|----------------|----------|----------|
|
|
| 11434 | 11434 | TCP | Service port |
|
|
|
|
|
|
### Volume Mappings
|
|
| Host Path | Container Path | Type | Purpose |
|
|
|-----------|----------------|------|----------|
|
|
| `/mnt/data/llama` | `/root/.ollama` | bind | Data storage |
|
|
|
|
|
|
## 🌐 Access Information
|
|
|
|
Service ports: 11434:11434
|
|
|
|
## 🔒 Security Considerations
|
|
|
|
- ⚠️ Consider adding security options (no-new-privileges)
|
|
- ⚠️ Consider running as non-root user
|
|
|
|
## 📊 Resource Requirements
|
|
|
|
No resource limits configured
|
|
|
|
### Recommended Resources
|
|
- **Minimum RAM**: 512MB
|
|
- **Recommended RAM**: 1GB+
|
|
- **CPU**: 1 core minimum
|
|
- **Storage**: Varies by usage
|
|
|
|
### Resource Monitoring
|
|
Monitor resource usage with:
|
|
```bash
|
|
docker stats
|
|
```
|
|
|
|
## 🔍 Health Monitoring
|
|
|
|
⚠️ No health check configured
|
|
Consider adding a health check:
|
|
```yaml
|
|
healthcheck:
|
|
test: ["CMD", "curl", "-f", "http://localhost:PORT/health"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
```
|
|
|
|
### Manual Health Checks
|
|
```bash
|
|
# Check container health
|
|
docker inspect --format='{{.State.Health.Status}}' CONTAINER_NAME
|
|
|
|
# View health check logs
|
|
docker inspect --format='{{range .State.Health.Log}}{{.Output}}{{end}}' CONTAINER_NAME
|
|
```
|
|
|
|
## 🚨 Troubleshooting
|
|
|
|
### Common Issues
|
|
**Service won't start**
|
|
- Check Docker logs: `docker-compose logs service-name`
|
|
- Verify port availability: `netstat -tulpn | grep PORT`
|
|
- Check file permissions on mounted volumes
|
|
|
|
**Can't access web interface**
|
|
- Verify service is running: `docker-compose ps`
|
|
- Check firewall settings
|
|
- Confirm correct port mapping
|
|
|
|
**Performance issues**
|
|
- Monitor resource usage: `docker stats`
|
|
- Check available disk space: `df -h`
|
|
- Review service logs for errors
|
|
|
|
### Useful Commands
|
|
```bash
|
|
# Check service status
|
|
docker-compose ps
|
|
|
|
# View real-time logs
|
|
docker-compose logs -f ollama
|
|
|
|
# Restart service
|
|
docker-compose restart ollama
|
|
|
|
# Update service
|
|
docker-compose pull ollama
|
|
docker-compose up -d ollama
|
|
|
|
# Access service shell
|
|
docker-compose exec ollama /bin/bash
|
|
# or
|
|
docker-compose exec ollama /bin/sh
|
|
```
|
|
|
|
## 📚 Additional Resources
|
|
|
|
- **Official Documentation**: Check the official docs for ollama
|
|
- **Docker Hub**: [ollama/ollama:latest](https://hub.docker.com/r/ollama/ollama:latest)
|
|
- **Community Forums**: Search for community discussions and solutions
|
|
- **GitHub Issues**: Check the project's GitHub for known issues
|
|
|
|
## 🔗 Related Services
|
|
|
|
Other services in the ai category on guava
|
|
|
|
---
|
|
|
|
*This documentation was originally auto-generated and has been updated with current deployment details.*
|
|
|
|
**Last Updated**: 2026-03-15
|
|
**Configuration Source**: `guava/portainer_yaml/llama_gpt.yaml`
|