281 lines
6.7 KiB
Markdown
281 lines
6.7 KiB
Markdown
# 🚀 Network Performance Tuning Guide
|
|
|
|
**🟠 Advanced Guide**
|
|
|
|
This guide documents the network performance testing and optimization between Calypso and Atlantis NAS units, connected via the TP-Link TL-SX1008 10GbE switch.
|
|
|
|
---
|
|
|
|
## 📊 Network Performance Test Results
|
|
|
|
### Test Configuration
|
|
- **Date**: January 2025
|
|
- **Tool**: iperf3 (via Docker: `networkstatic/iperf3`)
|
|
- **Connection**: Calypso ↔ TL-SX1008 ↔ Atlantis (10GbE)
|
|
- **MTU**: 1500 (standard)
|
|
|
|
### Baseline Results (Before Tuning)
|
|
|
|
| Direction | Speed | Notes |
|
|
|-----------|-------|-------|
|
|
| **Calypso → Atlantis** (upload) | 6.87 Gbps | ~3,570 TCP retransmits |
|
|
| **Atlantis → Calypso** (download) | 9.27 Gbps | Near line-rate ✅ |
|
|
|
|
### Optimized Results (After Tuning)
|
|
|
|
| Direction | Speed | Improvement |
|
|
|-----------|-------|-------------|
|
|
| **Calypso → Atlantis** (upload) | 7.35 Gbps | +7% |
|
|
| **Atlantis → Calypso** (download) | 9.27 Gbps | Unchanged |
|
|
|
|
---
|
|
|
|
## 🔧 Optimizations Applied
|
|
|
|
### 1. Ring Buffer Optimization (Calypso)
|
|
|
|
**Before:**
|
|
```
|
|
RX: 2048 (max: 8184)
|
|
TX: 4096 (max: 8184)
|
|
```
|
|
|
|
**After:**
|
|
```bash
|
|
sudo ethtool -G eth2 rx 8184 tx 8184
|
|
```
|
|
|
|
**Result:**
|
|
```
|
|
RX: 8184 ✅
|
|
TX: 8184 ✅
|
|
```
|
|
|
|
> ⚠️ **Note**: Changing ring buffers may briefly reset the NIC and drop connections.
|
|
|
|
### 2. TCP Buffer Tuning (Both NAS)
|
|
|
|
**Before:**
|
|
```
|
|
net.core.rmem_max = 212992
|
|
net.core.wmem_max = 212992
|
|
net.ipv4.tcp_rmem = 4096 87380 6291456
|
|
net.ipv4.tcp_wmem = 4096 16384 4194304
|
|
```
|
|
|
|
**Optimized settings:**
|
|
```bash
|
|
sudo sysctl -w net.core.rmem_max=16777216
|
|
sudo sysctl -w net.core.wmem_max=16777216
|
|
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
|
|
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
|
|
```
|
|
|
|
### 3. NIC Offloading Features (Verified Enabled)
|
|
|
|
```bash
|
|
ethtool -k eth2 | grep -E 'tcp-segmentation|generic-segmentation|generic-receive'
|
|
```
|
|
|
|
All offloading features should show `on`:
|
|
- `tcp-segmentation-offload: on`
|
|
- `generic-segmentation-offload: on`
|
|
- `generic-receive-offload: on`
|
|
|
|
### 4. Flow Control (Verified Enabled)
|
|
|
|
```bash
|
|
ethtool -a eth2
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
Pause parameters for eth2:
|
|
Autonegotiate: off
|
|
RX: on
|
|
TX: on
|
|
```
|
|
|
|
---
|
|
|
|
## 📋 Commands Reference
|
|
|
|
### Check Current Settings
|
|
|
|
```bash
|
|
# Ring buffers
|
|
ethtool -g eth2
|
|
|
|
# TCP buffers
|
|
sysctl net.core.rmem_max net.core.wmem_max net.ipv4.tcp_rmem net.ipv4.tcp_wmem
|
|
|
|
# Offloading
|
|
ethtool -k eth2
|
|
|
|
# Flow control
|
|
ethtool -a eth2
|
|
|
|
# MTU
|
|
cat /sys/class/net/eth2/mtu
|
|
```
|
|
|
|
### Apply Optimizations (Temporary)
|
|
|
|
```bash
|
|
# Max ring buffers
|
|
sudo ethtool -G eth2 rx 8184 tx 8184
|
|
|
|
# Increase TCP buffers
|
|
sudo sysctl -w net.core.rmem_max=16777216
|
|
sudo sysctl -w net.core.wmem_max=16777216
|
|
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
|
|
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
|
|
```
|
|
|
|
> ⚠️ These settings reset on reboot. See "Making Changes Persistent" below.
|
|
|
|
### Running iperf3 Tests
|
|
|
|
```bash
|
|
# Start server on Atlantis
|
|
sudo docker run -d --rm --name iperf3-server --network host networkstatic/iperf3 -s
|
|
|
|
# Run upload test from Calypso
|
|
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4
|
|
|
|
# Run download test from Calypso (reverse mode)
|
|
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4 -R
|
|
|
|
# Stop server
|
|
sudo docker stop iperf3-server
|
|
```
|
|
|
|
---
|
|
|
|
## 🔒 Making Changes Persistent
|
|
|
|
### On Synology DSM (Recommended)
|
|
|
|
For MTU and basic network settings, use DSM GUI:
|
|
- **Control Panel** → **Network** → **Network Interface**
|
|
- Select interface → **Edit** → Configure settings
|
|
|
|
### Via sysctl.conf
|
|
|
|
Create `/etc/sysctl.d/99-network-tuning.conf`:
|
|
```bash
|
|
# TCP buffer sizes for 10GbE
|
|
net.core.rmem_max = 16777216
|
|
net.core.wmem_max = 16777216
|
|
net.ipv4.tcp_rmem = 4096 87380 16777216
|
|
net.ipv4.tcp_wmem = 4096 65536 16777216
|
|
|
|
# Additional tuning
|
|
net.core.netdev_max_backlog = 250000
|
|
net.ipv4.tcp_max_syn_backlog = 30000
|
|
net.ipv4.tcp_tw_reuse = 1
|
|
```
|
|
|
|
Apply: `sudo sysctl -p /etc/sysctl.d/99-network-tuning.conf`
|
|
|
|
---
|
|
|
|
## 🎯 Jumbo Frames (MTU 9000)
|
|
|
|
### Why Jumbo Frames Help
|
|
|
|
Jumbo frames reduce per-packet overhead by sending larger packets (9000 bytes vs 1500 bytes). This can improve throughput by ~10-15% on 10GbE.
|
|
|
|
### Requirements
|
|
|
|
All devices in the path must support jumbo frames:
|
|
- ✅ **TL-SX1008**: Supports up to 9KB frames
|
|
- ✅ **Calypso**: Can be configured via DSM
|
|
- ✅ **Atlantis**: Can be configured via DSM
|
|
- ❌ **Archer BE19000**: Does NOT support jumbo frames
|
|
|
|
### Safe Configuration
|
|
|
|
Since Calypso and Atlantis communicate directly through the TL-SX1008 (not the router), jumbo frames can be enabled between them without affecting other devices:
|
|
|
|
```
|
|
Calypso (MTU 9000) ──► TL-SX1008 ──► Atlantis (MTU 9000)
|
|
│
|
|
▼
|
|
Archer (MTU 1500) ──► Other devices
|
|
```
|
|
|
|
### Enabling Jumbo Frames
|
|
|
|
**Via DSM GUI (Persistent):**
|
|
1. **Control Panel** → **Network** → **Network Interface**
|
|
2. Select your 10G interface → **Edit**
|
|
3. Set **MTU** to **9000**
|
|
4. Click **OK**
|
|
|
|
**Via CLI (Temporary):**
|
|
```bash
|
|
sudo ip link set eth2 mtu 9000
|
|
sudo ip link set ovs_eth2 mtu 9000
|
|
```
|
|
|
|
> ⚠️ **Synology OVS Note**: On Synology with Open vSwitch, the `ovs_eth2` bridge interface may not accept MTU changes via CLI. Use DSM GUI instead.
|
|
|
|
---
|
|
|
|
## 🔍 Troubleshooting
|
|
|
|
### High Retransmit Count
|
|
|
|
If you see many TCP retransmits in iperf3:
|
|
1. Check ring buffer sizes (increase to max)
|
|
2. Verify TCP buffers are tuned
|
|
3. Check for packet loss: `ethtool -S eth2 | grep -i error`
|
|
4. Verify flow control is enabled
|
|
|
|
### Asymmetric Speeds
|
|
|
|
If upload is slower than download:
|
|
- This can be normal due to NIC/driver asymmetry
|
|
- Check if one side has smaller buffers
|
|
- Synology OVS adds some overhead
|
|
|
|
### Speed Below Expected
|
|
|
|
1. Verify link speed: `ethtool eth2 | grep Speed`
|
|
2. Check for errors: `ethtool -S eth2`
|
|
3. Test with single stream first: `iperf3 -c IP -t 10` (no `-P`)
|
|
4. Check CPU usage during test (might be CPU-bound)
|
|
|
|
---
|
|
|
|
## 📈 Performance Summary
|
|
|
|
### Current Achieved Speeds
|
|
|
|
| Path | Speed | % of Line Rate |
|
|
|------|-------|----------------|
|
|
| Atlantis → Calypso | 9.27 Gbps | 93% ✅ |
|
|
| Calypso → Atlantis | 7.35 Gbps | 74% |
|
|
| NUC → Calypso (Tailscale) | 550 Mbps | N/A (WAN limited) |
|
|
| NUC → Calypso (SMB) | 1.1 Gbps | N/A (caching benefit) |
|
|
|
|
### For Streaming Use Cases
|
|
|
|
These speeds are more than sufficient for:
|
|
- **4K HDR streaming**: Requires ~80-150 Mbps ✅
|
|
- **4K Remux playback**: Requires ~100-150 Mbps ✅
|
|
- **Multiple concurrent 4K streams**: Easily supported ✅
|
|
|
|
---
|
|
|
|
## 📚 Related Documentation
|
|
|
|
- [Network Infrastructure Guide](networking.md)
|
|
- [10GbE Backbone Diagram](../diagrams/10gbe-backbone.md)
|
|
- [Storage Topology](../diagrams/storage-topology.md)
|
|
|
|
---
|
|
|
|
*Last updated: January 2025*
|