Sanitized mirror from private repository - 2026-04-08 10:30:03 UTC
This commit is contained in:
Gitea Mirror Bot
280
docs/infrastructure/network-performance-tuning.md
Normal file
280
docs/infrastructure/network-performance-tuning.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# 🚀 Network Performance Tuning Guide
|
||||
|
||||
**🟠 Advanced Guide**
|
||||
|
||||
This guide documents the network performance testing and optimization between Calypso and Atlantis NAS units, connected via the TP-Link TL-SX1008 10GbE switch.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Network Performance Test Results
|
||||
|
||||
### Test Configuration
|
||||
- **Date**: January 2025
|
||||
- **Tool**: iperf3 (via Docker: `networkstatic/iperf3`)
|
||||
- **Connection**: Calypso ↔ TL-SX1008 ↔ Atlantis (10GbE)
|
||||
- **MTU**: 1500 (standard)
|
||||
|
||||
### Baseline Results (Before Tuning)
|
||||
|
||||
| Direction | Speed | Notes |
|
||||
|-----------|-------|-------|
|
||||
| **Calypso → Atlantis** (upload) | 6.87 Gbps | ~3,570 TCP retransmits |
|
||||
| **Atlantis → Calypso** (download) | 9.27 Gbps | Near line-rate ✅ |
|
||||
|
||||
### Optimized Results (After Tuning)
|
||||
|
||||
| Direction | Speed | Improvement |
|
||||
|-----------|-------|-------------|
|
||||
| **Calypso → Atlantis** (upload) | 7.35 Gbps | +7% |
|
||||
| **Atlantis → Calypso** (download) | 9.27 Gbps | Unchanged |
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Optimizations Applied
|
||||
|
||||
### 1. Ring Buffer Optimization (Calypso)
|
||||
|
||||
**Before:**
|
||||
```
|
||||
RX: 2048 (max: 8184)
|
||||
TX: 4096 (max: 8184)
|
||||
```
|
||||
|
||||
**After:**
|
||||
```bash
|
||||
sudo ethtool -G eth2 rx 8184 tx 8184
|
||||
```
|
||||
|
||||
**Result:**
|
||||
```
|
||||
RX: 8184 ✅
|
||||
TX: 8184 ✅
|
||||
```
|
||||
|
||||
> ⚠️ **Note**: Changing ring buffers may briefly reset the NIC and drop connections.
|
||||
|
||||
### 2. TCP Buffer Tuning (Both NAS)
|
||||
|
||||
**Before:**
|
||||
```
|
||||
net.core.rmem_max = 212992
|
||||
net.core.wmem_max = 212992
|
||||
net.ipv4.tcp_rmem = 4096 87380 6291456
|
||||
net.ipv4.tcp_wmem = 4096 16384 4194304
|
||||
```
|
||||
|
||||
**Optimized settings:**
|
||||
```bash
|
||||
sudo sysctl -w net.core.rmem_max=16777216
|
||||
sudo sysctl -w net.core.wmem_max=16777216
|
||||
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
|
||||
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
|
||||
```
|
||||
|
||||
### 3. NIC Offloading Features (Verified Enabled)
|
||||
|
||||
```bash
|
||||
ethtool -k eth2 | grep -E 'tcp-segmentation|generic-segmentation|generic-receive'
|
||||
```
|
||||
|
||||
All offloading features should show `on`:
|
||||
- `tcp-segmentation-offload: on`
|
||||
- `generic-segmentation-offload: on`
|
||||
- `generic-receive-offload: on`
|
||||
|
||||
### 4. Flow Control (Verified Enabled)
|
||||
|
||||
```bash
|
||||
ethtool -a eth2
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Pause parameters for eth2:
|
||||
Autonegotiate: off
|
||||
RX: on
|
||||
TX: on
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Commands Reference
|
||||
|
||||
### Check Current Settings
|
||||
|
||||
```bash
|
||||
# Ring buffers
|
||||
ethtool -g eth2
|
||||
|
||||
# TCP buffers
|
||||
sysctl net.core.rmem_max net.core.wmem_max net.ipv4.tcp_rmem net.ipv4.tcp_wmem
|
||||
|
||||
# Offloading
|
||||
ethtool -k eth2
|
||||
|
||||
# Flow control
|
||||
ethtool -a eth2
|
||||
|
||||
# MTU
|
||||
cat /sys/class/net/eth2/mtu
|
||||
```
|
||||
|
||||
### Apply Optimizations (Temporary)
|
||||
|
||||
```bash
|
||||
# Max ring buffers
|
||||
sudo ethtool -G eth2 rx 8184 tx 8184
|
||||
|
||||
# Increase TCP buffers
|
||||
sudo sysctl -w net.core.rmem_max=16777216
|
||||
sudo sysctl -w net.core.wmem_max=16777216
|
||||
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
|
||||
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
|
||||
```
|
||||
|
||||
> ⚠️ These settings reset on reboot. See "Making Changes Persistent" below.
|
||||
|
||||
### Running iperf3 Tests
|
||||
|
||||
```bash
|
||||
# Start server on Atlantis
|
||||
sudo docker run -d --rm --name iperf3-server --network host networkstatic/iperf3 -s
|
||||
|
||||
# Run upload test from Calypso
|
||||
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4
|
||||
|
||||
# Run download test from Calypso (reverse mode)
|
||||
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4 -R
|
||||
|
||||
# Stop server
|
||||
sudo docker stop iperf3-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Making Changes Persistent
|
||||
|
||||
### On Synology DSM (Recommended)
|
||||
|
||||
For MTU and basic network settings, use DSM GUI:
|
||||
- **Control Panel** → **Network** → **Network Interface**
|
||||
- Select interface → **Edit** → Configure settings
|
||||
|
||||
### Via sysctl.conf
|
||||
|
||||
Create `/etc/sysctl.d/99-network-tuning.conf`:
|
||||
```bash
|
||||
# TCP buffer sizes for 10GbE
|
||||
net.core.rmem_max = 16777216
|
||||
net.core.wmem_max = 16777216
|
||||
net.ipv4.tcp_rmem = 4096 87380 16777216
|
||||
net.ipv4.tcp_wmem = 4096 65536 16777216
|
||||
|
||||
# Additional tuning
|
||||
net.core.netdev_max_backlog = 250000
|
||||
net.ipv4.tcp_max_syn_backlog = 30000
|
||||
net.ipv4.tcp_tw_reuse = 1
|
||||
```
|
||||
|
||||
Apply: `sudo sysctl -p /etc/sysctl.d/99-network-tuning.conf`
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Jumbo Frames (MTU 9000)
|
||||
|
||||
### Why Jumbo Frames Help
|
||||
|
||||
Jumbo frames reduce per-packet overhead by sending larger packets (9000 bytes vs 1500 bytes). This can improve throughput by ~10-15% on 10GbE.
|
||||
|
||||
### Requirements
|
||||
|
||||
All devices in the path must support jumbo frames:
|
||||
- ✅ **TL-SX1008**: Supports up to 9KB frames
|
||||
- ✅ **Calypso**: Can be configured via DSM
|
||||
- ✅ **Atlantis**: Can be configured via DSM
|
||||
- ❌ **Archer BE19000**: Does NOT support jumbo frames
|
||||
|
||||
### Safe Configuration
|
||||
|
||||
Since Calypso and Atlantis communicate directly through the TL-SX1008 (not the router), jumbo frames can be enabled between them without affecting other devices:
|
||||
|
||||
```
|
||||
Calypso (MTU 9000) ──► TL-SX1008 ──► Atlantis (MTU 9000)
|
||||
│
|
||||
▼
|
||||
Archer (MTU 1500) ──► Other devices
|
||||
```
|
||||
|
||||
### Enabling Jumbo Frames
|
||||
|
||||
**Via DSM GUI (Persistent):**
|
||||
1. **Control Panel** → **Network** → **Network Interface**
|
||||
2. Select your 10G interface → **Edit**
|
||||
3. Set **MTU** to **9000**
|
||||
4. Click **OK**
|
||||
|
||||
**Via CLI (Temporary):**
|
||||
```bash
|
||||
sudo ip link set eth2 mtu 9000
|
||||
sudo ip link set ovs_eth2 mtu 9000
|
||||
```
|
||||
|
||||
> ⚠️ **Synology OVS Note**: On Synology with Open vSwitch, the `ovs_eth2` bridge interface may not accept MTU changes via CLI. Use DSM GUI instead.
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### High Retransmit Count
|
||||
|
||||
If you see many TCP retransmits in iperf3:
|
||||
1. Check ring buffer sizes (increase to max)
|
||||
2. Verify TCP buffers are tuned
|
||||
3. Check for packet loss: `ethtool -S eth2 | grep -i error`
|
||||
4. Verify flow control is enabled
|
||||
|
||||
### Asymmetric Speeds
|
||||
|
||||
If upload is slower than download:
|
||||
- This can be normal due to NIC/driver asymmetry
|
||||
- Check if one side has smaller buffers
|
||||
- Synology OVS adds some overhead
|
||||
|
||||
### Speed Below Expected
|
||||
|
||||
1. Verify link speed: `ethtool eth2 | grep Speed`
|
||||
2. Check for errors: `ethtool -S eth2`
|
||||
3. Test with single stream first: `iperf3 -c IP -t 10` (no `-P`)
|
||||
4. Check CPU usage during test (might be CPU-bound)
|
||||
|
||||
---
|
||||
|
||||
## 📈 Performance Summary
|
||||
|
||||
### Current Achieved Speeds
|
||||
|
||||
| Path | Speed | % of Line Rate |
|
||||
|------|-------|----------------|
|
||||
| Atlantis → Calypso | 9.27 Gbps | 93% ✅ |
|
||||
| Calypso → Atlantis | 7.35 Gbps | 74% |
|
||||
| NUC → Calypso (Tailscale) | 550 Mbps | N/A (WAN limited) |
|
||||
| NUC → Calypso (SMB) | 1.1 Gbps | N/A (caching benefit) |
|
||||
|
||||
### For Streaming Use Cases
|
||||
|
||||
These speeds are more than sufficient for:
|
||||
- **4K HDR streaming**: Requires ~80-150 Mbps ✅
|
||||
- **4K Remux playback**: Requires ~100-150 Mbps ✅
|
||||
- **Multiple concurrent 4K streams**: Easily supported ✅
|
||||
|
||||
---
|
||||
|
||||
## 📚 Related Documentation
|
||||
|
||||
- [Network Infrastructure Guide](networking.md)
|
||||
- [10GbE Backbone Diagram](../diagrams/10gbe-backbone.md)
|
||||
- [Storage Topology](../diagrams/storage-topology.md)
|
||||
|
||||
---
|
||||
|
||||
*Last updated: January 2025*
|
||||
Reference in New Issue
Block a user