homelab-optimized/docs/infrastructure/network-performance-tuning.md

🚀 Network Performance Tuning Guide

🟠 Advanced Guide

This guide documents the network performance testing and optimization between Calypso and Atlantis NAS units, connected via the TP-Link TL-SX1008 10GbE switch.

---

📊 Network Performance Test Results

Test Configuration

• Date: January 2025
• Tool: iperf3 (via Docker: networkstatic/iperf3)
• Connection: Calypso ↔ TL-SX1008 ↔ Atlantis (10GbE)
• MTU: 1500 (standard)

Baseline Results (Before Tuning)

Direction	Speed	Notes
Calypso → Atlantis (upload)	6.87 Gbps	~3,570 TCP retransmits
Atlantis → Calypso (download)	9.27 Gbps	Near line-rate ✅

Optimized Results (After Tuning)

Direction	Speed	Improvement
Calypso → Atlantis (upload)	7.35 Gbps	+7%
Atlantis → Calypso (download)	9.27 Gbps	Unchanged

---

🔧 Optimizations Applied

1. Ring Buffer Optimization (Calypso)

Before:

RX: 2048 (max: 8184)
TX: 4096 (max: 8184)

After:

sudo ethtool -G eth2 rx 8184 tx 8184

Result:

RX: 8184 ✅
TX: 8184 ✅

⚠️ Note: Changing ring buffers may briefly reset the NIC and drop connections.

2. TCP Buffer Tuning (Both NAS)

Before:

net.core.rmem_max = 212992
net.core.wmem_max = 212992
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 16384 4194304

Optimized settings:

sudo sysctl -w net.core.rmem_max=16777216
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

3. NIC Offloading Features (Verified Enabled)

ethtool -k eth2 | grep -E 'tcp-segmentation|generic-segmentation|generic-receive'

All offloading features should show on:

• tcp-segmentation-offload: on
• generic-segmentation-offload: on
• generic-receive-offload: on

4. Flow Control (Verified Enabled)

ethtool -a eth2

Expected output:

Pause parameters for eth2:
Autonegotiate:  off
RX:             on
TX:             on

---

📋 Commands Reference

Check Current Settings

# Ring buffers
ethtool -g eth2

# TCP buffers
sysctl net.core.rmem_max net.core.wmem_max net.ipv4.tcp_rmem net.ipv4.tcp_wmem

# Offloading
ethtool -k eth2

# Flow control
ethtool -a eth2

# MTU
cat /sys/class/net/eth2/mtu

Apply Optimizations (Temporary)

# Max ring buffers
sudo ethtool -G eth2 rx 8184 tx 8184

# Increase TCP buffers
sudo sysctl -w net.core.rmem_max=16777216
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

⚠️ These settings reset on reboot. See "Making Changes Persistent" below.

Running iperf3 Tests

# Start server on Atlantis
sudo docker run -d --rm --name iperf3-server --network host networkstatic/iperf3 -s

# Run upload test from Calypso
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4

# Run download test from Calypso (reverse mode)
sudo docker run --rm --network host networkstatic/iperf3 -c 192.168.0.200 -t 10 -P 4 -R

# Stop server
sudo docker stop iperf3-server

---

🔒 Making Changes Persistent

On Synology DSM (Recommended)

For MTU and basic network settings, use DSM GUI:

• Control Panel → Network → Network Interface
• Select interface → Edit → Configure settings

Via sysctl.conf

Create /etc/sysctl.d/99-network-tuning.conf:

# TCP buffer sizes for 10GbE
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Additional tuning
net.core.netdev_max_backlog = 250000
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_tw_reuse = 1

Apply: sudo sysctl -p /etc/sysctl.d/99-network-tuning.conf

---

🎯 Jumbo Frames (MTU 9000)

Why Jumbo Frames Help

Jumbo frames reduce per-packet overhead by sending larger packets (9000 bytes vs 1500 bytes). This can improve throughput by ~10-15% on 10GbE.

Requirements

All devices in the path must support jumbo frames:

• ✅ TL-SX1008: Supports up to 9KB frames
• ✅ Calypso: Can be configured via DSM
• ✅ Atlantis: Can be configured via DSM
• ❌ Archer BE19000: Does NOT support jumbo frames

Safe Configuration

Since Calypso and Atlantis communicate directly through the TL-SX1008 (not the router), jumbo frames can be enabled between them without affecting other devices:

Calypso (MTU 9000) ──► TL-SX1008 ──► Atlantis (MTU 9000)
                           │
                           ▼
                    Archer (MTU 1500) ──► Other devices

Enabling Jumbo Frames

Via DSM GUI (Persistent):

1. Control Panel → Network → Network Interface
2. Select your 10G interface → Edit
3. Set MTU to 9000
4. Click OK

Via CLI (Temporary):

sudo ip link set eth2 mtu 9000
sudo ip link set ovs_eth2 mtu 9000

⚠️ Synology OVS Note: On Synology with Open vSwitch, the ovs_eth2 bridge interface may not accept MTU changes via CLI. Use DSM GUI instead.

---

🔍 Troubleshooting

High Retransmit Count

If you see many TCP retransmits in iperf3:

1. Check ring buffer sizes (increase to max)
2. Verify TCP buffers are tuned
3. Check for packet loss: ethtool -S eth2 | grep -i error
4. Verify flow control is enabled

Asymmetric Speeds

If upload is slower than download:

• This can be normal due to NIC/driver asymmetry
• Check if one side has smaller buffers
• Synology OVS adds some overhead

Speed Below Expected

1. Verify link speed: ethtool eth2 | grep Speed
2. Check for errors: ethtool -S eth2
3. Test with single stream first: iperf3 -c IP -t 10 (no -P)
4. Check CPU usage during test (might be CPU-bound)

---

📈 Performance Summary

Current Achieved Speeds

Path	Speed	% of Line Rate
Atlantis → Calypso	9.27 Gbps	93% ✅
Calypso → Atlantis	7.35 Gbps	74%
NUC → Calypso (Tailscale)	550 Mbps	N/A (WAN limited)
NUC → Calypso (SMB)	1.1 Gbps	N/A (caching benefit)

For Streaming Use Cases

These speeds are more than sufficient for:

• 4K HDR streaming: Requires ~80-150 Mbps ✅
• 4K Remux playback: Requires ~100-150 Mbps ✅
• Multiple concurrent 4K streams: Easily supported ✅

---

📚 Related Documentation

• Network Infrastructure Guide
• 10GbE Backbone Diagram
• Storage Topology

---

Last updated: January 2025