When a quiet Linux server suddenly isn’t quiet anymore
Most Linux servers don’t start their lives under pressure. In the beginning, traffic is predictable: a few users, a few cron jobs, a few dashboards. Then the system grows. We add APIs. We expose a login page. We publish a DNS record. We open a port for a partner integration. And slowly, without any dramatic moment, the server becomes “internet-real”.
That’s when DDoS stops being an abstract headline and becomes an operational problem. Not always a Hollywood-scale flood—often it’s smaller and nastier: connection exhaustion, SYN backlogs filling up, UDP noise, or an application that collapses because the kernel is spending its time juggling half-open sockets. The good news is that we can do a lot at the OS level to make Linux behave like a disciplined bouncer: accept what we want, drop what we don’t, and stay stable under stress.
This guide focuses on OS-level mitigation on Linux: kernel hardening, sane connection handling, and firewall controls that persist across reboots and can be verified in production.
Prerequisites and assumptions
Before we touch anything, we need to be explicit about the environment. OS-level DDoS mitigation is powerful, but it can also lock us out or break legitimate traffic if we apply it blindly.
- Platform: Linux (production servers). Commands below assume a modern distribution with
systemdandnftablesavailable. This is typical for Debian 11/12, Ubuntu 20.04/22.04/24.04, RHEL 8/9, Rocky/Alma 8/9. - Access: We need root privileges. If we use sudo, we must have passwordless sudo or be ready to enter the password interactively.
- Remote safety: We must have an out-of-band console (cloud serial console, IPMI/iDRAC/iLO, or hypervisor console). Firewall changes can cut SSH if we make a mistake.
- Service clarity: We should know which ports are actually required (typically SSH 22, HTTP 80, HTTPS 443, and maybe a small set of application ports). OS-level mitigation works best when the exposed surface is minimal.
- Stateful firewall expectation: We will use
nftableswith a persistent configuration. If the server currently uses another firewall manager, we must reconcile it to avoid conflicting rules. - Change control: We should schedule a maintenance window for production. Even “safe” kernel tuning can change behavior under load.
Step 1: Capture the current network reality
Before we harden anything, we want a baseline: which interface is external, which IPs are assigned, which ports are listening, and whether any firewall is already active. This prevents us from hardening the wrong interface or accidentally blocking required services.
set -eu
echo "=== Interfaces and addresses ==="
ip -br link
ip -br addr
echo "=== Default route (usually the external interface) ==="
ip route show default || true
echo "=== Listening TCP/UDP sockets ==="
ss -lntup
ss -lnu
echo "=== Existing nftables ruleset (if any) ==="
nft list ruleset 2>/dev/null || true
echo "=== Existing iptables rules (if any) ==="
iptables -S 2>/dev/null || true
ip6tables -S 2>/dev/null || trueWe now have a snapshot of what the server is doing. The default route typically tells us the external interface, and ss shows which services are exposed. If we see unexpected listeners, we should stop and fix that first—OS-level DDoS mitigation is not a substitute for reducing exposed services.
Step 2: Apply kernel-level network hardening with persistent sysctl
Now we will tune the kernel’s network behavior. The goal is not “magic DDoS protection”; it’s disciplined handling of common abuse patterns: spoofed traffic, SYN pressure, backlog exhaustion, and noisy ICMP behavior. We will apply settings via /etc/sysctl.d/ so they persist across reboots and are auditable.
We will create a dedicated sysctl file rather than editing /etc/sysctl.conf. This keeps changes clean and reversible.
cat > /etc/sysctl.d/99-ddos-mitigation.conf <<'EOF'
# OS-level DDoS mitigation and network hardening
# Apply with: sysctl --system
# --- Spoofing and routing hygiene ---
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# --- SYN pressure and backlog handling ---
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 8192
net.core.somaxconn = 8192
net.core.netdev_max_backlog = 16384
# --- Reduce exposure to low-effort connection abuse ---
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5
# --- ICMP behavior (keep diagnostics, reduce abuse surface) ---
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
# --- Log suspicious packets (rate-limited by kernel logging path) ---
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# --- IPv6 hygiene (do not disable IPv6 blindly; harden it) ---
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
EOF
sysctl --systemThe sysctl file is now on disk and will be applied on every boot. Running sysctl --system applied it immediately. The kernel will now be stricter about spoofing and redirects, more resilient under SYN pressure, and better tuned for handling bursts without collapsing into backlog exhaustion.
Verify the sysctl changes
We will verify a few key values to ensure the system accepted them. This also helps catch environments where another tool overwrites sysctl settings.
sysctl net.ipv4.tcp_syncookies
sysctl net.ipv4.tcp_max_syn_backlog
sysctl net.core.somaxconn
sysctl net.ipv4.conf.all.rp_filter
sysctl net.ipv4.conf.all.accept_redirects
sysctl net.ipv6.conf.all.accept_redirectsIf the output matches the values we set, the kernel hardening is active. If values differ, we likely have a conflicting sysctl file or a configuration management agent enforcing different settings.
Step 3: Build a controlled nftables firewall that can absorb common floods
Kernel tuning helps, but we still need a policy gate. At OS level, the firewall is where we enforce “only what we serve is allowed,” and where we can rate-limit noisy patterns without involving the application.
We will implement an nftables ruleset with these principles:
- Default deny inbound with explicit allows.
- Allow established/related so legitimate traffic stays fast.
- Allow loopback for local services.
- Allow SSH but rate-limit new connection attempts to reduce brute-force and low-grade floods.
- Allow HTTP/HTTPS with reasonable new-connection rate limiting (OS-level guardrail, not an application WAF).
- Drop invalid early to reduce kernel work.
- Keep IPv6 consistent so we don’t accidentally leave an open door.
Install and enable nftables
First we ensure nftables is installed and enabled. We will detect the package manager and proceed safely.
set -eu
if command -v apt-get >/dev/null 2>&1; then
apt-get update
apt-get install -y nftables
elif command -v dnf >/dev/null 2>&1; then
dnf install -y nftables
elif command -v yum >/dev/null 2>&1; then
yum install -y nftables
else
echo "No supported package manager found. Install nftables manually." 1>&2
exit 1
fi
systemctl enable --now nftables
systemctl status nftables --no-pagernftables is now installed (if it wasn’t already) and the service is enabled and running. This gives us a persistent firewall framework that loads rules on boot.
Detect the external interface and our SSH port safely
We will avoid hardcoding interface names like eth0 because modern Linux uses predictable naming. We will also detect the SSH port from the active sshd configuration so we don’t accidentally block access.
set -eu
EXT_IFACE="$(ip route show default 0.0.0.0/0 | awk 'NR==1{for(i=1;i<=NF;i++) if($i=="dev"){print $(i+1); exit}}')"
if [ -z "${EXT_IFACE}" ]; then
echo "Could not detect external interface from default route." 1>&2
exit 1
fi
echo "External interface: ${EXT_IFACE}"
SSH_PORT="$(ss -lntp 2>/dev/null | awk '/sshd/ {sub(/.*:/,"",$4); print $4; exit}')"
if [ -z "${SSH_PORT}" ]; then
SSH_PORT="22"
fi
echo "SSH port: ${SSH_PORT}"We now have EXT_IFACE and SSH_PORT set in the shell. This keeps the next steps copy/paste-safe while still adapting to real systems.
Apply a production-grade nftables ruleset
Now we will write a complete /etc/nftables.conf. We are intentionally being explicit: a clear baseline policy, IPv4 and IPv6 handling, and rate limits that reduce common abuse without breaking normal usage.
We will also include a small SSH allowlist hook as a commented option. In enterprise environments, restricting SSH to known management networks is one of the highest-impact OS-level controls.
set -eu
cp -a /etc/nftables.conf "/etc/nftables.conf.bak.$(date +%F-%H%M%S)" 2>/dev/null || true
cat > /etc/nftables.conf <<EOF
#!/usr/sbin/nft -f
flush ruleset
define EXT_IFACE = "${EXT_IFACE}"
define SSH_PORT = ${SSH_PORT}
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
# 1) Always allow loopback
iif "lo" accept
# 2) Drop invalid early
ct state invalid drop
# 3) Allow established/related traffic
ct state established,related accept
# 4) Allow essential ICMP/ICMPv6 for PMTU and basic network health (rate-limited)
ip protocol icmp icmp type { echo-request, echo-reply, destination-unreachable, time-exceeded, parameter-problem } limit rate 10/second burst 20 packets accept
ip6 nexthdr icmpv6 icmpv6 type { echo-request, echo-reply, destination-unreachable, packet-too-big, time-exceeded, parameter-problem, nd-neighbor-solicit, nd-neighbor-advert, nd-router-advert } limit rate 10/second burst 20 packets accept
# 5) SSH: allow but rate-limit new connections to reduce brute-force and low-grade floods
# Optional: restrict SSH to a management subnet by uncommenting and adjusting:
# ip saddr 203.0.113.0/24 tcp dport $SSH_PORT ct state new accept
iifname $EXT_IFACE tcp dport $SSH_PORT ct state new limit rate 15/minute burst 20 packets accept
iifname $EXT_IFACE tcp dport $SSH_PORT ct state new drop
# 6) Web: allow HTTP/HTTPS with a conservative new-connection rate limit
iifname $EXT_IFACE tcp dport { 80, 443 } ct state new limit rate 200/second burst 400 packets accept
iifname $EXT_IFACE tcp dport { 80, 443 } ct state new drop
iifname $EXT_IFACE tcp dport { 80, 443 } accept
# 7) If we run other services, we must explicitly allow them here.
# Example:
# iifname $EXT_IFACE tcp dport 5432 ct state new limit rate 30/second burst 60 packets accept
# 8) Log and drop everything else (rate-limited logging to avoid log floods)
limit rate 5/second burst 10 packets log prefix "nft-in-drop: " flags all counter drop
}
chain forward {
type filter hook forward priority 0; policy drop;
}
chain output {
type filter hook output priority 0; policy accept;
}
}
EOF
nft -f /etc/nftables.conf
nft list rulesetThe firewall policy is now active and persisted in /etc/nftables.conf. Inbound traffic is default-denied, established flows are fast, and new inbound connections to SSH and web ports are rate-limited. This doesn’t “stop DDoS” in the absolute sense, but it meaningfully reduces the chance that our kernel and services get overwhelmed by cheap connection churn.
Verify nftables persistence and behavior
We will confirm the service is enabled, the rules are loaded, and the policy is what we expect.
systemctl is-enabled nftables
systemctl status nftables --no-pager
nft list ruleset | sed -n '1,200p'If nftables is enabled and the ruleset shows our table inet filter with policy drop on input, the OS-level gate is in place and will survive reboots.
Step 4: Tighten connection tracking limits to match the host role
Many real incidents aren’t bandwidth floods—they’re state exhaustion. Connection tracking (conntrack) can fill up under heavy connection churn, causing legitimate traffic to fail. We will set sane conntrack limits based on available memory and expected concurrency.
We will first inspect current conntrack usage and limits, then apply a conservative increase. This is not “bigger is always better”: oversized conntrack tables consume RAM and can hide application inefficiencies.
Inspect current conntrack state
We will install conntrack tools if needed, then check current usage.
set -eu
if command -v apt-get >/dev/null 2>&1; then
apt-get update
apt-get install -y conntrack
elif command -v dnf >/dev/null 2>&1; then
dnf install -y conntrack-tools
elif command -v yum >/dev/null 2>&1; then
yum install -y conntrack-tools
fi
echo "=== conntrack current count ==="
cat /proc/sys/net/netfilter/nf_conntrack_count 2>/dev/null || true
echo "=== conntrack max ==="
cat /proc/sys/net/netfilter/nf_conntrack_max 2>/dev/null || true
echo "=== conntrack summary (if available) ==="
conntrack -S 2>/dev/null || trueWe now know whether conntrack is close to its maximum. If nf_conntrack_count approaches nf_conntrack_max during spikes, we will see dropped connections and intermittent failures.
Apply persistent conntrack sizing
Now we will set a conservative conntrack maximum. For many general-purpose web servers, 262144 is a reasonable starting point. We will persist it via sysctl so it survives reboots.
cat > /etc/sysctl.d/98-conntrack-sizing.conf <<'EOF'
# Conntrack sizing to reduce state exhaustion under connection churn
net.netfilter.nf_conntrack_max = 262144
EOF
sysctl --system
sysctl net.netfilter.nf_conntrack_max
cat /proc/sys/net/netfilter/nf_conntrack_maxThe conntrack maximum is now increased and persistent. This reduces the chance that connection tracking becomes the first bottleneck during a connection-heavy event.
Step 5: Validate under real operational checks
OS-level mitigation is only as good as our ability to verify it quickly during an incident. We will run a small set of checks that answer: “Is the firewall loaded?”, “Are ports still reachable?”, “Are we dropping too much?”, and “Is the kernel behaving as expected?”
Confirm listening services match intent
We will re-check listening sockets. The goal is to ensure we are only exposing what we meant to expose.
ss -lntup
ss -lnuIf we see unexpected listeners on the external interface, we should disable or bind them to localhost. OS-level controls are strongest when the exposed surface is small.
Confirm firewall counters and drops
We will inspect nftables counters. This helps confirm that rules are matching traffic and that we are not silently blocking legitimate flows.
nft list ruleset -a
journalctl -u nftables --no-pager -n 50
journalctl --no-pager -n 50 | grep -E "nft-in-drop" || trueIf counters increase on expected allow rules, traffic is flowing normally. If we see frequent nft-in-drop logs during normal business hours, we may be rate-limiting too aggressively or missing an allow rule for a required service.
Troubleshooting
Symptom: SSH access becomes intermittent or fails after applying nftables
- Likely cause: SSH is running on a non-standard port and our rules used the wrong port, or the SSH rate limit is too strict for our access pattern (for example, automation opening many short-lived sessions).
- Fix: Confirm the actual listening port and update the ruleset accordingly.
ss -lntp | awk '/sshd/ {print}'
grep -R "^[Pp]ort " /etc/ssh/sshd_config /etc/ssh/sshd_config.d 2>/dev/null || true
# If we need to relax SSH rate limiting, edit /etc/nftables.conf and reload:
nft -f /etc/nftables.conf
systemctl status nftables --no-pagerAfter reloading, SSH should stabilize. If we are operating in an enterprise environment, the stronger fix is to restrict SSH to management subnets rather than increasing rate limits globally.
Symptom: Web traffic drops during legitimate spikes
- Likely cause: The HTTP/HTTPS new-connection rate limit is too conservative for our workload (for example, many short-lived connections, no keep-alives, or a bursty client population).
- Fix: Increase the rate limit carefully, and confirm the application is using keep-alives and sane timeouts.
# Edit /etc/nftables.conf to adjust:
# tcp dport { 80, 443 } ct state new limit rate 200/second burst 400 packets accept
# Then reload:
nft -f /etc/nftables.conf
# Verify rules are loaded:
nft list ruleset | sed -n '1,200p'After the reload, the new thresholds apply immediately. We should also validate application-level keep-alive settings, because OS-level rate limiting is not a substitute for efficient connection reuse.
Symptom: “nf_conntrack: table full, dropping packet” appears in logs
- Likely cause: Conntrack table is still too small for the traffic pattern, or a flood is creating excessive state.
- Fix: Increase
nf_conntrack_maxand consider tightening inbound exposure so fewer flows become tracked.
journalctl -k --no-pager | grep -i conntrack | tail -n 50 || true
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max
# If needed, raise the limit (example: 524288) and apply persistently:
cat > /etc/sysctl.d/98-conntrack-sizing.conf <<'EOF'
net.netfilter.nf_conntrack_max = 524288
EOF
sysctl --system
sysctl net.netfilter.nf_conntrack_maxAfter increasing the limit, the kernel has more room for tracked flows. We should still treat this as a signal to reduce exposed services and ensure rate limits are appropriate.
Symptom: sysctl values revert after reboot
- Likely cause: Another sysctl file overrides our values, or a configuration management agent enforces different settings.
- Fix: Identify conflicting files and ensure our intended values are last-applied, or align with the management baseline.
sysctl --system 2>&1 | tail -n 80
grep -R "tcp_syncookies|somaxconn|tcp_max_syn_backlog|nf_conntrack_max" /etc/sysctl.conf /etc/sysctl.d 2>/dev/null || trueOnce we remove or reconcile conflicts, the settings will persist reliably across reboots.
Common mistakes
Mistake: Applying a firewall without confirming the SSH port
- Symptom: SSH disconnects immediately after firewall reload; console access is required.
- Fix: Use
ss -lntpto detect the sshd port, update/etc/nftables.conf, then reload withnft -f /etc/nftables.conf.
Mistake: Rate-limiting HTTP/HTTPS too aggressively
- Symptom: Users report random timeouts during peak traffic; logs show drops with prefix
nft-in-drop. - Fix: Increase the
limit rateandburstvalues for ports 80/443, then reload nftables. Confirm the application uses keep-alives to reduce new connection churn.
Mistake: Leaving unexpected services exposed
- Symptom: The server remains unstable under scanning or low-grade floods even after hardening.
- Fix: Use
ss -lntupand remove or bind non-essential services to localhost. Then explicitly allow only required ports in nftables.
Mistake: Disabling IPv6 without understanding the environment
- Symptom: Some clients fail to connect, or monitoring behaves inconsistently across networks.
- Fix: Keep IPv6 enabled and apply consistent firewall policy in
table inet. If IPv6 must be disabled for a controlled reason, do it as a separate, reviewed change with full impact analysis.
How do we at NIILAA look at this
This setup is not impressive because it is complex. It is impressive because it is controlled. Every component is intentional. Every configuration has a reason. This is how infrastructure should scale — quietly, predictably, and without drama.
At NIILAA, we help organizations design, deploy, secure, and maintain OS-level mitigation like this in real production environments—aligning kernel tuning, firewall policy, service exposure, and verification practices with operational reality. That means fewer surprises during incidents, cleaner change control, and systems that stay stable as traffic and risk grow.
Website: https://www.niilaa.com
Email: [email protected]
LinkedIn: https://www.linkedin.com/company/niilaa
Facebook: https://www.facebook.com/niilaa.llc