Ubuntu SSH Security Hardening: Industry Best Practices

Why SSH hardening stops being optional

SSH usually starts life as a convenience. We spin up an Ubuntu server, enable remote access, and move on. Weeks later it’s running a real workload. Months later it’s part of a fleet. Then the logs start telling a story: thousands of login attempts from everywhere, credential stuffing, noisy scanners, and the occasional “successful” login that nobody can explain.

Nothing dramatic happened on day one. Risk just accumulated quietly. The server gained value, the attack surface stayed open, and the default SSH posture remained “good enough” until it wasn’t. In production environments, SSH is not just a remote shell; it’s a control plane. If it’s weak, everything behind it is weak.

We are going to harden SSH on Ubuntu using industry best practices with an audit-ready mindset: key-based authentication, tight access control, sensible cryptography, firewall alignment, and fail2ban to reduce brute-force noise. Every step is designed to be persistent across reboots and safe to apply in real environments.

Prerequisites and assumptions

Before we touch SSH, we need to be explicit about the environment and the safety rails:

Platform: Ubuntu Server (recommended: Ubuntu 22.04 LTS or 24.04 LTS). Commands below assume systemd is present (default on modern Ubuntu).
Access method: We must have console access (cloud serial console / hypervisor console / physical access) or an out-of-band method in case we lock ourselves out. Production hardening without a backdoor plan is how outages happen.
Privileges: We must be able to run commands with sudo. If we are logged in as root, we should still follow the same steps but be extra careful when disabling root SSH.
Existing SSH session: We should keep one SSH session open while applying changes and use a second session to test new settings before closing the first. This is the simplest rollback strategy.
Network controls: If this server sits behind a security group, ACL, or upstream firewall, we must ensure SSH port changes are reflected there as well. Local firewall changes alone are not enough.
Identity model: We will use key-based authentication. That means we need an SSH keypair available on the admin workstation and we must install the public key on the server.
Audit readiness: We will keep configuration changes explicit, verify after each major step, and ensure services are enabled to persist across reboots.

Step 1: Establish a clean baseline and capture current SSH state

Before changing anything, we are going to confirm what SSH is currently doing: which port it listens on, whether it is running, and what configuration files are in play. This gives us a baseline for troubleshooting and audit trails.

set -euo pipefail

lsb_release -a || true
uname -a
systemctl status ssh --no-pager
ss -tulpn | grep -E '(:22s|sshd)' || true
sudo sshd -T | sed -n '1,120p'

We just confirmed the OS details, verified the SSH daemon status, checked listening sockets, and printed the effective SSH configuration as interpreted by sshd. That last command is especially useful because it shows what SSH will actually enforce after includes and defaults.

Step 2: Create a dedicated admin account and lock down privileges

In production, we avoid day-to-day SSH access as root. Instead, we use a named admin account with sudo privileges. This improves accountability (logs show who did what) and reduces the blast radius of credential mistakes.

We are going to create an admin user, add it to the sudo group, and ensure the home directory exists with correct permissions.

sudo adduser adminops
sudo usermod -aG sudo adminops
sudo passwd -l root || true
getent passwd adminops
id adminops
sudo ls -ld /home/adminops

We created adminops, granted sudo access, and confirmed the account exists. We also attempted to lock the root password (this does not disable root entirely; it prevents password-based root logins locally). The home directory permissions check helps prevent SSH key issues later.

Step 3: Enforce key-based authentication for SSH

Password-based SSH is the most common entry point for brute-force attacks. Even strong passwords get hammered, and the noise alone becomes operationally expensive. Key-based authentication is the baseline for production hardening.

We are going to install a public key for adminops. To keep this copy/paste-safe, we will first create the SSH directory and then add a key. Since public keys are unique, we will place the key using a controlled editor step. This is one of the few places where a placeholder is unavoidable because the key material must be specific to our environment.

Prepare the SSH directory with correct permissions

SSH is strict about permissions. If ~/.ssh or authorized_keys is writable by others, SSH may ignore the keys. We are going to create the directory and set ownership and mode correctly.

sudo -u adminops mkdir -p /home/adminops/.ssh
sudo chmod 700 /home/adminops/.ssh
sudo touch /home/adminops/.ssh/authorized_keys
sudo chmod 600 /home/adminops/.ssh/authorized_keys
sudo chown -R adminops:adminops /home/adminops/.ssh
sudo ls -la /home/adminops/.ssh

We created the SSH key directory and the authorized_keys file, then applied secure permissions. This ensures SSH will accept keys and prevents accidental exposure.

Add the public key

Now we will add the public key. We will open the file and paste the public key line (it starts with ssh-ed25519 or ssh-rsa depending on our key type). We should paste exactly one key per line.

sudoedit /home/adminops/.ssh/authorized_keys

We just updated the authorized keys for adminops. Next we will verify that the file is readable and still has correct permissions.

sudo ls -l /home/adminops/.ssh/authorized_keys
sudo -u adminops wc -l /home/adminops/.ssh/authorized_keys

We confirmed the file permissions and that at least one key line exists. At this point, we should test a new SSH login as adminops from a second terminal before tightening server-side SSH settings.

Step 4: Harden sshd configuration with an audit-ready approach

Ubuntu’s OpenSSH server supports a clean pattern for production: keep the vendor file readable, and place our hardening in a dedicated drop-in file. This makes upgrades safer and changes easier to audit.

We are going to create /etc/ssh/sshd_config.d/10-hardening.conf and explicitly set the controls we care about: disable root login, disable password auth, reduce authentication attempts, and tighten forwarding features. We will also keep the configuration compatible with modern clients.

Create a dedicated hardening config file

Before writing the file, we will confirm that the drop-in directory exists. Then we will write a complete hardening file.

sudo mkdir -p /etc/ssh/sshd_config.d
sudo ls -ld /etc/ssh/sshd_config.d

We ensured the drop-in directory exists. Now we will create the hardening configuration file.

sudo tee /etc/ssh/sshd_config.d/10-hardening.conf >/dev/null <<'EOF'
# Production SSH hardening (audit-friendly drop-in)
# This file is intentionally explicit to support change review and audits.

# Protocol and identity
Protocol 2

# Authentication: keys only
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no

# Root and privilege boundaries
PermitRootLogin no

# Reduce brute-force effectiveness
MaxAuthTries 3
LoginGraceTime 20
MaxSessions 5
MaxStartups 10:30:60

# Limit what SSH can be used for
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
PermitTunnel no
PermitUserEnvironment no

# Keep connections healthy without being noisy
ClientAliveInterval 300
ClientAliveCountMax 2

# Logging for investigations
LogLevel VERBOSE

# Explicitly define who can SSH
AllowUsers adminops
EOF

We created a dedicated hardening file that disables password-based access, blocks root SSH, reduces brute-force effectiveness, and limits forwarding features that are frequently abused for lateral movement. We also restricted SSH access to a single named account, which is a strong control in production.

Validate configuration before restarting SSH

Restarting SSH with a broken config is how lockouts happen. We are going to validate the configuration syntax first, then restart the service.

sudo sshd -t
sudo systemctl restart ssh
sudo systemctl status ssh --no-pager

sshd -t validated the configuration, and the service restart applied the new settings. The status output confirms SSH is running with the updated configuration.

Verify effective SSH settings

Now we will confirm that the effective configuration matches our intent, and that SSH is still listening.

sudo sshd -T | egrep -i 'permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|maxauthtries|allowtcpforwarding|x11forwarding|loglevel|allowusers'
ss -tulpn | grep -E 'sshd|:22s' || true

We confirmed that root login and password authentication are disabled, key authentication is enabled, and the access list is enforced. We also confirmed the daemon is listening as expected.

Step 5: Align the firewall with SSH access

Hardening SSH is stronger when the network path is also controlled. On Ubuntu, ufw is a practical host firewall that integrates with iptables/nftables. We are going to enable it carefully: first allow SSH, then enable the firewall, then verify rules.

If we are connected over SSH, we must ensure SSH is allowed before enabling ufw, otherwise we can lock ourselves out.

Install and enable UFW safely

We will install ufw, allow OpenSSH, and then enable the firewall.

sudo apt-get update
sudo apt-get install -y ufw
sudo ufw allow OpenSSH
sudo ufw --force enable
sudo ufw status verbose

We installed UFW, allowed SSH, enabled the firewall, and verified the active rules. This ensures SSH remains reachable while other unsolicited inbound traffic is blocked by default (depending on existing UFW defaults).

Optional: Restrict SSH to known source networks

In enterprise environments, we often restrict SSH to a management subnet or a bastion host. This is one of the highest-impact controls we can add. Because source networks differ per environment, we will first print the server’s primary IP and default interface so we can reason about routing, then we will show a safe pattern for adding a restricted rule using a variable.

SERVER_IP=$(ip -4 route get 1.1.1.1 | awk '{for(i=1;i<=NF;i++) if ($i=="src") {print $(i+1); exit}}')
EXT_IFACE=$(ip -4 route show default | awk '{print $5; exit}')
echo "SERVER_IP=${SERVER_IP}"
echo "EXT_IFACE=${EXT_IFACE}"

We identified the server’s primary source IP and default interface. Now we can add a restricted SSH rule if we have a known management CIDR. Since that CIDR is environment-specific, we will store it in a variable and apply it consistently.

MGMT_CIDR="192.0.2.0/24"
sudo ufw delete allow OpenSSH || true
sudo ufw allow from "${MGMT_CIDR}" to any port 22 proto tcp
sudo ufw status verbose

We replaced the broad SSH allow rule with a restricted rule limited to the management network. This reduces exposure dramatically. If the management CIDR is wrong, SSH access will fail, so we should only apply this after confirming the correct source network.

Step 6: Add fail2ban to reduce brute-force noise

Even with key-based authentication, SSH will still be scanned. fail2ban helps by temporarily banning IPs that generate repeated authentication failures. This reduces log noise and slows down opportunistic attacks. It is not a substitute for keys and firewalling, but it is a strong operational control.

We are going to install fail2ban, create a local jail configuration (so upgrades do not overwrite it), enable the SSH jail, and verify bans are working.

Install fail2ban and create a local jail

We will install the package and then create /etc/fail2ban/jail.d/sshd.local with explicit settings suitable for production.

sudo apt-get update
sudo apt-get install -y fail2ban
sudo systemctl enable --now fail2ban
sudo systemctl status fail2ban --no-pager

We installed fail2ban, enabled it to start on boot, and confirmed it is running. Next we will configure the SSH jail.

sudo mkdir -p /etc/fail2ban/jail.d
sudo tee /etc/fail2ban/jail.d/sshd.local >/dev/null <<'EOF'
[sshd]
enabled = true
mode = aggressive
port = ssh
logpath = %(sshd_log)s
backend = systemd

# Ban policy (tune to your environment)
maxretry = 5
findtime = 10m
bantime = 1h

# Optional: keep internal networks safe from accidental bans
ignoreip = 127.0.0.1/8 ::1
EOF

We enabled the sshd jail with an aggressive detection mode and a clear ban policy. We also set the backend to systemd for reliable log parsing on Ubuntu.

Restart fail2ban and verify jail status

Now we will restart fail2ban to apply the new jail configuration and verify that the SSH jail is active.

sudo systemctl restart fail2ban
sudo fail2ban-client status
sudo fail2ban-client status sshd

We confirmed fail2ban is running and that the sshd jail is enabled. The jail status output shows current bans and the filter activity, which is useful for audits and incident response.

Step 7: Make SSH logging and auditing operationally useful

Hardening is not only about blocking attacks; it is also about being able to explain what happened later. Ubuntu uses systemd-journald and can also write auth logs depending on configuration. We are going to verify where SSH authentication events are recorded and ensure we can query them quickly.

We will query recent SSH logs via journald and confirm that authentication events are visible.

sudo journalctl -u ssh -S "1 hour ago" --no-pager
sudo journalctl -u ssh -S "24 hours ago" | tail -n 50

We pulled recent SSH service logs from journald. In an audit or incident, this is often the fastest way to reconstruct access attempts and configuration reloads.

Step 8: Verification checklist for production readiness

Now we will verify the controls as a set. This is the part that prevents “we changed things” from turning into “we changed things and hope it worked.”

We confirm SSH is running and enabled on boot.
```
systemctl is-enabled ssh
systemctl status ssh --no-pager
```
SSH is confirmed enabled and active.
We confirm password authentication is disabled and root login is blocked.
```
sudo sshd -T | egrep -i 'passwordauthentication|kbdinteractiveauthentication|permitrootlogin|pubkeyauthentication'
```
The effective configuration should show PasswordAuthentication no and PermitRootLogin no, with PubkeyAuthentication yes.
We confirm the access control list is enforced.
```
sudo sshd -T | egrep -i '^allowusers'
```
We should see allowusers adminops (or our approved list). This is a strong guardrail against unexpected accounts gaining SSH access.
We confirm the firewall is active and SSH is allowed as intended.
```
sudo ufw status verbose
```
We should see an explicit allow rule for SSH (either broad or restricted to a management CIDR), and UFW should be active.
We confirm fail2ban is enabled and the SSH jail is running.
```
systemctl is-enabled fail2ban
sudo fail2ban-client status sshd
```
fail2ban should be enabled and the sshd jail should show as active.

Troubleshooting

When SSH hardening goes wrong, it usually fails in predictable ways. The goal here is to recover quickly without guessing.

Symptom: “Permission denied (publickey)” when logging in as adminops

Likely causes:
- The public key was not added correctly (wrong line, extra spaces, wrapped line).
- Wrong permissions on /home/adminops/.ssh or authorized_keys.
- We are trying the wrong username, or AllowUsers blocks the account.
Fix: We verify permissions and key presence, then re-test.

sudo ls -ld /home/adminops /home/adminops/.ssh
sudo ls -l /home/adminops/.ssh/authorized_keys
sudo -u adminops head -n 5 /home/adminops/.ssh/authorized_keys
sudo sshd -T | egrep -i '^allowusers|pubkeyauthentication|passwordauthentication'

We confirmed the filesystem permissions and that the key file contains valid entries. If AllowUsers does not include adminops, SSH will reject the login even if the key is correct.

Symptom: SSH service fails to restart after config changes

Likely causes:
- A syntax error in the hardening file.
- An unsupported directive for the installed OpenSSH version.
Fix: We validate config and inspect logs, then correct the file.

sudo sshd -t
sudo journalctl -u ssh -xe --no-pager | tail -n 80
sudo ls -l /etc/ssh/sshd_config.d
sudo sed -n '1,200p' /etc/ssh/sshd_config.d/10-hardening.conf

sshd -t points to the exact line causing the failure. The journal output provides context. Once corrected, we restart SSH and re-check status.

Symptom: We can’t connect after enabling UFW

Likely causes:
- SSH was not allowed before enabling UFW.
- We restricted SSH to the wrong management CIDR.
- Upstream firewall/security group does not match the server’s SSH port/rules.
Fix: Use console access to adjust UFW rules, then verify.

sudo ufw status numbered
sudo ufw allow OpenSSH
sudo ufw status verbose

We listed rules with numbers, ensured SSH is allowed, and verified the firewall state. If we used a restricted CIDR, we should correct it and re-apply the rule.

Symptom: fail2ban is running but no SSH jail is active

Likely causes:
- The jail file is not being loaded due to naming/location issues.
- Log backend mismatch (journald vs file-based logs).
Fix: Confirm jail file placement, restart fail2ban, and check status.

sudo ls -l /etc/fail2ban/jail.d
sudo systemctl restart fail2ban
sudo fail2ban-client status
sudo fail2ban-client status sshd
sudo journalctl -u fail2ban -S "1 hour ago" --no-pager | tail -n 120

We confirmed the jail file exists, restarted fail2ban, and checked both global and sshd-specific status. The fail2ban journal output typically explains why a jail did not start.

Common mistakes

Mistake: Disabling password authentication before confirming key login works

Symptom: Existing session works, new sessions fail with Permission denied (publickey).

Fix: Use the still-open session or console access to re-enable password auth temporarily, fix keys, then disable passwords again.

sudo tee /etc/ssh/sshd_config.d/99-breakglass.conf >/dev/null <<'EOF'
PasswordAuthentication yes
KbdInteractiveAuthentication yes
EOF
sudo sshd -t
sudo systemctl restart ssh
sudo sshd -T | egrep -i 'passwordauthentication|kbdinteractiveauthentication'

We added a temporary break-glass override, validated the config, restarted SSH, and confirmed password auth is enabled. Once key access is fixed, we should remove this file and restart SSH again.

Mistake: Forgetting that AllowUsers blocks all other accounts

Symptom: A valid key for another admin account still fails with Permission denied.

Fix: Add the required account(s) explicitly, validate, and restart.

sudo sed -n '1,200p' /etc/ssh/sshd_config.d/10-hardening.conf
sudo sshd -T | egrep -i '^allowusers'
sudo sshd -t
sudo systemctl restart ssh

We reviewed the allowlist and confirmed what SSH is enforcing. If additional accounts are required, we should update AllowUsers intentionally and document the change.

Mistake: Enabling UFW but relying on an upstream firewall rule that no longer matches

Symptom: SSH times out (not “denied”), and server logs show no connection attempts.

Fix: Confirm local listening port and local firewall rules, then validate upstream rules separately.

ss -tulpn | grep -E 'sshd|:22s' || true
sudo ufw status verbose
sudo journalctl -u ssh -S "30 minutes ago" --no-pager | tail -n 80

We confirmed whether SSH is listening and whether the host firewall allows it. If logs show no inbound attempts, the issue is likely upstream routing or firewall policy.

How do we at NIILAA look at this

This setup is not impressive because it is complex. It is impressive because it is controlled. Every component is intentional. Every configuration has a reason. This is how infrastructure should scale — quietly, predictably, and without drama.

At NIILAA, we help organizations design, deploy, secure, and maintain production environments where access paths like SSH are treated as part of the control plane: hardened by default, auditable by design, and aligned with operational reality. That includes baseline standards, secure remote access patterns, logging and monitoring integration, incident-ready runbooks, and ongoing maintenance that keeps security posture stable as systems evolve.

Website: https://www.niilaa.com
Email: [email protected]
LinkedIn: https://www.linkedin.com/company/niilaa
Facebook: https://www.facebook.com/niilaa.llc