Secure SSH Access on Ubuntu Server: Best Practices & Compliance

Secure SSH access is never urgent until it suddenly is

In the beginning, SSH is simple. We stand up an Ubuntu server, open port 22, and move on. Then the server becomes “important.” It starts hosting production workloads, it gets added to monitoring, more admins need access, and automation begins to touch it. Quietly, the risk profile changes.

Weeks later, we notice login attempts in the logs. Months later, a vendor asks for evidence of access controls. A security review asks why password authentication is still enabled. An incident response drill asks how quickly we can revoke access without breaking operations. This is where SSH security stops being a checkbox and becomes a discipline: controlled access, auditable changes, and predictable behavior under pressure.

In this guide, we will harden SSH on Ubuntu with a compliance mindset: least privilege, strong authentication, explicit network controls, and verification at every step. Everything we do is designed to survive reboots, scale to teams, and remain explainable to auditors and security teams.

Prerequisites and assumptions

Before we touch configuration, we need to be explicit about the environment we are securing. This avoids “it worked on my box” outcomes and prevents lockouts.

Platform: Ubuntu Server (supported and maintained release). The commands below assume systemd is present (default on modern Ubuntu).
Access method: We must have an existing working SSH session to the server, and ideally a second session open as a safety net while we apply changes.
Privileges: We must have a user that can run sudo without interactive surprises. If we are using a break-glass account, we keep it available until verification is complete.
Network: We must know how we reach the server (public IP, VPN, bastion, or private network). If the server is behind a security group or upstream firewall, we must ensure it will allow the final SSH port and source ranges we choose.
Key material: We must have an SSH key pair available on the admin workstation. If we do not, we will generate one. We will not rely on passwords for long-term access.
Change control: In enterprise environments, we should schedule a maintenance window and record the intended changes. We will also keep a rollback path.

Step 1: Establish a baseline and capture current SSH behavior

Before changing anything, we will capture the current SSH configuration and service state. This gives us a reference point for audits and makes troubleshooting faster if something behaves unexpectedly.

We are going to confirm the SSH daemon status, identify which port it is listening on, and back up the current configuration.

sudo systemctl status ssh --no-pager

This shows whether the SSH service is running and if systemd is reporting any startup errors.

sudo ss -tulpn | awk 'NR==1 || /sshd/'

This confirms which IPs and ports sshd is listening on. In many environments it will be 0.0.0.0:22 and [::]:22, which is functional but often broader than we want.

sudo cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config.bak.$(date +%F)

This creates a dated backup of the SSH daemon configuration so we can revert quickly if needed.

Step 2: Ensure we have key-based access before tightening controls

We are going to make sure key-based authentication works for our administrative account before we disable password authentication. This sequencing matters: disabling passwords first is a common cause of lockouts.

Generate an SSH key pair on the admin workstation if we do not already have one

We are going to create a modern Ed25519 key pair. It is fast, secure, and widely supported. We will protect it with a passphrase to align with compliance expectations.

ssh-keygen -t ed25519 -a 64 -f ~/.ssh/id_ed25519

This creates a private key at ~/.ssh/id_ed25519 and a public key at ~/.ssh/id_ed25519.pub. The -a 64 option increases key derivation rounds, making offline attacks against the passphrase more expensive.

Install the public key on the Ubuntu server

We are going to add our public key to the server account we use for administration. This is the account we will later allow in SSH policy.

First, we will print the public key so we can confirm we are using the correct one.

cat ~/.ssh/id_ed25519.pub

This outputs the public key line. We should see a single line starting with ssh-ed25519.

Next, we will install the key using ssh-copy-id. This is safer than manual edits because it handles permissions and formatting.

ssh-copy-id -i ~/.ssh/id_ed25519.pub "$USER"@<server_dns_or_ip>

This appends the public key to ~/.ssh/authorized_keys on the server for the specified account and ensures the SSH directory permissions are acceptable to sshd.

Now we will verify that key-based login works. We will explicitly request public key authentication and avoid passwords.

ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes "$USER"@<server_dns_or_ip>

If this succeeds without prompting for the account password, we have confirmed that key-based access is working and we can safely proceed to tighten SSH.

Step 3: Create a controlled admin group and enforce least privilege access

We are going to restrict SSH access to a specific Unix group. This is a clean, auditable control: membership becomes the access gate, and revocation becomes a single command.

First, we will create a dedicated group for SSH access.

sudo groupadd --force sshaccess

This ensures the group exists. If it already exists, the command completes without breaking anything.

Next, we will add our administrative user to that group. We will detect the current username on the server to avoid mistakes.

whoami

This prints the current account name. We will use it in the next command.

ADMIN_USER="$(whoami)"
sudo usermod -aG sshaccess "$ADMIN_USER"

This adds the current user to the sshaccess group without removing existing group memberships.

Now we will verify group membership. This matters because group changes may require a new login session to take effect.

id "$ADMIN_USER"

This shows the groups for the user. We should see sshaccess listed. If it is not listed, we should log out and log back in, then re-check.

Step 4: Harden sshd configuration with compliance-friendly defaults

We are going to apply a hardened SSH configuration that prioritizes: key-based authentication, explicit access control, reduced attack surface, and strong cryptography. We will also keep the configuration readable and defensible during audits.

On Ubuntu, it is safer to use a dedicated drop-in file under /etc/ssh/sshd_config.d/ rather than editing the main file heavily. This reduces merge conflicts during upgrades and makes intent clear.

Create a dedicated hardening drop-in file

We are going to write a complete drop-in configuration file. This will override defaults where needed while keeping the base file intact.

sudo tee /etc/ssh/sshd_config.d/10-niilaa-hardening.conf > /dev/null <<'EOF'
# NIILAA SSH hardening - production baseline
# Intent: controlled access, strong authentication, and audit-friendly behavior.

# Protocol and host keys
Protocol 2

# Authentication: keys only
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
UsePAM yes

# Root access: disallow direct login
PermitRootLogin no

# Limit who can log in via SSH
AllowGroups sshaccess

# Reduce exposure to brute force and resource abuse
MaxAuthTries 3
LoginGraceTime 30
MaxSessions 10
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable forwarding features unless explicitly required
AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no
PermitTunnel no

# Logging for investigations and compliance
LogLevel VERBOSE

# Strong crypto defaults (OpenSSH will ignore unsupported entries)
Ciphers [email protected],[email protected],aes256-ctr
MACs [email protected],[email protected]
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512
EOF

This creates a clear, versionable policy file. We have explicitly disabled password and interactive authentication, blocked root login, restricted access to the sshaccess group, tightened session behavior, and reduced forwarding features that often become lateral-movement tools in incidents.

Validate the SSH configuration before restarting

We are going to validate the SSH daemon configuration syntax. This is a critical safety step: it prevents a bad config from taking down remote access.

sudo sshd -t

If the command returns no output, the configuration is syntactically valid. If it prints errors, we must fix them before proceeding.

Restart SSH safely and verify service health

We are going to restart the SSH service so the new policy takes effect. We will then confirm it is running and listening as expected.

sudo systemctl restart ssh
sudo systemctl status ssh --no-pager

This applies the new configuration. The status output should show the service as active (running). If it is not, we should immediately consult logs and revert using the backup.

Now we will confirm the listening sockets again.

sudo ss -tulpn | awk 'NR==1 || /sshd/'

This confirms sshd is still listening. At this stage we have not changed the port, so we should still see port 22 unless the environment already used a different port.

Step 5: Enforce host-based network control with UFW

SSH hardening is incomplete without network control. Even with keys-only authentication, reducing who can reach the SSH port lowers noise, reduces attack surface, and aligns with compliance expectations around segmentation.

We are going to configure UFW to allow SSH only from a controlled source range. In enterprise environments, this is typically a bastion subnet, VPN subnet, or corporate egress IP range.

Install and enable UFW with a safe default policy

We are going to ensure UFW is installed, set a default deny inbound policy, and allow outbound traffic. We will also allow SSH before enabling the firewall to avoid locking ourselves out.

First, we will confirm whether UFW is present.

sudo ufw status verbose

If UFW is not installed, the command will indicate it. If it is installed, we will see whether it is active and what rules exist.

Now we will install UFW if needed.

sudo apt-get update
sudo apt-get install -y ufw

This ensures the firewall tooling is present and maintained via Ubuntu packages.

Next, we will set default policies.

sudo ufw default deny incoming
sudo ufw default allow outgoing

This establishes a baseline: inbound is blocked unless explicitly allowed, outbound remains functional for updates and telemetry.

Allow SSH from a controlled source

We are going to add an allow rule for SSH that is restricted to a known source range. Because source ranges differ per environment, we will first detect our current remote IP as seen by the server. This is not perfect in NAT scenarios, but it is a safe starting point for controlled environments.

First, we will print the current SSH client IP from the active session.

echo "$SSH_CONNECTION" | awk '{print $1}'

This outputs the client IP address for the current SSH session. If it is empty, we are likely not in an SSH session or the environment is unusual.

Now we will store it in a variable and allow SSH only from that IP. This is a conservative rule that we can later expand to a subnet or corporate range.

SSH_CLIENT_IP="$(echo "$SSH_CONNECTION" | awk '{print $1}')"
sudo ufw allow from "$SSH_CLIENT_IP" to any port 22 proto tcp

This creates a firewall rule allowing TCP/22 only from the detected client IP. Everyone else is blocked by default.

Now we will enable UFW.

sudo ufw enable

This activates the firewall and persists across reboots. Because we allowed SSH from our current client IP first, we should retain access.

We will verify the firewall status and rules.

sudo ufw status verbose

This confirms UFW is active, shows default policies, and lists the SSH allow rule we created.

Step 6: Add fail2ban for automated response to abusive behavior

Even with keys-only authentication, SSH endpoints attract constant scanning. We are going to add fail2ban to temporarily ban IPs that generate repeated authentication failures. This reduces log noise and slows down brute-force attempts, which is useful for both security and operational clarity.

Install fail2ban

We are going to install fail2ban from Ubuntu repositories so it receives security updates through standard patching processes.

sudo apt-get install -y fail2ban

This installs the service and its default configuration templates.

Configure an SSH jail with a clear policy

We are going to create a local configuration file so package updates do not overwrite our settings. We will enable the SSH jail and set a reasonable ban policy.

sudo tee /etc/fail2ban/jail.d/sshd.local > /dev/null <<'EOF'
[sshd]
enabled = true
mode = aggressive
port = 22
logpath = %(sshd_log)s
backend = systemd
maxretry = 5
findtime = 10m
bantime = 1h
EOF

This enables the SSH jail, uses systemd log backend, and applies a policy that is strict enough to reduce abuse without being overly sensitive for normal admin mistakes.

Now we will restart fail2ban and verify it is running.

sudo systemctl restart fail2ban
sudo systemctl status fail2ban --no-pager

This applies the configuration and confirms the service is active.

We will verify that the SSH jail is loaded and active.

sudo fail2ban-client status
sudo fail2ban-client status sshd

This confirms fail2ban is monitoring SSH and shows current ban statistics.

Step 7: Verify end-to-end behavior and persistence

We are going to verify that the controls work together: SSH access is restricted, passwords are disabled, group-based access is enforced, and firewall rules persist.

Confirm password authentication is disabled

We are going to query the effective SSH daemon configuration. This is more reliable than reading files because it reflects what sshd will actually use.

sudo sshd -T | egrep -i 'passwordauthentication|kbdinteractiveauthentication|challengeresponseauthentication|permitrootlogin|allowgroups|loglevel'

This prints the effective values. We should see passwordauthentication no, kbdinteractiveauthentication no, permitrootlogin no, and allowgroups sshaccess.

Confirm firewall and service persistence across reboots

We are going to confirm that both SSH and UFW are enabled at boot, which is a common compliance requirement.

sudo systemctl is-enabled ssh
sudo systemctl is-enabled ufw
sudo systemctl is-enabled fail2ban

This confirms the services are configured to start automatically after a reboot.

Confirm SSH access control via group membership

We are going to validate that access is tied to group membership. This is the operational control we will rely on for onboarding and offboarding.

getent group sshaccess

This prints the group entry and its members. If an admin needs access, we add them here. If access must be revoked, we remove them here.

Operational patterns that keep this secure over time

Onboarding: create a named user per person, add their public key, then add them to sshaccess. Avoid shared accounts.
Offboarding: remove the user from sshaccess immediately, then disable the account if required by policy.
Key rotation: treat keys like credentials. Rotate on schedule and on any suspicion of compromise.
Change control: keep SSH policy in a managed configuration system where possible, and record why each setting exists.
Logging: forward auth logs to a central system in enterprise environments. Local logs are not enough during incidents.

Troubleshooting

When SSH hardening goes wrong, it usually fails in predictable ways. We will focus on symptoms, likely causes, and fixes that are safe in production.

Symptom: “Permission denied (publickey)” after disabling passwords

Likely causes:
- The public key was not installed for the correct user.
- Incorrect permissions on ~/.ssh or authorized_keys.
- The user is not in the allowed group (sshaccess).
Fix: We will verify group membership and permissions from an existing session or console access.

ADMIN_USER="$(whoami)"
id "$ADMIN_USER"
sudo ls -ld "/home/$ADMIN_USER" "/home/$ADMIN_USER/.ssh" "/home/$ADMIN_USER/.ssh/authorized_keys"
sudo chmod 700 "/home/$ADMIN_USER/.ssh"
sudo chmod 600 "/home/$ADMIN_USER/.ssh/authorized_keys"
sudo chown -R "$ADMIN_USER:$ADMIN_USER" "/home/$ADMIN_USER/.ssh"

This confirms the user is in sshaccess and corrects common permission issues that cause OpenSSH to ignore keys.

Symptom: SSH service fails to restart after config changes

Likely causes:
- Syntax error in the hardening file.
- Unsupported crypto settings on the installed OpenSSH version.
Fix: We will validate config and inspect logs.

sudo sshd -t
sudo journalctl -u ssh -n 200 --no-pager

This surfaces the exact line or directive causing failure. If crypto directives are unsupported, OpenSSH may reject them. In that case, we remove or adjust the specific directive and re-validate with sshd -t.

Symptom: We can connect from one network but not another

Likely causes:
- UFW is restricting SSH to a single IP that changed (common on residential ISPs).
- Upstream firewall/security group does not allow the source range.
Fix: We will review UFW rules and adjust to a stable subnet (VPN range or corporate egress range).

sudo ufw status numbered

This lists rules with numbers so we can safely remove or modify them.

sudo ufw delete 1

This removes rule number 1. We should only do this after confirming which rule we are deleting.

Now we will add a new rule for a stable subnet. First we will print the server’s current default route interface to ensure we are not debugging the wrong host.

ip route | awk '/default/ {print; exit}'

This confirms the server’s default route and helps validate that the server is on the expected network.

Symptom: fail2ban bans legitimate admin IPs

Likely causes:
- Admins mistyped passphrases repeatedly.
- Automation is attempting password auth and failing.
Fix: We will unban the IP and then correct the client behavior.

sudo fail2ban-client status sshd

This shows currently banned IPs.

sudo fail2ban-client set sshd unbanip 203.0.113.10

This removes the ban for the specified IP. We should then ensure the client is using the correct key and not attempting password authentication.

Common mistakes

Mistake: Disabling password authentication before confirming key access

Symptom: Immediate lockout after restarting SSH; existing session works but new sessions fail.
Fix: Keep an existing session open, re-enable password auth temporarily from that session, install keys properly, verify key login, then disable passwords again.

Mistake: Forgetting the group restriction exists

Symptom: Correct key is presented but login still fails; logs show user not allowed.
Fix: Add the user to sshaccess and re-login to refresh group membership.

sudo usermod -aG sshaccess someuser
getent group sshaccess

This grants access via group membership and confirms the group contains the intended user.

Mistake: Over-restricting UFW to a changing IP

Symptom: SSH works from one location, fails from another, or fails after ISP IP changes.
Fix: Replace single-IP rules with a stable VPN subnet or corporate egress range, and confirm upstream firewalls match.

Mistake: Editing the main sshd_config directly and losing intent over time

Symptom: After upgrades, settings drift or become hard to audit; multiple conflicting directives exist.
Fix: Use a dedicated drop-in file under /etc/ssh/sshd_config.d/ and validate with sshd -T.

How do we at NIILAA look at this

This setup is not impressive because it is complex. It is impressive because it is controlled. Every component is intentional. Every configuration has a reason. This is how infrastructure should scale — quietly, predictably, and without drama.

At NIILAA, we help organizations design, deploy, secure, and maintain production-grade access patterns like this across fleets of Ubuntu systems: hardened baselines, identity-aligned access, audit-ready configuration, and operational runbooks that hold up during real incidents and real compliance reviews.

Website: https://www.niilaa.com
Email: [email protected]
LinkedIn: https://www.linkedin.com/company/niilaa
Facebook: https://www.facebook.com/niilaa.llc