Why NGINX becomes “urgent” only after everything is already working

In the beginning, a web server is simple. We install NGINX, open a port, and the site loads. Then the environment grows. A second site appears. A certificate expires at the worst possible time. Logs fill a disk quietly. A quick change becomes a permanent exception. And one day, a routine update restarts a service and the “it worked yesterday” server is suddenly down.

This is where production readiness stops being a buzzword and becomes a survival skill. We are going to deploy NGINX on Ubuntu in a way that stays predictable under change: secure by default, persistent across reboots, observable through logs, and tuned enough to handle real traffic without turning into a fragile snowflake.

Prerequisites and system assumptions

Before we touch packages or configuration, we need to be explicit about the baseline. Production issues often come from hidden assumptions, not from NGINX itself.

• Platform: Ubuntu Server (this guide assumes Ubuntu 22.04 LTS or Ubuntu 24.04 LTS). The commands are compatible with both.
• System state: A clean or minimally customized server is preferred. If NGINX or Apache is already installed, we will verify and resolve port conflicts.
• Access: We need a user with sudo privileges. We will not run day-to-day operations as root.
• Network: The server must have outbound internet access for package installation and certificate issuance. Inbound access will be restricted to SSH, HTTP, and HTTPS.
• DNS: For TLS certificates, we need a real DNS name pointing to this server’s public IP. If DNS is not ready, we can still deploy HTTP and add TLS later.
• Security posture: We will enable a firewall, harden TLS, and ensure logs rotate. We will also validate that NGINX starts on boot and reloads safely.

We will also keep everything copy/paste-safe. When a value varies per environment (like the primary network interface or a domain name), we will first detect it and then use shell variables so the flow remains safe and repeatable.

Step 1: Update the system and confirm the server identity

We start by updating packages and confirming which Ubuntu release we are on. This reduces surprises later, especially around OpenSSL, systemd behavior, and default paths.

sudo apt-get update
sudo apt-get -y upgrade
lsb_release -a
uname -a

At this point, the system is patched and we have confirmed the OS version and kernel. If a kernel upgrade occurred, we should plan a reboot after NGINX is installed and verified, not in the middle of configuration changes.

Step 2: Install NGINX and verify the service is healthy

Now we install NGINX from Ubuntu’s repositories. This is a stable choice for production environments that value predictable updates and security patches.

sudo apt-get install -y nginx
sudo systemctl enable nginx
sudo systemctl start nginx

NGINX is now installed, enabled to start on boot, and running. Next we verify service health and confirm it is listening on the expected ports.

sudo systemctl status nginx --no-pager
sudo nginx -t
sudo ss -ltnp | grep -E ':(80|443)s'

We have validated three things: systemd sees NGINX as active, the configuration syntax is valid, and the process is listening (typically on port 80 at this stage). If port 80 is already in use, we will address that in troubleshooting.

Step 3: Establish a firewall baseline with UFW

A production web server should not expose more than it needs. We will use UFW to allow SSH, HTTP, and HTTPS, and deny everything else by default. This is a simple control that prevents accidental exposure as the server evolves.

First, we confirm whether UFW is installed and check current rules.

sudo apt-get install -y ufw
sudo ufw status verbose

Now we allow SSH so we do not lock ourselves out, then allow NGINX traffic. Ubuntu’s NGINX package registers application profiles with UFW, which keeps rules readable.

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw --force enable

The firewall is now active, inbound traffic is restricted, and only SSH/HTTP/HTTPS are allowed. We verify the effective rules and confirm the NGINX profile is applied.

sudo ufw status verbose
sudo ufw app list
sudo ufw app info 'Nginx Full'

Step 4: Create a production-ready server block with safe permissions

We are going to create a dedicated site configuration (server block) instead of editing the default. This keeps changes isolated, makes future maintenance cleaner, and reduces the risk of breaking unrelated sites.

First, we detect the server’s primary IP and interface. This is useful for verification and for environments where multiple interfaces exist.

PRIMARY_IP=$(ip -4 route get 1.1.1.1 | awk '{print $7; exit}')
EXT_IFACE=$(ip -4 route ls default | awk '{print $5; exit}')
echo "Primary IP: ${PRIMARY_IP}"
echo "Primary interface: ${EXT_IFACE}"

Now we define the domain name as a variable. We will set it once and reuse it consistently. We will also create a web root owned by www-data with conservative permissions.

DOMAIN="example.com"
WEBROOT="/var/www/${DOMAIN}/public_html"

sudo mkdir -p "${WEBROOT}"
sudo chown -R www-data:www-data "/var/www/${DOMAIN}"
sudo find "/var/www/${DOMAIN}" -type d -exec chmod 0750 {} ;
sudo find "/var/www/${DOMAIN}" -type f -exec chmod 0640 {} ;

We have created a dedicated directory structure, ensured NGINX can read it, and avoided world-writable permissions. Next we place a simple index page so we can verify routing before adding application complexity.

sudo tee "${WEBROOT}/index.html" >/dev/null <<EOF
<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>${DOMAIN} - NGINX on Ubuntu</title>
</head>
<body>
  <h1>NGINX is serving ${DOMAIN}</h1>
  <p>If we can read this, the server block and permissions are working.</p>
</body>
</html>
EOF

Now we create the NGINX server block. We will include a few production-minded defaults: explicit server_name, a dedicated access/error log, sane timeouts, and a basic security header set. We will keep it HTTP-only for the moment and add TLS in the next step.

NGINX_SITE="/etc/nginx/sites-available/${DOMAIN}.conf"

sudo tee "${NGINX_SITE}" >/dev/null <<EOF
server {
    listen 80;
    listen [::]:80;

    server_name ${DOMAIN} www.${DOMAIN};

    root ${WEBROOT};
    index index.html;

    access_log /var/log/nginx/${DOMAIN}.access.log;
    error_log  /var/log/nginx/${DOMAIN}.error.log warn;

    # Basic hardening headers (TLS-specific headers will be added after HTTPS is enabled)
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;

    # Reasonable limits for production safety
    client_max_body_size 10m;
    client_body_timeout 10s;
    client_header_timeout 10s;
    send_timeout 10s;

    location / {
        try_files $uri $uri/ =404;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }
}
EOF

We have created a dedicated configuration file for the domain with its own logs and safe defaults. Next we enable the site and disable the default site to avoid ambiguity.

sudo ln -sfn "${NGINX_SITE}" "/etc/nginx/sites-enabled/${DOMAIN}.conf"
if [ -e /etc/nginx/sites-enabled/default ]; then sudo rm -f /etc/nginx/sites-enabled/default; fi

sudo nginx -t
sudo systemctl reload nginx

NGINX has now loaded our site configuration without a full restart. Reloading is safer in production because it keeps existing connections alive while applying new config.

We verify locally first, then from the network. Local verification avoids DNS and firewall confusion.

curl -i "http://127.0.0.1/" | sed -n '1,15p'
curl -i "http://${PRIMARY_IP}/" | sed -n '1,15p'

If DNS is already pointing to the server, we can also verify by domain name. This is the path that matters for TLS issuance.

curl -i "http://${DOMAIN}/" | sed -n '1,15p'

Step 5: Enable SSL with Let’s Encrypt and enforce HTTPS

HTTP works, but production traffic should be encrypted. We will use Let’s Encrypt via Certbot, which is widely adopted and automates renewals. We will also enforce HTTPS and add TLS-focused security headers.

First we install Certbot and the NGINX plugin. The plugin edits NGINX safely and can also validate domain ownership via HTTP challenge.

sudo apt-get install -y certbot python3-certbot-nginx

Now we request a certificate for both the apex and www hostnames. Certbot will prompt for an email and terms acceptance. This is one of the few places where interactive input is normal in production because it binds to account recovery and expiry notices.

sudo certbot --nginx -d "${DOMAIN}" -d "www.${DOMAIN}"

Certbot has now obtained certificates, updated NGINX to serve HTTPS, and typically created a redirect from HTTP to HTTPS depending on the option selected. Next we verify that NGINX configuration is still valid and that port 443 is listening.

sudo nginx -t
sudo systemctl reload nginx
sudo ss -ltnp | grep -E ':(80|443)s'

Now we verify the certificate status and renewal timer. This is where production readiness shows up: we do not want a certificate that works today but expires silently later.

sudo certbot certificates
sudo systemctl status certbot.timer --no-pager
sudo systemctl list-timers --all | grep -E 'certbot|CERTBOT' || true

At this point, HTTPS is active and renewals are scheduled. Next we harden TLS behavior and add HSTS carefully.

TLS hardening and security headers

We are going to add a small, controlled TLS hardening snippet. We will avoid exotic settings that break clients, and we will keep changes explicit and reversible. We will also add HSTS, but we will start with a conservative max-age to reduce risk during early rollout.

First we create a reusable snippet for TLS settings.

sudo tee /etc/nginx/snippets/tls-hardening.conf >/dev/null <<EOF
# TLS hardening (balanced for production compatibility)
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;

# Session settings
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;

# OCSP stapling (requires resolver)
ssl_stapling on;
ssl_stapling_verify on;

# DNS resolvers for stapling and upstream name resolution
resolver 1.1.1.1 8.8.8.8 valid=300s;
resolver_timeout 5s;
EOF

Now we add a headers snippet. We will include HSTS with a modest max-age first. Once we are confident HTTPS is stable across all subdomains and services, we can increase it.

sudo tee /etc/nginx/snippets/security-headers.conf >/dev/null <<EOF
add_header X-Content-Type-Options "nosniff" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;

# HSTS: start conservative; increase after validation
add_header Strict-Transport-Security "max-age=86400" always;
EOF

We have created two controlled snippets. Next we ensure our site’s TLS server block includes them. Certbot may have created or modified blocks, so we will update the site config in a way that is explicit and easy to audit.

First we inspect the active configuration so we know what we are changing.

sudo nginx -T 2>/dev/null | sed -n '1,120p'

Now we edit the site file and ensure the HTTPS server block includes the snippets. We will use a safe editor approach. If we prefer non-interactive changes, we can manage this with configuration management later, but for a single server we keep it controlled and readable.

sudo nano "/etc/nginx/sites-available/${DOMAIN}.conf"

After editing, the HTTPS server block should include these lines inside it:

include /etc/nginx/snippets/tls-hardening.conf;
include /etc/nginx/snippets/security-headers.conf;

Now we validate and reload. This ensures we do not deploy a broken config.

sudo nginx -t
sudo systemctl reload nginx

Finally we verify HTTPS behavior end-to-end and confirm the redirect is in place.

curl -I "http://${DOMAIN}/" | sed -n '1,20p'
curl -I "https://${DOMAIN}/" | sed -n '1,25p'

We should see an HTTP 301/308 redirect to HTTPS and a successful HTTPS response. We should also see the Strict-Transport-Security header on HTTPS responses.

Step 6: Production logging and log rotation

Logs are where production truth lives, but logs also grow. We will ensure NGINX logs rotate predictably, compress old logs, and keep a reasonable retention window. Ubuntu already ships a logrotate rule for NGINX, but we will verify it and ensure our per-site logs are included.

First we inspect the existing logrotate configuration.

sudo ls -l /etc/logrotate.d/nginx
sudo sed -n '1,200p' /etc/logrotate.d/nginx

If the rule targets /var/log/nginx/*.log, our per-site logs are already covered. Next we run a dry-run to confirm logrotate sees the files and would rotate them.

sudo logrotate -d /etc/logrotate.d/nginx | sed -n '1,200p'

Now we force a rotation once to confirm NGINX reopens log files correctly. This is a safe operational check because the logrotate script signals NGINX to reopen logs without stopping the service.

sudo logrotate -f /etc/logrotate.d/nginx
sudo systemctl status nginx --no-pager
sudo ls -lh /var/log/nginx | sed -n '1,200p'

We have confirmed that log rotation works and that NGINX remains healthy after rotation. This prevents the slow failure mode where disks fill up over weeks.

Step 7: Performance tuning that stays safe under load

Performance tuning is where many servers become fragile. We will apply a small set of changes that are broadly safe: worker sizing, connection handling, gzip compression, and file caching. The goal is not to chase benchmarks; it is to keep latency stable and resource usage predictable.

NGINX worker and event tuning

We will tune the global NGINX settings in /etc/nginx/nginx.conf. First we back it up so rollback is immediate.

sudo cp -a /etc/nginx/nginx.conf "/etc/nginx/nginx.conf.$(date +%F_%H%M%S).bak"
sudo sed -n '1,200p' /etc/nginx/nginx.conf

Now we edit the file and apply a conservative, production-friendly baseline. We will keep it readable and avoid aggressive values that can cause memory pressure.

sudo nano /etc/nginx/nginx.conf

We should ensure these key directives exist (either by adding or adjusting them). This is a complete, production-grade example of /etc/nginx/nginx.conf that we can use as a reference. If we already have custom includes, we keep them and only align the relevant parts.

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 4096;
    multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    keepalive_timeout 65;
    types_hash_max_size 2048;

    server_tokens off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Logging
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log warn;

    # Gzip compression (safe baseline)
    gzip on;
    gzip_comp_level 5;
    gzip_min_length 1024;
    gzip_proxied any;
    gzip_types
        text/plain
        text/css
        text/xml
        text/javascript
        application/javascript
        application/json
        application/xml
        application/rss+xml
        image/svg+xml;

    # File caching for static assets
    open_file_cache max=10000 inactive=30s;
    open_file_cache_valid 60s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    # Includes
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Now we validate and reload. Reloading applies changes without dropping active connections.

sudo nginx -t
sudo systemctl reload nginx

We have now improved baseline performance and reduced information leakage by disabling server tokens. Next we verify that NGINX is still stable and check basic resource usage.

sudo systemctl status nginx --no-pager
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -n 10
sudo ss -s

Step 8: Operational verification and persistence checks

Production readiness is not only about configuration; it is about behavior over time. We will verify that services start on boot, that the firewall persists, and that TLS renewal is scheduled.

sudo systemctl is-enabled nginx
sudo systemctl is-enabled ufw
sudo systemctl is-enabled certbot.timer
sudo ufw status verbose

These checks confirm persistence across reboots. If any of them are disabled, we enable them explicitly and re-check.

sudo systemctl enable nginx
sudo systemctl enable ufw
sudo systemctl enable certbot.timer
sudo systemctl status certbot.timer --no-pager

Now we validate the full request path again, because production issues often come from “it was enabled but not actually working.”

curl -I "https://${DOMAIN}/" | sed -n '1,25p'
sudo tail -n 50 "/var/log/nginx/${DOMAIN}.access.log" 2>/dev/null || true
sudo tail -n 50 "/var/log/nginx/${DOMAIN}.error.log" 2>/dev/null || true

We have confirmed that HTTPS responses are served, and we can see traffic and errors in the expected per-site logs.

Troubleshooting

When production issues happen, they usually show up as a small set of symptoms. We will keep fixes direct and verifiable.

Symptom: NGINX fails to start or reload

• Likely cause: Syntax error or duplicate directives in configuration.
• Fix: Validate config and read the exact error line, then correct the referenced file.

sudo nginx -t
sudo journalctl -u nginx --no-pager -n 200

After correcting the configuration, we reload safely.

sudo nginx -t
sudo systemctl reload nginx
sudo systemctl status nginx --no-pager

Symptom: Port 80 or 443 is already in use

• Likely cause: Another web server is running (often Apache) or a second NGINX instance is bound to the port.
• Fix: Identify the process and stop/disable the conflicting service.

sudo ss -ltnp | grep -E ':(80|443)s'
sudo systemctl list-units --type=service --state=running | grep -E 'apache2|nginx' || true

If Apache is the conflict and we do not need it, we stop and disable it.

sudo systemctl stop apache2 2>/dev/null || true
sudo systemctl disable apache2 2>/dev/null || true
sudo nginx -t
sudo systemctl restart nginx
sudo ss -ltnp | grep -E ':(80|443)s'

Symptom: Certbot fails with HTTP-01 challenge errors

• Likely cause: DNS does not point to the server, firewall blocks port 80, or the domain resolves to a different IP.
• Fix: Confirm DNS resolution, confirm inbound port 80 is reachable, and confirm UFW allows HTTP.

getent ahosts "${DOMAIN}" | head -n 5
echo "Server IP: ${PRIMARY_IP}"
sudo ufw status verbose
sudo ss -ltnp | grep -E ':80s'

After DNS and firewall are correct, we retry issuance.

sudo certbot --nginx -d "${DOMAIN}" -d "www.${DOMAIN}"
sudo certbot certificates

Symptom: HTTPS works, but redirect loops or mixed behavior occurs

• Likely cause: Conflicting redirect rules across multiple server blocks, or an upstream application also forcing redirects.
• Fix: Inspect the effective NGINX configuration and ensure only one HTTP-to-HTTPS redirect path exists.

sudo nginx -T 2>/dev/null | sed -n '1,220p'
curl -I "http://${DOMAIN}/" | sed -n '1,30p'
curl -I "https://${DOMAIN}/" | sed -n '1,30p'

Once we remove duplicate redirects and reload, the loop should stop.

sudo nginx -t
sudo systemctl reload nginx

Symptom: 403 Forbidden when loading the site

• Likely cause: Incorrect permissions on the web root or missing index file.
• Fix: Confirm NGINX can traverse directories and read files, and confirm the configured root matches the actual path.

sudo nginx -T 2>/dev/null | grep -n "root ${WEBROOT}" -n || true
sudo -u www-data test -r "${WEBROOT}/index.html" && echo "www-data can read index.html" || echo "www-data cannot read index.html"
sudo namei -l "${WEBROOT}/index.html"

After correcting ownership and permissions, reload and re-test.

sudo chown -R www-data:www-data "/var/www/${DOMAIN}"
sudo find "/var/www/${DOMAIN}" -type d -exec chmod 0750 {} ;
sudo find "/var/www/${DOMAIN}" -type f -exec chmod 0640 {} ;
sudo nginx -t
sudo systemctl reload nginx
curl -I "https://${DOMAIN}/" | sed -n '1,25p'

Common mistakes

• Mistake: Enabling UFW before allowing SSH.
  Symptom: SSH disconnects and cannot reconnect.
  Fix: Use console access (cloud serial console or hypervisor console), then run:

  sudo ufw allow OpenSSH
  sudo ufw --force enable
  sudo ufw status verbose

• Mistake: Leaving the default NGINX site enabled alongside a custom server block.
  Symptom: The wrong site loads, or requests go to the default page unexpectedly.
  Fix: Remove the default symlink and reload:

  if [ -e /etc/nginx/sites-enabled/default ]; then sudo rm -f /etc/nginx/sites-enabled/default; fi
  sudo nginx -t
  sudo systemctl reload nginx

• Mistake: Setting HSTS too aggressively too early.
  Symptom: Browsers refuse HTTP access for a long time, complicating rollback during early rollout.
  Fix: Start with a low max-age (as shown), validate stability, then increase gradually and only add includeSubDomains when ready.
• Mistake: Editing NGINX config and restarting instead of reloading.
  Symptom: Brief downtime during changes, dropped connections under load.
  Fix: Prefer:

  sudo nginx -t
  sudo systemctl reload nginx

• Mistake: Assuming certificates renew without checking the timer.
  Symptom: TLS works for weeks, then suddenly expires.
  Fix: Verify timer and run a dry-run renewal:

  sudo systemctl status certbot.timer --no-pager
  sudo certbot renew --dry-run

How do we at NIILAA look at this

This setup is not impressive because it is complex. It is impressive because it is controlled. Every component is intentional. Every configuration has a reason. This is how infrastructure should scale — quietly, predictably, and without drama.

At NIILAA, we help organizations design, deploy, secure, and maintain production-grade web platforms on Ubuntu: from hardened NGINX baselines and TLS lifecycle management to firewall policy, logging strategy, performance tuning, and operational runbooks that hold up under real change.

Website: https://www.niilaa.com
Email: [email protected]
LinkedIn: https://www.linkedin.com/company/niilaa
Facebook: https://www.facebook.com/niilaa.llc