Monitoring Troubleshooting

Effective monitoring and troubleshooting of your Nginx server ensures high availability, performance, and quick resolution of issues. In this guide, we’ll cover:

• Collecting and interpreting access and error logs
• Exposing and querying basic metrics with the stub_status module
• Integrating with external monitoring platforms
• Safely testing and reloading configurations
• Checking service health and firewall settings

HTTP Access Logs

Nginx’s HTTP access logs record every client request. By default, logs are written in the combined format, but you can define a custom pattern in the http block.

http {
    log_format custom '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
}

Common Log Variables

Enabling and Viewing Access Logs

1. Enable in your server block:

   server {
       listen 80;
       server_name example.com www.example.com;
   
       root /var/www/example.com/html;
       index index.html;
   
       access_log /var/log/nginx/access.log custom;
       error_log  /var/log/nginx/error.log;
   
       location / {
           try_files $uri $uri/ =404;
       }
   }
   

2. Tail the log to observe incoming requests:

   cd /var/log/nginx/
   tail -F access.log
   

Example entry (using the custom format):

192.168.1.1 - - [26/Jan/2025:14:23:35 +0000] "GET /images/logo.png HTTP/1.1" 200 512 "-" "Mozilla/5.0…Chrome/91.0…Safari/537.36"

Mapping fields:

• 192.168.1.1 → $remote_addr
• - → $remote_user
• [26/Jan/2025:14:23:35 +0000] → $time_local
• "GET /images/logo.png HTTP/1.1" → $request
• 200 → $status
• 512 → $body_bytes_sent
• "-" → $http_referer
• "Mozilla/5.0…Safari/537.36" → $http_user_agent

Error Logs

Error logs capture configuration syntax errors, runtime exceptions, and worker process messages. Enable them alongside access logs:

server {
    listen 80;
    server_name example.com www.example.com;

    root /var/www/example.com/html;
    index index.html;

    access_log /var/log/nginx/access.log custom;
    error_log  /var/log/nginx/error.log warn;

    location / {
        try_files $uri $uri/ =404;
    }
}

Monitor errors in real time:

cd /var/log/nginx/
tail -F error.log

Example log entries:

2025/01/29 23:44:57 [emerg] 71265#71265: invalid number of arguments in "root" directive in /etc/nginx/sites-enabled/default:24
2025/01/29 23:46:03 [emerg] 71439#71439: "server" directive is not allowed here in /etc/nginx/sites-enabled/default:39

Frequent [emerg] or [crit] entries usually point to misconfigurations that need immediate attention.

HTTP Stub Status Metrics

The stub_status module provides a lightweight metrics endpoint for basic Nginx statistics.

server {
    listen       81;
    server_name  localhost;

    location /nginx_status {
        stub_status;
        access_log off;
        allow 127.0.0.1;
        deny all;
    }
}

• Port 81 isolates the status endpoint
• access_log off reduces noise
• allow/deny restricts access to localhost

Fetch metrics:

curl http://127.0.0.1:81/nginx_status

Sample output:

Active connections: 309
server accepts handled requests
16630948 16630948 31070465
Reading: 11 Writing: 218 Waiting: 38

External Monitoring Agents

For advanced dashboards, alerts, and long-term retention, integrate Nginx with external platforms.

After installing the Datadog agent, you’ll receive out-of-the-box metrics for CPU, memory, disk I/O, network, requests, connections, and status codes.

Configuration Testing and Reloading

Validate every change before reloading Nginx:

nginx -t

Success:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Failure:

nginx: [emerg] unknown directive "worker_connection" in /etc/nginx/nginx.conf:7
nginx: configuration file /etc/nginx/nginx.conf test failed

Reload without dropping connections:

nginx -s reload

Service Status and Basic Connectivity

Check Nginx’s systemd status:

systemctl status nginx

If it’s inactive:

systemctl start nginx

Ensure no port conflicts on 80 or 443:

ss -tulpn | grep nginx

Test HTTP locally:

curl -I http://localhost

A 200 OK confirms that Nginx is serving content correctly.

Firewall Configuration

Open ports 80 and 443 on both OS and cloud firewalls.

Ubuntu (ufw):

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw reload

RHEL/CentOS (firewalld):

sudo firewall-cmd --permanent --add-port=80/tcp
sudo firewall-cmd --permanent --add-port=443/tcp
sudo firewall-cmd --reload

Blocked ports will prevent any external traffic from reaching your server.

Conclusion

You’ve learned how to:

• Define and interpret access and error logs
• Expose basic metrics with stub_status
• Integrate with external monitoring tools (e.g., Datadog, Prometheus)
• Validate and reload configurations safely
• Verify service status, connectivity, and firewall settings

With these practices in place, your Nginx server will be easier to monitor, troubleshoot, and maintain.