Demo Monitoring

In this lesson we install the Datadog Agent to collect host system metrics (CPU, memory, I/O, network, processes) and then configure the NGINX integration so Datadog can scrape NGINX status metrics (requests, connections, upstreams, etc.).

A Datadog system metrics dashboard showing multiple monitoring charts and widgets (CPU usage, memory breakdown, load averages, disk latency, network traffic, and a processes memory treemap). The dashboard is labeled "Data Dog System Metrics" and appears to be for an nginx instance.

Example NGINX access/error lines you might see in Datadog logs (for reference):

183,225 logs that match "source:nginx"

DATE    HOST    SERVICE
May 16 16:02:02.000    fse-ldmx-f7-18    fse-auto-process
2018/05/16 22:02:02 error 1244#1244: *203 open() "/var/www/html/order/2480" failed (2: No such file or directory), client: 127.0.0.1, server: _, request: "GET /order/2480 HTTP/1.1"
GET /order/2480 HTTP/1.1
GET /send/notice HTTP/1.1
GET /pass/beanserver/refund/2480 HTTP/1.1
GET /pass/beanserver/payment/7794 HTTP/1.1
2018/05/16 22:02:02 error 1244#1244: *186 open() "/var/www/html/bad" failed (2: No such file or directory), client: 127.0.0.1, server: _, request: "GET /bad HTTP/1.1"
GET /pass/welcome HTTP/1.1

Prerequisites

A Datadog account (free trials are typically available): https://www.datadoghq.com
Root or sudo access on the host where NGINX runs and where the Datadog Agent will be installed.
NGINX built with http_stub_status_module (or NGINX Plus for extended metrics).

Installation — Datadog Agent

Sign in to Datadog and choose your operating system; the UI will present an install command that includes your API key and site (e.g., datadoghq.com or datadoghq.eu).
Important: never share your API key publicly.

Generic install command (replace <YOUR_API_KEY> and <YOUR_SITE> with values from the Datadog UI):

DD_API_KEY=<YOUR_API_KEY> DD_SITE="<YOUR_SITE>" bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"

Example for the EU site:

DD_API_KEY=<YOUR_API_KEY> DD_SITE="datadoghq.eu" bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"

During installation you’ll see repository configuration, GPG key imports, package downloads, and unpacking. A successful install typically shows the datadog-agent package being installed. Example truncated console output:

Get:1 https://apt.datadoghq.com stable/7 amd64 datadog-agent amd64 1:7.xx.x-1 [xx MB]
Fetched xx MB in 3s
Selecting previously unselected package datadog-agent.
(Reading database ... )
Preparing to unpack .../datadog-agent_1%3a7.xx.x-1_amd64.deb ...
Unpacking datadog-agent (1:7.xx.x-1) ...
Setting up datadog-agent (1:7.xx.x-1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/datadog-agent.service → /lib/systemd/system/datadog-agent.service.

Verify the Agent is running:

# systemd systems
sudo systemctl status datadog-agent

# Datadog agent CLI
sudo datadog-agent status

It can take a few minutes for the agent to begin sending metrics and for them to appear in the Datadog UI (Dashboards → Hosts or Dashboards → Metrics). Quick commands reference

Purpose	Command
Install agent (example)	`DD_API_KEY=<YOUR_API_KEY> DD_SITE="<YOUR_SITE>" bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"`
Check service status	`sudo systemctl status datadog-agent`
Agent runtime status	`sudo datadog-agent status`
Check NGINX build for stub_status	`nginx -V 2>&1 \| grep -o http_stub_status_module \|\| true`

Enable the NGINX integration in Datadog

In the Datadog UI go to Integrations → Integrations and search for “Nginx”.
Datadog can auto-detect some services and will suggest integrations when applicable.

A screenshot of the Datadog Integrations page showing autodetected integrations (Nginx, SSH) and a grid of available integration tiles (.NET, 1Password, Active Directory, ActiveMQ, etc.). The left sidebar and top navigation are visible with search and category filters.

Expose NGINX metrics endpoint (stub_status) Datadog scrapes an HTTP endpoint exposed by NGINX. For open-source NGINX, enable stub_status and expose it only on the local interface so the agent can scrape it. Add a local-only server block (example uses port 81):

server {
    listen 81;
    server_name localhost;

    access_log off;
    allow 127.0.0.1;
    deny all;

    location /nginx_status {
        # For open-source NGINX
        stub_status;

        # For NGINX Plus, use:
        # status;

        # Ensure version information is accessible if necessary
        server_tokens on;
    }
}

Notes:

If you already have site server blocks for example.com (80/443), add this local server block in the same host config or included file—but do not expose /nginx_status publicly.
After editing config, test and reload NGINX:

sudo nginx -t
sudo nginx -s reload

Verify the stub_status endpoint locally:

curl http://127.0.0.1:81/nginx_status

Expected stub_status output:

Active connections: 1
server accepts handled requests
  9 9 13
Reading: 0 Writing: 1 Waiting: 0

Do not make the NGINX stub_status endpoint publicly accessible. Always restrict it to localhost or trusted networks to avoid exposing internal metrics.

Configure the Datadog NGINX integration

In Datadog: Integrations → NGINX → Configure.
Point the integration to the local endpoint (e.g., localhost:81 or 127.0.0.1:81) so the agent scrapes http://127.0.0.1:81/nginx_status.
If you changed the port or path, update the integration settings accordingly.

Verify the module is available in NGINX:

nginx -V 2>&1 | grep -o http_stub_status_module || true

If the command outputs http_stub_status_module, the module is present. If not, install an NGINX package that includes it or rebuild NGINX with that module. Viewing NGINX metrics in Datadog

After integration is configured and the agent is scraping the endpoint, allow a few minutes for metrics to appear.
Dashboards → NGINX or Dashboards → Hosts will show NGINX-specific charts (connections, requests, upstream statuses) alongside host-level metrics (CPU, memory, load).

If metrics are not visible immediately, try changing the dashboard time window (e.g., last 15 minutes) and confirm the host is selected in the integration settings.

A screenshot of the Datadog web UI showing an "Organize Dashboards with Lists" banner and a list of dashboards (e.g., NGINX, Host Counts) in the main pane. The page includes a left navigation menu, a search bar, and columns for author/teams/modified/popularity.

Troubleshooting and host details

Click an individual host in Datadog to open the host summary and view system metrics and integrations reporting for that host.
Confirm the Datadog agent is reporting and that the NGINX integration shows the configured endpoint.

A screenshot of the Datadog "NGINX - Overview" dashboard showing a large green NGINX banner, activity summary widgets (mostly showing no data), and a left-hand navigation menu with monitoring options.

Host map and host detail example:

A Datadog infrastructure Host Map screenshot showing a single host labeled "nginx" visualized as a green hexagon with nested components "agent" and "ntp." The lower panel shows host details and metrics like CPU utilization.

Final tips and recommended next steps

Ensure the Datadog Agent is running and that http://127.0.0.1:<PORT>/nginx_status is reachable from the host where the agent runs.
Keep /nginx_status restricted to localhost or a trusted IP range; do not expose it publicly.
For advanced metrics (status zones, upstream detail, extended metrics), consider NGINX Plus or additional log parsing and instrumentation.
Use load testing tools like ApacheBench (ab) or wrk to generate traffic and observe how metrics change in real time.
Check agent logs for troubleshooting: /var/log/datadog/ and run sudo datadog-agent status for runtime diagnostics.

If metrics do not appear immediately, wait a few minutes and verify the agent logs (/var/log/datadog/ or sudo datadog-agent status) and the integration configuration (host/port) in the Datadog UI.

This completes the Datadog Agent installation and NGINX integration setup for collecting host and NGINX metrics. For deeper troubleshooting or advanced monitoring (tracing, custom metrics, log processing), review Datadog’s documentation and integration-specific guides. Links and references

Datadog: https://www.datadoghq.com
Datadog Agent install docs: https://docs.datadoghq.com/agent/
NGINX stub_status: https://nginx.org/en/docs/http/ngx_http_stub_status_module.html
ApacheBench (ab): https://httpd.apache.org/docs/2.4/programs/ab.html
wrk: https://github.com/wg/wrk

Introduction

Install & Config

Intermediate Config

Security

Performance

Conclusion

Watch Video