Skip to main content
This lesson demonstrates PromQL label selectors and modifiers with practical examples. You’ll learn how to filter series by labels, use regular expressions, select historical samples with range vectors, and query past points in time using offset and the @ modifier.

​
Inspecting per-CPU metrics

The node_cpu_seconds_total metric exposes CPU time broken down by the cpu and mode labels. Use exact label matchers to select specific series. Select all series for CPU 0:
Example output (one sample per series):
Notes:
  • Label values must be quoted (for example, cpu="0").
  • To select every CPU except zero, use the negation operator !=:

​
Range vectors: selecting historical samples

Range vectors return all samples within a relative time window for each matching series. Append a range selector [...] to a vector selector. Get the last 2 minutes of samples for CPU 0:
This returns time series objects that include all samples in the last 2 minutes for each matching series.

​
Combining labels for more specific selection

You can combine multiple label matchers to narrow results. For example, to select CPU 0 when the CPU is idle:
Example output:

​
Matching with regular expressions

Use regex matchers to match label values that fit a pattern. The =~ operator matches regex; !~ negates it. Match all mountpoint values that start with /run (e.g., /run, /run/lock, /run/snapd/ns, /run/user/127):
Example output:
To get everything that does not start with /run, use the negated regex operator:
Example output:

​
Combining regex with range selectors

You can combine regex matchers with range vectors to fetch history for only those matching series. Fetch the last 5 minutes of samples for mountpoints that do not start with /run:
Example output (series identifiers shown):

​
Offsets: querying past time ranges

The offset modifier shifts the time window of a selector into the past. It is applied after the selector (and after any range vector). Query 5 minutes of data starting 30 minutes ago:
This returns the 5-minute range of samples from 30 minutes ago for all matching series.

​
Point-in-time queries with the @ modifier

Use the @ modifier to evaluate a selector at a specific instant (Unix timestamp). This is useful for fetching the value at an exact time in the past. Evaluate the series for mountpoint / at Unix timestamp 1620000000:
Tip: Use an online Unix epoch converter, such as https://www.epochconverter.com/, to translate a human-readable date/time into the correct Unix timestamp.

​
Quick reference: label matchers and examples

​
Notes about dashboards and PromQL

When using dashboards like Grafana, you usually don’t need to manually calculate Unix epoch timestamps or offset windows—Grafana translates the dashboard time range into appropriate PromQL expressions. When querying directly in Prometheus’s expression browser or via the Prometheus HTTP API, you’ll need to use offset or the @ modifier yourself to request historical data.

Watch Video

Practice Lab