Skip to main content
This lesson demonstrates practical examples of Prometheus relabeling: relabel_configs (runs before scraping) and metric_relabel_configs (runs after scraping). The same relabel rules apply regardless of the service discovery mechanism — only the discovered labels differ. Below are example discovered labels for node targets (shortened to the labels used in the examples):
The examples focus on two key labels:
  • env — environment (dev / staging / prod)
  • team — responsible team (web / database)
Using these labels you can decide which targets to scrape and how to shape target and metric metadata.
Relabeling runs at two distinct phases:
  • relabel_configs — applied to discovered targets before scraping (decide which targets to scrape, and transform target labels).
  • metric_relabel_configs — applied after scraping, giving you access to metrics and their labels (transform metric names and metric labels, drop or keep specific metrics).

​
How relabeling flows work

Quick reminders that will help you design rules:
  • When multiple source_labels are listed, their values are concatenated with ; before regex evaluation. Capture groups map to $1, $2, etc., in that order.
  • Use anchored regexes (^...$) when you need exact matches.
  • relabel_configs shape target metadata and control which endpoints are scraped.
  • metric_relabel_configs operate on scraped metric samples (including __name__) and are the place to rename metrics, drop metrics, or transform metric labels.
A short reference of common actions:

​
Filter targets by environment (relabel_configs — before scraping)

To keep only targets where env="prod", add a relabel_configs entry to the scrape job. Here source_labels values are concatenated and matched with regex. Use action: keep to preserve matches and drop all others.
If you instead use action: drop with the same regex, targets with env="prod" will be removed and everything else retained. Tip: To keep multiple environments, use alternation in the regex like ^(prod|staging)$. Example outcome after keep for env=prod (target list contains only production targets):

​
Combine labels into a new label (replace)

To create a composed label info that joins team and env (for example database-prod), use a replace action with two source_labels. The order of source_labels determines $1, $2, etc. Because source_labels are joined with ;, the regex should expect a semicolon between captures.
  • $1 is team, $2 is env.
  • target_label: info creates the new label info with the combined value.
Resulting target labels example:

​
Drop a discovered label (labeldrop / labelkeep)

To remove an unwanted discovered label such as size from target labels, use action: labeldrop with a regex that matches the label key:
  • labeldrop accepts a regex matching label keys to remove (e.g., ^size$).
  • labelkeep is complementary: it keeps only the label keys that match and drops all others.
After adding labeldrop: ^size$, the size label will no longer be present in target labels.
Be careful: labeldrop and labelkeep operate on label keys. An overly broad regex may remove labels you intended to keep. Test rules incrementally in a non-production environment when possible.

​
Metric relabeling (after scraping)

metric_relabel_configs run after scraping and operate on scraped samples. You can rename metric names (via the special __name__ label), keep or drop entire metrics, or transform metric labels.

​
Rename a metric name

To rename the metric node_cpu_seconds_total to host_cpu_seconds_total:
  • Matching __name__ lets you change the metric name.
  • Anchoring the regex with ^ and $ ensures exact matching.
After reloading Prometheus, queries and autocompletion will show host_cpu_seconds_total in place of node_cpu_seconds_total.

​
Keep only a specific metric (drop everything else)

To retain only node_arp_entries and drop all other scraped metrics for that scrape job:
  • action: keep preserves matching metrics and drops others for the job.
  • Note: some Prometheus internal or exporter-side metrics (e.g., up, scrape-related metrics) may still appear depending on the context.

​
Rename a metric label key (e.g., mountpoint → path)

To copy the mountpoint label value into a new label called path:
This adds path with the same value as mountpoint. To remove the original mountpoint label so only path remains, add a labeldrop rule:
  • The first rule creates path.
  • The second removes the original mountpoint label from the metric.

​
Summary and best practices

  • Use relabel_configs to shape discovered target metadata, control which endpoints are scraped, and build composite labels for better grouping.
  • Use metric_relabel_configs to transform scraped metric names and labels, or to drop/keep specific metrics.
  • Always anchor regexes (^...$) for exact matches, and remember source_labels are concatenated with ; (so capture groups must account for that separator).
  • Test rules progressively (start with non-production scrape jobs) and validate results in Prometheus targets and label explorers before rolling out broad changes.

Watch Video

Practice Lab