Skip to main content
In this lesson we cover the OpenTelemetry Collector filter processor to keep only the application metrics you care about (for example, temperature and humidity) and exclude the rest. We assume you already have metrics scraped from a Python application and are ingesting them via Prometheus or OTLP. Problem: the application emits many metrics (for example python_info, process_open_fds, process_cpu_seconds_total, etc.) that you likely do not want to collect. Below is a small excerpt of scraped output showing both useful and noisy metrics:
Resource metadata and other details are emitted as JSON-style resource attributes:
Goal: remove noisy metrics and keep only the ones you care about (for example current_temperature_fahrenheit_degF and current_humidity_percentage) using the filter processor. Overview
  • Add a filter processor under processors.
  • Configure rules for metrics (either include or exclude) and choose a match_type (strict, regexp, expr).
  • Wire the processor into the service pipeline so it runs for the intended telemetry type.
  • Prefer regexp for flexible pattern matching.
Processor ordering matters: filter early in the pipeline (usually before heavy processors like batch or attributes) so unwanted telemetry is removed as soon as possible.
Common startup errors stem from typos in the configuration keys. For example, use metric_names (not metric_namees). If the Collector fails on restart, inspect logs for configuration parse errors.

​
Filter out metric names with a regular expression

Below is a minimal example that excludes metrics whose names start with python or process. Note the correct key is metric_names.
Make sure the filter processor is included in your metrics pipeline so it executes:
If you still see metrics such as scrape_samples_scraped after applying the filter, either update the regex to exclude them or switch to an include-only strategy (see the next section). After the filter is applied, expected Collector output should include only application metrics like:

​
Include-only vs Exclude — which approach to use?

  • Exclude approach: use exclude to remove undesired metrics by pattern. Useful when noise is limited to a few predictable prefixes.
  • Include approach: use include to allow only the named metrics you want. Safer when you want to strictly control which metrics are collected.
Example: include-only for current_temperature_fahrenheit_degF and current_humidity_percentage:
Summary table — include vs exclude

​
Filtering logs by severity

The filter processor can target logs as well. If your application emits logs at many severities (DEBUG, INFO, WARN, ERROR, FATAL) and you want to collect only ERROR and above, configure the logs section of the filter processor. Example log severity samples:
Filter configuration to include only ERROR+ logs and exclude noisy metrics:
Wire the filter into both logs and metrics pipelines:
After restarting the Collector and running your logging workload, only ERROR and FATAL lines should appear in Collector output; INFO/DEBUG will be filtered out.

​
Full processors example (combined)

A consolidated processors block that demonstrates metrics exclusion, logs inclusion, batching, and an attributes inserter:
Reference the processors by name in each pipeline that should use them (order matters). Example service pipelines:
Best practices
  • Place the filter processor early in pipelines to minimize downstream processing on unwanted telemetry.
  • Prefer match_type: regexp for flexible pattern-based rules.
  • Use include-only when you require strict control over which metrics are collected.
  • Double-check configuration key names to avoid parse errors (e.g., metric_names).
Links and references Summary
  • The filter processor provides powerful include/exclude rules for metrics and logs.
  • Use regex patterns for flexible matching and include-only for tight control.
  • Verify processor ordering and configuration keys to avoid startup issues.

Watch Video