Demo OpenTelemetry Collector Configuration

This guide demonstrates how to configure an OpenTelemetry Collector using a single YAML file. It covers the top-level sections, a minimal working example, how to validate the configuration, running the Collector (Docker example), generating test telemetry with telemetrygen, and suggestions for extending the configuration for production. Core Collector configuration sections (in order):

receivers: how the Collector accepts telemetry (from applications, agents, or other collectors)
processors: optional transformers, batching, filtering, sampling, etc.
exporters: destinations for processed telemetry (backends, files, console)
service.pipelines: wiring that connects receivers → (processors) → exporters for each signal (traces, metrics, logs)

Summary table of the top-level sections:

Section	Purpose	Example snippet
`receivers`	Defines how telemetry is ingested (OTLP, Jaeger, Prometheus, Fluent Forward, etc.)	`yaml\nreceivers:\n otlp:\n protocols:\n grpc: {}\n`
`processors`	Optional processing (batching, sampling, resource enrichment)	`yaml\nprocessors:\n batch: {}\n`
`exporters`	Where telemetry is sent (OTLP, Prometheus, Jaeger, logging/debug)	`yaml\nexporters:\n otlp:\n endpoint: example:4317\n`
`service.pipelines`	Connects receivers → processors → exporters per signal	`yaml\nservice:\n pipelines:\n traces: { receivers: [otlp], exporters: [debug] }\n`

Minimal configuration skeleton

A minimal skeleton to remind the top-level layout:

receivers:

processors:

exporters:

service:
  pipelines:
    traces: {}
    metrics: {}
    logs: {}

Validate a Collector config with the built-in validator:

otelcol validate --config=customconfig.yaml

Note: Use the correct binary for your distribution (for example otelcol or otelcol-contrib).

If validation fails, carefully check YAML indentation and keys—most issues are typos or mis-indentation. You can also run otelcol --config customconfig.yaml --dry-run with some builds to surface runtime validation errors.

Receivers

Receivers determine how the Collector accepts telemetry. The Collector supports many receiver types (examples: Fluent Forward, Prometheus, Jaeger, Kafka, OpenCensus, OTLP, Zipkin). Below are example snippets for several common receivers. Example receiver snippets:

receivers:
  # Fluent Forward (logs)
  fluentforward:
    endpoint: 0.0.0.0:8006

  # Host-level metrics (Linux)
  hostmetrics:
    scrapers:
      cpu:
      disk:
      filesystem:
      load:
      memory:
      network:
      process:
      processes:
        paging:

  # Jaeger traces
  jaeger:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      thrift_binary:
      thrift_compact:
      thrift_http:

  # Generic Kafka receiver
  kafka:
    protocol_version: 2.0.0

OTLP receiver (recommended for examples & local testing — supports gRPC and HTTP):

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

Notes:

0.0.0.0 binds to all interfaces. For local-only testing, use 127.0.0.1 or a specific interface.
Add TLS (cert/key) or authentication under each protocol if required.

Processors

Processors run between receivers and exporters and can transform, filter, aggregate, or limit telemetry. Common processors: batch, memory_limiter, attributes, resource, probabilistic_sampler. Processors are optional—omit them for minimal configs. Example placeholder:

# processors:
#   batch:
#   attributes:
#   resource:

Exporters

Exporters send processed telemetry to destinations. Typical exporters include OTLP, Prometheus, Jaeger, Zipkin, Kafka, file, cloud vendor backends, and a debug exporter that prints telemetry to stdout. Example exporter snippets:

exporters:
  # Send to another OTLP endpoint (e.g., vendor ingest)
  otlp:
    endpoint: otel-collector-upstream:4317
    tls:
      cert_file: cert.pem
      key_file: cert-key.pem

  # Prometheus exporter (exposes a scrape endpoint)
  prometheus:
    endpoint: 0.0.0.0:8889
    namespace: default

  # Debug exporter: prints telemetry to stdout (useful for testing)
  debug:
    verbosity: detailed

Note: Historically some distributions used logging instead of debug—check the Collector version and distribution.

Wiring it together: service.pipelines

Pipelines connect receivers to processors and exporters. You must define a pipeline for each signal you want to process (traces, metrics, logs). Each pipeline lists receivers, optionally processors, and exporters. Minimal complete config (receives OTLP gRPC + HTTP; exports all signals to debug):

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors: {}

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
    metrics:
      receivers: [otlp]
      exporters: [debug]
    logs:
      receivers: [otlp]
      exporters: [debug]

With this configuration the Collector accepts traces, metrics, and logs via OTLP and prints them to the console for inspection.

Running the Collector (Docker example)

You can run the Collector as a binary, Docker image, Docker Compose service, or inside Kubernetes. Example using the contrib Docker image (includes many receivers/exporters):

docker pull otel/opentelemetry-collector-contrib:0.136.0

# Run the Collector interactively (example only):
docker run \
  -p 127.0.0.1:4317:4317 \
  -p 127.0.0.1:4318:4318 \
  -p 127.0.0.1:55679:55679 \
  -v "$(pwd)/otel-collector-config.yaml":/conf/otel-collector-config.yaml:ro \
  otel/opentelemetry-collector-contrib:0.136.0 \
  --config /conf/otel-collector-config.yaml 2>&1 | tee collector-output.txt

When OTLP is configured you should see logs indicating the gRPC and HTTP OTLP servers started and that the service is ready. Example (trimmed):

otel-collector | 2025-09-29T23:04:46.098Z info otlpreceiver@v0.135.0/otlp.go:121 Starting GRPC server
otel-collector | 2025-09-29T23:04:46.098Z info otlpreceiver@v0.135.0/otlp.go:179 Starting HTTP server
otel-collector | 2025-09-29T23:04:46.098Z info service@v0.135.0/service.go:234 Everything is ready. Begin processing data. {"resource":{"service.instance.id":"...","service.name":"otelcol-contrib","service.version":"0.135.0"}}

Be mindful of which image you use: otel/opentelemetry-collector (core) includes fewer components than otel/opentelemetry-collector-contrib. Choose the image that contains the receivers/exporters you need. Binding ports to 127.0.0.1 restricts access to localhost; remove 127.0.0.1: to expose to all interfaces.

Generating test telemetry with telemetrygen

Use telemetrygen to generate test traces, metrics, and logs to validate the Collector without instrumenting an application. Install telemetrygen (Go required):

# Optionally set GOBIN so go installs to a known dir
export GOBIN=$(go env GOPATH)/bin

# Install the telemetrygen CLI
go install github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen@latest

Generate traces (OTLP via HTTP, insecure—suitable for local testing):

$GOBIN/telemetrygen traces --otlp-http --otlp-insecure --traces 3

Expected telemetrygen output (informational):

2025-09-29T17:09:51.954-0600  INFO  traces/traces.go:40    starting HTTP exporter
2025-09-29T17:09:51.956-0600  INFO  traces/traces.go:118   generation of traces is limited {'per-second': 1}

Collector debug exporter will print received spans like:

otel-collector  | Span #1
otel-collector  | Trace ID     : 79b8fc5226d6465d3237317511b05a98
otel-collector  | Parent ID    : bac24f1a582a5e8c
otel-collector  | ID           : lets-go
otel-collector  | Kind         : Client
otel-collector  | Start time   : 2025-09-29 23:09:51.95737 +0000 UTC
otel-collector  | End time     : 2025-09-29 23:09:51.957493 +0000 UTC
otel-collector  | Attributes:
otel-collector  |    -> network.peer.address: Str(1.2.3.4)
otel-collector  |    -> peer.service: Str(telemetrygen-server)
otel-collector  | 2025-09-29T23:09:51.957493Z info  ResourceSpans #0
otel-collector  | Resource SchemaURL: https://opentelemetry.io/schemas/1.37.0
otel-collector  | Resource attributes:
otel-collector  |    -> service.name: Str(telemetrygen)

Generate logs:

$GOBIN/telemetrygen logs --otlp-http --otlp-insecure --logs 3

Collector debug output for logs:

otel-collector | 2025-09-29T23:11:22.712Z info Logs {"resource": {"service.name": "otelcol-contrib"}, "otelcol.component.id": "debug", "otelcol.signal": "logs", "resource logs": 1, "log records": 1}
otel-collector | LogRecord #0
otel-collector | Timestamp: 2025-09-29T23:11:21.730219Z
otel-collector | SeverityText: Info
otel-collector | Body: Str(the message)
otel-collector | Attributes:
otel-collector |   -> app: Str(server)

If you see the spans and log records printed, the pipelines are functioning and the Collector is receiving and exporting telemetry correctly.

Common receivers and exporters (quick reference)

Category	Examples	When to use
Receivers	`otlp`, `prometheus`, `jaeger`, `fluentforward`, `kafka`	Use based on the telemetry source (instrumentation, agent, or external pipeline)
Exporters	`otlp`, `prometheus`, `jaeger`, `zipkin`, `kafka`, `debug`	Send telemetry to vendor ingest, monitoring systems, or stdout for testing

Extending this configuration

Replace the debug exporter with a production backend exporter (e.g., OTLP to vendor ingest, Prometheus remote write, Jaeger, Zipkin, Kafka).
Add processors like batch, memory_limiter, resource, or attributes to shape telemetry.
Add multiple receivers and create pipelines that route different signals to different exporters.
For production, enable TLS/authentication for receivers and exporters, and tune processor parameters (e.g., batch sizes, sampling rate).

Final full example (copy/paste)

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors: {}

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
    metrics:
      receivers: [otlp]
      exporters: [debug]
    logs:
      receivers: [otlp]
      exporters: [debug]

Links and references

This concludes the basic OpenTelemetry Collector configuration guide: receivers, processors, exporters, wiring, validation, and local testing using the debug exporter and telemetrygen.

Course Introduction

Observability Core Concepts

OpenTelemetry Core Concepts

Span Anatomy and Context Propagation

Instrumentation

Metrics Data Model

Recording Measurements

OTel Collector Foundations

OTel Collector Core Components

Transforming Telemetry – Pipeline Processing

OpenTelemetry in Kubernetes

OpenTelemetry Collector: Debugging, Operations, and Scaling

Extended Topics

Demo OpenTelemetry Collector Configuration

Minimal configuration skeleton

Receivers

Processors

Exporters

Wiring it together: service.pipelines

Running the Collector (Docker example)

Generating test telemetry with telemetrygen

Common receivers and exporters (quick reference)

Extending this configuration

Final full example (copy/paste)

Links and references

Watch Video

Practice Lab

​Minimal configuration skeleton

​Receivers

​Processors

​Exporters

​Wiring it together: service.pipelines

​Running the Collector (Docker example)

​Generating test telemetry with telemetrygen

​Common receivers and exporters (quick reference)

​Extending this configuration

​Final full example (copy/paste)

​Links and references

Watch Video

Practice Lab

Minimal configuration skeleton

Receivers

Processors

Exporters

Wiring it together: service.pipelines

Running the Collector (Docker example)

Generating test telemetry with telemetrygen

Common receivers and exporters (quick reference)

Extending this configuration

Final full example (copy/paste)

Links and references