> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # Demo OpenTelemetry Collector Configuration > Guide to configuring an OpenTelemetry Collector with a single YAML file, covering receivers, processors, exporters, pipelines, validation, Docker usage, telemetrygen testing, and production extension tips. This guide demonstrates how to configure an OpenTelemetry Collector using a single YAML file. It covers the top-level sections, a minimal working example, how to validate the configuration, running the Collector (Docker example), generating test telemetry with telemetrygen, and suggestions for extending the configuration for production. Core Collector configuration sections (in order): * `receivers`: how the Collector accepts telemetry (from applications, agents, or other collectors) * `processors`: optional transformers, batching, filtering, sampling, etc. * `exporters`: destinations for processed telemetry (backends, files, console) * `service.pipelines`: wiring that connects `receivers` → `(processors)` → `exporters` for each signal (`traces`, `metrics`, `logs`) Summary table of the top-level sections: | Section | Purpose | Example snippet | | ------------------- | ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | `receivers` | Defines how telemetry is ingested (OTLP, Jaeger, Prometheus, Fluent Forward, etc.) | `yaml\nreceivers:\n otlp:\n protocols:\n grpc: {}\n` | | `processors` | Optional processing (batching, sampling, resource enrichment) | `yaml\nprocessors:\n batch: {}\n` | | `exporters` | Where telemetry is sent (OTLP, Prometheus, Jaeger, logging/debug) | `yaml\nexporters:\n otlp:\n endpoint: example:4317\n` | | `service.pipelines` | Connects receivers → processors → exporters per signal | `yaml\nservice:\n pipelines:\n traces: { receivers: [otlp], exporters: [debug] }\n` | ## Minimal configuration skeleton A minimal skeleton to remind the top-level layout: ```yaml theme={null} receivers: processors: exporters: service: pipelines: traces: {} metrics: {} logs: {} ``` Validate a Collector config with the built-in validator: ```bash theme={null} otelcol validate --config=customconfig.yaml ``` Note: Use the correct binary for your distribution (for example `otelcol` or `otelcol-contrib`). If validation fails, carefully check YAML indentation and keys—most issues are typos or mis-indentation. You can also run `otelcol --config customconfig.yaml --dry-run` with some builds to surface runtime validation errors. ## Receivers Receivers determine how the Collector accepts telemetry. The Collector supports many receiver types (examples: Fluent Forward, Prometheus, Jaeger, Kafka, OpenCensus, OTLP, Zipkin). Below are example snippets for several common receivers. Example receiver snippets: ```yaml theme={null} receivers: # Fluent Forward (logs) fluentforward: endpoint: 0.0.0.0:8006 # Host-level metrics (Linux) hostmetrics: scrapers: cpu: disk: filesystem: load: memory: network: process: processes: paging: # Jaeger traces jaeger: protocols: grpc: endpoint: 0.0.0.0:4317 thrift_binary: thrift_compact: thrift_http: # Generic Kafka receiver kafka: protocol_version: 2.0.0 ``` OTLP receiver (recommended for examples & local testing — supports gRPC and HTTP): ```yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 ``` Notes: * `0.0.0.0` binds to all interfaces. For local-only testing, use `127.0.0.1` or a specific interface. * Add TLS (cert/key) or authentication under each protocol if required. ## Processors Processors run between receivers and exporters and can transform, filter, aggregate, or limit telemetry. Common processors: `batch`, `memory_limiter`, `attributes`, `resource`, `probabilistic_sampler`. Processors are optional—omit them for minimal configs. Example placeholder: ```yaml theme={null} # processors: # batch: # attributes: # resource: ``` ## Exporters Exporters send processed telemetry to destinations. Typical exporters include OTLP, Prometheus, Jaeger, Zipkin, Kafka, file, cloud vendor backends, and a debug exporter that prints telemetry to stdout. Example exporter snippets: ```yaml theme={null} exporters: # Send to another OTLP endpoint (e.g., vendor ingest) otlp: endpoint: otel-collector-upstream:4317 tls: cert_file: cert.pem key_file: cert-key.pem # Prometheus exporter (exposes a scrape endpoint) prometheus: endpoint: 0.0.0.0:8889 namespace: default # Debug exporter: prints telemetry to stdout (useful for testing) debug: verbosity: detailed ``` Note: Historically some distributions used `logging` instead of `debug`—check the Collector version and distribution. ## Wiring it together: service.pipelines Pipelines connect `receivers` to `processors` and `exporters`. You must define a pipeline for each signal you want to process (`traces`, `metrics`, `logs`). Each pipeline lists `receivers`, optionally `processors`, and `exporters`. Minimal complete config (receives OTLP gRPC + HTTP; exports all signals to `debug`): ```yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: {} exporters: debug: verbosity: detailed service: pipelines: traces: receivers: [otlp] exporters: [debug] metrics: receivers: [otlp] exporters: [debug] logs: receivers: [otlp] exporters: [debug] ``` With this configuration the Collector accepts traces, metrics, and logs via OTLP and prints them to the console for inspection. ## Running the Collector (Docker example) You can run the Collector as a binary, Docker image, Docker Compose service, or inside Kubernetes. Example using the contrib Docker image (includes many receivers/exporters): ```bash theme={null} docker pull otel/opentelemetry-collector-contrib:0.136.0 # Run the Collector interactively (example only): docker run \ -p 127.0.0.1:4317:4317 \ -p 127.0.0.1:4318:4318 \ -p 127.0.0.1:55679:55679 \ -v "$(pwd)/otel-collector-config.yaml":/conf/otel-collector-config.yaml:ro \ otel/opentelemetry-collector-contrib:0.136.0 \ --config /conf/otel-collector-config.yaml 2>&1 | tee collector-output.txt ``` When OTLP is configured you should see logs indicating the gRPC and HTTP OTLP servers started and that the service is ready. Example (trimmed): ```plaintext theme={null} otel-collector | 2025-09-29T23:04:46.098Z info otlpreceiver@v0.135.0/otlp.go:121 Starting GRPC server otel-collector | 2025-09-29T23:04:46.098Z info otlpreceiver@v0.135.0/otlp.go:179 Starting HTTP server otel-collector | 2025-09-29T23:04:46.098Z info service@v0.135.0/service.go:234 Everything is ready. Begin processing data. {"resource":{"service.instance.id":"...","service.name":"otelcol-contrib","service.version":"0.135.0"}} ``` Be mindful of which image you use: `otel/opentelemetry-collector` (core) includes fewer components than `otel/opentelemetry-collector-contrib`. Choose the image that contains the receivers/exporters you need. Binding ports to `127.0.0.1` restricts access to localhost; remove `127.0.0.1:` to expose to all interfaces. ## Generating test telemetry with telemetrygen Use telemetrygen to generate test traces, metrics, and logs to validate the Collector without instrumenting an application. Install telemetrygen (Go required): ```bash theme={null} # Optionally set GOBIN so go installs to a known dir export GOBIN=$(go env GOPATH)/bin # Install the telemetrygen CLI go install github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen@latest ``` Generate traces (OTLP via HTTP, insecure—suitable for local testing): ```bash theme={null} $GOBIN/telemetrygen traces --otlp-http --otlp-insecure --traces 3 ``` Expected telemetrygen output (informational): ```plaintext theme={null} 2025-09-29T17:09:51.954-0600 INFO traces/traces.go:40 starting HTTP exporter 2025-09-29T17:09:51.956-0600 INFO traces/traces.go:118 generation of traces is limited {'per-second': 1} ``` Collector debug exporter will print received spans like: ```plaintext theme={null} otel-collector | Span #1 otel-collector | Trace ID : 79b8fc5226d6465d3237317511b05a98 otel-collector | Parent ID : bac24f1a582a5e8c otel-collector | ID : lets-go otel-collector | Kind : Client otel-collector | Start time : 2025-09-29 23:09:51.95737 +0000 UTC otel-collector | End time : 2025-09-29 23:09:51.957493 +0000 UTC otel-collector | Attributes: otel-collector | -> network.peer.address: Str(1.2.3.4) otel-collector | -> peer.service: Str(telemetrygen-server) otel-collector | 2025-09-29T23:09:51.957493Z info ResourceSpans #0 otel-collector | Resource SchemaURL: https://opentelemetry.io/schemas/1.37.0 otel-collector | Resource attributes: otel-collector | -> service.name: Str(telemetrygen) ``` Generate logs: ```bash theme={null} $GOBIN/telemetrygen logs --otlp-http --otlp-insecure --logs 3 ``` Collector debug output for logs: ```plaintext theme={null} otel-collector | 2025-09-29T23:11:22.712Z info Logs {"resource": {"service.name": "otelcol-contrib"}, "otelcol.component.id": "debug", "otelcol.signal": "logs", "resource logs": 1, "log records": 1} otel-collector | LogRecord #0 otel-collector | Timestamp: 2025-09-29T23:11:21.730219Z otel-collector | SeverityText: Info otel-collector | Body: Str(the message) otel-collector | Attributes: otel-collector | -> app: Str(server) ``` If you see the spans and log records printed, the pipelines are functioning and the Collector is receiving and exporting telemetry correctly. ## Common receivers and exporters (quick reference) | Category | Examples | When to use | | --------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------- | | Receivers | `otlp`, `prometheus`, `jaeger`, `fluentforward`, `kafka` | Use based on the telemetry source (instrumentation, agent, or external pipeline) | | Exporters | `otlp`, `prometheus`, `jaeger`, `zipkin`, `kafka`, `debug` | Send telemetry to vendor ingest, monitoring systems, or stdout for testing | ## Extending this configuration * Replace the `debug` exporter with a production backend exporter (e.g., OTLP to vendor ingest, Prometheus remote write, Jaeger, Zipkin, Kafka). * Add processors like `batch`, `memory_limiter`, `resource`, or `attributes` to shape telemetry. * Add multiple receivers and create pipelines that route different signals to different exporters. * For production, enable TLS/authentication for receivers and exporters, and tune processor parameters (e.g., `batch` sizes, sampling rate). ## Final full example (copy/paste) ```yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: {} exporters: debug: verbosity: detailed service: pipelines: traces: receivers: [otlp] exporters: [debug] metrics: receivers: [otlp] exporters: [debug] logs: receivers: [otlp] exporters: [debug] ``` ## Links and references * [OpenTelemetry Collector — official docs](https://opentelemetry.io/docs/collector/) * [OpenTelemetry Collector Contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib) * [telemetrygen (telemetry generator) — contrib command](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen) * [Kubernetes Concepts (for running Collector in K8s)](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) This concludes the basic OpenTelemetry Collector configuration guide: receivers, processors, exporters, wiring, validation, and local testing using the debug exporter and telemetrygen.