> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # Demo OTel Col Processors > Configuring OpenTelemetry Collector processors, especially the batch processor, to batch, filter, and modify telemetry to reduce exporter load and network chattiness while balancing latency and throughput. This guide shows how to configure a processor in the OpenTelemetry Collector to change how telemetry is exported. We'll focus on the batch processor to reduce exporter load and network chattiness by grouping items and flushing them on a timeout or when a batch size threshold is reached. Why this matters * By default, each telemetry signal (traces, metrics, logs) is forwarded to exporters immediately when received. * High-volume telemetry can create excessive network traffic or overwhelm an exporter. * The Collector can batch, filter, modify, or sample telemetry between receivers and exporters using processors. Processors operate between receivers and exporters. They can modify, filter, sample, or batch telemetry before it is sent to exporters. What processors do * Receive data from a receiver. * Modify, enrich, filter, sample, aggregate, or batch that data. * Forward the processed data to configured exporter(s). Example: current (simple) service pipelines configuration ```yaml theme={null} service: pipelines: traces: receivers: [otlp] exporters: [debug, otlp/jaeger] metrics: receivers: [otlp] exporters: [debug] logs: receivers: [otlp] exporters: [debug] ``` Common processors and when to use them Below are some processors the Collector supports. Multiple processors can be applied to a single pipeline and will execute in series (processor 1 -> processor 2 -> ...). | Processor | Use case | Example / notes | | ------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------ | | `attributes` | Enrich, remove, or mask attributes on telemetry | See the `attributes` example below for `insert`, `update`, `delete`, `hash` actions. | | `batch` | Buffer telemetry and flush on timeout or batch size | Useful to reduce exporter load and network chatter. | | `filter` | Keep or drop telemetry based on expressions | Fine-grained filtering for traces, metrics, logs. | Processors run in the order listed in a pipeline; the output of one processor becomes the input of the next. Attributes processor (example actions) ```yaml theme={null} processors: attributes: actions: - action: insert key: environment value: production - action: insert key: db.statement value: '{query}' - action: delete key: email - action: hash key: ssn ``` Filter processor (example usage) ```yaml theme={null} processors: filter: error_mode: ignore traces: span: - 'attributes["container.name"] == "app_container_1"' - 'resource.attributes["host.name"] == "localhost"' - 'name == "app_3"' span_event: - 'attributes["grpc"] == true' - 'ISMatch(name, .*grpc.*)' metrics: metric: - 'name == "my_metric" and resource.attributes["my_label"] == "abc123"' - 'type == METRIC_DATA_TYPE_HISTOGRAM' datapoint: - 'metric.type == METRIC_DATA_TYPE_SUMMARY' - 'resource.attributes["service.name"] == "my_service_name"' logs: log_record: - 'ISMatch(body, *password.*)' - 'severity_number >= SEVERITY_NUMBER_WARN' ``` Batch processor: behavior and configuration The batch processor buffers telemetry and will flush the buffer when either: * A timeout elapses (e.g., 15s), or * A configured `send_batch_size` is reached (e.g., 512 items). This protects exporters from bursts and reduces chattiness, but it can introduce buffering latency. Minimal Receiver + Batch configuration ```yaml theme={null} receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: timeout: 15s exporters: # exporters... ``` Add a batch size threshold ```yaml theme={null} processors: batch: timeout: 15s send_batch_size: 512 ``` Explanation * `timeout: 15s` — flush at most every 15 seconds (since first item in the batch). * `send_batch_size: 512` — flush sooner if the batch reaches 512 items. * The buffer flushes when either condition is met (whichever happens first). Important: batching increases latency for individual telemetry items; tune `timeout` and `send_batch_size` to balance latency and throughput. Batches reduce exporter load but introduce buffering delay. If you need near-real-time telemetry, set lower timeouts or avoid batching for that pipeline. Enabling processors in service pipelines After defining processors under `processors:`, reference them in the `service.pipelines` section where you want them applied. Processors listed for a pipeline execute sequentially. Example: enabling `batch` for traces ```yaml theme={null} service: pipelines: traces: receivers: [otlp] processors: [batch] exporters: [debug, otlp/jaeger] metrics: receivers: [otlp] exporters: [debug] logs: receivers: [otlp] exporters: [debug] ``` Multiple processors * When multiple processors are specified, they run in series: receiver -> processor A -> processor B -> exporter(s) Generate telemetry to compare behaviors First, observe behavior without the batch processor (immediate export). Using telemetrygen to generate traces and logs: ```bash theme={null} clear $GOBIN/telemetrygen traces --otlp-http --otlp-insecure --traces 3 $GOBIN/telemetrygen logs --otlp-http --otlp-insecure --logs 3 ``` Sample output you might see when exporters are writing immediately (debug exporter prints to console): ```text theme={null} jaeger {"level":"info","ts":1759188297.295914,"caller":"grpc@v1.75.0/balancer_wrapper.go:122","msg":"[core] [Channel #1] Channel switches to new LB policy \"pick_first\""} jaeger {"level":"info","ts":1759188297.296026,"caller":"gracefulSwitch/gracefulswitch.go:194","msg":"[pick-first-leaf-lb] Received new config {...}"} jaeger [Channel #1] SubChannel #9 SubChannel created jaeger [Channel #1] SubChannel #9 Connectivity change to CONNECTING jaeger [Channel #1] SubChannel #9 picks a new address "127.0.0.1:4317" to connect jaeger [Channel #1] SubChannel #9 Connectivity change to READY jaeger {"level":"info","ts":1759188297.296145,"caller":"grpc@v1.75.0/clientconn.go:563","msg":"[core] [Channel #1] Channel Connectivity change to READY"} ``` With no batch processor enabled, traces and logs are forwarded and printed immediately. Enable the batch processor Example configuration that enables batching for traces and sends to Jaeger: ```yaml theme={null} processors: batch: timeout: 15s send_batch_size: 512 exporters: otlp/jaeger: endpoint: http://jaeger:4317 tls: insecure: true ``` Behavior with batch enabled * If you generate three traces and the batch `timeout` is 15s, they will be buffered and will not reach exporters (or appear in the debug console) until: * 15 seconds have elapsed since the first item in the batch, or * 512 items have been collected. * Collector logs will show the receiver starting and the service becoming ready; after the timeout elapses the buffer is flushed and you’ll see debug exporter output. Collector log sample (startup and flush) ```text theme={null} otel-collector | 2025-09-29T23:33:17.065Z info otlpreceiver@v0.135.0/otlp.go:179 Starting HTTP server {"resource": {"service.instance.id": "4bcade82-9fb4-4fb9-9361-9b14efe2a42c", "service.name": "otelcol-contrib", "service.version": "0.135.0", "otelcol.component.id": "otlp", "otelcol.component.kind": "receiver"}} otel-collector | 2025-09-29T23:33:17.065Z info service@v0.135.0/service.go:234 Everything is ready. {"resource":{"service.instance.id":"4bcade82-9fb4-4fb9-9361-9b14efe2a42c","service.name":"otelcol-contrib","service.version":"0.135.0"}} ... # After timeout (batch flushed) you'll see the buffered spans/logs printed by the debug exporter ``` Summary * Processors let you inspect, modify, filter, sample, or batch telemetry between receivers and exporters. * The batch processor reduces exporter load and network chatter by grouping telemetry and flushing on a timeout or size threshold. * Add processors to specific pipelines to apply them per-signal; multiple processors run sequentially. * Tune `timeout` and `send_batch_size` to balance latency and throughput for your environment. Links and references * [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) * [OTLP protocol specification](https://opentelemetry.io/docs/reference/specification/protocol/otlp/) * [Jaeger Tracing](https://www.jaegertracing.io/) * telemetrygen (generate telemetry): [https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/telemetrygen)