OTel Col Load Balancing Exporter

This article explains the OpenTelemetry Collector load-balancing exporter: what it does, common use cases, routing key options, resolver types, and practical configuration patterns for reliable, sticky routing of telemetry to downstream collectors. What the load-balancing exporter does

Distributes telemetry (traces, metrics, logs) across a pool of downstream collectors (backends).
Ensures related telemetry stays “sticky” to the same backend using a routing key so downstream processors (e.g., tail-based samplers) can operate on complete data.
Creates an OTLP sub-exporter per backend so each backend gets independent queuing, retries, and resilience behavior.

Common uses

Tail-based sampling: keep all spans of a trace together so downstream samplers can decide with full trace context.
Sharding hot services: route high-volume services to a dedicated collector group.
Metric scaling: shard metric streams across backends to balance ingestion.

Routing keys overview Choose a routing key that preserves the consistency required by your downstream processing. The following table summarizes the most common routing key choices and when to use them.

Routing key	Applies to	Purpose / When to use
`traceID`	Traces	Default for traces. Keeps all spans of a trace on the same backend — essential for tail-based sampling.
`service`	Traces & Metrics (default for metrics)	Routes by `service.name`. Useful for grouping telemetry at the service level.
`resource`	Traces & Metrics	Routes by resource attributes (e.g., host, deployment). Use to group telemetry by resource identity.
`metric`	Metrics only	Routes by metric name. Good for coarse metric sharding.
`streamID`	Metrics only	Routes by metric stream ID (hash of attributes). Most granular for metrics.
`attributes`	Traces & Metrics	Custom attribute routing. Requires a `routing_attributes` list to define keys to hash on.

Choose the routing key that preserves the data consistency you need. For tail-based sampling, prefer traceID. For metric sharding, prefer metric or streamID.

Example: routing_key variants

# Example routing_key choices (pick one)
# 1) traceID - traces (default for traces)
exporters:
  loadbalancing:
    routing_key: "traceID"   # Keep all spans of a trace together

# 2) service - traces/metrics (default for metrics)
exporters:
  loadbalancing:
    routing_key: "service"   # Group by service.name

# 3) resource - traces/metrics
exporters:
  loadbalancing:
    routing_key: "resource"  # Group by resource attributes

# 4) metric - metrics only
exporters:
  loadbalancing:
    routing_key: "metric"    # Group by metric name

# 5) streamID - metrics only (most granular)
exporters:
  loadbalancing:
    routing_key: "streamID"

# 6) attributes - custom attribute routing
exporters:
  loadbalancing:
    routing_key: "attributes"
    routing_attributes:
      - "span.kind"
      - "http.method"

Resolvers — how the exporter discovers backend collectors Resolvers determine how the load-balancing exporter discovers its backend pool. Select a resolver that matches your infrastructure and operational model.

Resolver	Typical use case	Notes
`static`	Small or fixed deployments	Use when backend hostnames are known and stable.
`dns`	Cloud load-balancers or DNS-based discovery	Resolve a DNS name to multiple IPs; the exporter can poll DNS at a configured interval.
`k8s`	Kubernetes headless services	Discover pods from a headless K8s service by name and port(s).
Other cloud-specific resolvers	Vendor/cloud integrations	Some Collector builds include additional resolvers (check your distribution docs).

Example: resolver types

# Static resolver - fixed backends
exporters:
  loadbalancing:
    routing_key: "traceID"
  resolver:
    static:
      hostnames:
        - backend-1.example.com:4317
        - backend-2.example.com:4317
        - backend-3.example.com:4317

# DNS resolver - dynamic discovery
exporters:
  loadbalancing:
    routing_key: "service"
  resolver:
    dns:
      hostname: collectors.svc.cluster.local
      port: 4317
      interval: 5s   # poll interval for DNS updates
      timeout: 1s

# Kubernetes resolver - K8s service discovery
exporters:
  loadbalancing:
    routing_key: "traceID"
  resolver:
    k8s:
      service: otel-collector-headless
      ports:
        - 4317
        - 4318

How the exporter maintains consistency and reliability

Per-backend sub-exporters: the load-balancing exporter instantiates an OTLP sub-exporter per backend so each backend maintains independent queues and retry behavior.
Consistent mapping: the exporter hashes the selected routing key to consistently route items with the same key to the same backend.
Resilience: combined with exporter-level queues and retry_on_failure settings, this delivers robust forwarding even during transient backend issues.

The image presents information about a "Load-Balancing Exporter" and explains the importance of consistent routing for distributing traffic and ensuring data consistency across backends.

The load-balancing exporter guarantees stickiness using the selected routing key (for example, trace ID or service name). Choose a resolver type that matches your environment to discover the backend pool either statically or dynamically.

The image describes how a load-balancing exporter achieves consistent routing, using a routing key, supporting various resolvers, creating sub-exporters per backend, and maintaining consistent routing.

Tail-based sampling with a load-balancing exporter Tail-based sampling often runs on downstream collectors. If the load-balancing exporter keeps all spans of a trace on the same backend, downstream tail samplers can see the entire trace without cross-node coordination, enabling accurate sampling decisions.

The image is a slide titled "Load-Balancing Exporter: Tail Sampling Setup with Load Balancing," showing a typical setup involving tail-based sampling on downstream collectors and routing all spans for a trace to the same backend.

Illustration of consistent routing Within the load-balancing collector, spans (or metric streams) that share the same routing key are hashed or consistently mapped to the same backend so each backend receives all data for that key.

The image illustrates consistent routing using LB Exporter, showing a circular node diagram directing packets with different resolver badges to various server collectors.

Typical exporter configuration (production-minded) The example below shows a production-oriented configuration using OTLP transport, a static resolver, exporter-level queuing, and retry settings. Adjust routing_key, resolver type, and timeouts to fit your deployment.

exporters:
  loadbalancing:
    routing_key: "traceID"   # pick routing key appropriate for your use case

    protocol:
      otlp:
        timeout: 1s
        tls:
          insecure: false

    resolver:
      static:
        hostnames:
          - backend-1.example.com:4317
          - backend-2.example.com:4317
          - backend-3.example.com:4317

# Exporter-level resilience
sending_queue:
  enabled: true
  queue_size: 20000

retry_on_failure:
  enabled: true
  max_elapsed_time: 5m

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [loadbalancing]

Notes and caveats

metric and streamID routing keys are valid only for metrics; they are not applicable to traces or logs.
Select static for fixed backend pools, dns for DNS-based discovery in cloud deployments, and k8s for Kubernetes headless services.
Downstream backends must be able to handle the grouped load and any tail-sampling processors you plan to run.
Monitor and tune per-backend queue sizes, retry settings, and OTLP timeouts to avoid dropped telemetry during spikes.

Links and references

Course Introduction

Observability Core Concepts

OpenTelemetry Core Concepts

Span Anatomy and Context Propagation

Instrumentation

Metrics Data Model

Recording Measurements

OTel Collector Foundations

OTel Collector Core Components

Transforming Telemetry – Pipeline Processing

OpenTelemetry in Kubernetes

OpenTelemetry Collector: Debugging, Operations, and Scaling

Extended Topics

OTel Col Load Balancing Exporter

Watch Video