Skip to main content
In this lesson, we explain span sampling in OpenTelemetry: what it is, why it matters, where it applies, common samplers, and how to configure head-based sampling. Sampling controls which spans are recorded and exported so you can retain meaningful telemetry while reducing cost, load, and noise. A span is the basic unit of a trace. Sampling is the mechanism that decides whether a span (and usually its associated trace) is recorded or dropped. Effective sampling improves signal-to-noise, lowers network and storage costs, and keeps backend systems performant.
The image is an overview of sampling techniques with three sections that focus on span decisions, controlling data volume, and span creation time, each represented with icons and gradient colors.
Why sampling timing matters
  • Sampling decisions are normally made at span creation time (head-based sampling). Making decisions at the source reduces the amount of telemetry transmitted and processed.
  • The sampling decision affects child spans and downstream services—propagating a consistent decision prevents partial traces from being stored.
  • Delaying decisions (tail-based sampling) allows using whole-trace attributes to make smarter sampling choices but requires collecting more data up-front and is typically implemented in the Collector.
The image emphasizes the importance of sampling in large systems, highlighting issues like backend overload, increased network and storage costs, and added latency or CPU usage.
Where sampling applies
  • The sampler evaluates the current span at creation time and decides whether it should be recorded.
  • Sampling decisions commonly propagate to child spans: if a parent is not sampled, child spans are typically dropped to keep trace consistency.
  • Trace flags (for example, the sampled bit in the trace context) carry the sampling choice across process boundaries so the distributed system respects the decision end-to-end.
The image illustrates where sampling applies, featuring three categories: Current Span, Child Spans, and TraceFlags, each represented by a colored icon.
Built-in samplers (OpenTelemetry SDKs) The following samplers are commonly available across OpenTelemetry SDKs. Use the option that best balances visibility and cost for your environment.
SamplerPurpose / Use CaseExample
AlwaysOnSamplerRecords every span. Best for development, debugging, or short-lived environments where full visibility is required.AlwaysOnSampler()
AlwaysOffSamplerRecords no spans. Useful to keep instrumentation hooks enabled while disabling telemetry.AlwaysOffSampler()
TraceIdRatioBasedSamplerSamples a fixed fraction of traces (head-based). Good for scalable production sampling (e.g., 10% or 20%).TraceIdRatioBased(0.2)
ParentBasedSamplerAdopts the parent’s sampling decision if present; otherwise falls back to a configured root sampler. Useful for distributed systems to maintain consistent downstream behavior.ParentBased(root=TraceIdRatioBased(0.2))
The image lists built-in sampling strategies including AlwaysOnSampler, AlwaysOffSampler, TraceIdRatioBasedSampler, and ParentBasedSampler, with descriptions and use cases for each.
Configuring head-based sampling (SDK and environment) Python SDK examples:
  • Configure TraceIdRatioBased sampler at the SDK level for 20% sampling:
  • Combine ParentBased with TraceIdRatioBased as a fallback (parent decision preferred):
Agent / auto-instrumentation environment variables For auto-instrumentation or agent-based setups you can change sampling without modifying application code. Example to set trace-id-ratio sampling to 20%:
Head-based sampling makes the decision at span creation time (in the SDK). Tail-based sampling can make decisions later based on whole-trace information; tail-based sampling is typically implemented in the OpenTelemetry Collector.
Avoid AlwaysOn in high-volume production unless you have capacity for the inbound throughput, storage, and query costs. Use rate-limited or ratio-based sampling for scalable production environments.
Recap — best practices and recommendations
  • Use AlwaysOn for complete visibility during development and troubleshooting.
  • Use AlwaysOff to disable tracing while keeping instrumentation code intact.
  • Use TraceIdRatioBased or ParentBased with a ratio fallback for production to reduce telemetry volume while preserving representative traces.
  • Prefer making sampling decisions as close to the source as possible to minimize unnecessary network and backend load. If you need whole-trace signals (for anomaly detection or root-cause across entire traces), consider tail-based sampling implemented in the Collector.
The image contains key takeaways in a colorful sidebar format, discussing observability, SDK configuration, and visibility in production.
Tail-based sampling Tail-based sampling uses whole-trace attributes (durations, error rates, aggregated metrics) to decide which traces to keep. It typically runs in the OpenTelemetry Collector or a dedicated backend to avoid making suboptimal head-based decisions when trace-wide context is required. Tail-based approaches incur higher upfront collection cost but can yield more targeted sampling decisions for complex production use cases. Links and references

Watch Video