Logs SDK

All right, log-record experts — this lesson covers the OpenTelemetry Logs SDK: what it is, how its pieces fit together, and common patterns for initialization and shutdown. The Logs SDK implements the Logs API and connects instrumentation calls to telemetry export pipelines across supported languages. While most behavior is stable, some SDK details remain implementation-specific and continue to evolve.

A presentation slide titled "OTEL Logs SDK – Introduction" showing a vertical flow diagram: API Call → Logs SDK → Telemetry Data Store, with two side notes reading "01 Implements Logs API" and "02 Required across all supported languages."

Key concepts overview

Logs SDK: receives log calls from instrumentation and routes log records through processors to exporters.
LoggerProvider: factory and shared configuration container for loggers.
Logger / Instrumentation Scope: library/component-specific logger obtained from the provider.
LogRecordProcessor: manages buffering, batching, filtering, or enrichment.
LogRecordExporter: sends logs to final destinations (OTLP, cloud backends, files, etc.).
Lifecycle APIs: ForceFlush and Shutdown to ensure reliable termination.

LoggerProvider (factory and configuration)

LoggerProvider is the factory that creates named loggers and stores shared configuration like Resource (service name, environment, host metadata). Multiple providers can be used to isolate configuration for different subsystems.

Embed a Resource to annotate exported logs with service and environment metadata.
Use separate providers when you need isolated pipelines or export settings per subsystem.

The image is a slide titled "LoggerProvider Fundamentals" showing a factory icon labeled "Logger Provider" with the note "Allow isolated configurations." To the right is a flowchart: "Logger Request" → "Name Valid?" branching to "Return Logger (normal)" if yes and "Return Fallback Logger + Log Warning" if no.

When code requests a logger from the provider (for example GetLogger / get_logger), the provider returns a logger tied to that instrumentation scope name. Some SDKs validate names; invalid names may produce a fallback/no-op logger and a warning — behavior varies by language.

Example: getting a logger and Provider behavior (pseudocode)

provider = LoggerProvider(resource=Resource(service="orders"))
logger = provider.get_logger("my.lib.orders")  # instrumentation scope
logger.info("Order received", order_id=123)

A diagram titled "Logger Creation and Configuration" showing a LoggerProvider (with GetLogger()) and a config-update gear feeding three loggers (Logger1, Logger2, Logger3). Each logger is linked to a green "InstrumentationScope" pill.

Logger lifetime and dynamic provider updates

Loggers are obtained from the provider rather than created directly. Because loggers reference provider configuration, runtime updates to the LoggerProvider — such as adding a new LogRecordProcessor or changing exporter settings — will typically apply to all loggers obtained from that provider (including existing loggers). This allows you to change export behavior centrally without recreating logger instances. Example pattern:

Add a BatchingProcessor or switch exporters on the provider.
Existing loggers (Logger1, Logger2, Logger3) pick up the new pipeline immediately in most SDK implementations.

Lifecycle: ForceFlush and Shutdown

The SDK exposes lifecycle APIs to control export behavior at process termination:

ForceFlush() — force any buffered log records to be exported immediately (helpful before exit).
Shutdown() — stop accepting new log records and ensure processors/exporters finish and release resources.

Both APIs typically accept a timeout/deadline and return a completion status (success, failure, or timeout). Exact method names and return types vary by language SDK.

A presentation slide titled "Lifecycle: Shutdown and ForceFlush" showing two panels: ForceFlush() with "Export pending logrecords" and Shutdown() with "Stop new loggers and clean up." Below that is an "APIs Signals" row with three labels: Success, Failure, and Timeout.

Recommended shutdown sequence (pseudocode)

# On application termination:
provider.force_flush(timeout=5)   # ensure pending logs are exported
provider.shutdown(timeout=10)     # stop logging and cleanup resources

This pattern minimizes data loss by flushing pending records before releasing exporter resources.

LogRecordProcessor types and responsibilities

LogRecordProcessors determine when and how log records move from the SDK to exporters:

SimpleProcessor — exports each record synchronously (low-latency, high overhead; useful for debugging).
BatchingProcessor — buffers records and exports in batches based on size/interval thresholds (production-friendly).
CustomProcessor — implement enrichment, filtering, routing, or any specialized logic (analogous to custom span processors).

A diagram titled "LogRecordProcessor Architecture" showing a central "LogRecord Emission" box that emits records to three processor types. The processors are Simple Processor (sends logs immediately), Batching Processor (buffers and exports in intervals), and Custom Processor (enriches, filters, or routes logs).

Table: processor trade-offs

Processor type	Use case	Characteristics
SimpleProcessor	Local development, debugging	Immediate export, synchronous, higher overhead
BatchingProcessor	Production workloads	Buffered export, lower overhead, configurable size/timeout
CustomProcessor	Filtering, enrichment, routing	Implement domain-specific logic, can combine with batching

LogRecordProcessors are conceptually similar to span processors: they control retries, batching, and filtering, and determine when exporters are invoked.

LogRecordExporter responsibilities

The LogRecordExporter sends log batches to the final destination (OTLP endpoints, cloud logging services, files, etc.). Exporters typically expose:

Export(batch) — export one batch of log records (must be thread-safe and respect timeouts).
Shutdown() — release network/file resources on termination.
A clear export result/status so processors can decide to retry or drop records.

A presentation slide titled "LogRecordExporter Interface." It shows four numbered cards listing responsibilities: implement protocol-specific log transmission; expose Export(batch) and Shutdown() methods; follow concurrency rules and timeouts; and return ExportResult (Success or Failure).

Practical exporter considerations:

Ensure Export is safe for concurrent calls from multiple processors or threads.
Respect configured timeouts and return clear success/failure statuses.
Implement Shutdown() to prevent resource leaks and to allow graceful process exit.

Core components at a glance

Component	Purpose	Notable API / Behavior
LoggerProvider	Factory + global config	`get_logger(name)`, `force_flush()`, `shutdown()`
Logger	Emits log records	Bound to instrumentation scope (library/component)
LogRecordProcessor	Controls timing and transformation	`on_emit()` → `exporter.Export()` (Simple/Batching/Custom)
LogRecordExporter	Sends logs to sink	`Export(batch)`, `Shutdown()`

When integrating logging, ensure exporters and processors respect timeouts and are shut down cleanly during application termination. Call ForceFlush before Shutdown to minimize data loss.

You’ve now seen the Logs SDK’s main pieces and how they interact. The architecture mirrors other OpenTelemetry signals (traces/metrics), allowing consistent integration patterns across telemetry types.

A presentation slide titled "Summary and Next Steps" with a vertical numbered list. The points summarize a Logs SDK: core logging infrastructure, key components (Provider, Logger, Processors, Exporters), foundation for implementation, and hands-on coding examples.

References and further reading

OpenTelemetry Documentation
OpenTelemetry Logs Specification
SDK-specific docs (look up language-specific LoggerProvider, processors, and exporter implementations for best practices and API details)

Well, that’s it for this lesson — next, try a hands-on example wiring a LoggerProvider with a batching processor and an OTLP exporter in your preferred language.

Course Introduction

Observability Core Concepts

OpenTelemetry Core Concepts

Span Anatomy and Context Propagation

Instrumentation

Metrics Data Model

Recording Measurements

Logs

OTel Collector Foundations

OTel Collector Core Components

Transforming Telemetry – Pipeline Processing

OpenTelemetry in Kubernetes

OpenTelemetry Collector: Debugging, Operations, and Scaling

Extended Topics

Key concepts overview

LoggerProvider (factory and configuration)

Example: getting a logger and Provider behavior (pseudocode)

Logger lifetime and dynamic provider updates

Lifecycle: ForceFlush and Shutdown

Recommended shutdown sequence (pseudocode)

LogRecordProcessor types and responsibilities

LogRecordExporter responsibilities

Core components at a glance

References and further reading

Watch Video

​Key concepts overview

​LoggerProvider (factory and configuration)

​Example: getting a logger and Provider behavior (pseudocode)

​Logger lifetime and dynamic provider updates

​Lifecycle: ForceFlush and Shutdown

​Recommended shutdown sequence (pseudocode)

​LogRecordProcessor types and responsibilities

​LogRecordExporter responsibilities

​Core components at a glance

​References and further reading

Watch Video

Key concepts overview

LoggerProvider (factory and configuration)

Example: getting a logger and Provider behavior (pseudocode)

Logger lifetime and dynamic provider updates

Lifecycle: ForceFlush and Shutdown

Recommended shutdown sequence (pseudocode)

LogRecordProcessor types and responsibilities

LogRecordExporter responsibilities

Core components at a glance

References and further reading