Demo Exporting Logs to Collector

In this lesson you’ll replace the console log exporter with an OTLP log exporter so application logs are forwarded to an OpenTelemetry Collector. In production you typically send traces and logs to a Collector, and the Collector forwards them to a backend such as Elasticsearch, Splunk, or a vendor-specific ingest. Overview

Install the OTLP HTTP exporter package.
Import and configure the OTLP log exporter.
Wire the exporter into the SDK’s log pipeline.
Run the app and verify the Collector receives the logs.

Before running the code, install the OTLP HTTP exporter package:

pip install opentelemetry-exporter-otlp-proto-http

What we’ll change

Keep the span exporter (ConsoleSpanExporter) if you want to continue viewing spans locally.
Configure an OTLP log exporter to point at the Collector HTTP endpoint (default path for OTLP logs is /v1/logs on port 4318).
Create a BatchLogRecordProcessor and attach it to the SDK LoggerProvider.
Optionally capture stdlib logging records and forward them to OpenTelemetry logs.

Imports and configuration

Notes:

Use opentelemetry.exporter.otlp.proto.http._log_exporter.OTLPLogExporter for the OTLP HTTP logs exporter (module and class names are case-sensitive).
If you also want to send spans to OTLP, add the corresponding OTLP span exporter (for example, opentelemetry.exporter.otlp.proto.http.trace_exporter.OTLPSpanExporter) and a span processor.

Example configuration (consolidated):

import time
import logging

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk._logs.export import ConsoleLogExporter
from opentelemetry._logs.severity import SeverityNumber

# OTLP logs exporter (HTTP)
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter

# --- Tracing (optional: still export spans to console here) ---
span_exporter = ConsoleSpanExporter()
span_processor = SimpleSpanProcessor(span_exporter)

tracer_provider = TracerProvider()
tracer_provider.add_span_processor(span_processor)
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer("flight-booking", "0.1.2")

# --- Logging: configure OTLP exporter and processor ---
logger_provider = LoggerProvider()

# Send logs to the local Collector at http://localhost:4318/v1/logs
otlp_log_exporter = OTLPLogExporter(endpoint="http://localhost:4318/v1/logs")

log_processor = BatchLogRecordProcessor(otlp_log_exporter)
logger_provider.add_log_record_processor(log_processor)

# Make the SDK logger provider the global provider
set_logger_provider(logger_provider=logger_provider)

# Optionally capture Python `logging` records and forward them to OpenTelemetry logs
handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)
root_logger = logging.getLogger()
root_logger.setLevel(logging.INFO)
root_logger.addHandler(handler)

# Example usage
logger = logging.getLogger(__name__)
logger.info("starting booking process")

Docker Compose — Local Collector

This minimal Docker Compose example runs an OpenTelemetry Collector that accepts both OTLP gRPC (4317) and OTLP HTTP (4318). It mounts a local config file into the container.

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.85.0
    container_name: otel-collector
    ports:
      - "4317:4317" # OTLP gRPC
      - "4318:4318" # OTLP HTTP
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro
    command: ["--config", "/etc/otel-collector-config.yaml"]

Resource	Purpose	Example / Notes
Collector HTTP endpoint	Receives OTLP logs over HTTP	`http://localhost:4318/v1/logs`
Collector gRPC endpoint	Receives OTLP traces/metrics via gRPC	`localhost:4317`
Local config	Collector pipeline, receivers, exporters	Mount via `./otel-collector-config.yaml`

Verify the Collector received logs

Start the Collector via Docker Compose.
Run your application configured with the OTLP log exporter.
If the Collector routes logs to a stdout exporter in its config, you should see decoded log records in the Collector’s console.

Example Collector stdout output that indicates successful reception and decoding:

otel-collector | ScopeLogs #0
otel-collector | ScopeLogs SchemaURL:
otel-collector | InstrumentationScope root
otel-collector | LogRecord #0
otel-collector | ObservedTimestamp: 2025-09-18 23:38:13.04102 +0000 UTC
otel-collector | Timestamp: 2025-09-18 23:38:13.041005056 +0000 UTC
otel-collector | SeverityText: INFO
otel-collector | SeverityNumber: Info(9)
otel-collector | Body: Str(INFO:root:starting booking process)
otel-collector | Attributes:
otel-collector |  -> code.file.path: Str(/Users/sanjeev/Documents/courses/logs-demo/main.py)
otel-collector |  -> code.function.name: Str(book_flight)
otel-collector |  -> code.line.number: Int(42)
otel-collector | Trace ID: ca32c00c90b3aac91ea761e85eaa16c4
otel-collector | Span ID: 3e21b96358e13356
otel-collector | Flags: 1
otel-collector | LogRecord #1
otel-collector | ObservedTimestamp: 2025-09-18 23:38:13.041074 +0000 UTC
otel-collector | Timestamp: 2025-09-18 23:38:13.041068032 +0000 UTC
otel-collector | SeverityText: INFO
otel-collector | SeverityNumber: Info(9)
otel-collector | Body: Str(INFO:root:Searching for flights)
otel-collector | Attributes:
otel-collector |  -> code.file.path: Str(/Users/sanjeev/Documents/courses/logs-demo/main.py)
otel-collector |  -> code.function.name: Str(search_flights)
otel-collector |  -> code.line.number: Int(48)
otel-collector | Trace ID: ca32c00c90b3aac91ea761e85eaa16c4
otel-collector | Span ID: 48d7ae5a6846fa04
otel-collector | Flags: 1
otel-collector | LogRecord #2
otel-collector | ObservedTimestamp: 2025-09-18 23:38:13.041504 +0000 UTC
otel-collector | Timestamp: 2025-09-18 23:38:13.041498112 +0000 UTC
otel-collector | SeverityText: WARN
otel-collector | SeverityNumber: Warn(13)
otel-collector | Body: Str(WARNING:root:Reserving flight)
otel-collector | Attributes:
otel-collector |  -> code.file.path: Str(/Users/sanjeev/Documents/courses/logs-demo/main.py)
otel-collector |  -> code.function.name: Str(reserve_flight)
otel-collector |  -> code.line.number: Int(54)
otel-collector | Trace ID: ca32c00c90b3aac91ea761e85eaa16c4
otel-collector | Span ID: d6768916ca17ae49
otel-collector | Flags: 1
otel-collector | {"kind": "exporter", "data_type": "logs", "name": "logging"}

That output confirms:

The application forwarded logs via OTLP to the Collector.
The Collector decoded the OTLP payload and exposed metadata (severity, file/line attributes, trace/span IDs).

If you do not see logs in the Collector stdout:

Verify your application can reach http://localhost:4318.
Confirm the Collector config includes an OTLP receiver and a stdout (or other) exporter.
Check for network/firewall rules or Docker port mappings that might block port 4318.

Troubleshooting checklist

Is the OTLP exporter package installed? (pip show opentelemetry-exporter-otlp-proto-http)
Is the Collector running and listening on the expected ports? (docker ps and docker logs otel-collector)
Is the Collector config mounting correctly? Ensure --config /etc/otel-collector-config.yaml points to the expected file.
Are there authentication or proxy requirements in your environment? If so, adjust OTLPLogExporter(endpoint=...) or use environment variables per the exporter docs.

Links and references

OpenTelemetry Collector: https://opentelemetry.io/docs/collector/
OTLP HTTP logs exporter (PyPI): https://pypi.org/project/opentelemetry-exporter-otlp-proto-http/
OpenTelemetry Python documentation: https://opentelemetry.io/docs/instrumentation/python/
Docker Hub: https://hub.docker.com/

If you want, I can provide a ready-to-run otel-collector-config.yaml example that routes OTLP logs to stdout and to a simple file exporter for local debugging.

Course Introduction

Observability Core Concepts

OpenTelemetry Core Concepts

Span Anatomy and Context Propagation

Instrumentation

Metrics Data Model

Recording Measurements

Logs

OTel Collector Foundations

OTel Collector Core Components

Transforming Telemetry – Pipeline Processing

OpenTelemetry in Kubernetes

OpenTelemetry Collector: Debugging, Operations, and Scaling

Extended Topics

Demo Exporting Logs to Collector

What we’ll change

Imports and configuration

Docker Compose — Local Collector

Verify the Collector received logs

Troubleshooting checklist

Links and references

Watch Video

Practice Lab

​What we’ll change

​Imports and configuration

​Docker Compose — Local Collector

​Verify the Collector received logs

​Troubleshooting checklist

​Links and references

Watch Video

Practice Lab

What we’ll change

Imports and configuration

Docker Compose — Local Collector

Verify the Collector received logs

Troubleshooting checklist

Links and references