> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Demo Tracing with OpenTelemetry and Jaeger

> Guide to enabling OpenTelemetry tracing in Kubernetes and exporting application traces to Jaeger for diagnosing latency across microservices

Metrics tell you something is slow.\
Logs show what happened inside a single service.\
But when a request flows through multiple microservices and suddenly takes several seconds, how do you find the service responsible for the delay? Distributed tracing answers that question.

OpenTelemetry (OTel) provides a vendor-neutral way to instrument applications and emit traces and metrics. Jaeger is a popular tracing backend that collects, stores, indexes, and visualizes those traces. In short: OpenTelemetry generates and exports traces; Jaeger collects and displays them.

This guide shows how to enable OpenTelemetry in a Kubernetes deployment so traces from your application appear in Jaeger.

## What is running in the cluster

Start by confirming the observability components and your application workloads are running:

```bash theme={null}
# Check Jaeger and operator pods
kubectl get pods -n observability

# Check application pods
kubectl get pods -n workloads
```

Example output:

```bash theme={null}
NAME                                 READY   STATUS    RESTARTS   AGE
jaeger-fd6d64f9-824c6                1/1     Running   0          2m19s
jaeger-operator-648447789c-c4kqx     2/2     Running   0          2m32s

NAME                                     READY   STATUS    RESTARTS   AGE
frontend-svc-69cd76d4fb-ckklq            1/1     Running   0          2m28s
inventory-svc-84779cd768-jr52m           1/1     Running   0          2m28s
order-svc-5f46fb5764-mb6rs               1/1     Running   0          2m28s
payment-svc-68bd555596-v6z67             1/1     Running   0          2m28s
traffic-generator-86f5f97bfc-kpbkd       1/1     Running   0          2m27s
```

In this example:

* Jaeger runs in the `observability` namespace.
* Four microservices (frontend, order, inventory, payment) run in the `workloads` namespace.
* A traffic generator is exercising the services to generate traces.

If you open the Jaeger UI and a running service (for example, `frontend-svc`) does not appear in the service list, that usually means the application is not configured to emit traces — the OpenTelemetry environment variables are missing or incorrect.

## How to configure the frontend deployment to export traces

To export traces from the frontend to Jaeger via OTLP, add the appropriate OpenTelemetry environment variables to the frontend deployment. These variables tell the application’s OTel SDK what the service name is, which protocol and endpoint to use, and which exporter to enable.

Use this table to guide the required variables:

| Variable | Purpose | Example |
| - | - | - |
| `OTEL_SERVICE_NAME` | Service name that will appear in Jaeger (use a meaningful, unique name) | `frontend-svc` |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP collector endpoint to send traces to (cluster DNS) | `jaeger-collector.observability.svc.cluster.local:4317` |
| `OTEL_EXPORTER_OTLP_PROTOCOL` | Transport protocol for OTLP (must match collector port/protocol) | `grpc` or `http/protobuf` |
| `OTEL_TRACES_EXPORTER` | Which exporter to use in the SDK (for OTLP use `otlp`) | `otlp` |

Important: protocol and port must match the Jaeger collector. gRPC uses port `4317`; HTTP/Protobuf uses port `4318`. If you choose HTTP/Protobuf, some SDKs require an explicit scheme in the endpoint (for example `http://jaeger-collector.observability.svc.cluster.local:4318`).

Below is the patch fragment you can apply to the frontend deployment (save as `frontend-patch.yaml`):

```yaml theme={null}
spec:
  template:
    spec:
      containers:
      - name: frontend-svc
        env:
        - name: OTEL_SERVICE_NAME
          value: "frontend-svc"
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: "jaeger-collector.observability.svc.cluster.local:4317"
        - name: OTEL_EXPORTER_OTLP_PROTOCOL
          value: "grpc"
        - name: OTEL_TRACES_EXPORTER
          value: "otlp"
```

Apply the patch and monitor the rollout:

```bash theme={null}
kubectl patch deployment frontend-svc -n workloads --patch "$(cat frontend-patch.yaml)"
kubectl rollout status deployment/frontend-svc -n workloads
```

Confirm the Jaeger collector service and the ports it exposes:

```bash theme={null}
kubectl get svc -n observability
```

Example output showing the relevant ports:

```bash theme={null}
NAME                     TYPE        CLUSTER-IP       PORT(S)
jaeger-collector         ClusterIP   172.20.204.3     9411/TCP,14250/TCP,4317/TCP,4318/TCP
jaeger-query             ClusterIP   172.20.221.215   16686/TCP
jaeger-ui-nodeport       NodePort    172.20.161.35    16686:30086/TCP
```

Always ensure the `OTEL_EXPORTER_OTLP_PROTOCOL` value matches the collector port you set in `OTEL_EXPORTER_OTLP_ENDPOINT`: use `grpc` for port `4317` or `http/protobuf` for port `4318`. A mismatch will prevent traces from being delivered.

If your language's OpenTelemetry SDK exposes protocol-specific variables (for example `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`), prefer those over the generic `OTEL_EXPORTER_OTLP_PROTOCOL`. Check your SDK docs — using the wrong variable is a common cause of silent failures.

## Verify traces in Jaeger

After the deployment restarts with the new environment variables:

1. Open the Jaeger UI (query service typically at the `jaeger-query` service; many setups provide a UI route or NodePort).
2. Refresh the service dropdown — `frontend-svc` should now appear.
3. Search for recent traces for `frontend-svc`. Open a trace to inspect the waterfall.

A trace waterfall shows each service as a span (horizontal bar); the bar width corresponds to the duration spent in that span. The widest span is often the best starting point for investigating latency.

In this example, the waterfall shows frontend → order → inventory → payment, and the inventory span is the widest (about 5 seconds), revealing a likely bottleneck.

![A Jaeger UI trace screenshot showing a "frontend-svc: GET /" trace with multiple nested spans for services like frontend-svc, order-svc, inventory-svc and payment-svc across a \~5.21s timeline. The left panel lists services and operations while the right shows colored horizontal bars indicating each span's timing and duration.](https://mintcdn.com/kodekloud-c4ac6d9a/seqLTE3JlSrVAtw9/images/Prep-Course-Certified-Cloud-Native-Platform-Engineering-CNPE/Observability-and-Operations/Demo-Tracing-with-OpenTelemetry-and-Jaeger/jaeger-trace-frontend-order-inventory-payment.jpg?fit=max\&auto=format\&n=seqLTE3JlSrVAtw9\&q=85\&s=60195f4baa6100ff5df14a7129dd8f99)

If traces still do not appear, use this checklist:

* Confirm the environment variables are present on the running pod:
  ```bash theme={null}
  kubectl describe pod <frontend-pod-name> -n workloads
  ```
* Confirm you patched the correct container name (the patch targets the container named `frontend-svc` in this example).
* Verify the OTLP endpoint and protocol match the Jaeger collector service and port.
* Inspect application and OTel SDK logs for errors or warnings that indicate export failures.
* Ensure network policies or service mesh sidecars (if present) are not blocking export traffic to the collector.

## Summary and next steps

You enabled OpenTelemetry tracing for a Kubernetes service and exported traces to Jaeger by:

* Setting the service name (`OTEL_SERVICE_NAME`).
* Pointing the SDK at the Jaeger OTLP endpoint (`OTEL_EXPORTER_OTLP_ENDPOINT`).
* Choosing the correct transport protocol (`OTEL_EXPORTER_OTLP_PROTOCOL` or protocol-specific variables).
* Enabling the OTLP exporter (`OTEL_TRACES_EXPORTER`).

With traces visible in Jaeger, use the waterfall view to find latency hotspots and drill into span details to identify where to optimize. Try the same steps for other services (order, inventory, payment) and consider adding instrumentation libraries or auto-instrumentation for richer spans and attributes.

Links and references

* [OpenTelemetry](https://opentelemetry.io) — official site and SDK docs
* [Jaeger Tracing](https://www.jaegertracing.io) — Jaeger documentation and UI guides
* Kubernetes: [kubectl reference](https://kubernetes.io/docs/reference/kubectl/overview)

> **[Watch Video](https://learn.kodekloud.com/user/courses/prep-course-certified-cloud-native-platform-engineer-cnpe/module/9bd090c8-8d99-4742-b50c-ae63e516e6b9/lesson/9507aaae-93db-4f66-9d14-017d3e6651d8)**

> **[Practice Lab](https://learn.kodekloud.com/user/courses/prep-course-certified-cloud-native-platform-engineer-cnpe/module/9bd090c8-8d99-4742-b50c-ae63e516e6b9/lesson/4da36e12-ad6b-4d91-a8b8-451e1bf4b8fe)**


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.