> ## Documentation Index > Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt > Use this file to discover all available pages before exploring further. # Before OpenTelemetry Why Standardization Was the Missing Piece > Explains how OpenTelemetry standardized telemetry collection to solve fragmentation and vendor lock-in, offering vendor-neutral instrumentation and a configurable Collector for flexible observability. In this lesson we set the context for why OpenTelemetry (OTel) emerged and how it addresses long-standing challenges in collecting telemetry—metrics, logs, and traces—across modern distributed systems. As you read, ask: What was missing before OTel, and how does it fix those gaps? Telemetry is the process of recording and transmitting readings from an instrument. Think of weather systems: satellites, ground sensors, and weather stations all send measurements into a central system for analysis and forecasting. The image explains the basics of telemetry, showing how data from different sources such as weather stations and satellites is transmitted to a weather radar. Similarly, software systems emit signals that reveal the system’s operating condition. Telemetry can originate at multiple layers of the stack: application code, libraries, runtime, OS, kernel, and the underlying hardware. For example, a checkout service running on an EC2 instance is a workload composed of application code and libraries that call the OS and kernel and consume hardware resources. The image illustrates telemetry data sources in software systems, featuring a layered model from hardware to application and elements such as a shopping cart interface and EC2. In distributed systems the overall health is the aggregate of signals from these layers. For brevity we’ll call this unit—application, process, or service—a “software system.” How does telemetry leave a software system? Two common collection modes exist: * Push: an agent or SDK inside the system sends telemetry out (for example, via HTTP POST to a collector or vendor endpoint). * Pull: an external scraper polls an exposed endpoint (for example, Prometheus scraping `/metrics`) to collect metrics. You can install an agent on a VM or in a container to push telemetry, or configure your application to call an API/SDK directly. Alternatively, an external scraper can poll the application to collect metrics. The image illustrates push and pull telemetry data collection methods in software, featuring inbound and outbound data flows with components like agents, scrapers, APIs/SDKs, and polling. Comparison: Push vs Pull | Mode | How it works | Typical tools / examples | | ---- | ----------------------------------------------------------- | ---------------------------------------------------- | | Push | Application/agent sends telemetry to a collector or backend | Agents, SDKs, vendor agents (HTTP/gRPC exporters) | | Pull | External system scrapes exposed endpoints | `Prometheus` scraping `/metrics`, HTTP health checks | Once collected, telemetry is routed to dedicated systems for visualization and analysis: a metrics system (e.g., Prometheus), a logs pipeline and dashboard (e.g., the ELK stack), and a tracing backend (e.g., Jaeger or Zipkin). Each pipeline typically has its own ingestion, storage, and query tools. The image is a flowchart illustrating how metrics and logs are visualized from a software system, involving agents, APIs/SDKs, and dashboards for metrics and logs. It shows outbound data flow from the software system to the dashboards. Historically, observability developed in a pillar-based way: metrics, traces, and logs often used separate tools, standards, and vendor pipelines. Different teams adopted different instrumentation approaches—OpenTracing, OpenCensus, vendor-specific agents, or homegrown log agents—resulting in many isolated telemetry silos. The image shows a diagram of several "Software System" blocks connected by arrows, illustrating the challenges in pillar-based observability systems. It includes labels like OpenTracing, Open Source Telemetry, and vendor-specific agents and SDKs. This fragmentation led many organizations to consolidate on a single vendor to reduce operational complexity: deploy that vendor’s agent everywhere and send all telemetry to its backend. The image illustrates a single-vendor adoption model where multiple software systems use a vendor's agent to connect to a single observability backend, highlighting potential lock-in risks. While consolidation reduces short-term complexity, it creates vendor lock-in. Changing backends later can require replacing agents, re-instrumenting code, and migrating pipelines—costly and risky at scale. A better approach is a vendor-neutral layer between systems and backends. Each software system outputs metrics, logs, and traces into that neutral layer; from there, data can be routed to any backend—Vendor A, Vendor B, a data lake, or multiple destinations—without re-instrumenting the source. The image shows a flowchart titled "A Neutral Pipeline for All Observability Backends," illustrating a process involving Instrumentation, Collection, and Export stages for telemetry data. With a neutral pipeline, changing where telemetry ends up is a configuration change—not a code or agent replacement. Instrument once; control destinations via telemetry middleware configuration. OpenTelemetry provides that vendor-neutral instrumentation and processing layer. OTel SDKs and agents standardize how applications produce metrics, logs, and traces. The OpenTelemetry Collector acts as configurable middleware to receive, process, and export telemetry to one or many backends. As a result, routing telemetry becomes a matter of editing Collector configuration rather than reinstalling agents. The image illustrates the OpenTelemetry data pipeline, showing system software sending metrics, traces, and logs through OpenTelemetry agents or SDKs to the OpenTelemetry Collector, which then directs the data to various destinations like vendor backends and data lakes. OTel was designed to coexist with existing vendors and tools. The Collector can export to vendor backends, open-source systems, object stores, or multiple destinations simultaneously—making hybrid observability and gradual migration practical. The diagram illustrates how modern observability involves hybrid systems integrating seamlessly with OpenTelemetry. It shows different system software sending metrics, traces, and logs to various backends, including OpenTelemetry Collector and Prometheus. OTel supports hybrid environments where some services are instrumented with OpenTelemetry and others are not. You can continue accepting vendor-specific telemetry while incrementally adopting OTel, enabling coexistence and staged migration. To summarize the key challenges before standardization: * Fragmentation: multiple tools and standards across metrics, traces, and logs. * Vendor lock-in: switching backends often required replacing agents and re-instrumenting code. * No single standard: vendors and teams shipped their own agents/SDKs. * Limited flexibility: hard to send the same telemetry to multiple destinations. * Hybrid reality: many environments had a mix of instrumented and non-instrumented services. The image outlines key challenges before standardization in telemetry, highlighting issues like fragmentation, vendor lock-in, lack of standard approaches, low flexibility, and commonality of hybrid setups. OpenTelemetry addresses these issues by standardizing APIs and formats for metrics, logs, and traces, and by providing a flexible Collector to process and export telemetry to any backend. This reduces vendor lock-in, supports gradual adoption, and enables exporting the same telemetry to multiple destinations for future-proofing. OpenTelemetry provides a vendor-neutral instrumentation layer plus a configurable Collector. Instrument once; export anywhere. This enables flexibility, easier migrations, and hybrid coexistence with existing vendor ecosystems. That concludes this lesson. References and further reading: * OpenTelemetry: [https://opentelemetry.io](https://opentelemetry.io) * OpenTelemetry Collector docs: [https://opentelemetry.io/docs/collector/](https://opentelemetry.io/docs/collector/) * Prometheus: [https://prometheus.io](https://prometheus.io) * Jaeger: [https://www.jaegertracing.io](https://www.jaegertracing.io) * ELK Stack: [https://www.elastic.co/what-is/elk-stack](https://www.elastic.co/what-is/elk-stack)