> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Kafka Brokers The Foundation of Message Storage
> Explains Kafka brokers as durable, scalable servers for storing and serving event streams using an EV charging use case, covering replication, partitions, scalability, and operations
Welcome back. In this lesson we’ll drill into the core Kafka components that provide durable, scalable message storage: Kafka brokers and how they operate in a cluster. To make the concepts concrete, we’ll continue using our EV charging use case.
Producers (for example, EV charging stations) publish events to a logical stream called a topic. Consumers (the app, analytics, billing, or monitoring services) read those topics to act on data. Where are these events stored? On Kafka brokers — the servers that persist and serve message data for the cluster.
Use case: charging station status reports
* When a user searches for available chargers, the app must show free/occupied status for a given time window.
* Each charging station (producer) emits status and availability events to a topic (for example, `charger-status`).
* The authoritative, persisted stream of those events lives on Kafka brokers. Consumers read and materialize the current availability view as needed.
What is a Kafka broker?
A Kafka broker is the server process that stores topic data on disk, serves client requests for reads and writes, and coordinates with other brokers for replication and leadership. Brokers are the backbone of Kafka’s durability and throughput.
Key characteristics of Kafka brokers
| Feature | What it does | Relevance to EV charging use case |
| -------------------------------: | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| Message management | Persists messages to disk, serves producer writes and consumer reads, enforces retention and partition ordering | Ensures status events are durably stored and can be replayed to rebuild state |
| Cluster node | Each broker is a node that participates in the distributed cluster | Multiple brokers provide capacity and redundancy for high availability |
| Partition placement & leadership | Topics are split into partitions; a leader broker handles reads/writes for each partition, followers act as replicas | Partitioning allows parallelism (throughput) for many chargers; leaders handle client traffic |
| Scalability | Add brokers to distribute partitions and load horizontally | Scale ingestion as more chargers or regions are added |
| Fault tolerance & durability | Replication across brokers protects against data loss; followers can be promoted if a leader fails | Maintains accurate charger state despite broker failure |
| Dynamic membership | Brokers can join/leave; the cluster controller reassigns leadership/replicas as needed | Enables rolling upgrades and elastic growth without service disruption |
Operational examples
* Inspect topic partitions and replication:
```bash theme={null}
bin/kafka-topics.sh --bootstrap-server broker:9092 --describe --topic charger-status
```
* Produce a sample status event:
```bash theme={null}
echo '{"chargerId":"C-100","status":"AVAILABLE","timestamp":"2026-07-15T10:00:00Z"}' | \
bin/kafka-console-producer.sh --broker-list broker:9092 --topic charger-status
```
* Consume latest events to verify state:
```bash theme={null}
bin/kafka-console-consumer.sh --bootstrap-server broker:9092 --topic charger-status --from-beginning --max-messages 10
```
Retention vs durability: Kafka persists messages to disk and applies retention rules (time- or size-based). Replication across brokers provides durability — preventing data loss — while retention determines how long data remains queryable in the cluster.
Real-world scale and examples
Large companies run Kafka at vast scale to support real-time systems. Examples include:
* Netflix — thousands of brokers across many clusters for event-driven streaming and personalization.
* Pinterest — multi-trillion messages per day at very high throughput.
* PayPal — large broker fleets to provide reliable transaction and event delivery.
* LinkedIn — Kafka’s originator, operating large clusters for core real-time pipelines.
These deployments demonstrate how brokers deliver throughput, durability, and availability for streaming use cases — from payments to IoT and social platforms.
Summary
Kafka brokers are the durable storage and serving layer for event streams. In the EV charging scenario they store charger status events, enable replay and state reconstruction, and scale horizontally to support large fleets and high throughput.
Links and references
* [Apache Kafka documentation](https://kafka.apache.org/documentation/)
* [Kafka operations: topics, partitions, and replication](https://kafka.apache.org/documentation/#basic_ops)
* [Designing for durability and availability with Kafka](https://kafka.apache.org/intro)