Course Introduction

Welcome to this hands-on Learn By Doing course for Apache Kafka. I’m Vijin Palazhi, and I’ll guide you through this lesson created by Rakshith. This practical course is tailored for DevOps engineers, system administrators, and cloud professionals who want a focused, operational introduction to Apache Kafka — the distributed event-streaming platform used for building real-time data pipelines and streaming applications. This lesson emphasizes hands-on practice. You’ll set up a local Kafka environment, build simple producer and consumer applications, and work with topics and partitions so you understand how data moves through Kafka in production-like scenarios. What you’ll learn

How Apache Kafka fits into real-time data pipelines and streaming architectures.
Core Kafka components: brokers, producers, consumers, topics, and partitions.
Building and running basic Kafka producer and consumer applications.
Configuring Kafka brokers and creating topics for a local development environment.
Hands-on experience sending and receiving messages through Kafka topics.

Use the Overview tab for course material and background context. Use the Tasks tab as your step-by-step workbook. Each lab includes an embedded terminal that you’ll use to run commands. If you get stuck, open the Hint or Solution tab; use the Check button to validate tasks and progress.

How the course is structured

Each lab is self-contained with an Overview, Tasks, Hint, and Solution.
The Overview explains concepts and prerequisites; Tasks provide explicit steps to complete in the embedded terminal.
Progress is tracked via the Check button; completing tasks validates your understanding and unlocks the next steps.

Course modules at a glance

Module	Topic	What you’ll practice
1	Introduction to Kafka	Understand Kafka’s role in streaming systems and common use cases
2	Components & Architecture	Inspect brokers, zookeepers (where applicable), partitions, replication, and how data is stored
3	Producers & Consumers	Build simple producer and consumer apps; configure serializers and consumer groups
4	Topics & Partitions	Create topics, manage partitions, and observe ordering and retention behaviors
5	Kafka Environment Setup	Install and run Kafka locally, configure brokers and topics, and verify message flow

Quick terminal commands you may use in labs

# List pods (if a Kubernetes-based lab environment is provided)
kubectl get pods

# Show services
kubectl get svc

The image is a slide titled "Course Topics Overview" with a teal left panel and white content area. It lists five numbered Kafka course topics: Introduction to Kafka; Components & Architecture; Producers and Consumers; Kafka Topics & Partitions; and Kafka Environment Setup.

As you progress through the labs, you’ll develop and configure producers and consumers to send and receive messages from Kafka topics. Labs are ordered to move you from basic concepts to practical development and deployment, with each task building on the previous one so you can validate understanding incrementally. Next steps

Start the first lab in this lesson (Introduction to Kafka).
Follow the Overview for concepts, then complete Tasks using the embedded terminal.
Use Hint/Solution if needed, and run the Check step to confirm completion.

Links and references

Apache Kafka documentation: https://kafka.apache.org/documentation/
Confluent Kafka resources: https://www.confluent.io/resources/
Kafka tutorials and examples: https://kafka.apache.org/quickstart

Now, let’s jump in — navigate to the first lab and begin.

Watch Video