Hands On EFK Introduction

This lesson inspects what Bob currently has running in his Kubernetes cluster and explains why a persistent logging stack is needed. You’ll see how the app writes logs, where they live on the node, and which components (Elasticsearch, Fluentd, Kibana — the EFK stack) will collect, store, and visualize those logs. What is running in Bob’s cluster right now? The fastest way to find out is to ask the cluster. Bob lists pods in the default namespace:

$ kubectl get pods -n default

Kubernetes reports a single pod named app in Running state. There are no Elasticsearch, Fluentd, or Kibana pods yet — just the app.

$ kubectl get pods -n default
NAME    READY   STATUS    RESTARTS   AGE
app     1/1     Running   0          12m

To see what the pod is producing, Bob reads its stdout:

$ kubectl logs app -n default

Lines stream down the terminal; each line reports a user interaction with the web app:

[2026-10-13 08:12:01,118] INFO in event-simulator: USER3 logged in
[2026-10-13 08:12:02,204] INFO in event-simulator: USER2 is viewing page1
[2026-10-13 08:12:03,311] INFO in event-simulator: USER4 is viewing page3
[2026-10-13 08:12:04,409] INFO in event-simulator: USER3 logged out
[2026-10-13 08:12:05,402] INFO in event-simulator: USER1 logged in

Each log line contains a timestamp and username—fields you commonly want to search or filter by when investigating incidents. These lines are the key artifacts Bob will centralize with the EFK stack. Where do those lines go right now?

The app prints them to its standard output, so kubectl logs can read them.
The app also writes them to a folder on the kube node’s filesystem.

In this cluster, the app mounts a directory from the node, /var/log/webapp, into the container using a hostPath volume. Later, when Bob deploys a collector (Fluentd), that hostPath is exactly where the collector will read log files.

# relevant excerpt from the app manifest
- name: app
  image: kodekloud/event-simulator
  volumeMounts:
    - mountPath: /var/log/webapp
      name: log-volume

# Volume to store the logs
volumes:
  - name: log-volume
    hostPath:
      path: /var/log/webapp
      type: DirectoryOrCreate

HostPath volumes expose a directory from the Kubernetes node into a container. They are useful for simple demos and for collectors that need node-local files, but they tie data to the node and should be used with care in production.

Right now, Bob’s logs are tied to a single pod on a single node. If Kubernetes replaces the app pod (or the node fails), the pod-local stdout history and any node-local files may be lost. The EFK stack keeps logs beyond a pod’s lifetime by collecting, aggregating, and persisting them to a separate storage layer. Quick questions about the stack Bob will build:

Which Kubernetes object ensures a set of pods is kept running and can be scaled? A Deployment.
What component gathers log lines from many sources? Fluentd.
Where are logs stored and queried? Elasticsearch.
What provides a web UI for searching and visualizing logs? Kibana.

Deployments: the object that keeps copies running A Deployment declares the desired number of replicas; Kubernetes ensures that many pod instances exist and restarts them if needed. Kibana will be run as a Deployment in this lesson so it can be scaled and automatically recreated.

A dark-themed diagram titled "A Deployment Keeps Copies" showing a "Deployment" node with an arrow pointing down to two smiling blue rounded icons (representing copies) on a platform. A small "Q1" box with a green checkmark appears on the left.

Fluentd: log aggregation and buffering Fluentd aggregates log lines from multiple sources into a single stream and can filter/transform them via plugins. Its configuration uses .conf files (not YAML); plugins and buffering settings are defined there so Fluentd can hold data temporarily if Elasticsearch is unavailable.

Elasticsearch: document store, shards, and replicas Elasticsearch stores each log line as a document (a flexible JSON-like structure). To provide durability and scalability it replicates documents across nodes and splits indices into shards so searches run in parallel. A master node tracks cluster metadata, creates indices, and decides shard placement.

An illustration titled "One Line, One Document" showing a smiling blue chatbot icon with a "USER1 logged in" message on the left and a partially filled cylindrical database on the right.

An infographic titled "Copies and Shards" showing three smiling blue robot icons, each paired with a stacked-disk database labeled 1, 2, and 3 on separate platform bases. It visually represents database copies/shards or replicas.

A diagram titled "The Master Node" showing three server nodes represented by smiling blue mascots next to database icons labeled 1–3. The left node is marked "master" with arrows connecting it to the other two nodes.

Why Bob needs centralized logging All of this brings Bob back to his single busy app and its ephemeral logs. Two practical questions remain:

If the app pod were replaced right now, where would the “USER1 logged in” line go?
Where should log data live so it survives pod restarts and node failures?

Components at a glance

Component	Role	Notes / Example
Deployment	Ensures a set number of pod replicas are running	Use for Kibana or other stateless apps
Fluentd (collector)	Aggregates and forwards logs; can filter and buffer	Configured with `.conf` and plugins
Elasticsearch (store)	Indexes and stores log documents; supports shards & replicas	Scales by adding nodes; master manages metadata
Kibana (visualize)	UI for searching, filtering, and visualizing logs	Run as a Deployment for high availability

Introduction to EFK on Kubernetes

Deploying Elasticsearch

Collecting Logs With Fluentd

Visualizing Logs With Kibana

Monitoring and Alerting

Scaling the EFK Stack

Securing the EFK Stack

Advanced EFK Configuration

Hands On EFK Introduction

Watch Video

Practice Lab