> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Hands On EFK Introduction

> Intro to building an EFK logging stack on Kubernetes, showing how Fluentd collects container and hostPath logs, Elasticsearch stores them, and Kibana visualizes searches

This lesson inspects what Bob currently has running in his Kubernetes cluster and explains why a persistent logging stack is needed. You'll see how the app writes logs, where they live on the node, and which components (Elasticsearch, Fluentd, Kibana — the EFK stack) will collect, store, and visualize those logs.

What is running in Bob's cluster right now?

The fastest way to find out is to ask the cluster. Bob lists pods in the `default` namespace:

```bash theme={null}
$ kubectl get pods -n default
```

Kubernetes reports a single pod named `app` in `Running` state. There are no Elasticsearch, Fluentd, or Kibana pods yet — just the app.

```bash theme={null}
$ kubectl get pods -n default
NAME    READY   STATUS    RESTARTS   AGE
app     1/1     Running   0          12m
```

To see what the pod is producing, Bob reads its stdout:

```bash theme={null}
$ kubectl logs app -n default
```

Lines stream down the terminal; each line reports a user interaction with the web app:

```text theme={null}
[2026-10-13 08:12:01,118] INFO in event-simulator: USER3 logged in
[2026-10-13 08:12:02,204] INFO in event-simulator: USER2 is viewing page1
[2026-10-13 08:12:03,311] INFO in event-simulator: USER4 is viewing page3
[2026-10-13 08:12:04,409] INFO in event-simulator: USER3 logged out
[2026-10-13 08:12:05,402] INFO in event-simulator: USER1 logged in
```

Each log line contains a timestamp and username—fields you commonly want to search or filter by when investigating incidents. These lines are the key artifacts Bob will centralize with the EFK stack.

Where do those lines go right now?

1. The app prints them to its standard output, so `kubectl logs` can read them.
2. The app also writes them to a folder on the kube node's filesystem.

In this cluster, the app mounts a directory from the node, `/var/log/webapp`, into the container using a hostPath volume. Later, when Bob deploys a collector (Fluentd), that hostPath is exactly where the collector will read log files.

```yaml theme={null}
# relevant excerpt from the app manifest
- name: app
  image: kodekloud/event-simulator
  volumeMounts:
    - mountPath: /var/log/webapp
      name: log-volume

# Volume to store the logs
volumes:
  - name: log-volume
    hostPath:
      path: /var/log/webapp
      type: DirectoryOrCreate
```

HostPath volumes expose a directory from the Kubernetes node into a container. They are useful for simple demos and for collectors that need node-local files, but they tie data to the node and should be used with care in production.

Right now, Bob's logs are tied to a single pod on a single node. If Kubernetes replaces the `app` pod (or the node fails), the pod-local stdout history and any node-local files may be lost. The EFK stack keeps logs beyond a pod's lifetime by collecting, aggregating, and persisting them to a separate storage layer.

Quick questions about the stack Bob will build:

![A dark teal slide titled "Quick Questions" showing three circular icons labeled Elasticsearch, Fluentd, and Kibana, with two question cards labeled Q1 and Q2 below. The KodeKloud logo is visible in the bottom-left corner.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/quick-questions-elasticsearch-fluentd-kibana.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=f334adbf4cc97ceb943fd9acd009880a)

* Which Kubernetes object ensures a set of pods is kept running and can be scaled? A Deployment.
* What component gathers log lines from many sources? Fluentd.
* Where are logs stored and queried? Elasticsearch.
* What provides a web UI for searching and visualizing logs? Kibana.

Deployments: the object that keeps copies running
A Deployment declares the desired number of replicas; Kubernetes ensures that many pod instances exist and restarts them if needed. Kibana will be run as a Deployment in this lesson so it can be scaled and automatically recreated.

![A dark-themed diagram titled "A Deployment Keeps Copies" showing a "Deployment" node with an arrow pointing down to two smiling blue rounded icons (representing copies) on a platform. A small "Q1" box with a green checkmark appears on the left.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/deployment-keeps-two-copies-q1-check.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=c2bb79046e2eb119d47e6c152bce38b7)

Fluentd: log aggregation and buffering
Fluentd aggregates log lines from multiple sources into a single stream and can filter/transform them via plugins. Its configuration uses `.conf` files (not YAML); plugins and buffering settings are defined there so Fluentd can hold data temporarily if Elasticsearch is unavailable.

![A dark teal presentation slide titled "Fluentd Gathers Lines" featuring the Fluentd logo (a stylized blue bird) and the label "Fluentd" in the center. The phrase "Log aggregation" appears below, with a small KodeKloud logo in the bottom-left.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/fluentd-gathers-lines-log-aggregation-slide.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=df201b1d62f89c73f736e3d4426371ed)

![A presentation slide titled "Settings, Plugins, a Buffer" showing the Fluentd logo and two document icons: one labeled "fluent.conf" (highlighted) and another labeled "fluent.yaml" crossed out. It indicates using a .conf configuration for Fluentd instead of a .yaml file.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/fluentd-conf-not-yaml.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=1b391c7131de183044a0da5ffb69ab46)

Elasticsearch: document store, shards, and replicas
Elasticsearch stores each log line as a document (a flexible JSON-like structure). To provide durability and scalability it replicates documents across nodes and splits indices into shards so searches run in parallel. A master node tracks cluster metadata, creates indices, and decides shard placement.

![An illustration titled "One Line, One Document" showing a smiling blue chatbot icon with a "USER1 logged in" message on the left and a partially filled cylindrical database on the right.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/one-line-document-chatbot-user1-database.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=4f735c45f881187887467c50a7ae22a5)

![An infographic titled "Copies and Shards" showing three smiling blue robot icons, each paired with a stacked-disk database labeled 1, 2, and 3 on separate platform bases. It visually represents database copies/shards or replicas.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/copies-shards-robots-databases.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=b226ed14efa1224c1edfe5344e1d1809)

![A diagram titled "The Master Node" showing three server nodes represented by smiling blue mascots next to database icons labeled 1–3. The left node is marked "master" with arrows connecting it to the other two nodes.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/master-node-mascots-databases-replicas-diagram.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=2abce5910bd0d281d83404cafdc4b618)

Why Bob needs centralized logging
All of this brings Bob back to his single busy app and its ephemeral logs. Two practical questions remain:

* If the app pod were replaced right now, where would the "USER1 logged in" line go?
* Where should log data live so it survives pod restarts and node failures?

![A dark teal slide titled "Two Questions" showing a glowing blue robot-like icon on a platform and a notification that "USER1 logged in." A circular user avatar labeled "Bob" with a sad face is shown at the lower right and a KodeKloud logo appears in the bottom-left.](https://mintcdn.com/kodekloud-c4ac6d9a/YjasonRdhEsLdkKY/images/Learn-By-Doing-Deploying-and-Managing-the-EFK-Stack-on-Kubernetes/Introduction-to-EFK-on-Kubernetes/Hands-On-EFK-Introduction/two-questions-robot-user1-bob-sad.jpg?fit=max\&auto=format\&n=YjasonRdhEsLdkKY\&q=85\&s=ca552850459a5a141a211d0062547df4)

Components at a glance

| Component | Role | Notes / Example |
| - | - | - |
| Deployment | Ensures a set number of pod replicas are running | Use for Kibana or other stateless apps |
| Fluentd (collector) | Aggregates and forwards logs; can filter and buffer | Configured with `.conf` and plugins |
| Elasticsearch (store) | Indexes and stores log documents; supports shards & replicas | Scales by adding nodes; master manages metadata |
| Kibana (visualize) | UI for searching, filtering, and visualizing logs | Run as a Deployment for high availability |

Further reading and references

* Kubernetes docs — [https://kubernetes.io/docs/](https://kubernetes.io/docs/)
* Fluentd — [https://www.fluentd.org/](https://www.fluentd.org/)
* Elasticsearch — [https://www.elastic.co/elasticsearch/](https://www.elastic.co/elasticsearch/)
* Kibana — [https://www.elastic.co/kibana/](https://www.elastic.co/kibana/)

HostPath volumes are convenient for demos but are node-local. For production logging, prefer solutions that persist logs in centralized storage (e.g., Elasticsearch backed by persistent volumes or external storage) so data survives node failures and pod rescheduling.

> [card] [Watch Video](https://learn.kodekloud.com/user/courses/learn-by-doing-deploying-and-managing-the-efk-stack-on-kubernetes/module/1d399ab4-bb09-4d4a-832f-f8689597ab8c/lesson/025433f8-2aa4-4cb8-bd0a-8cf296303583)

> [card] [Practice Lab](https://learn.kodekloud.com/user/courses/learn-by-doing-deploying-and-managing-the-efk-stack-on-kubernetes/module/1d399ab4-bb09-4d4a-832f-f8689597ab8c/lesson/247ed6e4-d5a2-4ca3-ac01-588f55334fec)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.