elastic-stack. Two pods already run there: an app pod producing user events, and elasticsearch-0 (empty for now). All resources we create will live in the elastic-stack namespace.
Create a local workspace for Fluentd configuration:
fluentd ServiceAccount in the elastic-stack namespace with the label app: fluentd.

pods and namespaces across the cluster.
Apply the ClusterRole:
ServiceAccounts identify pods to the API server. ClusterRoles grant permissions cluster-wide; ClusterRoleBindings attach those roles to ServiceAccounts (or users/groups). Use the minimal set of verbs and resources necessary to reduce blast radius.
etc directory under /root/fluentd because the DaemonSet will mount this folder into each Fluentd pod.
- Source — tail container log files under
/var/log/containers/*.log.- A
pos_filekeeps read offsets so Fluentd resumes correctly after restarts. - Exclude Fluentd’s own logs to avoid self-collection.
read_from_head trueensures Fluentd reads existing files the first time it runs.
- A
- Parse — the
<parse>block above converts each raw JSON log line into fields such astime,stream, andlog. A typical parsed log record looks like:
- Match — send parsed records to Elasticsearch.
logstash_format trueandlogstash_prefix "fluentd"write documents to indices likefluentd-YYYY.MM.DD.- The
<buffer>section accumulates records and flushes them every 5 seconds. If Elasticsearch is unavailable,retry_forever trueensures records remain buffered until delivery succeeds.
- Namespace:
elastic-stack - Uses the
fluentdServiceAccount - Fluentd image built for Elasticsearch 7
- Environment variables control Elasticsearch connection and index prefix
- Toleration to allow running on control-plane nodes if necessary
- HostPath mounts:
/var/log,/var/lib/docker/containers(read-only), and the local/root/fluentd/etcmounted into the pod at/fluentd/etc - Memory limit:
512Mi
Using
hostPath mounts exposes node filesystem contents to the pod. Ensure you trust the container image and limit permissions (e.g., use read-only mounts where possible). Apply least-privilege principles for RBAC and pod security.
- Query the index pattern
fluentd-*in Kibana or via curl to confirm documents exist. - Inspect a sample document for expected
log,stream, andkubernetes.*metadata. - Verify timestamps align with the application events.
- Kubernetes RBAC: https://kubernetes.io/docs/reference/access-authn-authz/rbac/
- Fluentd Kubernetes integration: https://docs.fluentd.org/
- Elasticsearch HTTP API: https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html