Skip to main content
In this lesson we’ll learn how to monitor both applications and the Kubernetes cluster itself using Prometheus deployed inside Kubernetes. The two complementary goals are:
  • Monitor application-level metrics from services and web servers running in pods.
  • Monitor cluster-level metrics for control plane components, nodes, and Kubernetes object state.
Running Prometheus inside the cluster is generally recommended because it:
  • Places Prometheus close to scrape targets (lower latency, fewer network hurdles).
  • Reuses existing Kubernetes infrastructure (no separate VM required).
  • Lets you use Kubernetes-native deployment patterns (Helm charts, Operators, ServiceDiscovery).
A diagram of a Kubernetes cluster showing the control plane (API/controllers) on the left and multiple worker nodes with kubelet and pods on the right. Labels note deploying close to targets and using pre-existing Kubernetes infrastructure.
When monitoring Kubernetes with Prometheus we generally collect two categories of metrics:
  1. Application-level metrics — custom and framework metrics emitted by applications running inside pods.
  2. Cluster-level metrics — metrics about the Kubernetes control plane, nodes, and Kubernetes objects.
Important cluster-level components to expose metrics:
  • Control plane components (API server, kube-scheduler, CoreDNS, etcd, etc.).
  • kubelet / cAdvisor — provides container- and node-level metrics.
  • kube-state-metrics — emits metrics about Kubernetes objects (Deployments, Pods, Services, etc.). Kubernetes does not expose these object-level metrics by default.
  • node_exporter — exposes OS-level metrics (CPU, memory, disk, network) from each node.
A presentation slide titled "Kubernetes" that lists monitoring tasks for applications and cluster components. It names tools/components like the control-plane (api-server, coredns, kube-scheduler), kubelet/cAdvisor, kube-state-metrics, and node-exporter.
kube-state-metrics kube-state-metrics is a small service that listens to the Kubernetes API and translates object state into Prometheus metrics (for example, kube_deployment_status_replicas_available or kube_pod_status_phase). Deploy kube-state-metrics in your cluster to enable scraping of deployment/pod/service counts and status information.
A dark-themed slide titled "Kube-state-metrics" stating that the kube-state-metrics container must be deployed to collect cluster-level metrics. It includes a diagram showing a Kubernetes Cluster box with a smaller "Kube-state-metrics" component inside.
node_exporter Each node should run a node_exporter to expose OS-level metrics such as CPU, memory, disk and network usage. In Kubernetes, the recommended way to ensure one exporter per node is a DaemonSet because it automatically schedules the exporter pod on every node (including new nodes). Common options to run node_exporter:
  • Manual install on each host (not scalable).
  • Bake into the node image (possible but inflexible).
  • Use a Kubernetes DaemonSet (recommended).
A presentation slide titled "Node Exporter" explaining that each host should run a node_exporter to expose CPU, memory and network stats and recommending using a Kubernetes DaemonSet to run the exporter on every node. A diagram below shows a Kubernetes cluster containing three nodes.
Service discovery in Kubernetes Prometheus discovers scrape targets in Kubernetes by querying the Kubernetes API. It can find pods, services, endpoints, nodes (node exporters), and resources declared by the Prometheus Operator such as ServiceMonitor and PodMonitor. This dynamic discovery removes the need to hard-code target addresses.
A diagram titled "Service Discovery" showing Prometheus (center) connecting to the Kubernetes API on the left and to kube components, node exporters, and kube-state-metrics on the right. Arrows indicate Prometheus uses service discovery to find those targets.
Key cluster monitoring components — quick reference How to deploy Prometheus in Kubernetes You have two main approaches: Manual deployment requires creating all Kubernetes objects (Deployments/StatefulSets, Services, ConfigMaps, Secrets, RBAC, etc.), which can be educational but is operationally heavy.
A presentation slide titled "How to deploy" explaining manual deployment of Prometheus on Kubernetes, saying you must create deployments, services, configMaps, and secrets. It notes this is complex and requires a lot of configuration, with icons for Deployments, Services, and Secrets.
Helm + Prometheus Operator (recommended) The Prometheus community Helm chart kube-prometheus-stack installs Prometheus, Alertmanager, node-exporter, kube-state-metrics, Pushgateway, Grafana, and the Prometheus Operator — all configured and wired together. Helm is a Kubernetes package manager that installs versioned, templated charts. To install the community stack:
A presentation slide titled "Helm Charts" explaining that a helm chart is a collection of template and YAML files that convert into Kubernetes manifests. It also shows a GitHub link and repository details for the kube-Prometheus-stack.
The Prometheus Operator A Kubernetes Operator is a controller that extends the Kubernetes API to manage complex applications. The Prometheus Operator provides CRDs and controllers to simplify running Prometheus and Alertmanager in Kubernetes. With the Operator you gain higher-level resources:
  • Prometheus — CRD representing a Prometheus deployment.
  • Alertmanager — CRD for Alertmanager instances.
  • PrometheusRule — CRD for alerting and recording rules.
  • ServiceMonitor and PodMonitor — CRDs to declare scrape targets in a Kubernetes-native way.
Example Prometheus CRD (created by the Operator):
ServiceMonitor and PodMonitor CRDs let you declare which services or pods Prometheus should scrape. The Operator converts those CRDs into Prometheus scrape configs automatically, so you don’t need to edit Prometheus config files directly.
Running Prometheus inside Kubernetes (for example with the kube-prometheus-stack Helm chart and the Prometheus Operator) is recommended for most clusters. It reduces manual configuration and uses Kubernetes-native mechanisms such as DaemonSets, ServiceMonitor/PodMonitor, and RBAC.
When installing with Helm/Operator, review RBAC and resource limits before deploying to production. The stack creates cluster-level resources and strong permissions — apply least privilege and test upgrades in a staging cluster first.
Links and references

Watch Video