Skip to main content
This section teaches practical Kubernetes troubleshooting techniques you can apply to real-world clusters. The content is organized to move from application-level issues to cluster-level problems:
  • Diagnosing and fixing application failures (Pods, Deployments, and StatefulSets)
  • Troubleshooting control plane issues (API server, etcd, scheduler, controller-manager)
  • Resolving worker node problems (kubelet, container runtime, resource pressure)
  • Investigating network-related incidents (Services, NetworkPolicies, CNI plugins)
Each lesson emphasizes hands-on, pragmatic troubleshooting using common tools such as kubectl, log inspection, and cluster-level checks. Expect step-by-step labs that present realistic failure scenarios, reproducible steps (when applicable), and guided remediation exercises.
Before starting the labs, ensure you have access to a Kubernetes cluster and kubectl configured for that cluster. Familiarity with core Kubernetes concepts will help you get the most from these exercises.

​
How this section is organized

We follow a consistent troubleshooting workflow so you can approach problems methodically:
  1. Observe: Gather symptoms (pod status, events, metrics, logs).
  2. Hypothesize: Formulate likely causes (resource exhaustion, misconfiguration, network partition).
  3. Test: Reproduce or run focused checks (describe, logs, exec, health endpoints).
  4. Remediate: Apply fixes or workarounds (resource adjustments, restart controllers, rollbacks).
  5. Verify: Confirm system returns to expected state.
Use the checks and commands below as a starting point while working through each lab.

​
Labs overview

The section contains multiple labs focused on realistic failure modes. Each lab includes:
  • A scenario and symptoms to analyze
  • Steps to reproduce (where appropriate)
  • Guided exercises to diagnose and fix the issue
  • Suggested verification steps and follow-up hardening

​
Quick reference: common kubectl commands

Use these commands frequently while troubleshooting:
  • Inspect resources:
    • kubectl get pods --all-namespaces
    • kubectl get events --all-namespaces
    • kubectl get nodes -o wide
  • Describe and inspect:
    • kubectl describe pod <pod-name> -n <namespace>
    • kubectl logs <pod-name> -c <container> -n <namespace>
    • kubectl exec -it <pod-name> -n <namespace> -- /bin/sh
  • Rollouts and debugging:
    • kubectl rollout status deployment/<name> -n <namespace>
    • kubectl rollout undo deployment/<name> -n <namespace>
  • Cluster and control plane checks:
    • kubectl get componentstatuses (deprecated in newer versions — check API server health endpoints)
    • kubectl get apiservices
  • Node-level inspection:
    • kubectl describe node <node-name>
    • journalctl -u kubelet -n 200 (on the node)
Tip: When capturing logs, include timestamps and context (namespace, node, pod name) so you can correlate events across components and timelines.

​
Troubleshooting checklist (step-by-step)

  1. Confirm the scope
    • Is the issue isolated to a single pod, deployment, node, or cluster-wide?
  2. Gather evidence
    • kubectl get pods -A, kubectl get events -A, kubectl top pods (metrics-server)
  3. Inspect resource status and events
    • kubectl describe to reveal scheduling, image pull, readiness, or liveness failures
  4. Collect logs
    • Application logs: kubectl logs
    • Node and system logs: journalctl, container runtime logs
  5. Test connectivity
    • Use kubectl exec and networking tools (curl, nslookup, ping) to test DNS and service reachability
  6. Apply mitigations
    • Scale replicas, restart affected pods, adjust resource requests/limits, or rollback deployments
  7. Validate and document
    • Re-run status checks and events; capture the root cause and remediation steps for postmortem
  • kubectl (core CLI)
  • kubectl plugins (kubectl-trace, konnectivity tools)
  • Logging: EFK/ELK, Fluentd/Fluent Bit
  • Monitoring: Prometheus & Grafana
  • Tracing: Jaeger or Zipkin
  • Diagnostics: tcpdump, strace (node debugging), etcdctl (etcd cluster checks)
Use the labs in this section to practice the workflow above. Each lab will guide you through realistic scenarios and build confidence in diagnosing and remediating Kubernetes failures.

Watch Video