Skip to main content
Hello — in this lesson we’ll reproduce the debugging approach Elena used when she was paged at 2 a.m. The incident points to a payment gateway outage. This guide walks through fast, practical checks you can run to determine whether the problem is Kubernetes-level (pods, services, probes) or application-level (runtime errors, misconfiguration).
  1. Confirm cluster access
  • Before doing anything, verify you have the correct access to the production Kubernetes cluster. Access to production clusters typically requires an approval process; do not bypass company policy.
Accessing production clusters usually requires explicit approvals and auditing. Ensure you have the right permissions and follow your organization’s change and access control policies before running commands.
Run a quick command to confirm you can reach the cluster and list namespaces:
Example output:
Note: the payment namespace appears in the output — this aligns with the alert about the payment service/gateway being down. That makes payment the obvious place to continue your investigation.
  1. Inspect resources in the payment namespace List all workloads, services, and replica sets in the payment namespace:
Example output:
Observations from the output:
  • Several resources exist for payment: pods, services, deployments, replica sets.
  • All pods shown are in Running state, but Running does not guarantee the application is healthy — further checks are required.
  • Note a deployment payment-gateway-api with 0/0 ready pods and a corresponding replica set with 0/0 — this likely indicates a scaling or scheduling issue for that component.
Quick reference: what each resource type tells you
  1. Inspect how the application is exposed To confirm whether the service is reachable within the cluster, list services in payment:
Example output:
Notes:
  • These are ClusterIP services, meaning they are reachable only from within the cluster network (unless there is an Ingress or LoadBalancer in front).
  • A ClusterIP exists and points to a cluster-internal IP, but that does not guarantee the backend pods respond to requests on their ports.
  1. Recommended next debugging steps Choose one or more of the following, depending on whether you can access logs or need to test endpoints locally.
  • Tail pod logs to look for runtime errors or stack traces:
  • Inspect pod events and probe statuses to detect readiness/liveness probe failures or scheduling problems:
  • Port-forward a service to your local machine to test HTTP endpoints if the service is ClusterIP:
  • If the payment-gateway-api deployment shows 0 desired/available replicas, check its deployment spec and scheduler events:
Fast triage pattern: confirm cluster access → identify the relevant namespace → combine pod-level checks (logs, describe) with service-level checks (port-forward or internal probes). This helps you determine whether the issue is network/configuration-related (service, probes, scaling) or application-related (exceptions, crashes).
Links and references

Watch Video