Skip to main content
Welcome back. In the previous lesson we looked at the payment gateway UI and observed the error users were seeing. That user perspective is important during an incident. In this lesson we’ll start debugging the cluster-side components behind the UI by inspecting the payment namespace—pods, Deployments, and related resources—to find anything suspicious. Goal: confirm whether the API backend for the gateway is running, and if not, why.

​
1) Inspect pods in the payment namespace

List pods to see which application components have running pods:
Example output:
What to look for
  • Confirm presence of gateway UI and processor pods.
  • Check whether the API pod (the backend) is present. In this example it is missing; that’s a red flag.

​
2) Check Deployments for desired state

Deployments reveal the desired replica counts and whether each service is available. Run:
Example output:
Observation: payment-gateway-api shows 0/0 — no replicas desired and none available. The gateway UI and processor are running, but the API Deployment is scaled to zero. That explains gateway failures: there are no API pods to serve requests.

​
3) Describe the Deployment for details and events

Run a describe to inspect configuration, pod template, resource requests/limits, and events:
Relevant excerpt (truncated for clarity):
Key detail: Replicas: 0 desired — the Deployment is intentionally configured to have zero replicas.

​
Why would replicas be 0?

Common explanations:
  • Someone manually scaled the Deployment down: kubectl scale deployment payment-gateway-api --replicas=0.
  • A GitOps/CI change set the replica count to 0 in the declared manifest and the reconciler applied it.
  • Automation or operational policy temporarily scaled it down (less common).
  • A troubleshooting step was performed and not reverted.

​
Quick reference — useful kubectl commands

​
Investigative next steps

  1. Determine what changed the Deployment:
    • If API server audit logs are enabled, search for scale/update events targeting this Deployment.
    • If you run GitOps (ArgoCD or Flux), inspect the repo and recent commits that modify the Deployment manifest.
  2. Inspect rollout history and last change:
    • kubectl rollout history deployment/payment-gateway-api -n payment
  3. Confirm the declared desired replicas in the live manifest:
    • kubectl get deployment payment-gateway-api -n payment -o yaml
  4. Inspect ReplicaSets and events for clues:
    • kubectl get rs -n payment --selector=app=payment-gateway-api
    • kubectl get events -n payment --sort-by=.metadata.creationTimestamp
  5. If it should be running and you have verified it’s safe, scale it back up:
    • kubectl scale deployment payment-gateway-api --replicas=1 -n payment
If the Deployment was intentionally set to 0 (for maintenance or a planned outage), coordinate with the owning team or automation before scaling it up. If you can’t identify an owner, check your GitOps repo, CI/CD pipeline, or API audit logs for recent changes.

​
Wrap-up

From the pod and Deployment checks we can explain the missing API: payment-gateway-api has 0 desired replicas. To restore the service you must first identify whether the scale-down was intentional and by whom. If it was accidental or unauthorized, either:
  • Revert the change in the upstream manifest (GitOps), or
  • Scale the Deployment back up with kubectl scale, after confirming it’s safe to do so.
In the next lesson we’ll cover methods to discover the Deployment owner (audit logs, GitOps tools like ArgoCD/Flux, and ownership hints) and safe approaches to restore the desired state.

Watch Video