Demo Verifying the PaymentUI

Hello and welcome back. Earlier we began debugging a Kubernetes incident with Alina where the payment gateway was failing. In this lesson we’ll verify whether the Payment UI is reachable and confirm the user-facing error so we can trace the failure to the underlying infrastructure. First, ensure you can talk to the cluster. A quick way to sanity-check cluster access is:

# If using k3s:
k3s kubectl get nodes

# Or using the cluster's kubectl:
kubectl get nodes

What error do users see? The incident report says the payment gateway is down — let’s validate that from the frontend.

In the KodeKloud lab UI, open the playground container menu (three dots) → View Port → open the frontend port.
I recommend opening the resulting URL in an incognito/private window to avoid cached sessions or credentials affecting the test.
When the page loads, look for an incident banner and the service health box for the Payment API.

You should see a banner confirming an incident has been detected, which tells us the frontend static hosting is reachable. The service health widget, however, should show the Payment API as down — explaining the failed payments.

A dark-themed checkout screenshot for "AirPay" showing an order summary and card payment form on the right. A red incident banner and a service health box indicate the Payment API is down and the payment processing service is unreachable.

Reproduce the failure by entering dummy card details and clicking Pay Securely. The browser console or network tab will show the frontend failing to POST to the payment API, returning a 503:

Error: fetch POST /api/charges 503 Service Unavailable - upstream: payment-gateway-api:8080 - No healthy upstream / connection refused

This message means the frontend could not reach any healthy backend for the payment-gateway-api service (connection refused / no healthy upstream). That points to an infrastructure-level issue: the frontend cannot connect to healthy backend endpoints.

You are acting as an SRE/DevOps engineer during this lesson. Focus on identifying and fixing the infrastructure/service issue (pods, services, endpoints, readiness/liveness), rather than changing application code.

Quick checklist — what to inspect next:

Resource Type	Why check it	Example command
Pods	Are payment pods running / crashlooping / failing readiness?	`kubectl get pods -n payment`
Services	Is the Service targeting the right selector / port?	`kubectl get svc -n payment`
Endpoints	Does the Service have endpoints (IP:port) backing it?	`kubectl get endpoints -n payment`
Pod details	Describe pod to see events, readiness/liveness failures	`kubectl describe pod <pod-name> -n payment`
Logs	Check container logs for startup or runtime errors	`kubectl logs <pod-name> -n payment`
Deployment/ReplicaSet	Are replicas scheduled and available?	`kubectl get deploy,rs -n payment`
Network (optional)	Check network policies or DNS issues	`kubectl exec -it <pod> -n payment -- curl -sv payment-gateway-api:8080`

A typical workflow:

kubectl get pods -n payment — note pod status (Pending, CrashLoopBackOff, Running).
If pods are not Ready or are CrashLooping, kubectl describe pod <pod> -n payment and kubectl logs <pod> -n payment to find errors.
kubectl get svc -n payment and kubectl get endpoints -n payment — confirm the service has endpoint IPs and correct ports.
If Service has no endpoints, check pod labels and Service selectors or readiness probes that may be preventing endpoints from being considered healthy.
Inspect readiness/liveness probes in the Deployment spec and adjust if misconfigured.

Next step: explore the payment namespace (pods, services, endpoints) and identify any misconfiguration, failing pods, or missing endpoints that cause the 503. Resolve the root cause (for example, fix readiness probe, scale pods, correct service selector) and then re-check the frontend to confirm successful payments. Links and references

That’s it for this lesson — proceed to inspect the payment namespace resources and determine where the failure originates.

Introduction

The Problem Statement

Introduction to FinOps

Setting Up SkyLine Air Kubernetes Services

MCP Gateway for Kubernetes

LangChain MCP Agents

LLM Policy-Aware Label Audit

From Findings to Actionable Tickets

Demo Verifying the PaymentUI

Watch Video

Practice Lab