Skip to main content
Kubernetes networking lets your workloads (Pods) communicate with each other, with external services, and with users. Because every Pod gets a unique IP, application design is simplified — Pods can talk to each other without SNAT — but troubleshooting requires checking several layers: the application Pod, Service abstraction, DNS, CNI, and kube-proxy. This guide explains the core components to inspect and a practical, ordered troubleshooting sequence to follow when service connectivity breaks.

​
Core components at a glance

Start troubleshooting at the application Pod and work outward: Pod health → direct Pod connectivity → Service definition/endpoints → DNS → CNI → kube-proxy.

​
1) Check Pod health

A Service only sends traffic to Pods that are Ready. Begin by verifying the Pods backing the Service are Running and Ready. Example:
Sample output:
What to look for:
  • READY 1/1 — all containers are ready.
  • STATUS should be Running.
  • RESTARTS > 0 — may indicate container instability.
  • AGE — how long the Pod has been running.
If Pods are not healthy, inspect logs and Pod details:

​
2) Reach Pods directly (bypass the Service)

If Pods appear healthy, test direct connectivity to confirm the application is listening on the expected port. This removes the Service layer from the equation. Get Pod IPs:
Sample output:
Or pull Pod IPs using jsonpath:
From a temporary pod (BusyBox or similar) curl/wget the Pod IPs:
Interpretation:
  • If direct requests succeed, the app is serving traffic; investigate Service, kube-proxy, or DNS next.
  • If direct requests fail, the issue is likely the application itself or Pod-level networking (CNI).

​
3) Validate Service definition and endpoints

A Service must select the correct Pods and map port → targetPort correctly. Inspect the Service:
Typical Service spec snippet:
Check the Endpoints or EndpointSlices that the control plane exposes:
Sample output:
If the ENDPOINTS column is empty:
  • The Service selector is not matching any Pods — re-check Pod labels vs Service selector.
  • The targeted port may be incorrect — ensure targetPort matches the container port.

​
4) CoreDNS (cluster DNS)

CoreDNS resolves service names to ClusterIPs and Pod names to IPs. DNS failures prevent services from being reachable by name. CoreDNS runs in the kube-system namespace and is exposed via the kube-dns Service. Key resources: CoreDNS Deployment, Corefile (ConfigMap), Service, and RBAC.
A labeled Kubernetes architecture diagram titled "Debugging Service Issues: Check Service Definition and Endpoints," showing components like kube-system, ConfigMap, CoreDNS pods, Services, Control Plane, Worker Node, kube-proxy and client/worker pods. It visualizes how services, endpoints and DNS (kube-dns/CoreDNS) interact for debugging service connectivity.
Corefile example:
DNS troubleshooting checklist:
  1. Check CoreDNS Pod status:
  • Pending Pods may indicate CNI issues.
  • CrashLoopBackOff — inspect logs:
  1. Verify kube-dns endpoints (EndpointSlices):
If no endpoints are present, the Service selector might not match CoreDNS Pods — check labels and selectors.
  1. Inspect /etc/resolv.conf inside a Pod:
  • nameserver should be the kube-dns Service ClusterIP.
  • search manages short-name resolution.
  • ndots influences how unqualified names are searched.
  1. Test DNS resolution from a Pod:
If these fail, focus on CoreDNS (Deployment, Corefile, Service) and the network path to CoreDNS Pods.

​
5) CNI plugins

CNI plugins assign Pod IPs and set up Pod-to-Pod networking. A broken or misconfigured CNI commonly results in Pods stuck in Pending, or Pods running but lacking network connectivity. Popular CNIs:
  • Flannel — simple overlay (no default NetworkPolicy enforcement). Install example:
  • Calico — supports NetworkPolicy, can run with or without overlay. Install by applying the Calico manifest from the project.
Important: kubelet uses the first CNI config file it finds alphabetically in the CNI config directory. Multiple CNI config files can cause unexpected plugin selection.
If multiple CNI config files exist, remove or rename the unwanted ones so the intended plugin is the alphabetically first file.
CNI troubleshooting steps:
  • Ensure the CNI DaemonSet Pods are Running (namespace depends on plugin, e.g., kube-flannel, calico-system).
  • Check the CNI Pods’ logs for allocation errors.
  • If Pods remain Pending, inspect kubelet logs and CNI plugin logs for network allocation failures.

​
6) kube-proxy

kube-proxy runs on every node and programs packet-forwarding rules (iptables or IPVS) so Services reach Pod backends. It watches Services and Endpoints and keeps the node’s rules in sync.
An infographic titled "Troubleshooting CNI and kube-proxy" showing two panels: one for CNI troubleshooting (check DaemonSet pod status, analyze CNI pod logs) and one for kube-proxy rules (service-to-pod proxying). A warning triangle labeled "Network Issue" is shown beneath the panels.
kube-proxy troubleshooting steps:
  1. Check kube-proxy Pods:
  1. Inspect kube-proxy logs:
  1. Check kube-proxy configuration:
Example snippet:
  1. Verify rules on the node:
  • For iptables mode: SSH to the node and inspect iptables rules.
  • For ipvs mode: use ipvsadm -Ln to list IPVS virtual servers and destinations to confirm that Service VIPs map to Pod backends.
  1. Pod health: kubectl get pods → logs and describe if unhealthy.
  2. Direct Pod reachability: curl/wget Pod IPs from a debug Pod.
  3. Service validation: verify Service YAML (selector, port, targetPort). Check EndpointSlices/Endpoints.
  4. DNS: check CoreDNS Pods, Corefile, Service, and test resolution from a Pod.
  5. CNI: ensure CNI DaemonSet Pods are running and logs show successful Pod IP allocation.
  6. kube-proxy: check Pod status, logs, ConfigMap, and node-level iptables/IPVS rules.
Following this ordered sequence narrows the root cause quickly — whether it’s an application issue, Service misconfiguration, DNS problem, CNI allocation failure, or kube-proxy rule mismatch.

Watch Video

Practice Lab