Skip to main content
This guide walks through diagnosing and repairing a worker node that becomes NotReady by troubleshooting from the control plane, then inspecting and fixing kubelet-related issues on the affected worker node (node01). Follow the steps in order: identify the failing node from the control plane, then SSH to the worker and examine the kubelet service, logs, and configuration.
Troubleshooting order: start on the control plane with kubectl to find the failing node, then SSH into the worker node to check the kubelet service and its logs. The kubelet is the primary agent that keeps nodes registered and reporting to the control plane.

1) Set a short alias for kubectl

On the control plane node, create an alias to speed up commands during the session:
Use k for the rest of the session.

2) Inspect nodes from the control plane

List cluster nodes and their status:
Example output:
Since node01 is NotReady, SSH into that node to inspect the kubelet and related configuration.

3) Check kubelet systemd service on the worker

SSH to the worker and inspect the kubelet service:
If the kubelet is inactive (dead), you’ll see output like:
Start and re-check the kubelet:
After starting, expected status:
Return to the control plane and verify the node becomes Ready:

4) Intermittent failure — kubelet stuck in auto-restart (exit code)

If the node later regresses and the kubelet is repeatedly restarting, systemd may show an auto-restart state:
Collect recent kubelet logs to see why it fails early:
Look for fatal errors about loading config or CA files:
These errors point to the kubelet configuration referencing a missing or incorrect client CA file.

Inspect kubelet configuration files

Kubelet reads two key files:
  • /etc/kubernetes/kubelet.conf — kubeconfig that kubelet uses to authenticate to the API server
  • /var/lib/kubelet/config.yaml — KubeletConfiguration used by the kubelet process
List and inspect them:
Excerpt showing the problematic setting:
The authentication.x509.clientCAFile points to /etc/kubernetes/pki/WRONG-CA-FILE.crt, which does not exist and prevents the kubelet from starting.
Before editing kubelet config files, make a backup: cp /var/lib/kubelet/config.yaml /var/lib/kubelet/config.yaml.bak. Editing kubelet parameters can prevent the agent from starting—ensure you have console access to the node.

Fix the clientCAFile path and restart kubelet

Edit /var/lib/kubelet/config.yaml and correct the CA path to the cluster CA (typically /etc/kubernetes/pki/ca.crt):
Save the file, then restart kubelet:
After kubelet starts successfully, check the node status from the control plane:
You should see node01 return to Ready.

5) Recurrence — kubelet running but cannot register with API server

If service kubelet status shows active (running) yet the node is NotReady, check the kubelet logs for connection or registration errors:
These logs show kubelet is trying to reach the API server on port 6553 — the typical secure API port is 6443. This incorrect endpoint is usually defined in the kubelet kubeconfig.

Inspect the kubelet kubeconfig

Open the kubeconfig used by kubelet:
Excerpt showing the wrong server address:

Fix the API server port and restart kubelet

Edit /etc/kubernetes/kubelet.conf and change the server URL to the correct API server port (usually 6443):
Save, then restart kubelet:
Confirm kubelet logs show successful registration, and verify node status from the control plane:
Expected result:

6) Summary: troubleshooting checklist for worker nodes

Follow this checklist when a worker node is NotReady: Key things to watch in kubelet logs:
  • Errors loading config or client CA files (fix paths in /var/lib/kubelet/config.yaml)
  • TLS/client auth failures (verify cert paths in /etc/kubernetes/kubelet.conf and /var/lib/kubelet/pki/)
  • Connection refused or wrong port (ensure server points to :6443 or your API server address)
Useful references: That completes the lab. Start troubleshooting from the control plane to locate the NotReady node, then inspect the kubelet service, logs, and configuration on the worker to restore node readiness.

Watch Video