Demo Configure cloud instances Kubernetes

In this guide, you’ll learn how to integrate a Kubernetes cluster with Jenkins to provision dynamic build agents. We’ll cover plugin installation, cloud setup using both a full-admin kubeconfig and a least-privileged service account, and finalize key settings like pod retention and labels.

​

1. Install the Kubernetes Plugin

Jenkins requires the Kubernetes plugin to spin up agents in your cluster. If dependency errors occur (e.g., an out-of-date Credentials plugin), update those first and restart Jenkins:

​

2. Configure the Kubernetes Cloud

1. Go to Manage Jenkins → Manage Nodes and Clouds → Configure Clouds.
2. Click Add a new cloud and select Kubernetes.
3. Provide a name (e.g., prod-k8s-us-east).

You’ll see the Kubernetes cloud configuration form:

​

A. Connect Using a Kubeconfig File

1. Export your full kubeconfig:
   kubectl config view --raw > kubeconfig.yaml
   
2. In Jenkins, add a Secret file credential (kubeconfig-us-east) and upload kubeconfig.yaml.
3. Select this credential under Kubernetes Credentials and click Test Connection.

A successful connection shows your cluster version:

​

B. Connect Using a Service Account Token

Follow these steps to lock down permissions:

1. Create a namespace and service account:
   kubectl create namespace jenkins
   kubectl -n jenkins create serviceaccount jenkins-sa
   
2. Generate a long-lived token:
   kubectl -n jenkins create token jenkins-sa --duration=115d
   
3. In Jenkins, add a Secret text credential (jenkins-sa-token) with this token.
4. Back in the Kubernetes cloud config:
   • Kubernetes URL: your API server endpoint
   • Namespace: jenkins
   • Credentials: jenkins-sa-token
   • Click Test Connection.

Initially, you may encounter a certificate path error: You may then hit a 403 Forbidden error due to missing RBAC permissions:

5. Grant namespace-scoped admin rights:
   kubectl -n jenkins create rolebinding jenkins-admin-binding \
     --clusterrole=admin \
     --serviceaccount=jenkins:jenkins-sa
   
6. In Jenkins, click Test Connection again. You should see the credentials dropdown populated and a successful response:

​

3. Finalize Cloud Settings

Configure how Jenkins launches and cleans up pods:

• Jenkins URL / Jenkins tunnel: Host:port for JNLP/WebSocket agent connections.
• Pod Labels: e.g., organization=KodeKloudAzureArc—tags applied to every agent pod.
• Pod Retention:

Click Save. Jenkins will now provision build agents dynamically in your Kubernetes cluster!

---

​

References

• Jenkins Configuration as Code
• Kubernetes Credentials Plugin
• Kubernetes RBAC Overview