Lab Solution Multiple Scheduler

In this lesson you’ll practice deploying an additional Kubernetes scheduler alongside the cluster’s default scheduler. Follow each step to inspect the running scheduler, create configuration and RBAC resources, deploy a second scheduler Pod, and verify a test Pod is scheduled by the new scheduler. Goal summary:

Inspect the default kube-scheduler Pod to determine the image in use.
Create a ConfigMap containing the scheduler configuration.
Deploy an additional scheduler Pod that uses the same scheduler image.
Create a test Pod that is scheduled by the new scheduler.

Step	Task	Key command / resource
1	Inspect default kube-scheduler and identify its image	`kubectl describe pod <scheduler-pod> -n kube-system \| grep Image`
2	Verify service account / RBAC for the custom scheduler	`kubectl get sa my-scheduler -n kube-system`
3	Create ConfigMap from scheduler config file	`kubectl create configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml -n kube-system`
4	Edit Pod manifest and set `image` to match the control-plane scheduler	`/root/my-scheduler.yaml` (update `image`)
5	Create the new scheduler Pod	`kubectl create -f /root/my-scheduler.yaml`
6	Create a test Pod using `schedulerName: my-scheduler`	`kubectl create -f /root/nginx-pod.yaml`

1) Inspect the default scheduler Pod and identify its image

First, list Pods in the kube-system namespace and locate the kube-scheduler Pod:

kubectl get pods -n kube-system

Identify the specific kube-scheduler Pod name from that list, then describe it to find the image in use. Example (replace the pod name if different in your cluster):

kubectl describe pod kube-scheduler-controlplane -n kube-system | grep Image

Example output (this shows the exact image string you should use for the custom scheduler):

Image:         k8s.gcr.io/kube-scheduler:v1.23.0
Image ID:      docker-pullable://k8s.gcr.io/kube-scheduler@sha256:af8166ce28baa7cb902a2c0d16da865d5d7c892fe1b41187fd4be78ec6291c23

Always use the exact image string shown by the running kube-scheduler (including the tag) when configuring your custom scheduler Pod (for example, k8s.gcr.io/kube-scheduler:v1.23.0). This ensures compatibility between scheduler binary and cluster components.

2) Verify service account / RBAC are present

The lab environment includes a service account and ClusterRoleBinding for the custom scheduler. Confirm the service account exists:

kubectl get sa my-scheduler -n kube-system

Expected output:

NAME         SECRETS   AGE
my-scheduler 1         26s

If you need to inspect or troubleshoot RBAC, list role bindings and cluster role bindings:

kubectl get clusterrolebinding | grep my-scheduler
kubectl get rolebinding -n kube-system | grep my-scheduler

If the custom scheduler lacks the necessary RBAC (ClusterRole / ClusterRoleBinding), it will be unable to communicate with the API server and schedule Pods. Ensure the service account has appropriate permissions.

3) Create a ConfigMap for the scheduler configuration

A scheduler configuration file is provided at /root/my-scheduler-config.yaml. Create a ConfigMap named my-scheduler-config in the kube-system namespace from that file:

kubectl create configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml -n kube-system

Verify the ConfigMap was created:

kubectl get configmap my-scheduler-config -n kube-system

Expected output:

NAME                  DATA   AGE
my-scheduler-config   1      17s

For reference, the scheduler configuration file might look like this:

apiVersion: kubescheduler.config.k8s.io/v1beta2
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: my-scheduler
leaderElection:
  leaderElect: false

This configuration sets a custom scheduler profile named my-scheduler. leaderElect: false is often used for single-instance scheduler testing; in production you typically enable leader election for HA setups.

4) Prepare the Pod manifest for the additional scheduler

A Pod manifest is provided at /root/my-scheduler.yaml. It contains the container command, liveness/readiness probes, the volume mount for the ConfigMap, and essential spec fields. Open the file and update the container image field to match the image discovered in step 1 (for example k8s.gcr.io/kube-scheduler:v1.23.0). Example snippet (ensure you replace the image: line with the exact image string found earlier):

kind: Pod
metadata:
  labels:
    run: my-scheduler
  name: my-scheduler
  namespace: kube-system
spec:
  serviceAccountName: my-scheduler
  containers:
  - command:
    - /usr/local/bin/kube-scheduler
    - --config=/etc/kubernetes/my-scheduler/my-scheduler-config.yaml
    image: k8s.gcr.io/kube-scheduler:v1.23.0
    livenessProbe:
      httpGet:
        path: /healthz
        port: 10259
        scheme: HTTPS
      initialDelaySeconds: 15
    name: kube-second-scheduler
    readinessProbe:
      httpGet:
        path: /healthz
        port: 10259
        scheme: HTTPS
    resources:
      requests:
        cpu: '0.1'
    securityContext:
      privileged: false
    volumeMounts:
    - name: config-volume
      mountPath: /etc/kubernetes/my-scheduler
  hostNetwork: false
  hostPID: false
  volumes:
  - name: config-volume
    configMap:
      name: my-scheduler-config

Note: do not include any angle-bracket placeholders in the image field. Use the literal image tag from step 1.

5) Create the new scheduler Pod

Apply the manifest to create the Pod in the cluster:

kubectl create -f /root/my-scheduler.yaml

Expected response:

pod/my-scheduler created

Verify the new scheduler Pod appears and becomes ready in the kube-system namespace:

kubectl get pods -n kube-system

Example output showing the custom scheduler running:

NAME                                      READY   STATUS    RESTARTS   AGE
coredns-64897985d-6qwf5                   1/1     Running   0          11m
coredns-64897985d-9rdt8                   1/1     Running   0          11m
etcd-controlplane                         1/1     Running   0          11m
kube-apiserver-controlplane               1/1     Running   0          11m
kube-controller-manager-controlplane      1/1     Running   0          11m
kube-flannel-ds-rl72v                     1/1     Running   0          11m
kube-proxy-tzwkm                          1/1     Running   0          11m
kube-scheduler-controlplane               1/1     Running   0          11m
my-scheduler                              1/1     Running   0          7s

If the Pod remains in CrashLoopBackOff or Pending, describe the Pod for diagnostics:

kubectl describe pod my-scheduler -n kube-system
kubectl logs my-scheduler -n kube-system

Look for probe failures, permission errors, or config file mounting issues.

6) Create a test Pod that uses the custom scheduler

A test Pod manifest is provided at /root/nginx-pod.yaml. By default it looks like:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx

To ensure this Pod is scheduled by your new scheduler, add the schedulerName field with the value my-scheduler:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  schedulerName: my-scheduler
  containers:
  - image: nginx
    name: nginx

Create the Pod:

kubectl create -f /root/nginx-pod.yaml

Confirm the Pod is scheduled and running:

kubectl get pod nginx

Example output when the Pod is running:

NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          10s

If the Pod is not scheduled, inspect it for events and errors:

kubectl describe pod nginx

In the Events section, look for scheduling attempts and any errors returned by my-scheduler. If no scheduling attempts are recorded, verify the custom scheduler Pod is running and communicating with the API server.

That completes the steps to add an additional scheduler to the cluster and schedule a Pod using it.

Introduction

Core Concepts

Scheduling

Logging & Monitoring

Application Lifecycle Management

Cluster Maintenance

Security

Storage

Networking

Design and Install a Kubernetes Cluster

Install "Kubernetes the kubeadm way"

Troubleshooting

Helm Basics (2025 Updates)

Kustomize Basics (2025 Updates)

Other Topics

Mock Exams

Lab Solution Multiple Scheduler

1) Inspect the default scheduler Pod and identify its image

2) Verify service account / RBAC are present

3) Create a ConfigMap for the scheduler configuration

4) Prepare the Pod manifest for the additional scheduler

5) Create the new scheduler Pod

6) Create a test Pod that uses the custom scheduler

Links and References

Watch Video

​1) Inspect the default scheduler Pod and identify its image

​2) Verify service account / RBAC are present

​3) Create a ConfigMap for the scheduler configuration

​4) Prepare the Pod manifest for the additional scheduler

​5) Create the new scheduler Pod

​6) Create a test Pod that uses the custom scheduler

​Links and References

Watch Video

1) Inspect the default scheduler Pod and identify its image

2) Verify service account / RBAC are present

3) Create a ConfigMap for the scheduler configuration

4) Prepare the Pod manifest for the additional scheduler

5) Create the new scheduler Pod

6) Create a test Pod that uses the custom scheduler

Links and References