Configuring Scheduler Profiles

​

How the scheduler selects a Node (recap)

• The Pod enters the scheduling queue and is ordered by priority.
• The scheduler filters out Nodes that cannot meet the Pod’s requirements.
• Remaining Nodes are scored according to configured scoring rules.
• The scheduler binds the Pod to the Node with the highest resulting score.

• Only Nodes with at least 10 CPUs available will be considered.
• Pods are ordered in the scheduling queue by priority; higher priority Pods are processed first.

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000
globalDefault: false
description: "High priority class for important pods."

apiVersion: v1
kind: Pod
metadata:
  name: simple-webapp-color
spec:
  priorityClassName: high-priority
  containers:
  - name: simple-webapp-color
    image: simple-webapp-color
    resources:
      requests:
        memory: "1Gi"
        cpu: "10"

​

Scheduling pipeline: phases, extension points, and example plugins

​

Notes on scoring vs filtering

• Filter plugins determine whether a Node is feasible (yes/no).
• Score plugins assign numeric scores to feasible Nodes to indicate preference; they never reject a Node.

​

Scheduling queue and plugins lifecycle

• Queue sort (order Pods): PrioritySort
• preFilter, filter, postFilter
• preScore, score
• reserve, permit
• preBind, bind, postBind

​

Examples of Pod placement constraints

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  nodeName: controlplane
  containers:
  - name: nginx
    image: nginx

# Mark a node unschedulable (cordon)
$ kubectl cordon controlplane

# Example output from `kubectl describe node` showing the node is unschedulable
$ kubectl describe node controlplane
Name:                   controlplane
Roles:                  control-plane
CreationTimestamp:      Thu, 06 Oct 2022 06:19:57 -0400
Taints:                 node.kubernetes.io/unschedulable:NoSchedule
Unschedulable:          true
Lease:

​

Configuring custom behavior: scheduler binaries vs. scheduler profiles

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
- schedulerName: my-scheduler-2
- schedulerName: my-scheduler-3
- schedulerName: my-scheduler-4
- schedulerName: default-scheduler

​

Configuring plugins per profile

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
- schedulerName: my-scheduler-2
  plugins:
    score:
      enabled:
      - name: MyCustomPluginA
      - name: MyCustomPluginB
      disabled:
      - name: TaintToleration

- schedulerName: my-scheduler-3
  plugins:
    preScore:
      disabled:
      - name: '*'
    score:
      disabled:
      - name: '*'

- schedulerName: my-scheduler-4
  # This profile can have its own plugin customizations

• Use exact plugin names or wildcard patterns (for example, '*') to disable all plugins for an extension point.
• Each extension point under plugins: accepts enabled: and disabled: lists.

​

How Pods select a profile

• A Pod is scheduled by the profile whose schedulerName matches the Pod’s spec.schedulerName.
• If a Pod does not set spec.schedulerName, it is scheduled by the cluster’s default scheduler profile (typically default-scheduler).

​

Additional references

​

Summary

• The scheduler pipeline includes queue sorting, filtering, scoring, and binding, with several extension points between them.
• Plugins are the primary mechanism for customizing scheduling behavior and can be configured at multiple hooks.
• Multi-profile kube-scheduler (since Kubernetes 1.18) lets you run multiple logical schedulers (profiles) within a single kube-scheduler process; each profile can enable or disable plugins independently and is selected by Pods via spec.schedulerName.