Skip to main content
Welcome to this article on Kubernetes Jobs. In this guide, we’ll explore Jobs in Kubernetes and learn how they differ from long-running workloads. By the end, you’ll understand how to implement both simple operations and more complex batch processing tasks using Kubernetes Jobs.

​
Understanding Container Workloads

Containerized workloads are typically classified into two categories:
  • Long-running workloads: For example, web servers and database applications that continue to run until they are manually stopped.
  • Batch processing tasks: These execute a specific operation—such as computation, image processing, data analysis, or report generation—and then terminate.
We’ll start by examining how a simple workload behaves in Docker, and afterward, we’ll translate the concept into Kubernetes.

​
Simple Workload Example in Docker

When you run a Docker container tasked with a basic math operation—like adding two numbers—the container starts, performs the calculation, prints the output, and then exits. For example:
In this case, the task completed successfully as indicated by a return code of zero.

​
Replicating the Task in Kubernetes

To replicate the math addition in Kubernetes, you first need to create a pod definition file. When this pod is created, it launches a container that performs the computation and exits. Here is the pod definition:
Create the pod using:
Check the pod status with:
Even though the container finishes its task, Kubernetes will attempt to keep the pod running by default. After a short period, you will notice:
This is due to the default restart policy being set to Always. To prevent Kubernetes from restarting the container after the job finishes, override the restart policy by setting it to Never (or OnFailure, as needed):
Setting the restart policy to Never ensures that the pod does not restart automatically after the command has completed, which is ideal for one-off tasks.

​
Introducing Jobs for Batch Processing

For batch processing or large-scale data tasks, you might need multiple pods working together concurrently. Unlike ReplicaSets, which ensure a certain number of pods remain running, Kubernetes Jobs are designed to run pods until the specified task is completed successfully. To create a Job, start with a job definition file that uses the API version batch/v1 and kind Job. In the job specification, a template holds the pod definition. Here’s an example job definition that performs our math addition:
Create the job with:
Verify the job creation and its completion status:
Fetch the pod created by the job:
The job remains in a completed state with zero restarts, indicating that Kubernetes did not attempt to restart the pod after a successful completion. You can view the output of the command inside the container using the kubectl logs command with the pod name. To delete the job along with its associated pods, run:

​
Running Multiple Pods with Job Completions

In many real-world scenarios, you might require a job to run multiple pods simultaneously to process data in parallel or handle retries for failed operations. To run three pods for a single job, set the completions field to 3:
After creating the job:
And checking the pod status:
By default, pods for a job are created sequentially—each pod starts only after the previous one completes.

​
Handling Failures with Jobs

Consider a scenario where you use an image like kodekloud/random-error that randomly either completes successfully or fails. In such cases, if one pod fails, Kubernetes will create another pod until it achieves the specified number of successful completions. Here is an example job definition to handle this scenario:
Create the job:
Check the job status:
And review the pods:
Kubernetes continues to create new pods until three pods complete successfully.

​
Running Pods in Parallel with Jobs

For scenarios where you want the pods to run concurrently, you can set the parallelism property. This allows multiple pods to be created simultaneously. For example, to run up to three pods in parallel:
Create the job with:
Check the job status:
And inspect the pods:
Kubernetes will intelligently create new pods as necessary until all three completions are successful.
Kubernetes Jobs are ideal for managing batch tasks where tasks must complete successfully before termination. They allow for sequential or parallel pod execution and include robust failure handling.
This concludes our article on Kubernetes Jobs. Try these examples yourself to reinforce your understanding, and explore further to see how Jobs can help you manage batch processing in Kubernetes effectively. Happy learning!

Watch Video