A complete guide to Kubernetes CronJobs

Kubernetes CronJobs are objects that create jobs on a recurring schedule. The mechanism is modeled on the cron command-line scheduler, a staple of Unix environments.

CronJobs let you start pods, run a task, and benefit from automatic cleanup after the operation. They come with built-in features for controlling concurrency and tracking job failures and successes. These features make CronJobs the most convenient way to manage regular tasks and maintenance activities within your Kubernetes cluster.

This article will explain what CronJobs are, how you can set them up, and the caveats you should be aware of when using them in your cluster. One of the most common challenges is the terminology: CronJobs are intentionally similar to regular crontab scripts, but the two are not identical. As usual in the world of Kubernetes, CronJobs layer their own behaviors and unique characteristics over the basic principles of cron.

How do CronJobs work?

Most complex software systems end up incorporating processes that need to run outside their main loop at a regular cadence. You might be sending emails in batches, processing user photo uploads, or refreshing data from an external provider. In all of these cases, you need a way of running the operation on a time-based schedule.

Setting up such a schedule is easy in the non-containerized world of traditional VMs and OS-level services. You’d write a script, add it to your system’s crontab, and call it a day. Kubernetes CronJob objects are the container ecosystem’s answer to this ubiquitous requirement.

CronJobs solve the challenge by bringing the familiarity of cron to a cluster environment. Each CronJob consists of a schedule in cron format, such as */5 * * * * to run the schedule every five minutes, and a spec defining pods to start for each job execution. Kubernetes will schedule the pods to nodes in your cluster with spare capacity.

CronJobs are automatically managed by the cluster control plane. When it’s time to run your schedule, the cluster creates regular jobs with the pod spec from your CronJob object. Whereas standard jobs are used to encapsulate pods and retry execution until they terminate, a CronJob is a higher-level abstraction that repeats the cycle periodically.

When should a CronJob be used?

CronJobs are the right choice for anything you’d typically use cron to solve. They’re a fully automated approach to scheduled tasks and suitable for many different scenarios. Common use cases include application-level jobs such as sending emails and ingesting data, as well as infrastructure-bound operations like backing up a database or cleaning expired caches. CronJobs are always preferable to running a cron daemon within an existing container in your cluster, which goes against containerization best practices by giving the container multiple responsibilities.

Kubernetes CronJobs provide a way to run these tasks outside your main application containers, converting them to first-class components in your cluster. They also offer greater opportunities to inspect and manage your scheduled operations independently of your primary workload.

There are some cases where CronJobs might not be suitable. One particular limitation is Kubernetes’ lack of time zone support. While most clusters will respect the CRON_TZ and TZ environment variables, this is not standard and should not be taken for granted.

Jobs aren’t guaranteed to execute at the exact time indicated by their schedule. There are also cases where a CronJob could create multiple instances of its containers or start none at all. These rare scenarios arise from Kubernetes’ internal handling of scheduled operations and container creation. You should carefully consider whether to use a CronJob for critical operations where reliability is vital. All tasks run by a CronJob need to be idempotent, meaning they need to be able to run multiple times without side effects, because of the possibility that Kubernetes will complete a scheduled run more than once.

Creating a CronJob

You can add a CronJob to your cluster by writing a YAML file that describes its configuration. First, make sure you’ve got kubectl installed and configured with a connection to your cluster.

Create a YAML file in your working directory with the following content:

This sets up a simple CronJob that executes every minute and logs “Hello World” to the job output. You’ll see how to retrieve these logs later in this article. The spec.schedule field sets the CronJob schedule and should be a valid cron expression.

The spec.jobTemplate field defines the job that will be created each time the schedule occurs. Within the job template, the spec.containers section is where you set up the pods that will be started as part of the job.

In this example, a new container running the busybox:latest image will start each time the CronJob is scheduled (ie, every minute). The command /bin/sh -c “echo ‘Hello World’” will be executed inside the container. Once the command exits successfully, the container will stop, the job will be marked as complete, and the CronJob will finish its scheduled run in turn. If the command ends in a failure, the container will restart and repeat the command until it succeeds. This is because the restart policy is set to OnFailure.

Add your new CronJob object to your cluster using kubectl:

That’s it! Kubernetes will start running the CronJob in line with the requested spec.schedule. You’ve got a scheduled operation running in your cluster, independent of your regular long-running pods and their containers.

Common "gotchas" and items to tweak

Your example CronJob works as a model for many real-world jobs. In other cases, you might need to make a few more adjustments. Here are some things to think about as you define your CronJobs.

Concurrency

CronJobs come with built-in concurrency controls. These controls are a key difference from plain old Unix cron, where concurrent execution is always allowed and cannot be disabled. Kubernetes mirrors this and permits concurrency by default. Here’s an illustration of the possible effects in a CronJob that runs each minute:

The second run took longer than the scheduled minute to complete. As concurrency is allowed, the third run started anyway while the second was still incomplete. Concurrency can be undesirable when you need individual executions of a CronJob to run sequentially.

Setting the spec.concurrencyPolicy field on a CronJob object lets you control this behavior. It has three possible values:

• Allow: Allow concurrency as shown above (default).
• Forbid: Prevent concurrent runs. When a job is due to start but the previous run is incomplete, the new job is skipped. Kubernetes won’t try to start the job again until the next scheduled occurrence.
• Replace: A hybrid of the previous two. When a job is due to start but the previous run is incomplete, the containers created by the previous job are terminated, and the new job is allowed to proceed.

Here’s an example of selecting the concurrency policy for a CronJob:

Applying this object to your cluster will result in a CronJob where only one run can exist at any given time.

Starting deadlines

The starting deadline is another mechanism that determines whether a new scheduled CronJob run can begin. This Kubernetes-specific concept is used to determine how long a job run remains eligible to start after its scheduled time has passed. It’s used when the job doesn’t start on time, perhaps because its concurrency policy is set to Forbid and the previous run is still live.

This value is controlled by the spec.startingDeadlineSeconds field:

As an example, consider the schedule of the CronJob shown above. If the 09:00:00 run is still live at 09:05:00, that next run can’t start on time. Since the CronJob has a starting deadline of 30, its start can be delayed for up to thirty seconds. If the previous job terminates at 09:05:15, the 09:05:00 job can begin. However, if the previous run takes until 09:05:30 to end, the 09:05:00 job is skipped entirely.

History retention

Two other fields are also worth mentioning:

• spec.successfulJobsHistoryLimit
• spec.failedJobsHistoryLimit

These two values control how long Kubernetes retains the history of successful and failed jobs, respectively. They default to three successful jobs and one failed job. This means you can view the job objects and container logs of the last three successful CronJob runs on a rolling basis.

You might want to raise these values for jobs that run on a very frequent schedule. Higher values for the history will give you a better chance of successfully debugging issues before the logs get deleted.

Monitoring your CronJobs

You can monitor CronJobs using familiar Kubernetes mechanisms such as kubectl get:

$ kubectl get cronjob busybox-cron

This gives you all the details of the CronJob definition. To view the details of job runs, you can use the following command:

$ kubectl get jobs

Jobs belonging to a CronJob will display the CronJob’s name with their starting timestamp appended.

Once you’ve found the individual jobs, you can get their container logs using the kubectl logs job/<job-name> command:

Final thoughts

Kubernetes CronJobs let you automate recurring and periodic tasks inside your Kubernetes cluster. They have strong parallels with traditional Unix-based cron implementations but also offer extra functionality with some unique caveats.

This article explored what CronJobs are, the situations in which they’re used, and how you can create your own within your Kubernetes cluster. You also looked at the important choices and considerations to make when managing job concurrency and log retention.

Most containerized systems will end up using a CronJob sooner or later. Gaining an understanding of how they work and the options for configuring them will help you integrate scheduled activities into your application and anticipate any errors that could occur. Combining CronJobs with a monitoring platform is an excellent way to stay ahead of real-time events so you know when jobs are failing.