[7/24] J is for Jobs and CronJobs: Batch Processing in Kubernetes

📚 This is Post #7 in the Kubernetes A-to-Z Series

Reading Order: ← Previous: ReplicaSets → Next: Namespaces →

Series Progress: 7/24 complete | Difficulty: Intermediate | Time: 15 min | Part 2/6: Core Workloads

Deployments are great for long-running services like web servers. But what if you just want to run a script once? Or run a backup every night at 3 AM?

In this post, we’ll cover the “J” of Kubernetes: Jobs and CronJobs.

Jobs: Run Once and Done

A Job creates one or more Pods and ensures that a specified number of them successfully terminate.

Use Cases

  • Database migrations
  • Batch processing (video rendering, data analysis)
  • One-time setup scripts

Creating a Job

apiVersion: batch/v1
kind: Job
metadata:
  name: pi-calculator
spec:
  template:
    spec:
      containers:
      - name: pi
        image: perl:5.34.0
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  backoffLimit: 4
  • restartPolicy: Must be Never or OnFailure.
  • backoffLimit: How many times to retry if the pod fails (default is 6).

CronJobs: Scheduled Tasks

A CronJob creates Jobs on a repeating schedule. It’s exactly like the Linux cron daemon, but for clusters.

Use Cases

  • Nightly backups
  • Sending weekly emails
  • Cleaning up temporary files

Creating a CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
spec:
  schedule: "0 3 * * *" # Run at 3:00 AM every day
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: my-backup-tool:latest
            args: ["/bin/sh", "-c", "backup.sh"]
          restartPolicy: OnFailure

Cron Schedule Syntax

┌───────────── minute (0 - 59)
│ ┌───────────── hour (0 - 23)
│ │ ┌───────────── day of the month (1 - 31)
│ │ │ ┌───────────── month (1 - 12)
│ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday)
│ │ │ │ │
* * * * *

Advanced Job Patterns

Parallelism

You can run multiple pods in parallel to speed up work.

spec:
  completions: 10   # Total successful pods needed
  parallelism: 2    # Run 2 pods at a time

Handling Failures

If a Job fails, Kubernetes will create a new Pod to try again. This is why your jobs should be idempotent (running them multiple times should have the same result as running them once).

Cleanup

Jobs stick around after they finish so you can check logs. To clean them up automatically:

spec:
  ttlSecondsAfterFinished: 100 # Delete 100s after completion

Summary

  • Use Deployments for services that should run forever.
  • Use Jobs for tasks that should run once to completion.
  • Use CronJobs for tasks that should run on a schedule.

Next Steps

We’ve handled the workloads. Now let’s talk about maintaining the cluster itself. Next up is U is for Upgrades.

Series Navigation: