Skip to main content

Strimzi Backup Operator

The Strimzi Backup Operator is a Strimzi-native Kubernetes operator for backing up and restoring Kafka clusters managed by Strimzi. Instead of configuring bootstrap servers and TLS material by hand, it resolves connection details, cluster CA certificates, and KafkaUser credentials directly from your Strimzi custom resources.

The current release is v0.3.1 (0.3.0 introduced the stable API; 0.3.1 is the first chart published with the templated CRDs). Its default backup/restore Job image is osodevops/kafka-backup:v0.22.0 — see Compatibility for how to run a different engine and which engines are supported.

Which operator do I need?

This operator (kafkabackup.com/v1) is purpose-built for Strimzi-managed clusters and references Strimzi Kafka/KafkaUser resources. If you run Kafka outside Strimzi (MSK, Confluent, self-managed), use the OSO Kafka Backup Operator (kafka.oso.sh/v1alpha1) instead.

Custom Resource Definitions​

CRDAPI GroupDescription
KafkaBackupkafkabackup.com/v1Back up topics from a Strimzi cluster to S3, Azure Blob, GCS, or filesystem storage — one-shot or on a cron schedule
KafkaRestorekafkabackup.com/v1Restore a backup into a Strimzi cluster, with topic selection, renaming, and point-in-time recovery

The operator runs backups and restores as Kubernetes Jobs that execute the kafka-backup CLI, so every run is inspectable with standard tooling (kubectl get jobs,pods).

API versions​

kafkabackup.com/v1 is the stable API since operator 0.3.0: fields are never removed, renamed or changed in type, new fields are optional, and anything that would break that ships as a new version served alongside v1 with a deprecation window. kafkabackup.com/v1alpha1 is still served with an identical schema but deprecated (every request prints a warning) and will be removed no earlier than operator 0.5.0 or six months after 0.3.0. Migrate by changing apiVersion; nothing else changes. The full contract, graduation criteria and removal process are in the operator's API stability document; support, lifecycle and security policies are in SUPPORT.md and SECURITY.md.

Encryption availability​

Enterprise-only capability

spec.backup.encryption was mistakenly exposed by the v0.2.16 KafkaBackup CRD. The open source kafka-backup CLI has never supported that setting, so an enabled value fails reconciliation instead of creating an encrypted backup. Do not use it. The field is removed from the public CRD in v0.2.23.

OSO Kafka Backup Enterprise can preserve metadata from an existing Confluent Client-Side Field-Level Encryption deployment. That capability backs up KEKs, encrypted DEKs, encrypted subjects, and schema encryption rules; it does not encrypt Kafka backup segment files. See Confluent CSFLE Metadata Backup for the supported scope and configuration.

Installation​

# Add the OSO DevOps Helm repository
helm repo add oso-devops https://osodevops.github.io/helm-charts/
helm repo update

# Install the operator
helm install strimzi-backup-operator oso-devops/strimzi-backup-operator \
  --namespace kafka \
  --create-namespace \
  --version 0.3.1

Since 0.3.0 the chart renders the CRDs as templates (crds.install, default true; crds.keep keeps them on uninstall), so helm upgrade applies CRD changes. Upgrading from 0.2.x needs a one-off adoption of the CRDs the old static crds/ directory installed:

# Helm 3.17+ / Helm 4
helm upgrade strimzi-backup-operator oso-devops/strimzi-backup-operator \
  --namespace kafka \
  --version 0.3.1 \
  --take-ownership

With older Helm, label and annotate kafkabackups.kafkabackup.com and kafkarestores.kafkabackup.com for adoption first (app.kubernetes.io/managed-by=Helm, meta.helm.sh/release-name, meta.helm.sh/release-namespace). To keep managing the CRDs yourself, install with --set crds.install=false and apply the release asset:

kubectl apply --server-side -f https://github.com/osodevops/strimzi-backup-operator/releases/download/v0.3.0/crds.yaml

A minimal backup and restore​

apiVersion: kafkabackup.com/v1
kind: KafkaBackup
metadata:
  name: daily-backup
  namespace: kafka
spec:
  strimziClusterRef:
    name: my-cluster        # your Strimzi Kafka CR
  schedule:
    cron: "0 2 * * *"
  storage:
    type: s3
    s3:
      bucket: my-kafka-backups
      region: eu-west-1
      accessKeySecret:
        name: aws-credentials
        key: access-key-id
      secretKeySecret:
        name: aws-credentials
        key: secret-access-key
  backup:
    stopAtCurrentOffsets: true
    # Any option from the kafka-backup config reference can be set through
    # the free-form `config` map using native snake_case key names
    # (operator v0.2.20+; keys here win over the typed fields above).
    config:
      fetch_max_bytes: 16777216
      segment_max_records: 2000000
  logging:
    level: info
    format: json
  metrics:
    enabled: true
    keepAliveSeconds: 60
    maxPartitionLabels: 100
---
apiVersion: kafkabackup.com/v1
kind: KafkaRestore
metadata:
  name: restore-orders
  namespace: kafka
spec:
  strimziClusterRef:
    name: my-cluster
  backupRef:
    name: daily-backup
    backupId: backup-20260610-020000
  topics:
    include:
      - orders-*
  restore:
    stripOffsetHeaders: false   # true = header-for-header identical to the source (job image >= v0.19.0)

spec.backup.includeOffsetHeaders defaults to true, so every archived record gains x-original-offset / x-original-timestamp; spec.restore.stripOffsetHeaders removes them again on the way back. See the operator README's "Offset headers and record fidelity" section.

Advanced options passthrough​

The typed fields under spec.backup / spec.restore cover the common kafka-backup options with camelCase names (for example segmentSize maps to segment_max_bytes). Every other option in the config reference can be set through the free-form config map using kafka-backup's native snake_case key names (operator v0.2.20+), following the same pattern as Strimzi's spec.kafka.config:

spec:
  backup:
    compression: zstd
    config:
      fetch_max_bytes: 16777216       # native kafka-backup key names
      segment_max_records: 2000000

Keys set in config are passed through verbatim to the generated job config and take precedence over the typed fields (so config.segment_max_bytes wins over segmentSize). Keys the kafka-backup binary does not recognize are logged as warnings at job startup (kafka-backup v0.16.0+) instead of being silently ignored. spec.restore.config works the same way for the restore: section.

spec.logging.level and spec.logging.modules are applied to job pods via the RUST_LOG environment variable (operator v0.2.20+); an explicit RUST_LOG entry in spec.env still takes precedence.

Compatibility​

Each operator release ships with a default kafka-backup engine image. It is compiled into the operator, logged at start-up (default_job_image), exposed as the strimzi_backup_operator_engine_image_info metric, and is what every Job runs unless told otherwise. The operator generates the engine's config and runs the engine in a Job; backup and restore behaviour lives in the engine.

Choosing the engine​

Per resource, with spec.image on a KafkaBackup or KafkaRestore:

spec:
  image: osodevops/kafka-backup:v0.19.2   # any 0.x release newer than the default

For the whole installation, with the Helm value backupJobs.image (operator v0.2.25+):

helm upgrade strimzi-backup-operator oso-devops/strimzi-backup-operator \
  --namespace kafka --reuse-values \
  --set backupJobs.image=osodevops/kafka-backup:v0.19.2

Precedence is spec.image → backupJobs.image → the compiled-in default. The image a Job actually ran with is recorded in status.lastBackup.image and status.restore.image.

Policy​

  • Default engine — tested in CI on every commit (the configs the operator generates are run through it) and the only combination that is guaranteed.
  • Newer engine — any kafka-backup 0.x release newer than the default is supported. Pin it to pick up engine fixes without waiting for an operator release: since kafka-backup v0.16.0 an unknown config key is warned about rather than failing the run, and spec.backup.config / spec.restore.config are passed through verbatim, so new engine options are usable immediately. A nightly CI job runs the generated configs against the latest engine release.
  • Older engine — supported down to the minimum in the table, with degraded behaviour: options the older engine does not know are ignored with a warning. Below the minimum the operator still runs the Job but sets the EngineVersionSupported=False condition (reason EngineOlderThanMinimum) on the resource. Images whose tag is not a release (latest, a digest, a custom tag) get EngineVersionSupported=True with reason EngineVersionUnknown.
  • Behaviour changes — an engine bump can change data semantics (v0.18.0 stopped flattening null header values to empty; v0.19.0 added strip_offset_headers). Every default-image bump has a CHANGELOG entry in the operator repository saying what changed and whether existing archives need re-taking. Pin spec.image to keep an older engine.
  • Engine 1.x will require an operator release; operator 0.x does not support it.
OperatorDefault engineMinimum engineNotes
0.3.0 – 0.3.1v0.22.0v0.16.0kafkabackup.com/v1 API, templated CRDs; engine 0.22: prune/backup.retention, on_missing_topic, syncIntervalSecs honoured
0.2.25v0.19.1v0.16.0backupJobs.image, EngineVersionSupported condition, status.*.image
0.2.22 – 0.2.24v0.19.1v0.16.0
0.2.21v0.19.0v0.16.0stripOffsetHeaders needs ≥ v0.19.0
0.2.20v0.16.0v0.16.0spec.backup.config passthrough needs ≥ v0.16.0
≤ 0.2.19v0.15.x—not supported
FeatureNeeds engine
spec.backup.config / spec.restore.config passthrough≥ v0.16.0
null header values preserved (not flattened to empty)≥ v0.18.0
spec.restore.stripOffsetHeaders≥ v0.19.0
per-run incremental progress gauges (kafka_backup_snapshot_records_*)≥ v0.19.1

License​

The Strimzi Backup Operator is open source under the Apache License 2.0. The kafka-backup engine it runs inside each Job is MIT. Neither requires a commercial licence; only the Enterprise features of the engine do.

Next steps​