Skip to main content

KafkaRestore CRD

The KafkaRestore custom resource restores a backup into a target Kafka cluster.

Full Specification​

apiVersion: kafka.oso.sh/v1alpha1
kind: KafkaRestore
metadata:
  name: production-restore
  namespace: kafka-backup
spec:
  backupRef:
    name: production-backup
    namespace: kafka-backup
    backupId: "production-backup-20260413-120000"
    # storage can be set here instead of name/namespace for external backups.

  kafkaCluster:
    bootstrapServers:
      - kafka-0.kafka.svc:9092
      - kafka-1.kafka.svc:9092
    securityProtocol: SASL_SSL
    caSecret:                                    # optional: separate CA secret (e.g. Strimzi)
      name: cluster-ca-cert
      caKey: ca.crt
    tlsSecret:
      name: kafka-tls
      caKey: ca.crt
    saslSecret:
      name: kafka-credentials
      mechanism: SCRAM-SHA-512
    connection:
      tcpKeepalive: true
      keepaliveTimeSecs: 60
      keepaliveIntervalSecs: 20
      tcpNodelay: true
      connectionsPerBroker: 4

  topics:
    - orders
    - payments

  topicMapping:
    orders: restored-orders

  partitionMapping:
    0: 0

  repartitioning:
    restored-orders:
      strategy: murmur2
      targetPartitions: 6

  pitr:
    startTime: "2026-04-13T10:00:00Z"
    endTime: "2026-04-13T12:00:00Z"

  offsetReset:
    enabled: true
    consumerGroups:
      - order-processor
    strategy: manual

  rollback:
    snapshotBeforeRestore: true
    snapshotRetentionHours: 24
    autoRollbackOnFailure: false

  dryRun: false
  produceBatchSize: 1000
  produceAcks: -1
  produceTimeoutMs: 30000
  purgeTopics: false
  autoConsumerGroups: false
  createTopics: true
  defaultReplicationFactor: 3

  template:
    pod:
      hostAliases:
        - ip: "10.10.0.5"
          hostnames:
            - s3.internal
            - minio.internal

Spec Fields​

backupRef​

FieldTypeRequiredDescription
namestringYesKafkaBackup resource to restore from; use "" when using direct storage
namespacestringNoNamespace of the KafkaBackup; defaults to the restore namespace
backupIdstringNoSpecific backup ID
storageobjectNoDirect storage reference for external backups

kafkaCluster​

FieldTypeRequiredDescription
bootstrapServers[]stringYesTarget Kafka broker addresses
securityProtocolstringNoPLAINTEXT, SSL, SASL_PLAINTEXT, or SASL_SSL
tlsSecretobjectNoTLS certificate secret reference
caSecretobjectNoSeparate CA certificate secret (e.g. Strimzi cluster CA). Overrides caKey in tlsSecret when both are set
saslSecretobjectNoSASL credentials secret reference
connectionobjectNoKafka TCP connection tuning, including connectionsPerBroker

pitr​

FieldTypeRequiredDescription
startTimestampintNoStart timestamp in epoch milliseconds
endTimestampintNoEnd timestamp in epoch milliseconds
startTimestringNoStart timestamp as ISO 8601
endTimestringNoEnd timestamp as ISO 8601

Restore Tuning​

FieldTypeDefaultDescription
topics[]string[]Topics to restore; empty restores all topics in the backup
topicMappingmap{}Source topic to target topic names
partitionMappingmap{}Source partition to target partition IDs
repartitioningmap{}Per-target-topic repartitioning config
dryRunboolfalseValidate without executing the restore
produceBatchSizeint1000Producer batch size
produceAcksint-1-1 for all ISR, 1 for leader, 0 for no acknowledgements
produceTimeoutMsint30000Broker-side produce timeout
purgeTopicsboolfalseDelete records from target topics before restore
autoConsumerGroupsboolfalseLoad consumer groups from consumer-groups-snapshot.json and reset offsets after restore
createTopicsboolfalseCreate missing target topics
defaultReplicationFactorintunsetReplication factor for created topics
includeOriginalOffsetHeaderbooltrueAdd x-original-offset, x-original-timestamp and x-source-partition headers to every restored record (v1.3.0; this was always on before)
stripOffsetHeadersboolfalseRemove the headers kafka-backup added at backup time before producing; with includeOriginalOffsetHeader: false the restored records match the source header-for-header (v1.3.0)

repartitioning​

repartitioning:
  restored-orders:
    strategy: murmur2      # murmur2 or automatic
    targetPartitions: 6

targetPartitions must be greater than zero. strategy must be murmur2 or automatic.

offsetReset​

FieldTypeRequiredDescription
enabledboolNoEnable offset reset as part of restore
consumerGroups[]stringYesConsumer groups to reset
strategystringNomanual, auto, or dry_run; defaults to manual

rollback​

FieldTypeDefaultDescription
snapshotBeforeRestorebooltrueCreate an offset snapshot before restore
snapshotRetentionHoursint24Snapshot retention in hours
autoRollbackOnFailureboolfalseRoll back offsets automatically on restore failure
snapshotStorageobjectunsetOptional PVC storage override for snapshots

template.pod​

FieldTypeRequiredDescription
hostAliasesarrayNoPod-level Kubernetes hostAliases entries injected into restore Job pods. Use this for split-DNS or private endpoint storage access

Each hostAliases item uses the Kubernetes HostAlias shape:

FieldTypeRequiredDescription
ipstringYesIP address to write into the pod /etc/hosts file
hostnames[]stringNoHostnames mapped to ip

Examples​

Restore From a KafkaBackup​

apiVersion: kafka.oso.sh/v1alpha1
kind: KafkaRestore
metadata:
  name: restore-orders
spec:
  backupRef:
    name: production-backup
    backupId: "production-backup-20260413-120000"
  kafkaCluster:
    bootstrapServers:
      - kafka:9092
  topics:
    - orders
  createTopics: true
  produceAcks: -1

Restore From Direct Storage​

apiVersion: kafka.oso.sh/v1alpha1
kind: KafkaRestore
metadata:
  name: restore-external
spec:
  backupRef:
    name: ""
    backupId: "external-backup-20260413"
    storage:
      storageType: s3
      s3:
        bucket: kafka-backups
        region: us-west-2
        prefix: production
        credentialsSecret:
          name: s3-credentials
  kafkaCluster:
    bootstrapServers:
      - kafka:9092

Restore with Consumer Group Snapshot​

apiVersion: kafka.oso.sh/v1alpha1
kind: KafkaRestore
metadata:
  name: restore-with-groups
spec:
  backupRef:
    name: production-backup
  kafkaCluster:
    bootstrapServers:
      - dr-kafka:9092
  autoConsumerGroups: true
  createTopics: true

Restore from Split-DNS Storage​

Use template.pod.hostAliases when restore pods need the same private storage endpoint mapping as backup pods.

apiVersion: kafka.oso.sh/v1alpha1
kind: KafkaRestore
metadata:
  name: restore-private-s3
spec:
  backupRef:
    name: production-backup
    backupId: "production-backup-20260413-120000"
  kafkaCluster:
    bootstrapServers:
      - dr-kafka:9092
  createTopics: true
  template:
    pod:
      hostAliases:
        - ip: "10.10.0.5"
          hostnames:
            - s3.internal
            - minio.internal

Status​

status:
  phase: Completed
  message: "Restore completed successfully"
  startTime: "2026-04-13T12:00:00Z"
  completionTime: "2026-04-13T12:30:00Z"
  progressPercent: 100
  recordsRestored: 1000000
  bytesRestored: 1073741824
  segmentsProcessed: 8
  offsetMappingPath: "/tmp/offset-mapping.json"

Interrupted restores​

A restore runs inside the operator process. If the operator restarts while it runs (OOMKilled, evicted, rolled), the restore is set to phase: Failed with condition reason Interrupted from operator v1.6.5; earlier releases left it Running / "Restore in progress" for ever (#98).

It is not resumed automatically: the operator does not checkpoint restores, so running it again produces the records it had already restored a second time. Records restored before the interruption stay in the target topics. Clear them if you need an exact copy, then change the spec to run the restore again (a metadata-only change such as an annotation does not start a new run).

Next Steps​