Skip to main content

Confluent CSFLE metadata backup

OSO Kafka Backup Enterprise backs up metadata from Confluent's Client-Side Field-Level Encryption (CSFLE) system. It captures Key Encryption Keys (KEKs), Data Encryption Keys (DEKs), encrypted subjects, and the Schema Registry rules that describe which fields are encrypted.

Scope

This feature preserves existing Confluent CSFLE metadata. It does not encrypt Kafka backup segment files, and it does not add encryption to plaintext Kafka records. Confluent CSFLE encrypts fields in the producer or serializer before the records reach Kafka; Kafka Backup Enterprise preserves the metadata needed to understand and recover that setup.

The open source CLI does not provide backup encryption. The Strimzi Backup Operator does not support spec.backup.encryption.

What gets backed up​

ItemCaptured data
KEK RegistryKEK names, KMS type, KMS key identifier, properties, and status
DEK RegistrySubject, version, algorithm, and encrypted key material for each DEK
Encrypted subjectsSubjects with Confluent ENCRYPT rules, their KEK, algorithm, and encrypted-field count
Schema encryption rulesRaw Schema Registry versions containing encryption rules and metadata tags
Encryption manifestCounts, KMS types, backup timestamp, and per-subject summaries

The feature copies encrypted DEK material and KMS references. It does not export KMS master keys or other plaintext key material.

Prerequisites​

  • OSO Kafka Backup Enterprise v0.3.2 or later
  • A trial or license containing the encryption feature
  • Confluent Schema Registry with the DEK Registry API available
  • Existing Confluent CSFLE schemas and keys
  • enterprise.schema_registry in the same configuration file
  • A configured Kafka source and storage backend for the normal backup

The DEK Registry is part of the Confluent Schema Registry API. The encryption backup therefore cannot run without enterprise.schema_registry, even when dek_registry_url is set explicitly.

Configuration​

The smallest complete configuration looks like this:

mode: backup
backup_id: daily-2026-07-21

source:
  bootstrap_servers:
    - kafka:9092

storage:
  backend: s3
  bucket: kafka-backups
  region: eu-west-1

enterprise:
  schema_registry:
    enabled: true
    url: https://schema-registry:8081
    auth:
      type: basic
      username: ${SR_USERNAME}
      password: ${SR_PASSWORD}

  encryption:
    enabled: true
    backup:
      include_dek_registry: true
      include_encryption_rules: true
      kek_filter:
        - "prod-*"
      exclude_keks:
        - "prod-test-*"

Authentication and TLS settings inherit from enterprise.schema_registry by default. Set enterprise.encryption.auth or enterprise.encryption.tls only when the DEK Registry needs different connection settings.

Configuration reference​

FieldDefaultDescription
enabledtrueRuns the encryption metadata backup when the section is present
dek_registry_urlDerived from Schema RegistryOverrides the DEK Registry base URL
authSchema Registry authOptional DEK Registry authentication override
tls.ca_certSchema Registry TLSOptional path to a CA certificate for the DEK Registry
backup.include_dek_registrytrueExports KEKs and DEKs
backup.include_encryption_rulestrueExports schema versions containing encryption rules
backup.kek_filter["*"]Glob patterns selecting KEK names
backup.exclude_keks[]Glob patterns excluding KEK names
connection.timeout_ms30000HTTP request timeout in milliseconds
connection.max_retries3Maximum retry attempts
connection.retry_backoff_ms1000Initial retry backoff in milliseconds
connection.rate_limit_rps25Maximum DEK Registry requests per second

Environment-variable interpolation such as ${SR_USERNAME} and ${SR_PASSWORD} is supported in configuration values.

Run the backup​

kafka-backup backup --config backup.yaml

A successful run logs a summary similar to:

Starting encryption metadata backup
Encryption backup complete: 3 KEKs, 18 DEKs, 7 encrypted subjects

If encryption is configured but the current trial or license does not include the encryption feature, the enterprise binary logs a warning, skips the encryption metadata step, and continues the normal Kafka data backup.

Stored artifacts​

Artifacts are written below the normal backup ID:

daily-2026-07-21/
└── enterprise/
    └── encryption/
        ├── _manifest.json
        ├── kek-registry.json
        ├── dek-registry.json
        └── schemas/
            └── <url-encoded-subject>/
                └── v<version>.json

_manifest.json records the number of KEKs, DEKs, and encrypted subjects. The registry exports contain the full metadata returned by the DEK Registry. Schema files retain the raw encryption ruleSet and metadata tags.

Restore availability​

The supported v0.3.2 CLI workflow exports encryption metadata during backup. Automated encryption metadata import is not currently exposed by the kafka-backup restore command. Do not rely on enterprise.encryption.restore options for a production recovery workflow until CLI restore support is released and documented here.

Keep the source KMS keys and access policies available for recovery. The backup contains encrypted DEKs and KMS references, not the KMS master keys themselves.

Encryption responsibilities​

These controls protect different parts of the backup path:

ControlResponsibility
Kafka TLS or mTLSEncrypts traffic between Kafka and the backup process
Storage-provider encryptionEncrypts complete backup objects at rest using S3 SSE/KMS, Azure CMK, GCS CMEK, or an equivalent bucket policy
Confluent CSFLEEncrypts selected message fields before records are written to Kafka
Enterprise CSFLE metadata backupPreserves the KEK, DEK, subject, and schema-rule metadata for an existing CSFLE deployment

Configure storage encryption in the cloud provider or bucket policy. The OSS backup configuration and Strimzi KafkaBackup CR do not provide a client-side segment-encryption setting.

Troubleshooting​

enterprise.schema_registry is required​

Encryption backup requires enterprise.schema_registry to be configured

Add the Schema Registry URL and authentication settings to the same enterprise section. The DEK Registry is a sub-API of Schema Registry.

Encryption is configured but not licensed​

Encryption backup is configured but not licensed. Skipping encryption backup.

Check the active feature set:

kafka-backup license info

The output must include encryption, or an active trial must be in effect.

DEK Registry authentication fails​

The encryption client inherits Schema Registry authentication and TLS settings. Verify those credentials first. If the DEK Registry uses a separate endpoint or identity, configure dek_registry_url and the auth or tls override under enterprise.encryption.

No encrypted subjects are detected​

Confirm that the subjects contain Confluent Schema Registry ENCRYPT rules. The backup can still contain KEK and DEK Registry exports when no matching encrypted subjects are found.

Next steps​