Back up Apache Kafka with Docker

How Kafka backup in Docker works​

Kafka backup in Docker starts with the official image, ghcr.io/osodevops/kafka-backup:latest, published for linux/amd64 and linux/arm64 — it runs the same on Intel servers, AWS Graviton, and Apple Silicon. The container is a single static binary: mount a YAML config, point it at your brokers, and run backup or restore as the command.

Everything the CLI does works identically in the container. Backups compress records with Zstd or LZ4, preserve consumer group offsets, and target S3, S3-compatible endpoints, Azure Blob, GCS, or a mounted filesystem volume — every field is in the configuration reference. Credentials arrive as environment variables or Docker secrets, never baked into the image.

The image is built for hardened setups. It runs as a non-root user (UID 1000) by default and supports a read-only root filesystem, with one requirement: a /tmp tmpfs mount, because the offset tracking database that powers incremental backups lives there. Since v0.8.1 the engine handles SIGTERM gracefully — docker stop flushes in-progress segments and saves a checkpoint instead of corrupting a run.

Docker Compose Kafka backup configuration​

A production service is a few lines of Compose: restart policy, resource limits, and the hardening flags together.

mode: backup
backup_id: "docker-nightly"

source:
bootstrap_servers:
  - kafka-1.internal:9092
  - kafka-2.internal:9092
topics:
  include:
    - "*"
  exclude:
    - "__consumer_offsets"

storage:
backend: s3
bucket: kafka-backup-archive
region: us-west-2
prefix: backups/prod-cluster

backup:
compression: zstd
include_offset_headers: true

offset_storage:
db_path: /data/state/offsets.db

services:
kafka-backup:
  image: ghcr.io/osodevops/kafka-backup:latest
  command: ["backup", "--config", "/config/backup.yaml"]
  restart: unless-stopped
  user: "1000:1000"
  read_only: true
  tmpfs:
    - /tmp
  stop_grace_period: 60s
  environment:
    - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
    - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    - AWS_REGION=us-west-2
  volumes:
    - ./config:/config:ro
    - kafka-backup-state:/data/state
  networks:
    - kafka-network

networks:
kafka-network:
  external: true

volumes:
kafka-backup-state:

docker run --rm \
-e AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \
-e AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
-e AWS_REGION="us-west-2" \
-v $(pwd)/restore.yaml:/config/restore.yaml:ro \
ghcr.io/osodevops/kafka-backup:latest \
restore --config /config/restore.yaml

# restore.yaml can pin a point in time and preserve offsets:
#   restore:
#     time_window_end: 1784246400000
#     consumer_group_strategy: header-based

For scheduled backups, either call docker compose run --rm kafka-backup from host cron, or keep scheduling inside Docker with an Ofelia sidecar and job labels. Both patterns, plus a full development stack with a single-broker Kafka, are in the Docker deployment guide.

Container deployment checks​

A running container is not a tested backup. Schedule restore drills against a scratch cluster the same way you schedule the backups — the best practices guide covers cadence and validation.