Back up Apache Kafka to MinIO
Stream compressed topic data, consumer group offsets, and cluster metadata into a MinIO bucket on your own hardware — then restore any of it, to any cluster, at a precise moment in time. No cloud dependency anywhere in the pipeline.
Why MinIO for Kafka backups
MinIO made S3-compatible object storage a standard part of on-prem and Kubernetes estates: erasure coding across drives, bitrot protection, and the same API every S3 client already speaks. For Kafka backups that means a durable target that lives in your datacenter, under your compliance boundary, with no egress costs and no external dependency — which also makes it the natural choice for air-gapped environments.
OSO Kafka Backup talks to MinIO through the s3 backend with a custom endpoint. That endpoint
mechanism covers the whole MinIO family: the AGPLv3 community server (now distributed as source
only, with the upstream repository archived in early 2026) and MinIO's commercially supported
AIStor both expose the S3 API, so the backup workflow is identical against either. The
storage format reference covers the compressed layout written to
the bucket.
Configuration
Point the s3 backend at your MinIO endpoint with explicit credentials — MinIO deployments
use their own access keys rather than the AWS credential chain. The
configuration reference documents every option.
- Backup to MinIO
- Restore from MinIO
- Run it
mode: backup
backup_id: "onprem-daily"
source:
bootstrap_servers:
- kafka-1.internal:9092
- kafka-2.internal:9092
topics:
include:
- "*"
exclude:
- "__consumer_offsets"
storage:
backend: s3
bucket: kafka-backups
endpoint: https://minio.internal:9000
access_key: ${MINIO_ACCESS_KEY}
secret_key: ${MINIO_SECRET_KEY}
prefix: backups/onprem-cluster
backup:
compression: zstd
compression_level: 5
include_offset_headers: true
mode: restore
backup_id: "onprem-daily"
target:
bootstrap_servers:
- kafka-dr-1.internal:9092
storage:
backend: s3
bucket: kafka-backups
endpoint: https://minio.internal:9000
access_key: ${MINIO_ACCESS_KEY}
secret_key: ${MINIO_SECRET_KEY}
prefix: backups/onprem-cluster
restore:
# Restore up to the moment before the incident (Unix ms)
time_window_end: 1751630400000
consumer_group_strategy: header-based
kafka-backup backup --config backup.yaml
kafka-backup restore --config restore.yaml
What gets backed up
Every backup captures topic records with their timestamps and headers, consumer group offsets, and topic configuration — compressed with Zstd or LZ4 before upload. Restores can target the original cluster or a brand new one, and can stop at a precise millisecond, which is what makes recovery from bad deploys and accidental deletes possible.
Frequently asked questions
Does OSO Kafka Backup work with MinIO?
Yes. Set the s3 backend with a custom endpoint pointing at your MinIO server, provide the MinIO access and secret keys, and the full backup and restore workflow works unchanged — including point-in-time restore and offset preservation.
Does it work with both MinIO community edition and AIStor?
Yes. The integration is against the S3 API, which both the AGPLv3 community server and the commercial AIStor product expose. Existing community deployments and AIStor clusters use the same endpoint-based configuration.
Can I run Kafka backups fully air-gapped with MinIO?
Yes. The backup runs as a single binary or Kubernetes operator next to your cluster, and MinIO is the storage target — nothing in the pipeline requires internet access or a cloud provider account.
How are MinIO credentials provided?
Set access_key and secret_key in the storage configuration, typically via environment variable references. MinIO deployments manage their own keys, so the AWS credential chain fallback generally does not apply.
Does distributed MinIO with erasure coding need special configuration?
No. Erasure coding and healing happen below the S3 API, so the backup addresses the bucket the same way against a single-node or a distributed MinIO deployment. Size the erasure set for the durability your backups require.
Ready to protect your Kafka data?
Take your first backup in minutes with the open source CLI, or talk to us about Enterprise features like encryption, RBAC, and audit logging.