Back up Apache Kafka to Dell ECS
Store compressed topic data, consumer group offsets, and topic configuration on Dell ECS or ObjectScale through its S3-compatible API. Restore into any reachable Kafka cluster at a precise moment in time.
How Kafka backup to ECS works
Kafka backup to ECS uses the S3-compatible API that Dell ECS exposes. OSO Kafka Backup
connects through the s3 storage backend with a custom endpoint, the same mechanism the
configuration reference documents for S3-compatible stores. No
Dell-specific adapter is required.
Dell converged ECS and ObjectScale into one platform under the ObjectScale brand in 2025, on the same codebase and APIs. The configuration below applies to both: existing ECS namespaces and current ObjectScale deployments present the same S3 interface.
ECS keeps backup objects on hardware your team controls, inside your network boundary and away from Kafka brokers and their volumes. OSO compresses topic records with Zstd or LZ4 before upload and preserves consumer group offsets and topic configuration. The storage format reference explains the objects written to ECS.
ECS Kafka backup configuration
Create an object user in the ECS namespace with access to the backup bucket, and point the
s3 backend at your load-balanced ECS S3 endpoint.
- Back up to ECS
- Restore from ECS
- Run it
mode: backup
backup_id: "ecs-nightly"
source:
bootstrap_servers:
- kafka-1.internal:9092
- kafka-2.internal:9092
topics:
include:
- "*"
exclude:
- "__consumer_offsets"
storage:
backend: s3
bucket: kafka-backup-archive
region: us-east-1
endpoint: https://objectscale.dc1.example.com
access_key: ${ECS_ACCESS_KEY_ID}
secret_key: ${ECS_SECRET_ACCESS_KEY}
prefix: backups/prod-cluster
backup:
compression: zstd
compression_level: 5
include_offset_headers: true
mode: restore
backup_id: "ecs-nightly"
target:
bootstrap_servers:
- kafka-dr-1.internal:9092
storage:
backend: s3
bucket: kafka-backup-archive
region: us-east-1
endpoint: https://objectscale.dc1.example.com
access_key: ${ECS_ACCESS_KEY_ID}
secret_key: ${ECS_SECRET_ACCESS_KEY}
prefix: backups/prod-cluster
restore:
# Restore through the moment before the incident (Unix ms)
time_window_end: 1784246400000
consumer_group_strategy: header-based
kafka-backup backup --config backup.yaml
kafka-backup restore --config restore.yaml
The region value satisfies S3 request signing; ECS itself organizes storage by namespace
and replication group rather than AWS regions. Point endpoint at the load balancer in
front of your ECS nodes, and confirm the addressing style (path or virtual-hosted) your
site uses matches its DNS and TLS setup.
ECS deployment checks
| Check | Why it matters |
|---|---|
| Object user permissions | The backup identity needs object read, write, list, and delete in the bucket for normal lifecycle operations. |
| Retention and compliance locks | ECS retention policies can block object deletion. Size retention windows so pruning expired backups still works. |
| Replication group placement | Geo-replicated namespaces protect against site loss; know which sites hold backup data for recovery planning. |
| Load balancer and TLS | The S3 endpoint must resolve, terminate TLS with a matching certificate, and spread load across ECS nodes. |
| Recovery tests | A successful upload does not prove that a target cluster can be restored. |
ECS retention and geo-replication protect the stored objects. They do not replace backup lifecycle management — the backup tool still decides what to keep and what to prune. Use the S3 deployment guide for credential and connection patterns, and the best practices guide for schedules, retention, and restore testing.
Frequently asked questions
Does OSO Kafka Backup work with Dell ECS?
Yes. Select the s3 storage backend, set the ECS S3 endpoint, and provide an object user access key and secret key. ECS exposes an S3-compatible API, so no separate adapter is needed.
Does this configuration also apply to Dell ObjectScale?
Yes. Dell converged ECS and ObjectScale into a single platform under the ObjectScale brand, keeping the ECS codebase and S3 API. The same s3 backend configuration works against both.
What region should I set for an on-premises ECS endpoint?
Use a consistent placeholder such as us-east-1. The value satisfies S3 request signing; ECS organizes data by namespace and replication group, not AWS regions.
Do ECS compliance retention locks affect Kafka backups?
They can. Retention policies that forbid deletion also block pruning of expired backup objects. Keep the backup bucket retention window aligned with your backup retention schedule.
Can I restore an ECS backup into another Kafka cluster?
Yes. Point the restore configuration at any reachable target cluster. Restore can preserve consumer offsets and stop at a millisecond timestamp.
Ready to protect your Kafka data?
Take your first backup in minutes with the open source CLI, or talk to us about Enterprise features like encryption, RBAC, and audit logging.