Kafka Active-Passive DR Architecture: Design, Failover, and Failback

A Kafka active-passive DR architecture runs one production cluster and one standby that receives data continuously but serves no traffic until promotion. When the primary region fails, the team promotes the standby, redirects clients, and resumes processing.

Three decisions define the design: how the standby receives data, how clients switch to it, and how the team returns to the primary afterward. Get those three right and active-passive covers most Kafka workloads at a fraction of active-active cost.

Key takeaway

Active-passive succeeds or fails on details that idle standbys hide: data feed lag, consumer offset translation, and client cutover. Write the promotion runbook first, then prove it in a drill.

What Kafka active-passive means

In an active-passive design, the primary cluster serves every producer and consumer. The standby cluster receives a continuous copy of the data but handles no application traffic.

The contrast is active-active, where multiple clusters serve traffic at once with bidirectional replication. Active-active buys near-zero failover time at the price of offset synchronization, conflict handling, and roughly double the run cost. The Kafka disaster recovery guide compares the two patterns in depth; this guide builds the active-passive one.

One clarification matters before any architecture work. Kafka's replication factor copies partitions across brokers inside one cluster. Every replica shares that cluster's failure domain — its region, its control plane, its administrators. Intra-cluster replication keeps partitions online when a broker dies. It does nothing when the region fails or an operator deletes a topic.

Responsibility	Primary (steady state)	Standby (steady state)	Standby (after promotion)
Producer traffic	All	None	All
Consumer traffic	All	None	All
Data role	Source of record	Receives copy	Source of record
Topic and ACL changes	Applied directly	Synced from primary	Applied directly
Monitoring focus	Cluster health	Feed lag and drift	Cluster health

Choose how the standby receives data

The data feed determines the standby's cost, its failover speed, and what failures it can survive. Three tiers cover the practical options.

Warm standby keeps a running cluster fed by cross-cluster replication, usually MirrorMaker 2. Records arrive within seconds, so failover mostly consists of offset translation and client redirection.

Pilot light keeps a small running cluster plus a continuous backup stream in the standby region's object storage. Failover restores data into the standby cluster from the newest validated recovery point.

Cold standby keeps only the backup data. The cluster is provisioned at failover time, which adds infrastructure build-out to the recovery clock.

Tier	Data feed	Standby cost	Typical failover time	Survives logical corruption?
Warm standby	Cross-cluster replication	Full cluster running idle	Minutes	No — replication copies the corruption
Pilot light	Continuous backup to standby-region storage	Small cluster + storage	Tens of minutes to hours	Yes — restore an earlier recovery point
Cold standby	Backup to standby-region storage	Storage only	Hours	Yes — restore an earlier recovery point

The corruption column deserves emphasis. Replication is not backup. MirrorMaker 2 copies deletions, poison records, and application bugs to the standby with the same fidelity as good data. An independent backup preserves recovery points that replication has already overwritten.

For Tier 1 workloads, run both: replication for fast failover and an independent backup for recovery points replication cannot provide. For cost-sensitive tiers, a backup-fed pilot light alone is a defensible choice — the disaster recovery use case walks through that architecture.

Design the standby cluster

Size the standby for the promoted load, not the idle load. A standby that shrinks to save cost must either scale up during failover — adding minutes to the recovery clock — or run degraded after promotion. Either is acceptable if it is written down and approved; discovering it during an incident is not.

Configuration drift is the most common failover surprise. Keep these synchronized between clusters, and check them on a schedule:

• Kafka versions and broker configuration that affects clients
• Topic names, partition counts, retention, and compaction settings
• ACLs, quotas, SASL credentials, and TLS trust material
• Schema Registry subjects, versions, and compatibility settings

Schema Registry needs explicit treatment. Restored or replicated records reference schema IDs; consumers on the standby fail to deserialize when those subjects are missing. Include registry state in the standby feed, and verify it in every drill.

Topic naming is a design decision, not a default to accept. MirrorMaker 2 prefixes replicated topics with the source cluster alias (primary.orders) unless you configure an identity replication policy. Prefixed names force client changes at failover time. Identical names keep cutover simple but require care to avoid replication loops during failback.

Feed the standby with continuous backup

For the pilot-light and cold tiers — and as the corruption-proof layer under a warm standby — run a continuous backup that writes to object storage in the standby region:

standby-feed.yaml

mode: backup
backup_id: "orders-dr-feed"

source:
  bootstrap_servers:
    - kafka-0.kafka.svc:9092
    - kafka-1.kafka.svc:9092
  topics:
    include:
      - orders
      - payments

storage:
  backend: s3
  bucket: company-kafka-dr
  region: us-east-1        # standby region, not the primary's
  prefix: backups/prod

backup:
  continuous: true
  include_offset_headers: true
  consumer_group_snapshot: true
  source_cluster_id: "prod-west"

metrics:
  enabled: true
  port: 8080

Two settings carry the failover weight. include_offset_headers stores each record's original offset, which enables offset translation on the standby. consumer_group_snapshot writes every group's committed positions to storage after each cycle, giving the standby an authoritative offset source.

Watch the feed like production, because it is production. Track backup lag against your RPO budget with kafka-backup status --config standby-feed.yaml --watch, and alert on stalled progress before the gap exceeds the target. The RTO/RPO planning guide covers turning those targets into alert thresholds.

The failover decision and promotion runbook

Write the trigger criteria before the incident: who declares failover, on what evidence, and within what time budget. A standby that nobody is authorized to promote has an RTO of one management escalation chain.

Promotion runs in a fixed order. Data first, offsets second, verification third, clients last.

1. Stop the data feed. Halt replication or the backup stream so the recovery point is fixed and failback bookkeeping starts clean.
2. Establish the data boundary. Identify the newest validated recovery point and record it — this is your observed RPO evidence.
3. Restore or finalize data. Warm standbys skip this; pilot-light and cold tiers restore from storage into the standby cluster.
4. Reset consumer groups. Translate committed positions onto the standby's offsets before any consumer connects.
5. Verify with a controlled consumer. One application instance, production side effects disabled, confirming deserialization and processing.
6. Redirect clients. Flip DNS, service discovery, or pushed configuration to the standby's bootstrap servers.

For the restore step, a promotion config looks like this:

promote-standby.yaml

mode: restore
backup_id: "orders-dr-feed"

target:
  bootstrap_servers:
    - kafka-0.dr.svc:9092

storage:
  backend: s3
  bucket: company-kafka-dr
  region: us-east-1
  prefix: backups/prod

restore:
  create_topics: true
  default_replication_factor: 3
  consumer_group_strategy: header-based
  auto_consumer_groups: true
  offset_report: /var/log/kafka-backup/promotion-offsets.json

auto_consumer_groups loads the consumer-group snapshot from storage and resets those groups after the restore completes. The offset_report file becomes drill evidence and the input for troubleshooting any consumer that resumes in the wrong place.

Client redirection deserves preparation, not improvisation. Pre-stage the standby's connection configuration — bootstrap servers, TLS trust, credentials — in every client's deployment pipeline. A DNS flip that clients already trust beats editing twenty application configs at 3 a.m.

Consumer offsets: the hard part of failover

Offsets are cluster-local. Offset 4,200,000 on the primary's orders-3 partition means nothing on the standby, where the same records live at different offsets. Committed positions carried over without translation send consumers to arbitrary points — replaying hours of data or silently skipping it.

Each standby tier has a matching translation mechanism:

Standby feed	Offset translation mechanism
Backup restore	header-based strategy maps original offsets from record headers to restored offsets
Backup restore with group snapshot	auto_consumer_groups resets snapshot groups automatically after restore
MirrorMaker 2 replication	MM2 checkpoint connector emits offset mappings per group into the target cluster

For fine-grained control — or when only some groups should move — generate a reviewable plan instead of resetting everything:

kafka-backup offset-reset plan \
  --path s3://company-kafka-dr/backups/prod \
  --backup-id orders-dr-feed \
  --groups orders-service,payments-service \
  --bootstrap-servers kafka-0.dr.svc:9092

Review the plan, then execute it. The offset management guide documents the full plan/execute/rollback workflow, and the consumer offsets backup post explains why offsets belong in the backup scope at all.

Translation is not exactly-once. Consumers resume from the last translated committed position, so records processed after that commit are processed again. Size that duplicate window against your commit interval, and make important side effects — payments, emails, writes — safe to retry.

Failback to the primary

Failback is a second failover in reverse, and it is harder because the standby has now diverged. Plan it before the first failover, not after.

While the standby serves production, it produces data the primary has never seen. Back up the standby too — the same continuous configuration pointed at the standby cluster covers it. Without that, failback means choosing between losing the incident-period data and reconciling it by hand.

The failback sequence mirrors promotion. Repair and validate the primary, restore the delta produced during the incident, reset consumer groups on the primary, then redirect clients in a planned maintenance window. Unlike failover, failback is schedulable — take the time to verify rather than racing a clock.

Test the architecture before you need it

An untested standby is a hypothesis. Three practices turn it into evidence.

Run validation continuously. kafka-backup validate --path s3://company-kafka-dr/backups/prod --backup-id orders-dr-feed --deep verifies stored data integrity, and kafka-backup validate-restore --config promote-standby.yaml confirms the promotion config against reality — both belong in scheduled jobs, not just drills. The restore and PITR guide covers rehearsing the restore path itself.

Run a promotion drill quarterly. Promote the standby in isolation with production side effects disabled, run the acceptance checks, and measure each stage of the runbook separately. The slowest stage is the one to engineer next.

Keep the evidence. Observed RTO and RPO per drill, compared against approved targets, is what makes the architecture credible to auditors and to your own team.

Kafka active-passive DR checklist

• Standby tier chosen per workload, with cost and failover time approved
• Independent backup in place even where replication feeds the standby
• Standby sized for promoted load, or the degraded mode documented
• Configuration drift checked on a schedule (topics, ACLs, schemas, versions)
• Data feed lag monitored against the RPO budget with alerts
• Promotion runbook written, with named owners and trigger criteria
• Offset translation strategy tested for every consumer group
• Client redirection pre-staged in deployment pipelines
• Standby itself backed up, and the failback sequence written
• Quarterly promotion drill scheduled, with evidence retained

Active-passive is the default Kafka DR architecture for good reason: one cluster serves traffic, one waits, and the failure modes stay understandable. The pattern fails only when the waiting half is neglected.

Start with one workload. Choose its standby tier, feed it, write the promotion runbook, and run the drill. Every step after that is refinement.

Frequently asked questions

Does Kafka replicate message on each cluster?

Not by default. Kafka replicates partitions across brokers inside one cluster. Copying records to a second cluster requires a separate tool such as MirrorMaker 2 for replication or kafka-backup for an independent backup, which is what an active-passive standby is built on.

Why replication is required in Kafka?

Intra-cluster replication keeps partitions available when individual brokers fail, so it protects day-to-day availability. It does not protect against region failure, topic deletion, or corrupt records, because every replica shares the cluster and faithfully copies every change — those risks need a standby cluster and independent backups.

How to backup Kafka?

Run a continuous backup of the required topics to object storage in a separate region, with offset headers and consumer-group snapshots enabled. Validate the stored data on a schedule and prove the backup works by restoring it into a standby cluster during a drill.