Kafka MirrorMaker Best Practices: Production-Ready Cross-Cluster Replication

MirrorMaker 2 is the standard tool for Kafka cross-cluster replication, but the default configuration is rarely production-ready. These Kafka MirrorMaker best practices close the gap between a working tutorial and a replication flow you can trust for disaster recovery and migration. The short version: match tasks to partitions, sync offsets and configs, alert on lag before it hurts, and never run MirrorMaker on your broker nodes.

Key takeaway

Production MirrorMaker 2 comes down to five things: size tasks.max to the partition count, enable offset and config sync so failover works, run it on dedicated nodes, alert on replication lag (warning at 30s, critical at 5min), and test under load before you rely on it. MirrorMaker gives you a warm cluster for failover — it does not protect against corruption or accidental deletion, so pair it with point-in-time backup.

MirrorMaker 2 architecture — a quick refresher

MirrorMaker 2 (MM2) runs on the Kafka Connect framework and replicates data with three connectors working together. Understanding them is the foundation for every tuning decision below.

• MirrorSourceConnector copies topic records and topic configurations from the source cluster to the target.
• MirrorCheckpointConnector translates consumer group offsets so a consumer can resume at the right position after failover.
• MirrorHeartbeatConnector emits heartbeat records you use to measure end-to-end replication latency.

MM2 prefixes replicated topics with the source cluster alias. A topic named orders on a cluster aliased us-east becomes us-east.orders on the target. This prevents name collisions and makes the replication direction obvious. Offset translation is what separates MM2 from raw mirroring — the checkpoint connector maps source offsets to target offsets through an internal offset-syncs topic, so consumers do not replay or skip messages when they fail over.

Configuration best practices

Defaults get MM2 running; they do not make it production-ready. Start from the settings below and adjust to your cluster.

Setting	Recommended value	Why it matters
tasks.max	≥ total partition count of replicated topics	One task per partition maximizes parallelism; too few tasks caps throughput and grows lag
sync.topic.configs.enabled	true	Keeps partition counts and topic configs consistent across clusters
sync.group.offsets.enabled	true	Writes translated consumer offsets to the target so failover is clean
emit.checkpoints.enabled	true	Required for offset translation; without it, consumers cannot resume correctly
emit.heartbeats.enabled	true	Provides the heartbeat topic used to measure replication latency
replication.factor	Match target cluster standard (usually 3)	Protects replicated topics from broker loss on the target
offset-syncs.topic.replication.factor	3	Internal topic; a single replica loses offset translation on broker failure
consumer.auto.offset.reset	earliest	Prevents silently missing records when a task starts on a new topic
refresh.topics.interval.seconds	60–300	Controls how fast new source topics are picked up; lower means faster, more metadata load

A minimal but production-shaped MM2 properties file looks like this:

clusters = us-east, eu-west
us-east.bootstrap.servers = broker1.us-east:9092,broker2.us-east:9092
eu-west.bootstrap.servers = broker1.eu-west:9092,broker2.eu-west:9092

# Enable one-way replication us-east -> eu-west
us-east->eu-west.enabled = true
us-east->eu-west.topics = orders,payments,inventory

# Throughput and correctness
tasks.max = 16
replication.factor = 3
offset-syncs.topic.replication.factor = 3
heartbeats.topic.replication.factor = 3
checkpoints.topic.replication.factor = 3

# Keep clusters consistent and failover clean
sync.topic.configs.enabled = true
sync.group.offsets.enabled = true
emit.checkpoints.enabled = true
emit.heartbeats.enabled = true

# Producer tuning for throughput
us-east->eu-west.producer.batch.size = 65536
us-east->eu-west.producer.linger.ms = 50

Set tasks.max to at least the total partition count of the topics you replicate. MM2 assigns partitions to tasks, so more tasks than partitions wastes resources, and fewer tasks than partitions leaves throughput on the table. Tune producer.batch.size and producer.linger.ms together — larger batches and a small linger raise throughput at the cost of a few milliseconds of latency, which is the right trade for bulk replication.

How do you monitor MirrorMaker 2 replication lag?

You monitor MirrorMaker 2 lag through the JMX metrics its source connector exposes, backed by heartbeat-based end-to-end latency. Lag is the single most important signal — it tells you how far behind your DR target is at any moment.

MM2 publishes these metrics under the kafka.connect.mirror domain per replicated topic and partition:

• replication-latency-ms — time between a record being produced at the source and written at the target. This is your primary lag signal.
• record-age-ms — age of the record when MM2 consumes it, useful for spotting consumer-side backpressure.
• record-count — records replicated, which you turn into a throughput rate.
• byte-count — bytes replicated, for bandwidth planning.

Scrape these with the Prometheus JMX exporter and alert on replication-latency-ms. Use heartbeat records for a second, independent latency measurement that does not depend on data topics having traffic. A sensible starting alert policy:

Signal	Warning	Critical
Replication latency	30s	5min
Connector/task state	any task FAILED	connector FAILED
Throughput drop	50% below baseline for 5min	replication stalled (0 records)

Watch connector and task health through the Kafka Connect REST API at /connectors/<name>/status. A task that flips to FAILED stops replicating its partitions silently, so treat any failed task as a page-worthy event, not a warning. For how these signals map to OSO Kafka Backup's own reporting, see the metrics reference.

Operational best practices

Configuration gets replication correct; operations keep it correct under change and load.

• Run MirrorMaker on dedicated nodes. Co-locating MM2 with brokers means a replication spike competes with broker I/O and page cache. Give it its own Connect workers so a traffic burst never degrades the clusters it connects.
• Use separate consumer groups per replication flow. Each source-to-target flow should have its own group so lag and offsets are isolated and debuggable.
• Fail over gracefully. Stop producers to the source, let MM2 drain until replication latency reaches zero, confirm consumer offsets have synced, then promote the target. Cutting over before lag hits zero loses in-flight records.
• Decide topic creation deliberately. Let MM2 auto-create topics for speed, or pre-create them when you need specific partition counts, retention, or compaction that differ from the source.
• Upgrade with rolling restarts. MM2 runs as Connect workers, so restart them one at a time and verify tasks rebalance and resume before moving to the next.

Graceful failover is where most DR plans fall apart. The moment to verify lag is zero is before you promote the target, not after users report missing data. Build the drain-and-verify step into a runbook and rehearse it — see the Kafka disaster recovery guide for how MirrorMaker fits a full DR architecture.

Common pitfalls and how to avoid them

These are the failures that turn a working MM2 setup into a 2am incident.

• Topic name collisions in active-active. When both clusters replicate to each other, distinct source aliases keep us-east.orders and eu-west.orders separate. Reusing an alias creates loops and overwrites.
• Unbounded lag during traffic spikes. If tasks.max is sized for average load, a peak buries the replica. Pre-scale tasks to peak partition throughput, not the average, and alert early so you can add capacity before lag becomes hours.
• Offset translation gaps. If the checkpoint connector is not running, consumers fail over to wrong offsets and replay or skip data. Keep emit.checkpoints.enabled and sync.group.offsets.enabled on, and monitor the checkpoint connector's health like any other.
• Security config mismatch between clusters. SASL or TLS parity is not optional. A source on SASL_SSL and a target expecting PLAINTEXT fails silently at connector startup. Verify security configs match on both sides before enabling a flow.

Where MirrorMaker stops and backup begins

MirrorMaker 2 is a replication tool, and replication is not backup. This distinction is the one that costs teams data. MM2 faithfully copies every write from source to target — including the write from a buggy producer and the DELETE from a fat-fingered operator. If corruption lands on the source, it lands on the replica milliseconds later. A warm replica of corrupted data recovers nothing.

MirrorMaker protects availability: lose a region, fail over to the target, keep serving traffic. It does not protect recoverability: it cannot rewind a topic to the moment before a bad deploy. For that you need an independent, restorable copy. OSO Kafka Backup writes topic data and consumer group offsets to object storage — Amazon S3 and S3-compatible stores, Azure Blob, GCS, or a filesystem — with Zstd or LZ4 compression, and restores to a chosen timestamp with millisecond precision. It is open source under the MIT license, ships a Kubernetes operator, and is Strimzi compatible.

The production pattern pairs the two: MirrorMaker 2 for fast regional failover, point-in-time backup for the corruption, deletion, and compliance cases replication cannot cover. For the full comparison, see OSO Kafka Backup vs Kafka MirrorMaker 2 and the Kafka backup tools compared breakdown.

Conclusion

MirrorMaker 2 is a capable, free replication tool, but the defaults will not carry you to production. Size tasks.max to your partitions, turn on offset and config sync so failover is clean, run it on dedicated nodes, and alert on replication lag before it becomes a recovery problem. Rehearse graceful failover, and verify lag is zero before you promote a target.

Above all, remember what MirrorMaker is for. It keeps you available when a region goes down. It does not make you recoverable when data goes bad. Pair it with continuous point-in-time backup, test both under load, and you have replication you can actually rely on. The replication across data centers guide covers the multi-region topology, and the getting started guide has the exact steps to add the backup layer.

Frequently asked questions

What are the best practices for Kafka MirrorMaker?

The core MirrorMaker 2 best practices are: size tasks.max to at least the total partition count of replicated topics, enable sync.group.offsets.enabled and emit.checkpoints.enabled so consumer offsets translate for clean failover, enable sync.topic.configs.enabled to keep clusters consistent, run MirrorMaker on dedicated nodes rather than broker nodes, and alert on replication latency with a warning at 30 seconds and critical at 5 minutes. Test replication under load before relying on it for disaster recovery.

How do you monitor MirrorMaker 2 replication lag?

Monitor MirrorMaker 2 lag through the JMX metrics its source connector exposes under the kafka.connect.mirror domain, primarily replication-latency-ms, which measures the time between a record being produced at the source and written at the target. Scrape these metrics with the Prometheus JMX exporter, use heartbeat records for an independent end-to-end latency measurement, and watch connector and task state through the Kafka Connect REST API. Alert on latency at 30 seconds (warning) and 5 minutes (critical).

How many tasks should MirrorMaker use?

Set tasks.max to at least the total partition count of the topics you replicate, aiming for one task per partition. MirrorMaker 2 assigns partitions to tasks, so more tasks than partitions wastes resources while fewer tasks than partitions caps throughput and lets replication lag grow. Size tasks to peak partition throughput rather than average load so traffic spikes do not bury the replica.

What is MirrorMaker offset translation?

Offset translation is how MirrorMaker 2 maps a consumer group offset on the source cluster to the equivalent offset on the target cluster, so a consumer resumes at the correct position after failover instead of replaying or skipping messages. The MirrorCheckpointConnector performs this translation through an internal offset-syncs topic. It requires emit.checkpoints.enabled to be true, and sync.group.offsets.enabled writes the translated offsets to the target automatically.

Should you run MirrorMaker on broker nodes?

No. Run MirrorMaker 2 on dedicated Kafka Connect worker nodes, not on your broker nodes. Co-locating MirrorMaker with brokers means a replication spike competes with broker I/O and page cache, so a traffic burst can degrade the very clusters MirrorMaker connects. Dedicated nodes isolate replication load and let you scale MirrorMaker independently of the brokers.