What Is Kafka MirrorMaker? Architecture, Use Cases, and Limits

Kafka MirrorMaker 2 is Apache Kafka's framework for copying records and selected metadata between independent Kafka clusters. It runs on Kafka Connect, reads from a source cluster, and writes to a target cluster through a directional replication flow.

MirrorMaker helps with regional availability, cluster migration, and centralized data access. It does not create an independent historical backup, and it cannot restore a topic to a time before a bad record or deletion.

Key takeaway

Use MirrorMaker 2 when another Kafka cluster needs a recent copy of selected topics. Use native Kafka replication for broker failures, and use independent backup when you must recover an earlier state.

What is Kafka MirrorMaker?

Kafka MirrorMaker is a cross-cluster replication system included with Apache Kafka. A replication flow consumes records from one cluster and produces them to another. The source and target remain separate Kafka clusters with their own brokers, metadata, storage, and clients.

When engineers discuss MirrorMaker now, they generally mean MirrorMaker 2. This version replaced the older consumer-and-producer design with connectors built on Kafka Connect. That change added cluster discovery, remote topic naming, topic setting sync, consumer offset checkpoints, and heartbeat records.

MirrorMaker flows are directional. A flow from primary to secondary does not copy data back unless you also enable a secondary to primary flow. That makes one-way disaster recovery and two-way regional topologies possible from the same model.

MirrorMaker is separate from native partition replication. Native replication keeps copies of a partition on brokers inside one cluster. MirrorMaker moves records across cluster boundaries, usually over a network link between sites or regions.

The Kafka replication explained guide covers leaders, followers, and in-sync replicas inside one cluster. MirrorMaker begins where that failure boundary ends.

How does the MirrorMaker 2 architecture work?

MirrorMaker 2 uses three Kafka Connect connector types. Each one has a narrow role in the replication path.

Connector	Main job	Why it matters
MirrorSourceConnector	Reads selected source topics and writes their records to remote topics	Creates the cross-cluster data copy and can synchronize allowed topic settings
MirrorCheckpointConnector	Emits mappings between source and target consumer offsets	Helps consumer groups find the corresponding position after a cluster switch
MirrorHeartbeatConnector	Emits heartbeat records between clusters	Shows that the replication path is reachable and supports latency checks

The source connector discovers matching topics and partitions. Its tasks consume records, preserve the record key and value bytes, and produce the records to the target. More tasks allow work to be spread across MirrorMaker processes, subject to the number of replicated partitions and available capacity.

The checkpoint connector handles a different problem. Source and target offsets are not automatically identical because remote topics have their own logs. Checkpoint records map a consumer group's source position to a corresponding target position. Operators can use those mappings when preparing a failover.

Heartbeats confirm that a source-to-target path is active. They do not prove that every business topic is current. Monitor connector state, per-topic replication lag, and target consumption as well.

How remote topic names prevent loops

The default replication policy prefixes a remote topic with its source cluster alias. If orders comes from primary, the target topic becomes primary.orders.

The prefix preserves origin and prevents a replicated topic from cycling between clusters. It also prevents two clusters' local orders topics from being merged by accident. Applications must account for remote names when they switch clusters or aggregate data.

Apache Kafka also supports a custom replication policy. Treat any naming change as an architecture decision. Producers, consumers, access rules, dashboards, and runbooks all need to agree on the resulting topic names.

A minimal Kafka MirrorMaker 2 flow

MirrorMaker's dedicated mode commonly reads a connect-mirror-maker.properties file. This example defines two clusters and a one-way flow for the orders, payments, and inventory topics:

connect-mirror-maker.properties

clusters = primary, secondary

primary.bootstrap.servers = primary-1:9092,primary-2:9092
secondary.bootstrap.servers = secondary-1:9092,secondary-2:9092

primary->secondary.enabled = true
secondary->primary.enabled = false

primary->secondary.topics = orders,payments,inventory
primary->secondary.groups = order-service,payment-service

tasks.max = 4
replication.factor = 3
checkpoints.topic.replication.factor = 3
heartbeats.topic.replication.factor = 3
offset-syncs.topic.replication.factor = 3

Start the dedicated MirrorMaker process with the script included in an Apache Kafka distribution:

bin/connect-mirror-maker.sh connect-mirror-maker.properties

The cluster aliases identify each side of a flow. Bootstrap servers let the Kafka clients discover brokers. The topics and groups lists define the data and consumer groups in scope. Internal-topic replication factors should match the target cluster's durability policy and available broker count.

This is only a shape for the flow. A production deployment also needs TLS or SASL settings, access rules, worker storage topics, capacity planning, metrics, and a tested restart process. The current Apache Kafka geo-replication guide documents the supported property syntax and deployment modes.

When should you use Kafka MirrorMaker?

MirrorMaker fits problems where Kafka records must remain available or useful in another cluster.

Active-passive disaster recovery

A primary cluster serves applications while MirrorMaker copies selected topics to a standby cluster. After a primary-site failure, operators stop or fence the old writers, translate consumer positions, and direct applications to the standby.

The copy is asynchronous. Any records still in flight when the source disappears may be outside the recovery point. Measure lag under normal load and degraded network conditions before setting a recovery point objective.

Active-active regional service

Two directional flows can distribute records between regions. Each region can serve nearby applications and receive remote topics from the other.

MirrorMaker prevents topic loops through its remote naming policy, but it does not resolve application-level write conflicts. Define which region owns each topic or key space. Also define how consumers treat local and remote topics.

Cluster migration

MirrorMaker can keep a new cluster current while applications move in stages. Teams can verify target data, switch consumers, switch producers, and retire the old cluster after lag reaches the accepted threshold.

Migration still requires a cutover plan. Topic settings, security, quotas, client metadata, and consumer positions must be checked separately from record flow.

Aggregation and regional data access

Teams can copy selected topics from several operational clusters into a central Kafka cluster for analytics or downstream processing. They can also place data near regional consumers without making those consumers read over a long-distance link.

Check data residency rules before enabling a broad topic pattern. MirrorMaker can discover new matching topics automatically, so a future topic may enter the flow without a separate deployment.

MirrorMaker 2 strengths and tradeoffs

MirrorMaker uses Kafka's own client and Connect APIs, so it works with Apache Kafka clusters without adding a separate storage format. Its topic filters, directional flows, remote naming, checkpoints, and task model cover many common cross-cluster designs.

Those strengths come with operating costs. The target needs enough broker, disk, and network capacity for the copied workload. MirrorMaker workers need their own capacity, security configuration, monitoring, and upgrade path. A slow or broken link increases lag rather than turning the copy into a synchronous write.

Requirement	Native Kafka replication	MirrorMaker 2	Independent backup
Survive one broker loss	Yes, with suitable replica placement	Not its main role	Restore is slower than leader election
Keep data in another Kafka cluster	No	Yes	Only after a restore
Switch applications to a warm cluster	No	Yes, with a tested cutover	No warm cluster by itself
Recover before a bad write or deletion	No	No	Yes, with retained restore points
Store data outside a live Kafka cluster	No	No	Yes

The right choice is often a combination. Native replicas keep each active cluster available. MirrorMaker keeps another cluster near the present. Backup keeps an older recoverable state outside the live replication path.

For operating guidance after the architecture decision, read Kafka MirrorMaker best practices. It covers task sizing, checkpoints, monitoring, internal topics, and failover drills.

What does MirrorMaker not protect against?

MirrorMaker copies the source log's current record stream. That behavior means a valid but harmful event can reach the target just like a good event. A mistaken producer write, tombstone, topic deletion, or retention decision can remove the standby's value as a clean recovery point.

MirrorMaker also does not write an archival copy to object storage. Both copies remain active Kafka topics governed by cluster retention and administration. You cannot ask MirrorMaker to reconstruct the topic exactly as it existed before an incident.

Consumer checkpoints help with position translation, but they do not switch an application. A failover still needs target credentials, network routes, topic names, producer fencing, consumer validation, and a clear decision owner.

These limits do not make MirrorMaker a poor DR tool. They define its role. MirrorMaker is an availability and data-mobility layer. Independent backup is a historical recovery layer.

OSO Kafka Backup stores topic data and consumer group offsets in Amazon S3, S3-compatible storage, Azure Blob, GCS, or a filesystem. It supports point-in-time recovery with millisecond precision. See the disaster recovery documentation and the backup comparison for the protection boundary.

How to decide whether MirrorMaker fits

Start with the failure and recovery outcome, not the tool name.

1. Define the failure boundary. Broker loss needs native replication. Cluster or region loss may need MirrorMaker. Logical data loss needs an independent restore point.
2. Set an RPO from measured lag. Decide how much cross-cluster delay the business accepts, then test the flow at peak throughput and during link loss.
3. Choose direction and ownership. Define one-way or two-way flows. For two-way flows, assign topic or key ownership before applications write.
4. Plan the target as a real cluster. Size storage, brokers, network, quotas, and retention for both replicated traffic and failover traffic.
5. Treat checkpoints as one cutover input. Test consumer translation together with credentials, DNS or service routing, topic names, and producer fencing.
6. Monitor the whole path. Watch connector tasks, replication lag, heartbeat age, target broker health, and the ability of a test consumer to read.
7. Add a separate recovery history. Keep backups when deletion, corruption, ransomware, or rollback to an earlier time is in scope.

The cross-datacenter replication guide helps place MirrorMaker in a multi-site design. The Kafka backup strategies guide covers retention, restore testing, and the boundary between replication and backup.

MirrorMaker in one sentence

Kafka MirrorMaker 2 copies selected Kafka records and supporting metadata between independent clusters through directional Kafka Connect flows.

That makes it useful for a warm standby, regional distribution, migration, and aggregation. Design the application cutover and monitor lag as carefully as the replication process. Add independent backup when recovery must reach further back than the target cluster's current log.

Frequently asked questions

Does Kafka replicate messages on each cluster?

Native Kafka replication copies each partition across brokers inside one cluster. It does not automatically copy messages to every separate Kafka cluster. MirrorMaker 2 creates explicit directional flows that copy selected records from a source cluster to a target cluster.

What is replication in Kafka?

Kafka replication keeps additional copies of partition data for availability. Native replication works among brokers in one cluster. Cross-cluster replication tools such as MirrorMaker 2 consume records from one Kafka cluster and produce them to another independent cluster.

How does Kafka replication work?

Inside a cluster, followers copy the ordered partition log from its leader and can take over after a broker failure. Between clusters, MirrorMaker 2 connector tasks consume selected source partitions and produce those records to remote topics in a target cluster.

Why is replication required in Kafka?

Replication keeps data available when a broker or a larger failure domain becomes unavailable. Native replicas address broker and rack failures. Cross-cluster replication can address cluster or site failure. Neither provides a historical restore point, so backups remain necessary when earlier recovery is required.