Kafka MirrorMaker 2 Setup Tutorial: Configure, Run, and Verify Replication

A Kafka MirrorMaker 2 setup needs two cluster aliases, a directional flow, topic and group filters, and a running dedicated Connect process. You must then verify remote topics, copied records, translated consumer offsets, and measured replication delay.

MirrorMaker 2 provides asynchronous cross-cluster replication. It does not keep a historical restore point, and a running connector does not prove that an application can fail over safely.

Key takeaway

A valid MirrorMaker 2 setup copies the right records and supports a rehearsed consumer cutover. Check topic names, offsets, task health, and replication delay before treating the target as a standby.

Kafka MirrorMaker 2 setup at a glance

This tutorial creates a one-way flow from a primary cluster to a secondary cluster. It replicates orders and a dedicated mm2-smoke test topic. It also translates offsets for the order-service consumer group.

The introductory Kafka MirrorMaker guide explains the connector architecture. This tutorial starts with the deployment decisions and ends with a failover drill.

Stage	Action	Evidence to keep
Plan	Choose flow direction, topics, groups, and cluster aliases	Approved source-to-target map
Connect	Allow network access and grant Kafka permissions	Successful client connection to both clusters
Configure	Write connect-mirror-maker.properties	Reviewed filters and replication factors
Start	Launch the dedicated MirrorMaker process	Healthy process and connector tasks
Verify	Check remote topics, records, and offsets	Repeatable smoke-test output
Rehearse	Cut a test consumer over to the target	Timed drill with rollback steps

Before starting, confirm these prerequisites:

• Two reachable Kafka clusters with known bootstrap servers
• A Kafka distribution that includes the MirrorMaker 2 scripts
• A test topic named mm2-smoke on the source cluster
• Broker counts that support the chosen internal-topic replication factors
• Permissions to read the source and write the target
• A target cluster sized for replicated traffic and failover traffic

The default replication policy adds the source alias to remote topic names. orders therefore becomes primary.orders on the target. Clients must use the remote name unless you deliberately adopt another replication policy.

Step 1: Plan the replication flow

A flow has a source and a target. primary->secondary copies records from the primary cluster to the secondary cluster. It does not copy records in the other direction.

Use clear aliases that remain stable across configuration files, dashboards, and runbooks. An alias becomes part of each remote topic name under the default policy. Renaming it later changes what applications expect to consume.

Start with explicit topic and group allowlists. Broad patterns can copy internal or short-lived data that the standby never needs. They can also increase network, storage, and operational load without warning.

This tutorial uses an active-passive pattern:

Decision	Tutorial value	Reason
Source alias	primary	Identifies the active cluster
Target alias	secondary	Identifies the standby cluster
Enabled flow	primary->secondary	Keeps the example one-way
Topics	orders,mm2-smoke	Covers one application topic and one test topic
Consumer groups	order-service	Limits checkpoint work to the failover group

Run MirrorMaker close to the target cluster when the deployment permits it. The consumer reads remotely while the producer writes locally. This placement keeps producer traffic on the lower-latency side of the link.

Step 2: Create connect-mirror-maker.properties

Create a file named connect-mirror-maker.properties beside your deployment scripts. Replace the broker names with endpoints from your clusters.

connect-mirror-maker.properties

clusters = primary, secondary

primary.bootstrap.servers = primary-1:9092,primary-2:9092,primary-3:9092
secondary.bootstrap.servers = secondary-1:9092,secondary-2:9092,secondary-3:9092

primary->secondary.enabled = true
secondary->primary.enabled = false

primary->secondary.topics = orders,mm2-smoke
primary->secondary.groups = order-service

tasks.max = 4
replication.factor = 3
checkpoints.topic.replication.factor = 3
heartbeats.topic.replication.factor = 3
offset-syncs.topic.replication.factor = 3

primary->secondary.emit.checkpoints.enabled = true
primary->secondary.emit.heartbeats.enabled = true
primary->secondary.sync.group.offsets.enabled = true
primary.consumer.isolation.level = read_committed

The first block defines both Kafka clusters. The second block enables only the primary-to-secondary flow. MirrorMaker will not copy records back to primary.

The topic and group properties are regular-expression filters. Comma-separated literal names work for this small allowlist. Test any wider pattern against an inventory before deploying it.

tasks.max sets the maximum task count for each connector. Start from measured partition load, worker capacity, and the number of processes. More tasks do not help after every source partition has an active reader.

The four replication-factor properties cover remote and internal topics. A factor of three requires at least three eligible brokers in every cluster that stores one of those topics. Reduce it for a two-broker lab, but do not copy that lab setting into production.

Checkpoints map consumer positions between the source and target logs. Offset syncs provide the mapping that checkpoint translation needs. Direct group-offset sync writes translated positions into the target cluster while that target group is inactive.

read_committed prevents the MirrorMaker consumer from copying records from aborted transactions. It matters when source producers use Kafka transactions.

Apache Kafka documents the current syntax in its geo-replication guide. Review the version matching your Kafka distribution before deployment.

Understand the remote topic name

The default DefaultReplicationPolicy prefixes each source topic with its cluster alias. These are the names created by the tutorial:

Source topic	Target topic	Target consumers read
orders	primary.orders	primary.orders
mm2-smoke	primary.mm2-smoke	primary.mm2-smoke

The prefix records origin and prevents loops in multi-cluster topologies. It also means a failover consumer cannot always keep its original topic subscription. Include the target topic name in the application cutover plan.

Step 3: Add authentication and authorization

Most Kafka clusters reject plaintext clients. Add the client settings required by each cluster before starting MirrorMaker. This TLS shape uses truststores and contains no embedded passwords:

primary.security.protocol = SSL
primary.ssl.truststore.location = /etc/mm2/primary-truststore.p12
primary.ssl.truststore.type = PKCS12

secondary.security.protocol = SSL
secondary.ssl.truststore.location = /etc/mm2/secondary-truststore.p12
secondary.ssl.truststore.type = PKCS12

Provide passwords through the secret method supported by your deployment. Do not commit credentials to the MirrorMaker file or bake them into an image.

MirrorMaker needs permission to discover and read the selected source topics. It also needs access to the selected consumer groups and offset-sync data. On the target, it must create, describe, and write remote and internal topics.

ACL synchronization needs additional authority and a separate security review. Disable that feature when target access rules must remain independently managed. Test permissions with the same principal that the MirrorMaker process will use.

Step 4: Start MirrorMaker 2

Start dedicated mode with the script shipped in the Kafka distribution:

bin/connect-mirror-maker.sh connect-mirror-maker.properties

This command stays in the foreground and writes logs to the process output. It works for an initial test. A production deployment needs a supervisor, resource limits, restart policy, log collection, and a controlled shutdown path.

MirrorMaker discovers topics and groups after startup, so the first copy may not appear immediately. Wait for connector initialization before judging the flow. Configuration changes require a MirrorMaker process restart.

For more than one process, use an identical configuration for processes sharing a target. Conflicting flow definitions can race through the target cluster's shared Connect state.

Step 5: Verify topic and record replication

A running process is only the first signal. Verify the remote topic, then send a known record through the full path.

Describe the target topic:

bin/kafka-topics.sh \
  --bootstrap-server secondary-1:9092 \
  --describe \
  --topic primary.mm2-smoke

Confirm the partition count, assigned replicas, and in-sync replicas. Compare the target partition count with the source. Also review important topic settings such as retention before approving the standby.

Write one test record to the source:

printf 'mm2-smoke-2026-07-13\n' | bin/kafka-console-producer.sh \
  --bootstrap-server primary-1:9092 \
  --topic mm2-smoke

Read the remote topic on the target:

bin/kafka-console-consumer.sh \
  --bootstrap-server secondary-1:9092 \
  --topic primary.mm2-smoke \
  --from-beginning \
  --max-messages 1

The consumer should print the test value. Topic existence alone is insufficient. An old remote topic can remain present while a connector task is stopped or blocked.

Use a unique smoke topic or unique keyed record in repeatable checks. A shared application topic can return an older record and create a false pass.

Step 6: Verify checkpoints and rehearse failover

MirrorMaker offset translation connects a source consumer position to its target position. The source and remote logs have different offsets, so copying the raw number is unsafe.

Inspect the application group on the source:

bin/kafka-consumer-groups.sh \
  --bootstrap-server primary-1:9092 \
  --describe \
  --group order-service

Then inspect the translated group on the target:

bin/kafka-consumer-groups.sh \
  --bootstrap-server secondary-1:9092 \
  --describe \
  --group order-service

Allow at least one configured checkpoint and group-sync interval before comparing the results. The target group must be inactive for MirrorMaker to write synced offsets into its __consumer_offsets topic.

Do not compare source and target numbers directly. Confirm that the target group points at the expected remote partitions. Start a test consumer and verify that it neither skips accepted records nor replays beyond your tolerance.

A basic failover drill follows this order:

1. Pause test producers or define a clear final source record.
2. Wait until measured replication delay meets the drill threshold.
3. Stop the source consumer and fence any process that could resume it.
4. Confirm the target group has translated offsets.
5. Start the consumer against primary.orders on the secondary cluster.
6. Validate processing, downstream effects, credentials, and service routing.
7. Record elapsed time, duplicate handling, lost-record checks, and rollback steps.

The Kafka disaster recovery guide explains how this drill fits RTO and RPO planning. Repeat it after client, security, topic, and deployment changes.

Production checks after the first successful flow

MirrorMaker inherits Kafka Connect metrics and adds metrics in the kafka.connect.mirror group. Monitor the replication path and the target Kafka cluster together.

Signal	What it answers	Failure indicated
replication-latency-ms	How long records take to reach the target	Cross-cluster delay exceeds the RPO budget
record-age-ms	How old a record is when copied	Source reading is falling behind
Connector and task state	Is every assigned task running?	One or more partitions may have stopped copying
Heartbeat age	Can MirrorMaker still traverse the flow?	Link, worker, or target failure
Target broker health	Can the standby accept and retain writes?	Replication exists but failover capacity is weak
Consumer position	Can the application resume correctly?	Checkpoint or group-sync failure

Test behavior during link loss, a worker restart, new topic creation, and a target broker failure. After restoring the link, measure how long MirrorMaker takes to clear its backlog.

The MirrorMaker best-practices guide covers worker placement, lag alerting, scaling, and recurring drills in more detail.

Common MirrorMaker 2 setup failures

Most first-run failures come from flow direction, filters, permissions, or topic naming. Start with those checks before changing throughput settings.

Symptom	Likely cause	Check or fix
No remote topic	Flow disabled or topic filter matches nothing	Check primary->secondary.enabled and the topic allowlist
Internal topic creation fails	Replication factor exceeds eligible broker count	Match the factor to each cluster's broker count
Authorization errors	MirrorMaker principal lacks source or target permissions	Review broker audit logs and ACLs for the exact principal
Target topic looks empty	Consumer reads orders instead of primary.orders	Use the remote topic name from the replication policy
Test group has no checkpoint	Console consumer groups are excluded by default	Test with the allowlisted application group
Target offsets do not update	The target group is active	Stop target consumers before direct group-offset sync
Topic exists but new data stops	Connector task failed after initial creation	Check every connector task and replication metric

Avoid changing several settings at once. Preserve the failing configuration and logs, isolate one cause, and rerun the same smoke check.

MirrorMaker 2 replication is not backup

MirrorMaker keeps another Kafka cluster near the current source state. That is useful for regional availability, migrations, aggregation, and warm standby designs.

It also copies the live stream forward. Bad records can reach the target. Topic deletions, retention changes, compromised credentials, or application mistakes may affect both environments.

Independent backup adds a separate storage and retention boundary. It can support recovery to an earlier point after both live clusters contain the same bad state. The Kafka backup strategies guide compares these failure scopes.

OSO Kafka Backup stores topic data and consumer group offsets in S3, S3-compatible storage, Azure Blob, GCS, or a filesystem. It supports point-in-time restore with millisecond precision.

Add recovery beyond live replication

Keep an independent, restorable copy of Kafka topics and consumer offsets outside both live clusters. Start with OSO Kafka Backup.

Finish with evidence, not process state

A successful Kafka MirrorMaker 2 setup produces evidence at every layer. The process runs, the target topic exists, a new record arrives, offsets translate, and a test consumer resumes at the expected position.

Keep the smoke test and failover drill in the operating runbook. Measure delay under normal load and failure conditions. Pair replication with independent backup when recovery must reach an earlier state.

Frequently asked questions

Does Kafka replicate messages on each cluster?

Native Kafka replication copies partition data among brokers inside one cluster. It does not automatically copy records to separate clusters. MirrorMaker 2 creates an explicit directional flow from a source cluster to a target cluster.

How does Kafka replication work?

Within one cluster, follower replicas fetch partition data from a leader. MirrorMaker 2 handles a different scope. Its Kafka Connect tasks consume selected source records, write them to remote target topics, and emit metadata used for consumer offset translation.

How do I verify Kafka follower replication is accurate?

For MirrorMaker 2, describe the remote topic, send a unique source record, consume that record from the target, and inspect connector metrics. Also verify translated consumer positions with a failover drill. Topic existence or a running process alone is not enough.

Why is replication required in Kafka?

Replication keeps data available across a defined failure boundary. Native replicas cover broker failures inside a cluster. MirrorMaker 2 can cover cluster or site availability. Independent backup is still needed when recovery must return to an earlier state.