MSK ZooKeeper to KRaft: there is no in-place path

Why MSK ZooKeeper to KRaft is a cluster move​

An MSK ZooKeeper to KRaft migration cannot happen in place. Self-managed Kafka migrates with a rolling dual-write procedure (KIP-866), but MSK does not expose that path. The only route is a new KRaft-mode cluster — MSK supports KRaft from version 3.7.x — and a controlled move of everything on the old one.

That reframes the problem. The hard part is not provisioning the new cluster. It is moving topic data, topic configurations, ACL bindings, and consumer group offsets so that applications resume exactly where they stopped. Offsets are the usual casualty: partition offsets on the new cluster start from zero, so every committed offset from the old cluster points at the wrong record.

Three ways to move an MSK cluster​

MirrorMaker 2 is the standard answer, and it is genuinely good at what it was built for: continuous, low-lag replication between live clusters, with no license cost. For an ongoing active/active topology it is the right tool. A migration is a different job — it needs a clean cutover, exact offset translation, and proof that nothing was lost.

The pipeline runs as a journaled state machine: plan, precheck, topology copy, seed, tail, drain, cutover, validation, finalize. Before cutover the source cluster is never modified, so rollback stays open the whole way. The full lifecycle is documented in the MSK KRaft migration docs.

Start with the free plan and precheck​

plan and precheck run without a license against your real clusters. They are read-only: you get a generated runbook, IAM policy templates, a cost estimate, and a readiness report before deciding anything.

enterprise:
msk_kraft_migration:
  source:
    cluster_arn: arn:aws:kafka:us-east-1:123456789012:cluster/my-zk-cluster/abc-123
    auth:
      mode: iam
  target:
    cluster_arn: arn:aws:kafka:us-east-1:123456789012:cluster/my-kraft-cluster/def-456
    auth:
      mode: iam
  backup:
    s3_bucket: my-migration-segments
    s3_prefix: migrations/
  evidence:
    s3_bucket: my-migration-evidence
    s3_prefix: evidence/

kafka-backup migrate msk-kraft plan \
--config migration.yaml \
--format all \
--out-dir ./migration-plan

kafka-backup migrate msk-kraft precheck --config migration.yaml

kafka-backup migrate msk-kraft execute \
--config migration.yaml \
--journal-dir ./journal

Precheck flags blockers and warnings — unreachable brokers, S3 permission gaps, cross-region egress, compacted topics — each with a remediation code in the precheck reference. Cross-auth moves work too: a SCRAM source can migrate to an IAM target, with ACLs emitted as an IAM policy map.

What "done" looks like​

Before the migration finalizes, five automated checks run: topic parity, record counts and offsets, spot-check record equality, cutover sentinel presence, and consumer group reconciliation. The result is an Ed25519-signed evidence bundle in S3 — the answer to "prove every record made it and every consumer resumes in the right place."

If you are still planning the wider move — or your Kafka is not on MSK — start with the platform-by-platform ZooKeeper to KRaft migration page or the complete migration guide on the blog.