ZooKeeper to KRaft Migration: The Complete Guide (MSK, Strimzi, Confluent)
How you run a ZooKeeper to KRaft migration depends entirely on where your Kafka lives. Self-managed clusters migrate in place with a rolling, dual-write procedure (KIP-866) through the Kafka 3.9 bridge release. Strimzi drives that same procedure on Kubernetes with a single annotation, Confluent Platform runs it on versions 7.6.1 through 7.9 before the KRaft-only 8.0 upgrade — and AWS MSK has no in-place path at all, which turns the migration into a cluster move.
The deadline is upstream and fixed: ZooKeeper mode was deprecated in Apache Kafka 3.5 and removed in 4.0. If you are still weighing the two modes, the KRaft vs ZooKeeper comparison settles that question. This guide assumes the decision is made and covers the part that actually carries risk: moving every cluster you have, on each platform, without losing metadata, offsets, or your rollback option.
All in-place migrations are the same KIP-866 machinery: a KRaft controller quorum copies metadata out of ZooKeeper while dual-writing keeps a way back until you finalize. Know your platform's last exit — self-managed before finalization, Strimzi at KRaftPostMigration, Confluent at dual-write, MSK before client cutover — and take an independent backup before touching cluster metadata.
Every ZooKeeper to KRaft migration path at a glance
| Platform | Migration path | Minimum versions | Last rollback point |
|---|---|---|---|
| Self-managed Apache Kafka | In-place KIP-866 rolling migration via the 3.9 bridge release | Production-ready since 3.6; migrate on the latest 3.9 patch | Before controllers restart without the migration flag |
| Strimzi on Kubernetes | Same flow, driven by the strimzi.io/kraft annotation | Strimzi 0.40+ with Kafka 3.7+; must finish before Strimzi 0.46 | KRaftPostMigration state |
| Confluent Platform | In-place migration on CP 7.5–7.9 | GA at 7.6.1, recommended 7.7.0+; required before CP 8.0 | While the cluster is in dual-write mode |
| AWS MSK | No in-place conversion — new KRaft cluster plus data, config, ACL, and offset migration | KRaft-mode clusters available from Kafka 3.7.x | Any time before client cutover; the source cluster is never modified |
The first three rows are the same mechanism wearing different clothes. Strimzi and Confluent both orchestrate the upstream KIP-866 procedure; what differs is who performs the restarts and where the safety checks live. MSK is the odd one out: AWS does not expose the migration machinery, so the only route to KRaft is a new cluster and everything that entails.
How the migration actually works (KIP-866 dual-write)
The first thing to internalize: an in-place migration moves cluster metadata, not data. Topic partitions stay on the same disks throughout. Consumer group offsets live in __consumer_offsets — a regular Kafka topic — so they survive untouched. What moves is the cluster's brain: topic configurations, partition assignments, ACLs, and controller state, copied from the ZooKeeper ensemble into a KRaft controller quorum.
The Kafka 3.9 documentation describes the migration as a sequence of phases:
- Initial phase — all brokers in ZooKeeper mode, a ZooKeeper-based controller in charge.
- Initial metadata load — the new KRaft quorum reads the full metadata snapshot out of ZooKeeper.
- Hybrid phase — some brokers still in ZooKeeper mode, but the KRaft controller now leads.
- Dual-write phase — every broker runs in KRaft mode, and the controller keeps writing metadata to ZooKeeper as well.
- Finalized — ZooKeeper writes stop. The migration is complete and irreversible.
Dual-writing is the safety net. Because ZooKeeper stays current through phase 4, you can revert to ZooKeeper mode at any step before finalization. The moment you finalize, that net is gone.
Two hard rules from the upstream docs apply to every in-place path. Do not change the inter.broker.protocol.version or metadata version while a migration is in flight — the docs are blunt that this may break the cluster. And if a broker runs multiple log directories, a directory failure during the migration shuts that broker down until the directory is repaired (KAFKA-16431), so enter the window with healthy disks.
Migrating self-managed Kafka from ZooKeeper to KRaft
Migration has been production-ready since Kafka 3.6, but 3.9 is the bridge release every ZooKeeper cluster should migrate through — it received the final round of migration fixes, and 4.0 will not start against ZooKeeper at all. So step zero is a standard rolling upgrade of all brokers to 3.9, with inter.broker.protocol.version set to match.
Provision the KRaft controllers. The new controller quorum must carry the same cluster ID as the existing cluster. Read it from any broker's meta.properties, or ask ZooKeeper directly:
bin/zookeeper-shell.sh localhost:2181 get /cluster/id
Format each controller's storage with kafka-storage using that ID (a single test controller can use kafka-storage format --standalone). One constraint that catches people: brokers and controllers share one node ID namespace, so controller node.id values must not collide with any existing broker.id.
Each controller starts with migration support enabled and a connection to ZooKeeper:
# controller.properties — ready for migration
process.roles=controller
node.id=3000
controller.quorum.bootstrap.servers=controller1:9093
controller.listener.names=CONTROLLER
listeners=CONTROLLER://:9093
# Enable the migration
zookeeper.metadata.migration.enable=true
# ZooKeeper client configuration
zookeeper.connect=zk1:2181
# Lets the KRaft controller send RPCs to the ZooKeeper-mode brokers
inter.broker.listener.name=PLAINTEXT
Enter migration mode on the brokers. Rolling-restart every broker — still in ZooKeeper mode — with the migration flag and the controller quorum details added (zookeeper.metadata.migration.enable=true, controller.quorum.bootstrap.servers, controller.listener.names, and the controller listener added to the security protocol map). The migration begins automatically once the final broker comes back. Completion is announced by a literal INFO line on the active controller:
Completed migration of metadata from Zookeeper to KRaft
The docs also recommend turning on TRACE logging for the migration components on each controller while the window is open: log4j.logger.org.apache.kafka.metadata.migration=TRACE.
Move the brokers to KRaft. Restart each broker as a KRaft broker: keep the same numeric ID but declare it as node.id, add process.roles=broker, and remove zookeeper.connect and the inter.broker.protocol.version line (KRaft manages that through the metadata.version feature flag). If you use Kafka ACLs, this is also the moment to swap the authorizer from kafka.security.authorizer.AclAuthorizer to org.apache.kafka.metadata.authorizer.StandardAuthorizer.
Finalize. Once every broker runs in KRaft mode, remove zookeeper.metadata.migration.enable and zookeeper.connect from each controller and restart them one at a time. After this step the docs are unambiguous: reverting to ZooKeeper mode is no longer possible. Until this step, it is — the 3.9 documentation includes a revert table for every earlier stage. Only decommission the ZooKeeper ensemble after finalizing, and only if nothing else uses it.
Strimzi: KRaft migration with one annotation
On Kubernetes with Strimzi, the answer to "how do I migrate?" is one annotation — the operator performs the entire KIP-866 sequence for you. The prerequisites: Strimzi 0.40 or newer running Kafka 3.7 or newer, node pools enabled on the Kafka resource (strimzi.io/node-pools: enabled), and the Unidirectional Topic Operator.
There is also a version cliff to respect. Strimzi 0.45.x is the last release line that runs ZooKeeper-based clusters at all — the 0.46.0 release notes state that support for ZooKeeper-based clusters and for the KRaft migration itself has been removed. Migrate on 0.45.x first, then upgrade the operator.
The flow:
# 1. Deploy a KafkaNodePool with the controller role for the new quorum
# 2. Start the migration
kubectl annotate kafka my-cluster strimzi.io/kraft="migration" --overwrite
# 3. Watch progress
kubectl get kafka my-cluster -w
The operator deploys the KRaft controllers with the migration flag set, rolls the brokers into migration mode, and reports progress through the resource's status.kafkaMetadataState field: KRaftMigration → KRaftDualWriting → KRaftPostMigration. At KRaftPostMigration you choose — finalize with strimzi.io/kraft="enabled", or roll back.
Rollback works only from KRaftPostMigration: annotate with rollback (the cluster returns to KRaftDualWriting), delete the controller KafkaNodePool, then annotate with disabled to return fully to ZooKeeper. If you plan to retry later, delete the /migration znode in ZooKeeper first — the Strimzi docs warn that skipping this cleanup can cause metadata loss on the next attempt. Once the state reaches KRaft, there is no way back; that boundary comes from Kafka itself, not Strimzi.
Before the run, take a backup of the cluster — the Strimzi backup walkthrough covers the operator-native way to do it, and the Strimzi integration page shows how the pieces fit.
Confluent Platform: migrate before 8.0
Confluent Platform's migration is the same in-place procedure with commercial version boundaries around it. The documented process applies to CP 7.5 through 7.9. Production migration became generally available in CP 7.6.1, and Confluent recommends migrating on 7.7.0 or later — explicitly not on anything running Kafka 3.6.1 or earlier.
The forcing function is CP 8.0: it is built on Kafka 4.0 and ships without ZooKeeper entirely, so CP 7.9.x is the last release that can run your ZooKeeper-mode cluster. Migration is a prerequisite for the upgrade, not an option alongside it. (Confluent's support policy gives existing ZooKeeper deployments a runway after the 7.9.0 release — two years on Standard support, three on Platinum — but that is a support-window fact, not a technical one.)
Recent CP patch releases (7.7.5+, 7.8.5+, 7.9.2+) include a kafka-migration-check utility that pre-flights the cluster configuration and reports migration status:
kafka-migration-check status --controller-config ./etc/kafka/kraft/controller.properties
If you run Confluent's tooling, both Confluent for Kubernetes and Confluent Ansible automate the migration. The Ansible playbook is worth studying even if you don't use it, because its structure encodes the risk model: a two-phase run that deliberately pauses at dual-write mode — the last point where rollback to ZooKeeper is possible — for validation before completing the move to KRaft.
AWS MSK: there is no in-place path
MSK supports KRaft mode from Kafka 3.7.x onward — for new clusters only. There is no in-place ZooKeeper-to-KRaft conversion on MSK: none of the migration configuration above is exposed to customers, and AWS has not built a managed equivalent. A ZooKeeper-mode MSK cluster that needs Kafka 4.x is therefore facing a cluster move, not a migration procedure. The broader context of running Kafka on AWS is covered in our Kafka on AWS guide.
That changes the problem completely. Everything the in-place path preserves for free now has to be moved deliberately:
- Topic data — every partition, every record
- Topic configurations — retention, compaction, replication factors
- ACL bindings — recreated on the new cluster, or translated to IAM policies
- Consumer group offsets — the part hand-rolled migrations lose
Offsets are the usual casualty. An offset is only meaningful within the cluster that assigned it, so copying data to a new cluster invalidates every committed position. Without translation, consumers restart from earliest or latest — reprocessed duplicates or silent data loss, often surfacing days later in downstream systems.
This is the gap the MSK KRaft migration pipeline in Kafka Backup Enterprise closes: it seeds the new KRaft cluster through S3, tails the source until lag is near zero, coordinates a short producer freeze for cutover, translates every consumer group offset so applications resume exactly where they stopped, runs five automated validation checks, and signs an evidence bundle proving what moved. The plan and precheck stages are free to run, no license required:
kafka-backup migrate msk-kraft plan --config migration.yaml --format all --out-dir ./migration-plan
kafka-backup migrate msk-kraft precheck --config migration.yaml
The rollback story is the one bright side of the new-cluster path: the source cluster is never modified, so before cutover, rolling back means pointing clients back at it. The MSK ZooKeeper to KRaft migration page summarizes this cluster-move path with the configs to start from. The step-by-step runbook walks the full sequence, and the troubleshooting guide covers the failure modes. For how Kafka Backup connects to MSK in general, see the AWS MSK integration page.
Rollback windows and the point of no return
The single most important fact to carry into a migration plan is where your last exit is:
- Self-managed: any step before the controllers restart without
zookeeper.metadata.migration.enable. The 3.9 docs provide revert instructions per completed step. - Strimzi: the
KRaftPostMigrationstate. Annotaterollback, delete the controller node pool, annotatedisabled— and clean up the/migrationznode before any retry. - Confluent Platform: while the cluster is in dual-write mode; the two-phase Ansible flow pauses there for exactly this reason.
- MSK new-cluster move: any time before client cutover, because the source cluster is untouched.
The shared rule underneath all four: finalization deletes the safety net. Once metadata writes to ZooKeeper stop, no platform — and no vendor — can put you back, because Apache Kafka itself has no KRaft-to-ZooKeeper path.
Back up before you touch cluster metadata
A ZooKeeper to KRaft migration is a planned, high-blast-radius change: multiple rolling restarts, an authorizer swap, and a metadata handover between two consensus systems. The in-place path doesn't touch topic data, and offsets survive in __consumer_offsets — but the risk being insured is not the mechanism, it's the operator. A wrong config on the third restart of a fleet-wide roll is exactly the kind of mistake that replication faithfully copies to every follower in milliseconds.
An independent, point-in-time backup restores from before the mistake. Take one immediately before the migration window opens, whichever path you're on — the ZooKeeper to KRaft migration page has a ready-made pre-migration backup config. The migration use cases page covers the backup-and-restore route for cluster moves generally, and the cluster migration solution shows the same pattern applied end to end.
Frequently asked questions
What Kafka version do I need to migrate from ZooKeeper to KRaft?
The migration has been production-ready since Apache Kafka 3.6, but you should migrate through Kafka 3.9 — the bridge release with the final migration fixes and the last version that supports ZooKeeper. Kafka 4.0 cannot start against ZooKeeper at all. On Strimzi you need 0.40+ with Kafka 3.7+, and on Confluent Platform the migration is GA from 7.6.1 with 7.7.0+ recommended.
Is the ZooKeeper to KRaft migration production ready?
Yes. Apache Kafka 3.6.0 was the first release to declare the ZooKeeper-to-KRaft migration production ready, and the procedure matured through the 3.7–3.9 releases. Kafka 3.9 is the recommended bridge release for running it. Strimzi and Confluent Platform both ship supported, automated versions of the same procedure.
Do consumer offsets survive a ZooKeeper to KRaft migration?
In an in-place migration, yes — consumer group offsets are stored in the __consumer_offsets topic, which is regular topic data, and the migration only moves cluster metadata. On AWS MSK the situation is different: because MSK requires a new cluster, offsets from the old cluster are meaningless on the new one and must be explicitly translated, or consumers will restart from earliest or latest.
Can I roll back a ZooKeeper to KRaft migration?
Yes, until you finalize. The migration dual-writes metadata to ZooKeeper, so reverting to ZooKeeper mode is possible at every step before the KRaft controllers restart without the migration flag. On Strimzi the rollback window is the KRaftPostMigration state; on Confluent Platform it is dual-write mode. After finalization, no platform can revert — Apache Kafka has no KRaft-to-ZooKeeper path.
Can I migrate from ZooKeeper to KRaft with zero downtime?
On self-managed Kafka, Strimzi, and Confluent Platform the migration is performed with rolling restarts, so the cluster keeps serving traffic throughout — the same availability profile as a rolling upgrade. On AWS MSK there is no in-place path, so a migration involves a cluster move; a tooled cutover reduces the impact to a short coordinated producer freeze rather than an outage.
The deadline is fixed — the risk is optional
Four platforms, one mechanism, different last exits. Self-managed clusters and Strimzi and Confluent deployments all ride the same KIP-866 dual-write machinery and keep a rollback path until finalization. MSK trades the in-place procedure for a cluster move, where the hard part is offset continuity rather than restart choreography. What none of them offer is a way back after the metadata handover completes — so the preparation, not the migration itself, is where the risk gets managed.
The plan and precheck stages of Kafka Backup's MSK KRaft migration pipeline are free — point them at your production clusters and get a generated runbook, readiness report, and cost estimate before you commit to anything. Get started.