ZooKeeper to KRaft Migration: The Complete Guide (MSK, Strimzi, Confluent)

How you run a ZooKeeper to KRaft migration depends on where your Kafka lives. Self-managed clusters migrate in place with a rolling, dual-write procedure (KIP-866) through the Kafka 3.9 bridge release. Strimzi drives that same procedure on Kubernetes with a single annotation. Confluent Platform runs it on versions 7.6.1 through 7.9, before the KRaft-only 8.0 upgrade.

Eligible AWS MSK clusters migrate in place as well. The cluster stays available, and no topic data moves. The change cannot be reversed.

The deadline is upstream and fixed: ZooKeeper mode was deprecated in Apache Kafka 3.5 and removed in 4.0. If you are still weighing the two modes, the KRaft vs ZooKeeper comparison settles that question.

This guide assumes the decision is made. It covers the risky part: moving every cluster you have, on each platform, without losing metadata or offsets. It also names the last point where rollback still exists.

Key takeaway

Self-managed, Strimzi, and Confluent in-place migrations share the KIP-866 machinery. A KRaft controller quorum copies metadata out of ZooKeeper, and dual-writing keeps a way back until you finalize. MSK's in-place migration is an AWS operation, and you cannot switch that cluster back to ZooKeeper. Know the last exit on your platform, and take an independent backup before you touch cluster metadata.

Every ZooKeeper to KRaft migration path at a glance

Platform	Migration path	Minimum versions	Last rollback point
Self-managed Apache Kafka	In-place KIP-866 rolling migration via the 3.9 bridge release	Production-ready since 3.6; migrate on the latest 3.9 patch	Before controllers restart without the migration flag
Strimzi on Kubernetes	Same flow, driven by the strimzi.io/kraft annotation	Strimzi 0.40+ with Kafka 3.7+; must finish before Strimzi 0.46	KRaftPostMigration state
Confluent Platform	In-place migration on CP 7.5–7.9	GA at 7.6.1, recommended 7.7.0+; required before CP 8.0	While the cluster is in dual-write mode
AWS MSK	In-place via UpdateClusterKafkaVersion for eligible clusters; a new cluster when the cluster is ineligible or a separate destination is preferred	Kafka 3.9.x in ZooKeeper mode, then a 3.9.x.kraft target	In-place: none. The migration cannot return to ZooKeeper. A new-cluster move can roll back before client cutover

The first three rows are the same mechanism wearing different clothes. Strimzi and Confluent both orchestrate the upstream KIP-866 procedure. What differs is who performs the restarts and where the safety checks live.

MSK is different again. Eligible clusters migrate in place through an AWS API, and you cannot return that cluster to ZooKeeper. A new cluster is the route when that API will not accept the cluster, or when you want a separate destination.

How the migration actually works (KIP-866 dual-write)

The first thing to internalize: an in-place migration moves cluster metadata, not data. Topic partitions stay on the same disks throughout. Consumer group offsets live in __consumer_offsets — a regular Kafka topic — so they survive untouched. What moves is the cluster's brain: topic configurations, partition assignments, ACLs, and controller state, copied from the ZooKeeper ensemble into a KRaft controller quorum.

The Kafka 3.9 documentation describes the migration as a sequence of phases:

1. Initial phase — all brokers in ZooKeeper mode, a ZooKeeper-based controller in charge.
2. Initial metadata load — the new KRaft quorum reads the full metadata snapshot out of ZooKeeper.
3. Hybrid phase — some brokers still in ZooKeeper mode, but the KRaft controller now leads.
4. Dual-write phase — every broker runs in KRaft mode, and the controller keeps writing metadata to ZooKeeper as well.
5. Finalized — ZooKeeper writes stop. The migration is complete and irreversible.

Dual-writing is the safety net. Because ZooKeeper stays current through phase 4, you can revert to ZooKeeper mode at any step before finalization. The moment you finalize, that net is gone.

Two hard rules from the upstream docs apply to every KIP-866 path you run yourself. Do not change the inter.broker.protocol.version or metadata version while a migration is in flight — the docs are blunt that this may break the cluster. And if a broker runs multiple log directories, a directory failure during the migration shuts that broker down until the directory is repaired (KAFKA-16431), so enter the window with healthy disks.

Migrating self-managed Kafka from ZooKeeper to KRaft

Migration has been production-ready since Kafka 3.6, but 3.9 is the bridge release every ZooKeeper cluster should migrate through — it received the final round of migration fixes, and 4.0 will not start against ZooKeeper at all. So step zero is a standard rolling upgrade of all brokers to 3.9, with inter.broker.protocol.version set to match.

Provision the KRaft controllers. The new controller quorum must carry the same cluster ID as the existing cluster. Read it from any broker's meta.properties, or ask ZooKeeper directly:

bin/zookeeper-shell.sh localhost:2181 get /cluster/id

Format each controller's storage with kafka-storage using that ID (a single test controller can use kafka-storage format --standalone). One constraint that catches people: brokers and controllers share one node ID namespace, so controller node.id values must not collide with any existing broker.id.

Each controller starts with migration support enabled and a connection to ZooKeeper:

# controller.properties — ready for migration
process.roles=controller
node.id=3000
controller.quorum.bootstrap.servers=controller1:9093
controller.listener.names=CONTROLLER
listeners=CONTROLLER://:9093

# Enable the migration
zookeeper.metadata.migration.enable=true

# ZooKeeper client configuration
zookeeper.connect=zk1:2181

# Lets the KRaft controller send RPCs to the ZooKeeper-mode brokers
inter.broker.listener.name=PLAINTEXT

Enter migration mode on the brokers. Rolling-restart every broker — still in ZooKeeper mode — with the migration flag and the controller quorum details added (zookeeper.metadata.migration.enable=true, controller.quorum.bootstrap.servers, controller.listener.names, and the controller listener added to the security protocol map). The migration begins automatically once the final broker comes back. Completion is announced by a literal INFO line on the active controller:

Completed migration of metadata from Zookeeper to KRaft

The docs also recommend turning on TRACE logging for the migration components on each controller while the window is open: log4j.logger.org.apache.kafka.metadata.migration=TRACE.

Move the brokers to KRaft. Restart each broker as a KRaft broker: keep the same numeric ID but declare it as node.id, add process.roles=broker, and remove zookeeper.connect and the inter.broker.protocol.version line (KRaft manages that through the metadata.version feature flag). If you use Kafka ACLs, this is also the moment to swap the authorizer from kafka.security.authorizer.AclAuthorizer to org.apache.kafka.metadata.authorizer.StandardAuthorizer.

Finalize. Once every broker runs in KRaft mode, remove zookeeper.metadata.migration.enable and zookeeper.connect from each controller and restart them one at a time. After this step the docs are unambiguous: reverting to ZooKeeper mode is no longer possible. Until this step, it is — the 3.9 documentation includes a revert table for every earlier stage. Only decommission the ZooKeeper ensemble after finalizing, and only if nothing else uses it.

Strimzi: KRaft migration with one annotation

On Kubernetes with Strimzi, the answer to "how do I migrate?" is one annotation — the operator performs the entire KIP-866 sequence for you. The prerequisites: Strimzi 0.40 or newer running Kafka 3.7 or newer, node pools enabled on the Kafka resource (strimzi.io/node-pools: enabled), and the Unidirectional Topic Operator.

There is also a version cliff to respect. Strimzi 0.45.x is the last release line that runs ZooKeeper-based clusters at all — the 0.46.0 release notes state that support for ZooKeeper-based clusters and for the KRaft migration itself has been removed. Migrate on 0.45.x first, then upgrade the operator.

The flow:

# 1. Deploy a KafkaNodePool with the controller role for the new quorum
# 2. Start the migration
kubectl annotate kafka my-cluster strimzi.io/kraft="migration" --overwrite

# 3. Watch progress
kubectl get kafka my-cluster -w

The operator deploys the KRaft controllers with the migration flag set, rolls the brokers into migration mode, and reports progress through the resource's status.kafkaMetadataState field: KRaftMigration → KRaftDualWriting → KRaftPostMigration. At KRaftPostMigration you choose — finalize with strimzi.io/kraft="enabled", or roll back.

Rollback works only from KRaftPostMigration: annotate with rollback (the cluster returns to KRaftDualWriting), delete the controller KafkaNodePool, then annotate with disabled to return fully to ZooKeeper. If you plan to retry later, delete the /migration znode in ZooKeeper first — the Strimzi docs warn that skipping this cleanup can cause metadata loss on the next attempt. Once the state reaches KRaft, there is no way back; that boundary comes from Kafka itself, not Strimzi.

Before the run, take a backup of the cluster — the Strimzi backup walkthrough covers the operator-native way to do it, and the Strimzi integration page shows how the pieces fit. The Strimzi KRaft migration page condenses this section into a task checklist with the backup CR to run first.

Confluent Platform: migrate before 8.0

Confluent Platform's migration is the same in-place procedure with commercial version boundaries around it. The documented process applies to CP 7.5 through 7.9. Production migration became generally available in CP 7.6.1, and Confluent recommends migrating on 7.7.0 or later — explicitly not on anything running Kafka 3.6.1 or earlier.

The forcing function is CP 8.0: it is built on Kafka 4.0 and ships without ZooKeeper entirely, so CP 7.9.x is the last release that can run your ZooKeeper-mode cluster. Migration is a prerequisite for the upgrade, not an option alongside it. (Confluent's support policy gives existing ZooKeeper deployments a runway after the 7.9.0 release — two years on Standard support, three on Platinum — but that is a support-window fact, not a technical one.)

Recent CP patch releases (7.7.5+, 7.8.5+, 7.9.2+) include a kafka-migration-check utility that pre-flights the cluster configuration and reports migration status:

kafka-migration-check status --controller-config ./etc/kafka/kraft/controller.properties

If you run Confluent's tooling, both Confluent for Kubernetes and Confluent Ansible automate the migration. The Ansible playbook is worth studying even if you don't use it, because its structure encodes the risk model: a two-phase run that deliberately pauses at dual-write mode — the last point where rollback to ZooKeeper is possible — for validation before completing the move to KRaft. The Confluent Platform KRaft migration page condenses the version windows and the pre-migration backup into one task landing.

AWS MSK: in-place ZooKeeper to KRaft migration

AWS MSK can migrate an existing ZooKeeper cluster to KRaft in place. The migration guide uses the UpdateClusterKafkaVersion API. The cluster stays available throughout, and no topic data is moved. MSK provisions KRaft controllers at no additional cost, copies cluster metadata onto that quorum, and then decommissions the ZooKeeper nodes.

The operation can take several hours, depending on cluster size. Watch it with DescribeClusterOperation. It cannot be reversed. The Kafka on AWS guide covers the wider AWS picture.

Prerequisites

Meet every item on the AWS prerequisite list before you start:

• Apache Kafka 3.9.x, still in ZooKeeper mode. If it is older, upgrade to 3.9.x with the normal version upgrade first.
• Standard brokers deployed across three Availability Zones. Express brokers already meet this.
• Status ACTIVE, with no operation still pending.
• Clients connect with --bootstrap-server, not --zookeeper. The ZooKeeper connection string disappears after migration.
• Tools that open ZooKeeper directly, such as older Cruise Control, move to the Kafka Admin APIs instead.
• Public Access and Open Monitoring are not both enabled.
• No kafka.t3.small brokers.
• advertised.listeners has not been set dynamically. Run the check below, with $bs set to the bootstrap servers, and confirm the output does not list that property.

bin/kafka-configs.sh --bootstrap-server $bs --entity-type brokers --describe

Standard (provisioned) clusters must also turn off ZooKeeper client access before the migration. Express clusters do not need this step. Set ZookeeperAccess.Enabled to false with UpdateConnectivity.

Cluster versions are not plain integers. Read the current version from describe-cluster first. An example version looks like KTVPDKIKX0DER.

aws kafka describe-cluster --cluster-arn ClusterArn

aws kafka update-connectivity \
  --cluster-arn ClusterArn \
  --current-version Current-Cluster-Version \
  --zookeeper-access '{"Enabled": false}'

A successful connectivity update generates a new cluster version. Read that version again before the migration command.

Migrate with the AWS CLI

Confirm the cluster is eligible. Targets that end in .kraft mean the in-place migration is available.

aws kafka get-compatible-kafka-versions --cluster-arn ClusterArn

Start the migration with the KRaft target version. Pass the current cluster version from describe-cluster, not a guess.

aws kafka update-cluster-kafka-version \
  --cluster-arn ClusterArn \
  --current-version Current-Cluster-Version \
  --target-kafka-version "3.9.x.kraft"

The call returns a cluster operation ARN. Poll it until the state reaches UPDATE_COMPLETE. While it runs, the state is UPDATE_IN_PROGRESS.

aws kafka describe-cluster-operation --cluster-operation-arn ClusterOperationArn

Take a point-in-time backup before you issue that update. There is no switch back to ZooKeeper after the migration completes. The pre-migration backup config is the same one the other platforms use.

After migration

The cluster is in KRaft mode, and the ZooKeeper nodes are gone. Clients that already use --bootstrap-server need no change. MSK does not publish DNS names for the KRaft controllers on a migrated cluster.

Controller JMX metrics are therefore not available through open monitoring. If you use open monitoring, scrape the broker endpoints only.

When a new cluster is the right path

The in-place API is not available for every cluster. Use a new cluster when that API will refuse this one. That includes a version below 3.9.x that you cannot upgrade, kafka.t3.small brokers, Public Access and Open Monitoring together, and a dynamic advertised.listeners setting.

The same route fits when you want a new cluster on purpose. Typical reasons are a different size, a different auth mode, or a cutover you can still abandon.

That move has to carry what an in-place migration leaves where it is:

• Topic data — every partition, every record
• Topic configurations — retention, compaction, replication factors
• ACL bindings — recreated on the new cluster, or translated to IAM policies
• Consumer group offsets — meaningless on the new cluster until they are translated

Offsets are the usual casualty. An offset only means something inside the cluster that assigned it. Copying records to a new cluster invalidates every committed position.

Without translation, consumers restart from earliest or latest. Duplicates or silent gaps often show up days later, downstream.

Kafka Backup's MSK ZooKeeper to KRaft migration is this source-to-destination path. It is not a stand-in for the AWS in-place API. It seeds the new cluster through S3, and tails the source until lag is inside the configured tolerance. It uses a short producer freeze at cutover.

It translates consumer group offsets so each group resumes from the equivalent position. It runs five automated validation checks, and signs an evidence bundle. plan and precheck need no license:

kafka-backup migrate msk-kraft plan --config migration.yaml --format all --out-dir ./migration-plan
kafka-backup migrate msk-kraft precheck --config migration.yaml

Before cutover, the source cluster is never modified, so rollback is pointing clients back at it. The MSK ZooKeeper to KRaft migration page is the short version of this cluster move, with the configs to start from. The step-by-step runbook walks the sequence, and the troubleshooting guide covers the failure modes. For how Kafka Backup connects to MSK in general, see the AWS MSK integration page.

Rollback windows and the point of no return

The single most important fact to carry into a migration plan is where your last exit is:

• Self-managed: any step before the controllers restart without zookeeper.metadata.migration.enable. The 3.9 docs provide revert instructions per completed step.
• Strimzi: the KRaftPostMigration state. Annotate rollback, delete the controller node pool, annotate disabled — and clean up the /migration znode before any retry.
• Confluent Platform: while the cluster is in dual-write mode; the two-phase Ansible flow pauses there for exactly this reason.
• MSK in-place: none. Once the cluster is on KRaft, you cannot switch it back to ZooKeeper. Take a point-in-time backup before you start.
• MSK new-cluster move: any time before client cutover, because the source cluster is untouched.

The KIP-866 paths share one rule. Finalization deletes the safety net. Once metadata writes to ZooKeeper stop, Apache Kafka has no path back.

MSK does not give you a ZooKeeper rollback after the in-place migration. Once the cluster is on KRaft, you cannot switch it back.

Back up before you touch cluster metadata

A ZooKeeper to KRaft migration is a planned, high-blast-radius change: multiple rolling restarts, an authorizer swap, and a metadata handover between two consensus systems. The in-place path doesn't touch topic data, and offsets survive in __consumer_offsets — but the risk being insured is not the mechanism, it's the operator. A wrong config on the third restart of a fleet-wide roll is exactly the kind of mistake that replication faithfully copies to every follower in milliseconds.

An independent, point-in-time backup restores from before the mistake. Take one immediately before the migration window opens, whichever path you're on — the ZooKeeper to KRaft migration page has a ready-made pre-migration backup config.

On MSK, the in-place migration cannot be reversed. That backup is the copy you still hold of the data from before the change. The migration use cases page covers the backup-and-restore route for cluster moves generally, and the cluster migration solution shows the same pattern applied end to end.

Frequently asked questions

What Kafka version do I need to migrate from ZooKeeper to KRaft?

The migration has been production-ready since Apache Kafka 3.6, and you should run it on 3.9, the last release that can use ZooKeeper. Kafka 4.0 cannot start against ZooKeeper at all. Strimzi needs 0.40 or newer with Kafka 3.7 or newer, and Confluent recommends 7.7.0 or later after 7.6.1. On Amazon MSK, the in-place migration needs Apache Kafka 3.9.x in ZooKeeper mode.

Is the ZooKeeper to KRaft migration production ready?

Yes. Apache Kafka 3.6.0 was the first release to declare the ZooKeeper-to-KRaft migration production ready, and the procedure matured through the 3.7 to 3.9 releases. Kafka 3.9 is the recommended bridge release, and Strimzi and Confluent both automate that procedure. Amazon MSK runs its own in-place migration for eligible clusters on Kafka 3.9.x.

Do consumer offsets survive a ZooKeeper to KRaft migration?

In an in-place migration, yes. Consumer group offsets live in the __consumer_offsets topic, which is ordinary topic data, and only cluster metadata moves. An eligible MSK in-place migration works the same way, because no topic data moves. On a new MSK cluster those offsets do not apply, so translate them or consumers restart from earliest or latest.

Can I roll back a ZooKeeper to KRaft migration?

On the KIP-866 paths, yes, until you finalize. Dual-writing keeps ZooKeeper current, so you can revert before the controllers restart without the migration flag. Strimzi rolls back from KRaftPostMigration, and Confluent Platform rolls back from dual-write mode. An MSK in-place migration cannot be switched back to ZooKeeper, while a new-cluster move can roll back before client cutover.

Can I migrate from ZooKeeper to KRaft with zero downtime?

On self-managed Kafka, Strimzi, and Confluent Platform, rolling restarts keep the cluster serving traffic. An eligible MSK in-place migration also keeps the cluster available, moves no data, and can take several hours. A move to a new MSK cluster is a separate job. A tooled cutover there limits the impact to a short, coordinated producer freeze.

The deadline is fixed — the risk is optional

Four platforms, and the last exit is not the same on each. Self-managed clusters, Strimzi, and Confluent use KIP-866 dual-write, and they keep a rollback path until finalization. Eligible MSK clusters migrate in place with UpdateClusterKafkaVersion, and that change cannot return to ZooKeeper.

A new MSK cluster is the path when the cluster is not eligible, or when you want a separate destination. Offset translation is the hard part of that move. After a KIP-866 finalization, or after an MSK in-place migration, there is no path back to ZooKeeper. Preparation is where you manage that risk.

Moving to a new MSK cluster

The plan and precheck stages of Kafka Backup's MSK KRaft migration pipeline are free. Point them at your clusters for a generated runbook, readiness report, and cost estimate before you commit. Get started.