Skip to main content

Kafka Without ZooKeeper: Is ZooKeeper Deprecated?

· 9 min read
OSO Engineering
The team behind OSO Kafka Backup

Yes — running Kafka without ZooKeeper is not just possible, it is now the only supported way forward. Kafka has run without ZooKeeper in production since version 3.3, ZooKeeper mode was deprecated in 3.5, and Kafka 4.0 removed it entirely. A 4.x broker will not even start against a ZooKeeper ensemble.

The question arrives in many forms: is ZooKeeper still required? Is it removed? Can I run Kafka without it today? The answer depends entirely on which Kafka version and platform you are on. This post gives the version-by-version answer, lists what still requires ZooKeeper, and shows how to start a cluster with no ZooKeeper anywhere.

Key takeaway

Every new Kafka cluster should run KRaft — ZooKeeper-free — and has been able to since Kafka 3.3. Existing ZooKeeper-mode clusters are on a fixed deadline: 3.9 is the last release that supports ZooKeeper, and 4.0 will not start against it. The only place ZooKeeper still runs is legacy 3.x deployments that have not migrated yet.

Can Kafka run without ZooKeeper?

Yes. Since Kafka 3.3, new clusters can run KRaft mode in production — no ZooKeeper process, no ensemble, no zookeeper.connect anywhere. In KRaft mode, a quorum of controller nodes replicates cluster metadata through an internal Raft-based log. Brokers read their view of the cluster from those controllers. The coordination work ZooKeeper used to do now happens inside Kafka itself.

That coordination work was real: ZooKeeper elected the controller, tracked which brokers were alive, and stored topic definitions, configs, and ACLs. If you want the full picture of what the old layer did — and what it never stored — read What Is ZooKeeper in Kafka?. The short version is that ZooKeeper held the cluster's coordination metadata, never its message data.

Removing it changes the operational shape of every deployment. There is no separate three- or five-node ensemble to size, patch, secure, and monitor. There is one system instead of two, one security model instead of two, and one set of quorum math governing availability.

Is ZooKeeper deprecated in Kafka? The version-by-version answer

ZooKeeper mode has been deprecated since Kafka 3.5 and removed since Kafka 4.0. Between those two points sits a multi-year capability ramp, and "can I run without ZooKeeper?" has a different answer at each step:

Kafka versionZooKeeper statusCan you run without ZooKeeper?
2.8 (2021)Required (KRaft early access)Dev and test only
3.3Optional for new clustersYes — KRaft production-ready (KIP-833)
3.5ZooKeeper mode deprecatedYes
3.6Deprecated; ZooKeeper-to-KRaft migration production-ready (KIP-866)Yes, but JBOD brokers cannot move yet
3.7Deprecated; JBOD in KRaft early access (KIP-858)Yes, though multi-disk brokers still wait
3.8Deprecated; JBOD in KRaft production-readyYes, including multi-disk brokers
3.9Bridge release — last version with ZooKeeperYes
4.0Removed — will not start against ZooKeeperKRaft is the only mode

Two rows are the real inflection points. Kafka 3.3 is the "you may" release: KIP-833 marked KRaft production-ready for new clusters, and from that point on, starting a fresh cluster with ZooKeeper was a choice rather than a requirement. Kafka 4.0 is the "you must" release: ZooKeeper support is gone from the codebase, so any cluster that wants 4.x features has to be running KRaft first.

The JBOD rows explain why some teams could not move early. Brokers that spread partitions across multiple log directories — just a bunch of disks — had no KRaft equivalent when the migration went production-ready in 3.6. KIP-858 shipped JBOD support in KRaft as early access in 3.7 and made it production-ready in 3.8, closing one of the last feature-parity gaps between the two modes.

What still requires ZooKeeper today

The inventory is short, and every item on it has a deadline attached.

Existing ZooKeeper-mode clusters on 3.x. A cluster built around a ZooKeeper ensemble keeps requiring that ensemble until it migrates. There is no way to upgrade past 3.9 in place: the metadata has to move to a KRaft quorum first.

Confluent Platform before 8.0. ZooKeeper-mode Confluent Platform clusters must migrate — the migration is GA from 7.6.1, with 7.7.0 or later recommended — because Confluent Platform 8.0 is KRaft-only.

ZooKeeper-mode MSK clusters. AWS MSK offers KRaft only for new clusters, from Kafka 3.7.x onward, with no in-place conversion. An existing ZooKeeper-mode MSK cluster that needs 4.x is facing a cluster move: a new KRaft cluster plus a data, config, ACL, and offset migration.

Strimzi before 0.46. Strimzi could run ZooKeeper-mode Kafka on Kubernetes until 0.46 removed ZooKeeper support. The operator drives the migration itself through the strimzi.io/kraft annotation on Strimzi 0.40+ with Kafka 3.7+.

Pre-3.0 tooling habits. Kafka 3.0 removed the --zookeeper flag from the main admin tools. Any script still passing it to kafka-topics.sh is two eras behind; everything goes through --bootstrap-server now.

One thing was never on this list: your applications. Producers and consumers have connected only to brokers via bootstrap.servers for years, so client code does not change at all when ZooKeeper goes away.

If you hold one of the items above, the path out is a migration, not an upgrade. The ZooKeeper to KRaft migration guide covers the procedure for self-managed Kafka, Strimzi, Confluent Platform, and MSK. The condensed task version lives at migrate ZooKeeper to KRaft. For the two modes side by side, see the KRaft vs ZooKeeper comparison.

How to run Kafka without ZooKeeper

For a new cluster, running Kafka without ZooKeeper takes three commands. KRaft clusters carry their identity in their own formatted log directories, so the first step generates a cluster ID and stamps it onto the storage:

KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties
bin/kafka-server-start.sh config/kraft/server.properties

There is no ZooKeeper start step, because there is no ZooKeeper. Four properties define each KRaft node:

  • process.roles — whether this node is a broker, a controller, or both.
  • node.id — the node's unique identifier in the cluster.
  • controller.quorum.voters — the controller quorum, as a list of id@host:port entries.
  • controller.listener.names — which listeners carry controller traffic.

A note on paths: the config/kraft/ subdirectory existed only while the two modes coexisted. In Kafka 4.0, KRaft is the default and only mode, so the KRaft configuration is simply config/server.properties.

Combined mode — process.roles=broker,controller in a single process — is convenient for development and testing. Production clusters run dedicated controller nodes, separate from the brokers, so metadata quorum health does not compete with data traffic.

The kafka-storage.sh format step is the tell that the architecture changed. A ZooKeeper-mode cluster's identity lived in an external tree; a KRaft cluster's identity lives in its own disks, formatted before first boot.

What removing ZooKeeper does not change: your data protection

Neither ZooKeeper nor the KRaft quorum ever held your topic data — and since Kafka 0.9, consumer offsets live in the internal __consumer_offsets topic on the brokers, not in any metadata store. Going ZooKeeper-free changes the metadata plane, not what you must protect. And replication is not backup, in either mode: replicas copy corruption and deletion as faithfully as they copy good data.

Two concrete implications follow. First, a KRaft cluster still needs an independent, off-cluster copy of topic data and offsets. Kafka Backup writes both to S3, S3-compatible storage, Azure Blob, GCS, or filesystem targets. Consumer group offset preservation is built in, with point-in-time recovery to the millisecond. Second, if your ZooKeeper-free future involves a migration — or a full cluster move, as on MSK — take that independent backup before the change. The migration use case covers using a backup as the safety net under exactly this kind of planned, high-blast-radius work.

On Kubernetes, the same applies: the Kafka Backup operator is Strimzi compatible and works the same against KRaft-mode clusters.

FAQ

Frequently asked questions

Does Kafka still need ZooKeeper?

No. Since Kafka 3.3, new clusters run KRaft mode in production with no ZooKeeper at all, and Kafka 4.0 removed ZooKeeper support entirely — a 4.x broker cannot use it. Only existing ZooKeeper-mode clusters on 3.x still need their ensemble, and only until they migrate.

Is ZooKeeper removed from Kafka?

Yes, as of Kafka 4.0. The removal followed a staged timeline: KRaft became production-ready for new clusters in 3.3, ZooKeeper mode was deprecated in 3.5, the migration became production-ready in 3.6, and 3.9 was the last release to ship ZooKeeper support.

How do I run Kafka without ZooKeeper?

Start a KRaft cluster: generate a cluster ID with kafka-storage.sh random-uuid, format each node's log directories with kafka-storage.sh format, and start the server with a KRaft server.properties that sets process.roles, node.id, controller.quorum.voters, and controller.listener.names. No ZooKeeper process is involved at any step.

Is ZooKeeper still used in Kafka?

Only in legacy deployments. ZooKeeper-mode Kafka 3.x clusters, Confluent Platform clusters before 8.0, and ZooKeeper-mode MSK clusters still use it until they migrate. Every currently supported path for new clusters — Apache Kafka 4.x, MSK from 3.7.x, Strimzi 0.46+, Confluent Platform 8.0 — is KRaft-only.

Conclusion

"Can Kafka run without ZooKeeper?" stopped being a capability question in 3.3 and became a deadline in 4.0. The state of the world is now simple: KRaft for everything new, a fixed migration window for everything old, and ZooKeeper surviving only in legacy 3.x estates that have not moved yet.

Whichever side of that line you are on, the metadata layer was never where your data lived. Protect the topics and offsets independently, and the metadata mode becomes an implementation detail you can change on your own schedule.

Back Up Kafka in Any Metadata Mode

ZooKeeper or KRaft, your topic data and consumer offsets live on the brokers — and neither metadata layer protects them. Kafka Backup captures both to S3, Azure, GCS, or filesystem storage with point-in-time recovery. Get started.