Kafka 4.0: What Changed and What It Means for Your Cluster
Kafka 4.0, released on March 18, 2025, is the first Apache Kafka that runs entirely without ZooKeeper — and it raises the floor on almost everything else at the same time. Old clients, old message formats, old Java versions, and the original MirrorMaker all exit in the same release.
That makes 4.0 less a feature release than a floor-raising release. The interesting question is not "what shiny thing do I get?" but "what do I have to be running before I can get there?" This post covers both: what shipped, what was removed, and the upgrade path that connects your cluster to it.
Kafka 4.0 is KRaft-only and will not start against ZooKeeper. The KIP-848 consumer rebalance protocol is GA, Java baselines moved to 11 (clients/Streams) and 17 (brokers/Connect/tools), and clients must be 2.1 or newer. Direct upgrades come only from KRaft clusters on 3.3.x–3.9.x — ZooKeeper clusters must migrate first, via the 3.9 bridge release.
What changed in Kafka 4.0?
In one line: Kafka 4.0 completed the KIP-500 project by removing ZooKeeper, shipped a new consumer rebalance protocol as GA, previewed two durability and queuing features, and cleared out a decade of deprecated baggage.
| Change | KIP | Status in 4.0 |
|---|---|---|
| ZooKeeper mode removed — KRaft is the only mode | KIP-500 (completed) | Removed |
| New consumer rebalance protocol | KIP-848 | GA, server default |
| Queues for Kafka (share groups) | KIP-932 | Early access |
| Eligible Leader Replicas (ELR) | KIP-966 | Preview |
Duration-based offset reset (by_duration) | KIP-1106 | New |
| Old client protocol API versions removed — clients need 2.1+ | KIP-896 | Removed |
| Message formats v0 and v1 removed | KIP-724 | Removed |
| MirrorMaker 1 removed | KIP-720 | Removed |
| Logging migrated from Log4j to Log4j2 | KIP-653 | Changed |
| Java 11 for clients/Streams, Java 17 for brokers/Connect/tools | — | Raised |
The 4.x line has continued since — 4.1, 4.2, and 4.3 all build on the same foundation. Everything in this table is the floor of the 4.x era, which is why it still decides whether your cluster can move at all.
Kafka 4.0 is KRaft-only
A Kafka 4.0 broker cannot start against a ZooKeeper ensemble — the code that talked to ZooKeeper is gone, and the --zookeeper option is gone from the admin CLI tools with it. Cluster metadata lives in KRaft mode: a quorum of controllers replicating an internal metadata log.
This was the endpoint of a staged timeline, not a surprise. KRaft became production-ready for new clusters in 3.3, ZooKeeper mode was deprecated in 3.5, and 3.9 was the last release to support it — the full version-by-version matrix is in Kafka Without ZooKeeper. What 4.0 changed is the enforcement: the choice became a requirement.
New in 4.0: the rebalance protocol, queues, and duration offsets
The headline feature is KIP-848, the new consumer rebalance protocol, GA in 4.0. The classic protocol synchronized every rebalance through a global barrier: one consumer joining or leaving could stall the whole group while assignments were renegotiated. The new protocol moves assignment logic to the group coordinator and rebalances incrementally, so membership changes stop being stop-the-world events.
Adoption is split between the two sides. The new protocol is enabled by default on the server side; consumers switch to it by setting group.protocol=consumer. Until a consumer sets that, it keeps using the classic protocol — both work against a 4.0 broker.
KIP-932, Queues for Kafka, ships as early access. Share groups let multiple consumers process a topic queue-style, without the partition-ownership model that normally caps parallelism at the partition count. Early access means exactly that: a feature to evaluate, not one to run in production.
KIP-966, Eligible Leader Replicas, ships as preview. ELR tracks the subset of the ISR that is guaranteed to have complete data up to the high-watermark, giving leader election a safer candidate list after failures.
KIP-1106 adds a fourth answer to "where does a consumer with no committed offset start?" — a fixed duration back from now:
group.protocol=consumer
auto.offset.reset=by_duration:P30D
The duration uses ISO-8601 notation (PnDTnHnMn.nS — P30D is 30 days, PT1H is one hour), and negative durations are not allowed. It is a small feature, but it replaces a common pattern of seeking by timestamp in application code.
Removals and raised baselines
The removals are where upgrade work actually comes from. Each one sets a new floor:
| Removed or raised | New floor in 4.0 |
|---|---|
| ZooKeeper mode | KRaft mode only |
| MirrorMaker 1 | MirrorMaker 2 is the only version |
| Message formats v0 and v1 (deprecated in 3.0) | Message format v2 |
| Client protocol API versions below 2.1 (KIP-896) | Java clients 2.1+ |
| Java 8/11 for servers | Java 17 for brokers, Connect, and tools; Java 11 for clients and Streams |
| Log4j logging | Log4j2 |
--zookeeper CLI options | --bootstrap-server only |
The client baseline deserves the most attention, because it cuts in both directions. The upstream guidance from KIP-896: make sure brokers are 2.1 or higher before upgrading Java clients to 4.0, and make sure Java clients are 2.1 or higher before upgrading brokers to 4.0. A 2.0-era client that has quietly worked for years stops connecting the day the brokers move. Audit the client fleet — including Connect workers and Streams applications — before touching the brokers.
The upgrade path to Kafka 4.0
The rule is strict: brokers upgrade directly to 4.0 only from KRaft-mode clusters running 3.3.x through 3.9.x, with the metadata version also at 3.3 or higher. Clusters still in ZooKeeper mode have to be migrated to KRaft before they can be upgraded, and Kafka 3.9 is the last bridge release that supports the migration.
| Where you are today | Your path to 4.0 |
|---|---|
| KRaft mode, 3.3.x–3.9.x | Rolling upgrade to 4.0, then bump metadata.version |
| ZooKeeper mode, any 3.x | Upgrade to 3.9 → migrate to KRaft → then upgrade to 4.0 |
| KRaft mode, metadata version below 3.3 | Raise the metadata version first |
| Client fleet with anything below 2.1 | Upgrade clients before brokers |
The migration itself is a documented, rolling procedure — the ZooKeeper to KRaft migration guide covers it end to end, with the condensed task version at migrate ZooKeeper to KRaft.
One property of the final step deserves respect: it is a one-way door. Once the cluster's metadata version is bumped to a 4.0 level, downgrade is not supported, because those versions carry metadata format changes. The rollback you have is the one you made yourself — a verified, off-cluster backup taken before the upgrade window opens.
What Kafka 4.0 means for your backup and DR posture
Nothing in 4.0 moves the data plane. Topic data still lives in partition logs on the brokers, and consumer offsets still live in the internal __consumer_offsets topic, exactly as they did in 3.x. The release changes the coordination plane and the compatibility baselines — not what you have to protect. And replication is not backup in either era: replicas copy corruption and accidental deletion as faithfully in a 4.x cluster as they did in a 3.x one.
Two practical implications follow. First, the upgrade itself is the risk window. Between the client audit, the KRaft migration, and the one-way metadata bump, a 4.0 move touches every layer of the cluster. A point-in-time backup of topic data and offsets, taken before the window opens, is the only rollback that survives the metadata change. Kafka Backup writes both to S3, S3-compatible storage, Azure Blob, GCS, or filesystem targets, with recovery to the millisecond.
Second, in-place is not the only road. Some teams reach 4.0 by standing up a fresh KRaft cluster and moving data into it, retiring the old cluster afterwards — the pattern the migration use case is built around, with consumer group offsets carried across as part of the restore.
On Kubernetes, the Kafka Backup operator is Strimzi compatible and works the same against 3.x and 4.x KRaft clusters — backups taken today restore the same way after the upgrade.
FAQ
Frequently asked questions
When was Kafka 4.0 released?
Apache Kafka 4.0.0 was released on March 18, 2025. It was the first major release to run entirely without ZooKeeper, and the 4.x line has continued since with 4.1, 4.2, and 4.3 on the same KRaft-only foundation.
What are the new features in Kafka 4.0?
The new consumer rebalance protocol (KIP-848) became GA, Queues for Kafka share groups (KIP-932) shipped as early access, Eligible Leader Replicas (KIP-966) shipped as preview, and consumers gained a duration-based offset reset (KIP-1106). Alongside those, 4.0 removed ZooKeeper mode, MirrorMaker 1, message formats v0/v1, and pre-2.1 client protocol versions.
What is the ISO-8601 duration format (PnDTnHnMn.nS) in Kafka 4.0 configuration?
It is the duration syntax used by the by_duration offset reset option added in KIP-1106. Setting auto.offset.reset=by_duration:P30D starts a consumer with no committed offset 30 days back from the current time; PT1H would be one hour. The duration must be a valid ISO-8601 value and cannot be negative.
What Java version does Kafka 4.0 require?
Kafka clients and Kafka Streams require Java 11, while Kafka brokers, Connect, and the command-line tools require Java 17. Deployments still on Java 8 must move before upgrading anything to 4.0.
Do older Kafka clients work with Kafka 4.0 brokers?
Java clients must be version 2.1 or newer, because KIP-896 removed the older protocol API versions in 4.0. The check applies in both directions: brokers should be 2.1+ before clients upgrade to 4.0, and clients should be 2.1+ before brokers upgrade to 4.0.
Conclusion
Kafka 4.0 closes the ZooKeeper era and resets the baselines the next several years of Kafka will build on. The features are incremental — a better rebalance protocol, previews of queues and safer leader election. The prerequisites are not: KRaft mode, modern clients, modern Java, and a migration deadline that has already passed for anyone starting the journey now.
The right response is an audit before an upgrade: metadata mode, broker version, client fleet, Java runtimes. And because the final metadata bump cannot be rolled back, the audit should end with a verified copy of your data that lives outside the cluster you are about to change.
Kafka 4.0 upgrades are one-way once the metadata version bumps. Kafka Backup gives you a point-in-time copy of topic data and consumer offsets in S3, Azure, GCS, or filesystem storage before you commit. Get started.