Running Kafka in KRaft Mode with Docker Compose
Running Kafka in KRaft mode with Docker Compose takes one service and a handful of environment variables. The official apache/kafka image runs KRaft out of the box — no ZooKeeper container, no migration steps, and docker run -p 9092:9092 apache/kafka:4.3.1 alone gives you a working broker. A short Compose file makes that broker reproducible, and a slightly longer one gives you a 3-controller, 3-broker cluster that mirrors production topology.
This post walks through both files, the env-var conventions behind them, and how to keep data across restarts. For what KRaft mode actually is, see our KRaft guide.
Use apache/kafka (JVM) for everyday work and apache/kafka-native (GraalVM, sub-second startup) for integration tests. Configure through KAFKA_* environment variables — but the moment you override anything, the image's defaults vanish and you own the full config. Pin CLUSTER_ID and mount a volume on the log directory if data should survive docker compose down.
The fastest KRaft container: docker run
One command starts a combined-mode node — a single process acting as both broker and controller — with a client listener on localhost:9092:
docker pull apache/kafka:4.3.1
docker run -p 9092:9092 apache/kafka:4.3.1
Two official images exist, and the version history matters. Kafka 3.7.0 shipped apache/kafka (KIP-975): a JVM image on a slim JRE base, published for linux/amd64 and linux/arm64. Kafka 3.8.0 added apache/kafka-native (KIP-974): the broker compiled ahead-of-time to a native binary with GraalVM. The native image starts in under a second, which is excellent for integration tests and CI — but the project recommends it for development and testing only, not production.
The image bundles the full CLI under /opt/kafka/bin, so you can smoke-test without installing anything on the host:
docker exec --workdir /opt/kafka/bin/ -it <container> sh
./kafka-topics.sh --bootstrap-server localhost:9092 \
--create --topic smoke-test --partitions 1 --replication-factor 1
How the apache/kafka image configures KRaft
The image reads any Kafka server property from an environment variable. The naming convention: prefix with KAFKA_, replace . with _, _ with __, and - with ___. So process.roles becomes KAFKA_PROCESS_ROLES, and controller.quorum.voters becomes KAFKA_CONTROLLER_QUORUM_VOTERS.
Precedence runs in three layers. Environment variables win over a property file you mount at /mnt/shared/config, which in turn replaces the defaults baked into the image.
One caveat catches almost everyone. The defaults are all-or-nothing: the moment you override any property, none of the built-in configuration applies, and you must supply a complete working config — listeners, quorum voters, the lot. This is why a bare docker run works with zero variables, then "just adding one setting" in Compose breaks startup.
The container also formats its own storage. On a bare-metal host you generate a cluster ID and run kafka-storage.sh format before first boot — the flow our Kafka without ZooKeeper post walks through. The image does the equivalent at startup, driven by the CLUSTER_ID environment variable.
Single-node KRaft with Docker Compose
Here is a complete, tested single-node file. It adapts the official example from the Apache Kafka repository, with one change: data lives on a named volume, so the broker keeps its topics when the container is recreated.
services:
broker:
image: apache/kafka:4.3.1
container_name: broker
ports:
- "9092:9092"
environment:
# Identity: one node, both roles, one pinned cluster ID
KAFKA_NODE_ID: 1
KAFKA_PROCESS_ROLES: 'broker,controller'
CLUSTER_ID: '4L6g3nShT-eMCtK--X86sw'
# Listeners: host clients, in-network clients, quorum traffic
KAFKA_LISTENERS: 'CONTROLLER://:29093,PLAINTEXT_HOST://:9092,PLAINTEXT://:19092'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT_HOST://localhost:9092,PLAINTEXT://broker:19092'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
# KRaft quorum: this node votes for itself
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
# One broker means internal topics need replication factor 1
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR: 1
KAFKA_SHARE_COORDINATOR_STATE_TOPIC_MIN_ISR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
# Data on a named volume (survives docker compose down)
KAFKA_LOG_DIRS: '/var/lib/kafka/data'
volumes:
- kafka-data:/var/lib/kafka/data
volumes:
kafka-data:
docker compose up -d starts it. The three listeners are the part worth understanding, because nearly every "cannot connect" problem traces back to them:
| Listener | Who connects | Address |
|---|---|---|
PLAINTEXT_HOST | Apps on your laptop, outside Docker | localhost:9092 |
PLAINTEXT | Containers on the same Compose network | broker:19092 |
CONTROLLER | KRaft quorum traffic only | broker:29093 — never advertised to clients |
The rule of thumb: a Kafka client bootstraps to whatever address you give it, but then reconnects to whatever the broker advertises. A host app must receive localhost:9092; a containerized app must receive broker:19092. If you change the host port mapping, change the PLAINTEXT_HOST advertised listener to match, or clients will bootstrap fine and then time out.
That last row of environment variables matters too. Kafka's internal topics — offsets, transaction state, share-coordinator state — default to replication factor 3, and a one-broker cluster cannot satisfy that. The overrides set them to 1. Docker itself must be version 20.10.4 or newer; older versions hit permission errors when the container starts.
A 3-controller, 3-broker cluster in Compose
A single combined node is right for app development. Reach for a real cluster when you want to test partition rebalancing, client failover, or the same isolated-controller topology you run in production. The official examples ship exactly this: three dedicated controllers, three brokers, static quorum.
The controllers are minimal — no client listeners, just quorum traffic:
services:
controller-1:
image: apache/kafka:4.3.1
environment:
KAFKA_NODE_ID: 1
KAFKA_PROCESS_ROLES: 'controller'
KAFKA_LISTENERS: 'CONTROLLER://:9093'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
CLUSTER_ID: '4L6g3nShT-eMCtK--X86sw'
kafka-1:
image: apache/kafka:4.3.1
hostname: kafka-1
ports:
- "29092:9092"
environment:
KAFKA_NODE_ID: 4
KAFKA_PROCESS_ROLES: 'broker'
KAFKA_LISTENERS: 'PLAINTEXT://:19092,PLAINTEXT_HOST://:9092'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka-1:19092,PLAINTEXT_HOST://localhost:29092'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@controller-1:9093,2@controller-2:9093,3@controller-3:9093'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
CLUSTER_ID: '4L6g3nShT-eMCtK--X86sw'
depends_on:
- controller-1
- controller-2
- controller-3
The repetition follows a fixed pattern. Controllers 2 and 3 differ only in KAFKA_NODE_ID; brokers take node IDs 4–6, host ports 29092, 39092, and 49092, and each advertises its own hostname. Every node — controller and broker — carries the identical KAFKA_CONTROLLER_QUORUM_VOTERS line, because that static voter list is the quorum definition. What those three voters do, and how to inspect them with kafka-metadata-quorum.sh, is covered in our controller quorum guide.
Be honest about scope: this is a dev and test topology. Everything shares one host, so it demonstrates failover mechanics without providing fault tolerance. Multi-host production clusters belong on Kubernetes or dedicated machines.
Persisting data: CLUSTER_ID and volumes
By default, everything is ephemeral. The official examples write logs to /tmp/kraft-combined-logs inside the container, so docker compose down discards your topics, consumer offsets, and cluster metadata together.
Making data survive takes two coordinated changes, and the single-node file above already includes both. First, point KAFKA_LOG_DIRS at a stable path and mount a named volume there. Second, pin CLUSTER_ID. The image formats storage using that ID on first boot — if a recreated container generates a mismatched ID against already-formatted storage, Kafka refuses to start rather than risk mixing clusters.
Don't ship the example ID from documentation. Generate your own once and commit it with the Compose file:
docker run --rm apache/kafka:4.3.1 /opt/kafka/bin/kafka-storage.sh random-uuid
We verified the single-node file end to end while writing this post: create a topic, produce a message, docker compose down, docker compose up -d — the topic and message are still there. In the cluster case, give every broker and controller its own volume; they format and own their storage independently.
Which Kafka Docker image should you use?
The image question comes up constantly because years of tutorials point at images with very different maintenance stories today.
| Image | KRaft support | Maintained | Best for |
|---|---|---|---|
apache/kafka | Default since day one | Yes — official ASF image, amd64 + arm64 | The default choice for dev, test, and self-managed use |
apache/kafka-native | KRaft combined mode by default | Yes — official ASF image | Integration tests and CI needing sub-second startup; not production |
confluentinc/cp-kafka | Yes | Yes — Confluent's release cadence | Teams already running the Confluent Platform ecosystem |
bitnami/kafka | Yes | No — frozen at bitnamilegacy since 2025-08-28 | Existing setups migrating off it |
The Bitnami row deserves a fair word. For years bitnami/kafka was the answer most tutorials gave, and its KAFKA_CFG_* variable scheme is all over Stack Overflow. In August 2025 the free image line moved to the bitnamilegacy namespace, where it receives no further updates; maintained builds moved to a commercial subscription. Existing setups keep working, but new Compose files should start from apache/kafka — and old tutorials showing KAFKA_CFG_* variables need translating to the KAFKA_* convention above.
Your Compose cluster runs. What about the data in it?
Local KRaft clusters start as throwaways. Then the seed data gets curated, integration fixtures accumulate, and the same Compose pattern quietly graduates to a shared team environment — and suddenly the cluster holds state someone would miss.
The pattern that runs Kafka in Compose runs its backup too. OSO Kafka Backup ships a container image designed to sit beside the broker in the same Compose file: it writes compressed topic data and consumer group offsets to S3, Azure Blob, GCS, or a mounted volume, and restores to a point in time with millisecond precision. The Docker deployment guide covers the full setup, including scheduled runs.
And keep one distinction straight as the cluster grows: replication factor 3 in the cluster file protects against a broker container dying. It does not protect against deleting the topic itself, a bad producer writing garbage, or a down -v typo — every replica applies the mistake faithfully. Replication is not backup.
FAQ
Frequently asked questions
How do I run Kafka in KRaft mode with Docker Compose?
Define one service from the apache/kafka image with KAFKA_PROCESS_ROLES set to broker,controller, a CONTROLLER listener, KAFKA_CONTROLLER_QUORUM_VOTERS pointing at itself, and a pinned CLUSTER_ID. No ZooKeeper service is needed. A complete tested docker-compose.yml is about 30 lines.
What is the difference between apache/kafka and apache/kafka-native?
apache/kafka, introduced with Kafka 3.7, runs the broker on a JVM and is the general-purpose official image. apache/kafka-native, introduced with Kafka 3.8, compiles the broker to a native binary with GraalVM for sub-second startup. The native image is recommended for development and testing only, not production.
Does the apache/kafka Docker image need ZooKeeper?
No. The official image runs in KRaft mode, where controllers manage cluster metadata natively. None of the official Docker or Compose examples include a ZooKeeper container.
How do I persist Kafka data in Docker Compose?
Mount a named volume at the path set in KAFKA_LOG_DIRS and pin a fixed CLUSTER_ID in the environment. The volume keeps topic data and metadata across container recreation, and the pinned ID keeps the recreated container consistent with the already-formatted storage.
Is the Bitnami Kafka Docker image still maintained?
The free bitnami/kafka image line was frozen in August 2025 and moved to the bitnamilegacy namespace, which receives no further updates. Maintained Bitnami builds require a commercial subscription. New setups should use the official apache/kafka image instead.
Conclusion
KRaft removed the last excuse for a heavyweight local Kafka. One apache/kafka service gives you a combined-mode broker in seconds; the isolated cluster file gives you production-shaped topology when you need it. The mechanics worth internalizing are small: the KAFKA_* naming convention, the all-or-nothing override rule, the three-listener split, and the CLUSTER_ID-plus-volume pair for persistence.
Once the cluster stops being disposable, treat its data accordingly — config in git deserves data with a restore path.
The same Compose pattern that runs KRaft Kafka runs OSO Kafka Backup beside it — compressed topic data and consumer group offsets written to S3, Azure, GCS, or a local volume, restorable to a point in time. Get started.