Invalid replication factor: why topic creation failed
Kafka refused to create the topic because it cannot place that many replicas on the brokers it can see. The fix depends on why the brokers are missing — count them before you change anything.
What the error means
org.apache.kafka.common.errors.InvalidReplicationFactorException:
Replication factor: 3 larger than available brokers: 1.
Kafka raises the invalid replication factor error when a topic is created with more replicas than there are brokers able to host them. Each replica of a partition must live on a different broker, so a factor of 3 needs at least three registered, live brokers.
On KRaft clusters the same failure often reads differently:
INVALID_REPLICATION_FACTOR (Unable to replicate the partition 3 time(s):
All brokers are currently fenced.)
"Fenced" means the brokers exist but are not heartbeating to the controller quorum, so none of them can take a replica. Either way, the question is the same: how many brokers can the cluster actually place replicas on right now?
How to fix invalid replication factor errors in Kafka
1. Count the brokers the cluster can see.
kafka-broker-api-versions.sh --bootstrap-server kafka:9092 | grep -c "id:"
2. Match the count to the cause.
| Situation | Why it fails | Fix |
|---|---|---|
Single-broker dev cluster, explicit --replication-factor 3 | Only one broker to place replicas on | Create with --replication-factor 1 on dev; keep 3 for production |
Fresh single-broker setup fails creating __consumer_offsets | Internal topics default to a factor of 3 (offsets.topic.replication.factor) | Set offsets.topic.replication.factor=1, transaction.state.log.replication.factor=1, and transaction.state.log.min.isr=1 in the broker config for dev |
| KRaft: "All brokers are currently fenced" | Brokers registered but not heartbeating — controller quorum unreachable, listener misconfiguration, or brokers mid-shutdown | Check broker logs for heartbeat errors; verify controller.quorum settings and advertised listeners; wait for registration before creating topics |
| Production cluster after losing brokers | Live broker count dropped below the requested factor | Recover the brokers first; topic creation is rarely the urgent fix during a broker outage |
| Restore job fails while auto-creating topics | default_replication_factor in the restore config exceeds the target cluster's broker count | Set it at or below the target's broker count — see below |
3. Re-run the creation. The error is not retriable by producers — the topic simply does not exist until a creation request succeeds with a valid factor.
The restore-time version of this error
This exception has a habit of appearing during restores and migrations: a backup taken on a six-broker production cluster is restored into a three-broker staging cluster, with the restore configured to recreate topics at production's replication factor. The target cannot place the replicas, and topic creation fails.
Restored topics are created with the default_replication_factor you set —
so set it for the cluster you are restoring into, not the one you backed
up from.
- restore.yaml
- Rehearse first
mode: restore
backup_id: "production-backup"
target:
bootstrap_servers:
- staging-kafka-1:9092
- staging-kafka-2:9092
storage:
backend: s3
bucket: my-kafka-backups
region: us-west-2
prefix: backups/production
restore:
create_topics: true
default_replication_factor: 3 # Must not exceed the target's broker count
topic_mapping:
orders: orders-staging
kafka-backup validate-restore --config restore.yaml
kafka-backup restore --config restore.yaml
validate-restore checks the plan against the live target, so a factor the
cluster cannot satisfy surfaces before any data moves. Every key above is in
the configuration reference, and the full
restore walkthrough is on the
restore a Kafka topic page.
After the fix
Pick replication factors deliberately: 1 is fine for throwaway dev, 3 is the production norm, and anything above your broker count will always fail. If an existing topic needs a different factor, that is a partition reassignment rather than a setting change — the change replication factor guide covers the procedure. And before you resize clusters or rebuild dev environments, a verified backup means a failed topic creation is an inconvenience, not an incident.
Frequently asked questions
What causes InvalidReplicationFactorException in Kafka?
A topic creation request asked for more replicas than the cluster has available brokers. Each replica must live on a different broker, so a replication factor of 3 needs three live, registered brokers. It also fires when brokers exist but are fenced, and during restores that recreate topics with a factor sized for a bigger cluster.
Why does Kafka say "All brokers are currently fenced"?
On KRaft clusters, a broker is fenced when it stops heartbeating to the controller quorum. Fenced brokers cannot host new replicas, so topic creation fails even though the brokers are running. Check broker logs for heartbeat failures and verify the controller quorum and advertised listener configuration.
Can the replication factor be higher than the broker count?
No. Kafka never places two replicas of the same partition on one broker, so the factor is capped by the number of brokers able to host replicas. Add brokers or lower the factor.
Why does __consumer_offsets fail to create on a single broker?
The internal offsets topic defaults to a replication factor of 3 via offsets.topic.replication.factor. On a one-broker dev cluster that default cannot be satisfied, so the first consumer group triggers this error. Set the offsets and transaction-log factors to 1 in the broker config for development clusters.
How do I set the replication factor when restoring topics from a backup?
Set default_replication_factor under the restore section with create_topics enabled. Restored topics are created with that value, so size it to the target cluster — a backup from a six-broker cluster restores into a three-broker cluster as long as the configured factor fits the target.
Ready to protect your Kafka data?
Take your first backup in minutes with the open source CLI, or talk to us about Enterprise features like encryption, RBAC, and audit logging.