How to Change Kafka Replication Factor Safely (Step by Step)
To change Kafka replication factor, there is no single alter command. You change it by reassigning partitions: write a JSON plan that lists the full target replica set for each partition, then execute and verify that plan with kafka-reassign-partitions.sh. Done casually, the same procedure can saturate your network, stall producers, or leave throttles choking replication for weeks.
This is the runbook for doing it safely. It covers the preflight checks, the JSON plan, throttled execution, progress monitoring, verification, rollback, and the failure modes at each step. If you are still deciding what value to run, the replication factor guide covers choosing; this post covers changing.
A replication factor change is a partition reassignment. Save the rollback JSON that --execute prints, always set a throttle, watch replica lag until every partition reports completed, and finish with --verify — it is the step that removes the throttle.
Why there is no --alter flag for replication factor
kafka-topics.sh --alter can add partitions to a topic, but it cannot change how many brokers hold each partition. That is deliberate. Adding a partition creates an empty log; adding a replica copies every byte of an existing partition across the network.
Kafka makes that cost explicit by routing all replica changes through partition reassignment. The Apache Kafka operations documentation describes the mechanism: you hand kafka-reassign-partitions.sh a JSON file naming the complete replica list you want, and the brokers copy data until reality matches the file.
The same tool handles increases, decreases, and plain broker-to-broker moves. Only the contents of the replica lists differ.
Before you start: the preflight checklist
Ten minutes of checking prevents most reassignment incidents. Work through this table before touching production:
| Check | Why it matters | How to check |
|---|---|---|
| Enough registered brokers | The target factor cannot exceed the brokers able to host replicas | Count brokers; a plan naming a missing broker ID never completes |
| Disk headroom on destinations | Each new replica stores a full copy of the partition | Compare partition size against free space on each target broker |
| Rack spread | Three replicas in one rack still fail together | Check broker.rack and place each partition's replicas across racks |
| Topic currently healthy | Reassigning an already-degraded topic compounds the problem | --describe shows Isr matching Replicas before you begin |
| Throttle value chosen | An unthrottled move competes with production traffic | Start well below spare network capacity; you can raise it mid-flight |
| Rollback location agreed | --execute prints the original assignment exactly once | Decide where the rollback JSON gets saved before running anything |
min.insync.replicas compatibility | A decrease below the min-ISR floor breaks acks=all producers | Compare the target factor against the topic's min.insync.replicas |
One more precaution for large moves: take an independent backup first. A reassignment shuffles live data with no undo, and the disaster recovery use case shows what a restorable copy adds when a maintenance window goes wrong.
How to change Kafka replication factor step by step
The example below raises the topic orders from a replication factor of 2 to 3 across brokers 1, 2, and 3. The same six steps apply to any topic and any target factor.
Step 1: Capture the current assignment
bin/kafka-topics.sh --bootstrap-server localhost:9092 \
--describe --topic orders
Topic: orders Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2
Topic: orders Partition: 1 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: orders Partition: 2 Leader: 3 Replicas: 3,1 Isr: 3,1
Save this output. It is your record of the starting state and your sanity check at the end.
Step 2: Write the reassignment JSON
List the complete target replica set for every partition you are changing — existing replicas plus the new one, not just the addition:
{
"version": 1,
"partitions": [
{ "topic": "orders", "partition": 0, "replicas": [1, 2, 3] },
{ "topic": "orders", "partition": 1, "replicas": [2, 3, 1] },
{ "topic": "orders", "partition": 2, "replicas": [3, 1, 2] }
]
}
Two details matter here. Keep each partition's existing replicas in the list, so Kafka only copies data for the one new replica. And rotate the order — the first broker in each list is the preferred leader, so identical lists would pile every leader onto one broker.
Step 3: Execute with a throttle
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
--reassignment-json-file increase-rf.json \
--execute --throttle 50000000
Current partition replica assignment
{"version":1,"partitions":[{"topic":"orders","partition":0,"replicas":[1,2],"log_dirs":["any"]}, ...]}
Save this to use as the --reassignment-json-file option during rollback
The inter-broker throttle limit was set to 50000000 B/s
Successfully started partition reassignments for orders-0,orders-1,orders-2
The tool prints the original assignment and tells you to save it — do exactly that, into a file like rollback-orders.json. This output is the only rollback plan you get. The throttle here caps replica movement at 50 MB/s between brokers so the copy cannot starve production traffic.
Step 4: Monitor progress
Check status with the same JSON file:
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
--reassignment-json-file increase-rf.json --verify
While replicas copy, partitions report still in progress. You can also see every active reassignment cluster-wide with --list. Under the hood, watch the follower lag metric — kafka.server:type=FetcherLagMetrics,name=ConsumerLag — which should fall steadily for the new replicas.
Expect under-replicated partition alerts during the move: a replica that has not caught up yet is under-replicated by definition. Alerts that persist after completion are a different story, covered in the under-replicated partitions guide.
Step 5: Complete and clear the throttle
Re-run --verify until every partition reports completed:
Status of partition reassignment:
Reassignment of partition orders-0 is completed
Reassignment of partition orders-1 is completed
Reassignment of partition orders-2 is completed
Clearing broker-level throttles on brokers 1,2,3
Clearing topic-level throttles on topic orders
Those last two lines are why --verify is not optional. The throttle configs stay on the cluster until this command clears them, and a leftover throttle silently limits normal replication — including recovery after the next broker failure.
Step 6: Confirm the result
bin/kafka-topics.sh --bootstrap-server localhost:9092 \
--describe --topic orders
Every partition should now show three brokers in Replicas and the same three in Isr. If Isr is still shorter, the new replicas are catching up — give them time before declaring the change done.
Throttling: protect production traffic during the move
The --throttle flag configures five settings for you. Knowing what they are helps when you need to inspect or adjust a move in flight:
| Config | Level | What it limits |
|---|---|---|
leader.replication.throttled.rate | Broker (dynamic) | Bytes/sec a leader sends to throttled replicas |
follower.replication.throttled.rate | Broker (dynamic) | Bytes/sec a follower fetches for throttled replicas |
replica.alter.log.dirs.io.max.bytes.per.second | Broker (dynamic) | Disk-to-disk moves inside one broker |
leader.replication.throttled.replicas | Topic | Which replicas the leader throttle applies to |
follower.replication.throttled.replicas | Topic | Which replicas the follower throttle applies to |
The tool applies the leader throttle to the replicas that existed before the move and the follower throttle to the destinations. You can inspect all five with kafka-configs.sh --describe.
One throttle rule is worth memorizing: if the incoming producer rate on a broker exceeds the throttle, the new replica can never catch up. The operations documentation states the stall condition as max(BytesInPerSec) > throttle. When lag stops falling, raise the limit mid-flight by re-running with --additional:
bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
--reassignment-json-file increase-rf.json \
--additional --execute --throttle 100000000
Rolling back or cancelling a reassignment
If the move is causing trouble, you have two levers. To stop in-flight moves, run --cancel with the same JSON file; the tool cancels the active reassignments for those partitions. To return to the starting layout, execute the rollback JSON you saved in step 3 — a rollback is just another reassignment.
Two related flags matter for bigger operations. --preserve-throttles keeps throttle configs in place between batches of a multi-stage move. And --disallow-replication-factor-change adds validation for plain broker-to-broker moves, so a typo in a replica list cannot silently change a partition's factor.
Decreasing the replication factor
Reducing the factor uses the same runbook with shorter replica lists. It is fast — Kafka deletes copies instead of creating them — but it deserves more caution, not less.
Keep the target factor at or above the topic's min.insync.replicas, or acks=all producers start failing with not-enough-replicas errors the moment the change lands. The interplay between the factor, the ISR, and the min-ISR floor is covered in the Kafka replication guide. Prefer keeping the current leader in each replica list to avoid unnecessary leader elections, and remember you are permanently deleting redundancy that took real time to build.
Internal topics and clusters that grew out of dev defaults
Replication factor changes are most often needed on clusters that started small. The broker setting default.replication.factor ships as 1, so auto-created topics on a cluster that began life as a single-broker install carry a factor of 1 into production.
The consumer offsets topic deserves special attention. offsets.topic.replication.factor defaults to 3, but __consumer_offsets is created once — a cluster that started with one broker keeps its single-copy offsets topic even after growing to ten brokers. Losing it loses every consumer group's position. The fix is exactly the runbook above, applied to __consumer_offsets.
Auditing is one command:
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe \
| grep "ReplicationFactor: 1"
Common failures and what they mean
| Symptom | Likely cause | Fix |
|---|---|---|
InvalidReplicationFactorException at topic creation | Requested factor exceeds registered brokers | Add brokers or lower the factor |
| Reassignment never completes | Throttle below producer inflow, or a target broker is offline | Raise the throttle with --additional, or fix the dead broker ID in the plan |
Isr shorter than Replicas after completion | A new replica keeps falling behind | Diagnose with the under-replicated partitions guide |
| Producers fail after a decrease | Factor dropped below min.insync.replicas | Restore the factor or lower the min-ISR floor deliberately |
One boundary to keep straight while you work: reassignment moves live copies around, and every copy carries the same records — including any delete or bad write that already happened. More replicas mean more failure tolerance, never a way back to yesterday's data. That layer comes from an independent backup, and the comparison of alternatives maps which tool covers which failure.
A reassignment moves live replicas; it cannot undo a mistake. Kafka Backup keeps independent, point-in-time restorable copies of your topics and consumer offsets while you reshape the cluster — start with the disaster recovery use case.
Frequently asked questions
How do I increase the replication factor of a Kafka topic?
Write a reassignment JSON that lists the complete target replica set for each partition, keeping the existing replicas and adding the new broker IDs. Execute it with kafka-reassign-partitions.sh --execute and a --throttle value, then re-run with --verify until every partition reports completed and the throttles are cleared.
How do I reduce the replication factor of a topic in Kafka?
Use the same reassignment procedure with fewer broker IDs in each partition's replica list. Keep the target factor at or above the topic's min.insync.replicas setting, otherwise producers using acks=all start failing, and prefer keeping the current leader in each list to avoid leader elections.
Can kafka-topics.sh --alter change the replication factor?
No. kafka-topics.sh --alter can increase a topic's partition count but cannot change its replication factor. Replication factor changes require a partition reassignment executed with kafka-reassign-partitions.sh, because Kafka must copy existing partition data onto the new replicas.
What causes InvalidReplicationFactorException in Kafka?
Kafka raises InvalidReplicationFactorException when a topic is created with a replication factor larger than the number of brokers able to host replicas. Add brokers or request a smaller factor. A reassignment plan that names a nonexistent broker ID fails differently: it simply never completes.
What replication factor should the __consumer_offsets topic use?
The broker default for offsets.topic.replication.factor is 3, and production clusters should keep it there or higher. The topic is created once, so a cluster that started with a single broker can carry a one-copy __consumer_offsets topic for years; fix it with the standard partition reassignment runbook.
Wrapping up
Changing the Kafka replication factor safely comes down to respecting what it really is: a bulk data copy wearing a config change's clothes. Plan the full replica lists, save the rollback JSON before anything moves, throttle the transfer, watch lag until every partition completes, and let --verify clean up after you. Do those five things and a replication factor change becomes a routine, reversible operation — and with an independent backup taken first, even the worst case stays recoverable.