Kafka In-Sync Replicas (ISR) Explained
In-sync replicas (ISR) are the subset of a Kafka partition's replicas that are currently caught up with the leader — the leader itself, plus every follower that has fetched to the leader's latest offset within replica.lag.time.max.ms (30 seconds by default). The ISR is the mechanism behind every guarantee Kafka makes: only ISR members confirm acks=all writes, and only ISR members can become leader without losing data.
Replication factor is a static promise; the ISR is the live measurement of how much of that promise currently holds. This guide explains how replicas join and leave the ISR, what the high watermark commits, how min.insync.replicas and acks form a durability contract, and what happens during leader election when the ISR has shrunk.
The ISR defines what "committed" means in Kafka. A record is committed once every ISR member has it, consumers can only read committed records, and any ISR member can take over as leader with nothing lost. Everything downstream — durability, availability, failover safety — is a function of who is in that set.
What is an in-sync replica in Kafka?
Every Kafka partition has one leader replica and, at replication factor above one, follower replicas on other brokers. The in-sync replicas are the leader plus the followers that are keeping up with it. A follower keeps up by continuously fetching records; it counts as in sync while it has caught up to the leader's log end offset within the replica.lag.time.max.ms window. The leader is always a member of its own ISR.
The distinction from replication factor matters. Replication factor says how many copies each partition should have — that choice is covered in the Kafka replication factor guide. The ISR says how many of those copies are currently trustworthy. A topic with replication factor 3 and an ISR of 2 is one broker failure away from having a single good copy.
You can see both sets for any topic with one command:
kafka-topics.sh --bootstrap-server localhost:9092 \
--describe --topic orders
Topic: orders Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: orders Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
Topic: orders Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 3,1
Replicas is the assignment; Isr is the reality. Partition 2 above has lost broker 2 from its ISR — it is under-replicated, and the under-replicated partitions guide covers the incident workflow for exactly that situation.
How replicas join and leave the ISR
Followers replicate by sending the leader a stream of fetch requests, appending whatever the leader returns in the same order. There is no separate synchronization protocol: staying in the ISR simply means this fetch loop is keeping pace.
A follower leaves the ISR — an ISR shrink — in two situations. Either its broker has failed outright, or it is running but has not caught up to the leader's log end offset within replica.lag.time.max.ms. The default lag window is 30 seconds; KIP-537 raised it from 10 seconds in Kafka 2.5 because routine garbage-collection pauses were knocking healthy followers out of the ISR. The leader detects the lag, and the membership change is recorded in the cluster metadata so every broker agrees on the current set.
Rejoining — an ISR expand — is automatic. A recovered follower resumes fetching, works through the backlog until it reaches the leader's log end offset, and is added back to the set. No operator action is required; the only thing to watch is that catch-up is actually progressing.
Two broker metrics summarize this lifecycle: IsrShrinksPerSec and IsrExpandsPerSec. Both should sit near zero in a healthy cluster. Sustained shrink-expand flapping means a follower is chronically on the edge of the lag window — usually saturated disk or network rather than a dead broker. The under-replicated partitions guide walks through finding that broker and fixing each cause.
The high watermark: what "committed" means
The ISR does more than track health — it defines commitment. Each partition leader maintains a high watermark: the highest offset that every current ISR member has replicated. Records at or below the high watermark are committed; records above it exist only on some replicas and are not yet guaranteed.
Consumers can only read up to the high watermark. That rule is what makes failover invisible to applications: a consumer never sees a record that could vanish because it lived only on a leader that then failed. Whatever was readable before an election is still present after it, because every ISR member had it.
This is also the precise sense in which a shrunken ISR weakens durability. The high watermark advances when every ISR member has a record — so a smaller ISR means fewer copies must confirm before Kafka calls a record committed. With replication factor 3 and an ISR of 2, "committed" means two copies, not three. The full replication flow behind this is covered in Kafka replication explained.
min.insync.replicas and acks: the durability contract
Producer acks and the topic setting min.insync.replicas turn ISR membership into an enforceable contract. The subtlety that surprises most people: acks=all waits for every replica currently in the ISR — not every assigned replica. If the ISR has shrunk to just the leader, acks=all is satisfied by one copy.
min.insync.replicas supplies the floor that makes the guarantee real. It sets the smallest ISR that may accept an acks=all write; below it, the broker rejects writes with NotEnoughReplicas and producers see errors instead of silently reduced durability. The default is 1, which enforces nothing — production topics with replication factor 3 almost always set it to 2. Since Kafka 3.0, the producer default is acks=all with idempotence enabled, so the broker-side floor is the piece you must set deliberately.
| Configuration | Behavior when replicas fail | What it protects |
|---|---|---|
RF 3, min ISR 1, acks=all | Writes continue down to a single copy | Availability, not durability |
RF 3, min ISR 2, acks=all | One replica may be lost; writes stop below 2 copies | The production baseline |
RF 3, min ISR 2, acks=1 | Leader alone acknowledges; min ISR never checked | Little — the floor only applies to acks=all |
RF 3, min ISR 3, acks=all | Any missing replica stops writes | Maximum copies, lowest write availability |
The third row deserves emphasis: min.insync.replicas is only enforced for acks=all requests. A producer using acks=1 can receive success from a leader that fails before any follower fetches the record, whatever the topic's minimum says. Choosing the replication factor side of this equation is covered in the replication factor guide.
ISR and leader election: who is allowed to take over
When a leader fails, the controller elects a replacement — and under normal configuration, only ISR members are candidates. The logic follows directly from the high watermark: every ISR member has every committed record, so promoting any of them loses nothing. This is a clean leader election, and it is why Kafka can fail over in seconds without breaking its durability promise.
The hard case is a partition whose ISR members are all gone. The remaining choice is stark: leave the partition offline until an ISR member returns, or promote an out-of-sync replica that is missing committed records. unclean.leader.election.enable controls that choice, and its default is false — Kafka prefers unavailability over data loss. Setting it to true restores availability faster in a disaster, at the documented cost that records already acknowledged to producers can disappear.
Unclean election should be a deliberate, per-topic decision, not a cluster-wide reflex. Enabling it means accepting that under a sufficiently bad failure, replication alone will discard committed data — a fact worth remembering when weighing replication against independent backups.
One related setting is easy to confuse with ISR eligibility. The preferred leader is simply the first replica in the assignment list, used to spread leadership evenly across brokers. Preference decides who leads when everything is healthy; ISR membership decides who is allowed to lead when things fail.
ISR configuration reference
Four settings define ISR behavior end to end:
| Setting | Scope | Default | What it controls |
|---|---|---|---|
replica.lag.time.max.ms | Broker | 30000 | How far a follower may lag before leaving the ISR |
min.insync.replicas | Broker or topic | 1 | Smallest ISR that may accept an acks=all write |
unclean.leader.election.enable | Broker or topic | false | Whether a non-ISR replica may become leader |
acks | Producer | all (since Kafka 3.0) | How many acknowledgements a producer waits for |
Treat the middle two as a pair. min.insync.replicas=2 on a replication factor 3 topic buys real durability only while unclean.leader.election.enable stays false — enabling unclean election reintroduces exactly the loss the minimum ISR was set to prevent.
A full ISR is not a backup
An ISR at full strength means every replica agrees with the leader. It says nothing about whether the data itself is worth agreeing on. A bad deploy writing corrupt records, an accidental topic deletion, a retention misconfiguration expiring data early — the ISR replicates each of these to every member in milliseconds, with all health metrics reading green throughout.
Unclean leader election makes the limitation explicit from the other direction: Kafka's own configuration admits that under a bad enough failure, replication either sacrifices availability or discards committed records. Replication protects against broker failure. It has no answer for logical damage or for failures larger than the cluster.
That layer comes from an independent backup — copies in object storage whose durability does not depend on the cluster's own health. OSO Kafka Backup writes continuous backups to S3, GCS, or Azure Blob with point-in-time recovery and consumer group offset preservation, so a restore returns both the data and the consumer positions to a moment before the damage. For recovering a whole Kafka estate after the failures replication cannot absorb, see the disaster recovery use case.
The ISR is Kafka's live durability ledger
The in-sync replica set defines what committed means, who may lead, and whether writes proceed. Followers join by keeping up and leave by falling 30 seconds behind; the high watermark commits what every member holds; min.insync.replicas sets the floor beneath acks=all; and clean election promotes only members, so failover loses nothing. Configure the floor deliberately, watch the shrink and expand rates, keep unclean election off unless you have consciously chosen availability over data — and pair the ISR with an independent backup for the failures it cannot express.
Frequently asked questions
What is in sync replica in Kafka?
An in-sync replica (ISR) is a partition replica that is currently caught up with the leader — it has fetched to the leader log end offset within replica.lag.time.max.ms, which defaults to 30 seconds. The ISR always includes the leader itself. Only ISR members acknowledge acks=all writes and only ISR members can be elected leader without data loss.
Why are Kafka replicas not in sync?
A replica falls out of sync when its broker is down or when it cannot keep up with the leader for more than replica.lag.time.max.ms. Common causes for a running but lagging follower are saturated disk I/O, network congestion, long garbage-collection pauses, or too few replica fetcher threads. Run kafka-topics.sh --describe --under-replicated-partitions to find which broker is missing from the ISR.
When does a NotEnoughReplicas error happen in Kafka?
A producer using acks=all receives NotEnoughReplicas when the partition ISR has shrunk below the topic min.insync.replicas setting. With replication factor 3 and min.insync.replicas=2, that means two of the three replicas are unavailable or lagging. Writes resume automatically once enough replicas catch up and rejoin the ISR.
How does Kafka maintain minimum in-sync replicas?
Kafka does not keep replicas in sync to meet the minimum — min.insync.replicas is a write gate, not a repair mechanism. The leader tracks which followers are caught up, and if the ISR drops below the configured minimum, the broker rejects acks=all writes until recovered followers catch up and rejoin. Records above the high watermark that were never committed are truncated when replicas resynchronize after a leader change.