Kafka Point-in-Time Recovery, Explained
Kafka point-in-time recovery (PITR) restores topic data to a specific moment — down to the millisecond — by filtering backed-up records on the timestamps they already carry. Kafka itself has no rewind button. Retention deletes forward, and replication copies corruption as faithfully as it copies good data. PITR therefore needs two things the platform does not provide: an immutable copy of the data outside the cluster, and a restore engine that filters that copy by time.
This post explains the mechanism end to end: why native Kafka features cannot rewind topic state, how timestamp filtering works, what happens to consumer groups, and the caveats — clock skew, transactions, compaction — that decide whether the restore you get matches the restore you planned.
PITR is an immutable backup plus a timestamp filter applied at restore time. The time window controls where you cut. Backup cadence controls how fresh that cut can be. They are different guarantees — plan them separately.
What point-in-time recovery means for Kafka
Point-in-time recovery rebuilds a topic as it existed at a chosen moment. Four incident shapes call for it:
- A bad deploy started writing corrupt records at 14:02:37. You want everything up to 14:02:37.411 and nothing after.
- An accidental deletion removed a topic, or a retention misconfiguration purged data early. You want the topic back as of this morning.
- An investigation needs the exact records produced between 10:00 and 11:00, in a copy nobody else touches.
- An audit asks for the data as it existed on a specific date, months after the fact.
The raw material for all four is the same. Every Kafka record has carried a timestamp since Kafka 0.10 (KIP-32). Store those records in an immutable copy, and "restore to a moment" becomes "replay the records whose timestamps fall inside a window."
Why Kafka has no native point-in-time recovery
Every durability feature Kafka ships protects the live log. None of them can rewind it.
Retention only moves forward. Old segments are deleted on schedule, so the cluster holds recent data, never past states.
Replication follows the log. Followers, and mirrors built with MirrorMaker 2, apply every write in order — including the corrupt ones. A bad record reaches every replica within seconds, which is why replication is not backup. There is no replica that represents "the topic as of an hour ago."
Offset replay looks closest to PITR and is the most common point of confusion. A consumer can rewind to an earlier offset and re-read the log. That helps when the data is intact and still within retention. It cannot resurrect a deleted topic, and it cannot remove poisoned records that are already in the log — replay delivers them again.
Tiered storage extends retention by moving older segments to object storage, but the tiered data still belongs to the live cluster. Delete the topic and the tiered segments go with it. The tiered storage guide covers that boundary in depth.
| Mechanism | Rewinds topic state? | Limits |
|---|---|---|
| Retention | No | Forward-only; deletes old segments on schedule |
| Replication (RF 3, MirrorMaker 2) | No | Applies the log as-is; corruption and deletes propagate in seconds |
| Consumer offset replay | Partially | Needs records still in retention; cannot undo deletes or skip bad records |
| Tiered storage | No | Extends retention on the live cluster; topic deletion still deletes |
| Backup + timestamp filter (PITR) | Yes | Needs an immutable copy maintained outside the cluster |
The last row is the only one that works after the log itself is damaged. That is the row this post is about.
How timestamp-based PITR works
A PITR restore reads an immutable backup, skips everything outside the requested time window, and streams the remainder into the target topic. Three steps make that fast and precise:
- The backup preserves timestamps. Records are stored with their original timestamps, and the backup manifest records the time range each segment file covers.
- You set the window.
time_window_startandtime_window_endtake Unix milliseconds. Both are optional: omit the start to include everything earlier, omit the end to include everything later. - The engine filters twice. Segment files entirely outside the window are skipped before they are read or decompressed. Records inside the remaining segments are filtered one by one as they stream toward the target.
Because the filter runs at restore time, any backup taken with OSO Kafka Backup supports a time-windowed restore — incremental backups included. There is no special PITR mode to enable in advance.
Here is the restore for the bad-deploy scenario, cutting one millisecond before the first corrupt write:
mode: restore
backup_id: "prod-backup-20241201"
target:
bootstrap_servers:
- broker-1.kafka.svc:9092
storage:
backend: s3
bucket: company-kafka-backups
region: us-west-2
prefix: production/prod-cluster
restore:
create_topics: true
# Bad deploy started writing at 14:02:37.412 UTC — cut just before it
time_window_end: 1733061757411 # 2024-12-01T14:02:37.411Z
With no time_window_start, every record older than the cut comes back. Every record from 14:02:37.412 onward stays out. Different topics can even use different windows in the same run, with per-topic time_window_start and time_window_end entries.
Before you choose a window, check what the backup actually covers:
kafka-backup describe \
--path s3://company-kafka-backups/production/prod-cluster \
--backup-id prod-backup-20241201 \
--format json | jq '.time_range'
One detail worth knowing before you trust the window: which timestamp is being filtered. Kafka sets this per topic through message.timestamp.type: CreateTime (the default) is stamped by the producer, LogAppendTime by the broker on arrival. PITR filters whichever value the record carries.
The same window can land in the original topic, in a side topic through topic_mapping, or on a different cluster entirely. Those restore-target patterns, with configs for each, live on the point-in-time recovery solution page. For the internals — manifest layout, segment selection, streaming filter — see the PITR implementation docs.
What happens to consumer groups
A restored partition is a new log: the same records now live at different offsets. Committed consumer positions refer to the old log, so leaving them untouched means consumers resume at meaningless offsets.
The restore config decides this explicitly with consumer_group_strategy: skip leaves groups alone (right for inspection copies), header-based repositions groups using original offsets preserved in record headers, and timestamp-based and manual cover clusters where headers are absent. Set reset_consumer_offsets: true alongside a strategy to apply it. The mechanics are documented in offset translation, and the wider topic has its own post: backing up Kafka consumer offsets.
The caveats that decide whether PITR works
Clock skew moves records in time. With CreateTime, timestamps come from producer machines. A producer running five minutes fast stamps its records five minutes into the future, and your carefully chosen window edge catches the wrong data. Run NTP on every producer host, or shift the topic to broker-side timestamps:
bin/kafka-configs.sh --bootstrap-server kafka:9092 \
--entity-type topics --entity-name orders \
--alter --add-config message.timestamp.type=LogAppendTime
The trade-off: LogAppendTime records when the broker received the record, not when the event happened.
Transactions can be split. The filter works on timestamps, not transaction boundaries. A window edge that falls inside a transaction can restore some of its records and exclude the rest. If transactional integrity matters, place edges in quiet periods and add a few seconds of buffer.
Compacted topics restore backup-time state. A backup of a compacted topic holds the records that existed when the backup ran. A windowed restore filters those records — it does not reconstruct the compaction history the source cluster went through.
Precision is one millisecond. Kafka timestamps are millisecond-resolution, and so is the window. Records sharing the same millisecond are either all in or all out, and sub-millisecond ordering is not guaranteed.
PITR precision is not RPO
A time window can only cut within data that a backup contains. That puts two hard bounds on any PITR plan. The newest reachable moment is the newest backed-up record — set by how often backups run. The oldest reachable moment is the oldest retained backup — set by your backup retention policy.
If the last backup finished at 02:00 and the incident hit at 14:02, no window setting reaches 14:01. Millisecond precision tells you where you can cut; backup cadence tells you how much recent data exists to cut into. Size both against your loss tolerance in the RTO/RPO planning guide.
And rehearse before the incident, not during it. Set dry_run: true to validate the whole restore plan without producing a record, and run kafka-backup validate --deep to prove every segment is readable. Backup verification shows how to make both checks routine.
FAQ
Frequently asked questions
How far back can Kafka point-in-time recovery go?
As far as your backup retention allows. The window can target any moment between the oldest record in your retained backups and the newest backed-up record. Run kafka-backup describe to see the exact time range a backup covers.
How is Kafka PITR different from replaying a topic from an earlier offset?
Offset replay re-reads the live log, so it only works while the records are intact and still within retention. PITR rebuilds records from an immutable backup, so it works after a topic deletion and can exclude corrupt records by ending the window before they were written.
Does point-in-time recovery improve your Kafka RPO?
No. Your recovery point objective is bounded by how recently a backup captured data. PITR controls where you cut inside the backed-up data with millisecond precision, while backup frequency determines how fresh that data can be.
Which timestamp does Kafka point-in-time recovery filter on?
The timestamp stored in each record. Kafka sets this per topic through message.timestamp.type: CreateTime, the default, is set by the producer, while LogAppendTime is set by the broker on arrival. PITR filters the stored value regardless of type.
Does Kafka PITR respect transaction boundaries?
No. Filtering is timestamp-based, so a window edge that falls inside a transaction can restore part of it. If transactional integrity matters, place window edges in quiet periods, add a few seconds of buffer, and verify the result before moving consumers over.
Conclusion
Kafka protects the live log well and rewinds it not at all. Point-in-time recovery is what fills that gap: an immutable copy outside the cluster, filtered by the timestamps every record already carries, streamed back with millisecond precision. The mechanism is simple; the discipline around it is what pays off — synchronized clocks, window edges away from transactions, and a backup cadence that matches the data loss you can actually tolerate.
OSO Kafka Backup filters any backup by time window at restore — millisecond precision, consumer group repositioning included. See the restore patterns or get started.