Running Kafka on AWS means choosing who operates the brokers. Self-managed EC2 leaves the install, the quorum, and the disks with your team. Amazon MSK provisioned and MSK Serverless leave broker operations with AWS. Confluent Cloud on AWS leaves them with Confluent.

Topic data stays your responsibility on every option. A live replica is not a restore point. Multi-AZ replication, MirrorMaker, and MSK Replicator copy deletes and bad writes with the good records. AWS Backup's supported services do not include MSK.

Kafka MirrorMaker 2 offset sync is the mechanism to translate consumer offsets between clusters. It maps a consumer group's committed position on the source cluster to the equivalent position on the target cluster. The same record almost never sits at the same offset in both logs, so MirrorMaker 2 records source-to-target offset pairs in an internal offset-syncs topic and emits per-group checkpoints that failover consumers use to resume without reprocessing or skipping data.

Topic data that fails over without consumer positions is an incomplete disaster recovery story. Applications either replay hours of duplicates or silently skip records. This guide explains the machinery behind offset translation: the three internal topics, the checkpoint flow, automated group offset sync, the tuning knobs with their real defaults, and the places where the mechanism breaks down.

In-sync replicas (ISR) are the subset of a Kafka partition's replicas that are currently caught up with the leader — the leader itself, plus every follower that has fetched to the leader's latest offset within replica.lag.time.max.ms (30 seconds by default). The ISR is the mechanism behind every guarantee Kafka makes: only ISR members confirm acks=all writes, and only ISR members can become leader without losing data.

Replication factor is a static promise; the ISR is the live measurement of how much of that promise currently holds. This guide explains how replicas join and leave the ISR, what the high watermark commits, how min.insync.replicas and acks form a durability contract, and what happens during leader election when the ISR has shrunk.

Kafka under-replicated partitions (URPs) are partitions whose in-sync replica (ISR) count has dropped below their replication factor. One or more follower replicas has either died or fallen behind the leader for longer than replica.lag.time.max.ms — 30 seconds by default. Every broker reports the count through the UnderReplicatedPartitions JMX gauge, and any value above zero means the cluster is running with less redundancy than you configured.

URP is the most-watched health metric in Kafka operations for a simple reason: it measures how much failure margin remains before an outage becomes data loss. This guide covers what the metric means, the commands that find the failing broker, a fix for each root cause, and the alert thresholds that separate routine maintenance from a real incident.

Kafka backup monitoring starts with one fact: OSO Kafka Backup serves Prometheus text-format metrics on port 8080 while a backup or restore is running. Scrape that endpoint, watch lag, progress, throughput, and errors, and alert when the pipeline stalls — then add a durable success signal, because one-shot jobs take their metrics with them when they exit.

An unmonitored backup fails silently. Nobody notices the stalled consumer or the storage errors, and the failure is discovered months later, during a restore, when it can no longer be fixed. This guide covers the endpoint configuration, the metrics that matter, copy-paste PromQL, two alerts, and how scraping changes on Kubernetes.

A Kafka disaster recovery plan is only credible after it survives a test. This playbook turns the written plan into a scheduled program of drills that climbs four levels: backup validation, restore drills, failover simulation, and a full game day.

Most Kafka DR plans are documents nobody has executed. The first real execution happens during a real incident, with the clock running and half the team asleep. Testing moves that first execution to a Tuesday afternoon, where mistakes cost nothing.

Kafka active-active replication runs two or more clusters that all serve producers and consumers, with bidirectional replication keeping them converged. Every region writes locally, reads locally, and receives the other regions' data within seconds.

Kafka has no native multi-master mode, so active-active is an architecture you assemble — from a cross-cluster replication tool plus application-level discipline. Three problems define the assembly: replication loops, consumer offset divergence, and the complete absence of conflict resolution.

A Kafka active-passive DR architecture runs one production cluster and one standby that receives data continuously but serves no traffic until promotion. When the primary region fails, the team promotes the standby, redirects clients, and resumes processing.

Three decisions define the design: how the standby receives data, how clients switch to it, and how the team returns to the primary afterward. Get those three right and active-passive covers most Kafka workloads at a fraction of active-active cost.

Kafka RTO/RPO planning starts with two limits. RPO sets the maximum acceptable gap between an incident and the newest usable recovery point. RTO sets the maximum acceptable time until the workload serves traffic again.

Set both targets per workload, then prove them with an isolated restore drill. Backup frequency alone does not prove RPO. A fast restore command alone does not prove RTO.

A Kafka Schema Registry backup must protect registry state alongside the Kafka records that depend on it. Capture subjects, versions, IDs, references, compatibility settings, and modes through the supported Registry API. Restore that state before starting consumers against recovered records.

Backing up records alone can leave valid bytes that applications cannot decode. Treat Kafka data and Schema Registry as one recovery set with one tested recovery point.