46 posts tagged with "Kafka Backup"

Strimzi Kafka backup is the piece Strimzi deliberately leaves to you. Strimzi manages the lifecycle of Kafka on Kubernetes — brokers, topics, users, certificates — but it does not back up the message data inside your topics. Re-applying your Kafka and KafkaTopic resources rebuilds an empty cluster, not your data.

This guide maps what Strimzi protects and what it does not, explains why volume snapshots are not a substitute, and walks through the declarative alternative: a Strimzi-native operator that runs backups and restores as custom resources next to your Kafka CR.

The Kafka Connect S3 sink connector consumes Kafka topics and writes their records to Amazon S3 as partitioned object files in Avro, JSON, Parquet, or raw bytes. If you already run a Kafka Connect cluster, it is the standard way to archive topic data for analytics, long-term retention, and compliance records.

This guide covers how the connector works, a complete working configuration, the partitioners and formats that shape your archive, the exact conditions for exactly-once delivery, licensing, and the one boundary every team should plan around: an S3 sink archive has no first-class path back into Kafka.

Kafka tiered storage (KIP-405) splits each partition's log into two tiers: a local tier on broker disk and a remote tier in external storage such as S3. When a segment rolls, the broker copies it to remote storage and can delete the local copy once local retention expires — so how long you keep data stops being a question of how much broker disk you can afford.

The feature has been production-ready since Apache Kafka 3.9 and is transparent to producers and consumers. This guide covers how the two-tier architecture works, which versions and platforms support it, the exact configs to enable and disable it, the documented limitations, and the line between what tiered storage protects and what it does not.

Running Kafka on AWS means choosing between three deployment models: Amazon MSK provisioned, where AWS manages brokers you size yourself; MSK Serverless, where capacity follows throughput automatically; and self-managed Kafka on EC2, where everything is yours to run. MSK removes broker patching, metadata-quorum management, and hardware failure from your plate. It does not remove responsibility for the data inside the topics.

That second point surprises teams more than it should. AWS Backup does not support MSK as a resource type, there is no snapshot API for topic data, and multi-AZ replication faithfully copies deletions and corruption along with everything else. This guide walks the MSK architecture, the failure scenarios AWS absorbs, the ones it leaves to you, and the options for closing the gap.

Kafka backup monitoring starts with one fact: OSO Kafka Backup serves Prometheus text-format metrics on port 8080 while a backup or restore is running. Scrape that endpoint, watch lag, progress, throughput, and errors, and alert when the pipeline stalls — then add a durable success signal, because one-shot jobs take their metrics with them when they exit.

An unmonitored backup fails silently. Nobody notices the stalled consumer or the storage errors, and the failure is discovered months later, during a restore, when it can no longer be fixed. This guide covers the endpoint configuration, the metrics that matter, copy-paste PromQL, two alerts, and how scraping changes on Kubernetes.

A Kafka disaster recovery plan is only credible after it survives a test. This playbook turns the written plan into a scheduled program of drills that climbs four levels: backup validation, restore drills, failover simulation, and a full game day.

Most Kafka DR plans are documents nobody has executed. The first real execution happens during a real incident, with the clock running and half the team asleep. Testing moves that first execution to a Tuesday afternoon, where mistakes cost nothing.

Kafka active-active replication runs two or more clusters that all serve producers and consumers, with bidirectional replication keeping them converged. Every region writes locally, reads locally, and receives the other regions' data within seconds.

Kafka has no native multi-master mode, so active-active is an architecture you assemble — from a cross-cluster replication tool plus application-level discipline. Three problems define the assembly: replication loops, consumer offset divergence, and the complete absence of conflict resolution.

A Kafka active-passive DR architecture runs one production cluster and one standby that receives data continuously but serves no traffic until promotion. When the primary region fails, the team promotes the standby, redirects clients, and resumes processing.

Three decisions define the design: how the standby receives data, how clients switch to it, and how the team returns to the primary afterward. Get those three right and active-passive covers most Kafka workloads at a fraction of active-active cost.

Kafka RTO/RPO planning starts with two limits. RPO sets the maximum acceptable gap between an incident and the newest usable recovery point. RTO sets the maximum acceptable time until the workload serves traffic again.

Set both targets per workload, then prove them with an isolated restore drill. Backup frequency alone does not prove RPO. A fast restore command alone does not prove RTO.

A Kafka Schema Registry backup must protect registry state alongside the Kafka records that depend on it. Capture subjects, versions, IDs, references, compatibility settings, and modes through the supported Registry API. Restore that state before starting consumers against recovered records.

Backing up records alone can leave valid bytes that applications cannot decode. Treat Kafka data and Schema Registry as one recovery set with one tested recovery point.