Kafka for Database Replication (CDC) — and What It Means for Backup
Kafka database replication almost never means what the words suggest. Searchers using that phrase are usually not asking about Kafka's own broker-to-broker replication — they're asking how to get changes out of a database and into Kafka, so downstream systems can react to them. That's change data capture (CDC), and Kafka Connect with a connector like Debezium is the standard way to do it.
This post untangles the terms, walks through the CDC architecture that "Kafka database replication" actually describes, and covers something that architecture's own documentation rarely does: what happens to those topics when you need to recover from a failure.
CDC topics are often the only replayable record of what changed in your database and when. Losing them doesn't mean replaying a Kafka log — it means re-snapshotting a production database. That makes them a backup priority, not an afterthought.
"Kafka database replication" is a vocabulary collision
Kafka has its own replication, and it has nothing to do with databases. A replication factor tells brokers how many copies of each partition to keep, and followers copy the partition leader's log byte-for-byte inside one cluster. Move that copying between two Kafka clusters and you get mirroring — the job MirrorMaker does — which is a different mechanism again, covered in Kafka mirroring vs replication. Neither one touches a database.
"Kafka database replication" is a third thing layered on top of that already-crowded vocabulary. It describes using Kafka as the transport layer for database change events: a connector reads changes from a source database and publishes them as Kafka records, so anything downstream — a data warehouse, a search index, another service's cache — can replicate that database's state without querying it directly. If you actually meant moving data between two Kafka clusters, Kafka-to-Kafka replication is the post you want instead.
How database-to-Kafka replication actually works
The standard architecture is change data capture: a connector reads a database's own transaction log and turns each row-level insert, update, or delete into a Kafka record. Debezium is the connector most teams reach for, running as a source connector inside Kafka Connect, but the pattern isn't specific to any one product.
| Approach | How it captures changes | Database load | Schema-change handling |
|---|---|---|---|
| Log-based CDC (Debezium and similar) | Reads the database's transaction log (binlog, WAL, redo log) | Low — no query load on top of normal writes | Detected from the log; can propagate automatically |
| Polling / query-based CDC | Periodically queries for rows changed since the last poll | Higher — adds recurring query load | Requires a reliable "last modified" column or timestamp |
| Outbox pattern | Application writes events to an outbox table in the same transaction as its business data; CDC captures that table | Low, plus one extra write per transaction | Explicit — the outbox schema is the contract, not the whole database schema |
Log-based CDC wins on latency and database load because it never queries the tables it's watching — it tails a log the database is already writing. The trade-off is that connectors need enough access to read that log (binlog access for MySQL, logical replication for PostgreSQL, and so on), and every table's full schema, including columns no downstream consumer needs, becomes visible in the resulting events.
The outbox pattern solves that visibility problem, and a different one: without it, an application that needs to write to its database and publish an event has to do both and hope neither one fails alone — the classic dual-write problem. With an outbox table, the application writes its event as one more row in the same database transaction as the business change. Debezium then captures just that table, and its documented Outbox Event Router transform reshapes each row into the event topic a downstream consumer expects. The application chooses exactly what event contract it exposes, instead of every column in every source table being fair game.
Where the change events end up
A log-based connector typically produces one Kafka topic per source table, named by convention as <prefix>.<schema>.<table> — a connector configured with prefix inventory watching PostgreSQL's public.customers table publishes to inventory.public.customers. Each record carries the row's state, so a downstream consumer can rebuild what changed without going back to the database.
The connector's own position in the source log lives somewhere else entirely: Kafka Connect's internal offset-storage topic, connect-offsets by default in distributed mode. For a source connector like Debezium, what gets stored there isn't a Kafka consumer-group offset — there's no Kafka topic being consumed on the source side — it's a database-specific position: a binlog filename and byte offset (or a GTID) for MySQL, a logical-replication LSN for PostgreSQL. That's a different topic, and a different kind of "offset," from the sink-side consumer group offsets covered in Kafka consumer offsets backup. Losing connect-offsets doesn't lose data, but it does force the connector to restart from a stale position or take a fresh snapshot, which is its own kind of expensive recovery.
Kafka Connect also keeps connect-configs and connect-status, tracking connector configuration and task health. All three are worker bookkeeping. None of them are the CDC data itself — that lives in the per-table topics the connector produces, and treating "the Connect topics" as one disposable bucket is a mistake specific to source connectors: a sink connector's consumer offsets are usually cheap to rebuild by reprocessing input from earliest, but a source connector's offset determines how far back in the database's own log it can resume from, and that log doesn't keep everything forever.
Why CDC topics deserve their own backup plan
Ordinary Kafka backup guidance often treats certain topics as safely skippable because they're regenerable — Kafka Streams repartition topics, for instance, rebuild automatically from input topics that are still sitting in Kafka. CDC topics don't get that shortcut. Regenerating a lost CDC topic means re-snapshotting the source database, and a snapshot re-reads production tables rather than replaying Kafka records, which means real load on a live database, potential locking depending on the connector and database engine, and a fresh copy that reflects the database's current state rather than the change history you actually lost.
That makes the case for backing these topics up like any other Kafka data that matters:
- Per-table CDC topics are usually the priority — they're the only durable, replayable record of what changed and when, especially once the source database's own log has rotated past that point.
connect-offsetsmatters if a fast, in-place connector resume is part of your recovery plan; restoring it alongside the data topics avoids forcing a resnapshot after every recovery.connect-configsandconnect-statusare usually low-priority for backup — they're reconstructable from the connector configuration you already manage as code, not from data that only exists inside Kafka.
The general mechanics are the same as any Kafka backup: scope topics.include to the CDC topics and Connect bookkeeping topics you've decided matter, using the same include/exclude patterns documented in the configuration reference. Kafka Backup for Amazon MSK Connect walks through the equivalent setup when your connectors run on managed Kafka Connect.
The operational trap: an idle connector can hurt the source database
There's a failure mode here that has nothing to do with Kafka's own retention settings. PostgreSQL's logical replication depends on a replication slot, and that slot tells the database how much write-ahead log (WAL) it still needs to keep around for whichever consumer is reading it. If the CDC connector stalls, crashes, or is simply misconfigured and stops consuming, the slot doesn't expire on its own — PostgreSQL keeps retaining WAL for it, and without a safeguard like max_slot_wal_keep_size configured, that retained WAL can grow until it consumes disk on the production database itself. Other databases manage their own CDC-source logs differently — MySQL binlog retention and Oracle redo retention are governed by their own separate settings — but the underlying risk is the same shape: a stalled downstream consumer can turn into an upstream storage problem you didn't budget for.
Monitoring the connector's lag, and treating its Kafka-side topics as backed-up data rather than a disposable cache, both reduce how often that trap gets triggered — a healthy connector doesn't let the slot fall behind, and a backed-up topic set means a connector restart doesn't have to fall back on the source database at all. The disaster recovery use cases cover the broader set of failure modes a backup plan needs to account for beyond this one.
When you actually want this pattern
Reach for database-to-Kafka CDC when you need a downstream system — a data warehouse, a search index, a cache, another microservice — to stay current with a database without querying it directly or accepting the dual-write problem. It is not a substitute for Kafka's own replication factor, and it is not the same job as mirroring topics between two Kafka clusters; if either of those was what you actually needed, How Kafka Replication Works and Kafka-to-Kafka Replication cover them directly. And regardless of which kind of "replication" put data into your Kafka cluster, none of it is a backup — replication and CDC both move data forward, none of them give you a point-in-time copy to restore from. Getting started with OSO Kafka Backup covers building that plan for the topics these pipelines create.
Frequently asked questions
Does "Kafka database replication" mean the same thing as Kafka replication factor?
No. Replication factor controls how many copies of a partition Kafka keeps inside one cluster. "Kafka database replication" almost always refers to change data capture (CDC) — using a connector like Debezium on Kafka Connect to publish database changes as Kafka records, which is a completely separate mechanism from broker replication.
Do CDC topics from Debezium need to be backed up like other Kafka topics?
Yes, and often with higher priority. A per-table CDC topic is frequently the only replayable record of a database's change history once the source database's own transaction log has rotated past that point. Losing it without a backup means re-snapshotting the production database instead of replaying Kafka data.
What happens if I lose a Debezium connector's offset topic?
You don't lose data, but you lose the connector's resume position. Kafka Connect stores source-connector offsets in an internal topic (connect-offsets by default) — for Debezium, that's a database-specific position like a binlog offset or LSN, not a Kafka consumer-group offset. Without it, the connector typically has to fall back to a fresh database snapshot to resume safely.
What is the outbox pattern, and how does it relate to CDC?
The outbox pattern has an application write its events to a dedicated outbox table in the same transaction as its business data, avoiding the dual-write problem of writing to a database and a message broker separately. Debezium then captures just that table via log-based CDC, and its Outbox Event Router transform reshapes each row into the event topic downstream consumers expect.
Can an idle CDC connector cause problems on the source database?
Yes, at least for PostgreSQL logical replication. A replication slot tells the database how much write-ahead log to retain for its consumer. If the connector stops consuming and no safeguard like max_slot_wal_keep_size is configured, retained WAL can grow and consume disk on the production database — a downstream failure turning into an upstream storage problem.