Kafka Backup Compliance: What GDPR, SOX, and HIPAA Actually Require

Kafka backup compliance comes down to four obligations that appear, in different words, in almost every regulation: retain topic data for a defined period, restore it on demand, prove both with tamper-evident records, and erase personal data when the law requires it. No regulation names Kafka. Auditors simply treat the events in your topics as regulated records — and broker retention, replication factor, and disk durability satisfy none of the four obligations.

Most Kafka estates that fail an audit fail on process, not technology. Retention was set by disk budget instead of regulation, no copy exists outside the cluster, and nobody can produce evidence that a restore has ever worked. This guide maps GDPR, SOX, HIPAA, and PCI DSS to the specific backup controls that close those gaps.

Key takeaway

Regulations converge on four verbs: retain, restore, prove, erase. Broker retention does none of them. Scope backup policies to the regulated topics, store the copies immutably outside the cluster, and automate signed restore evidence.

Why broker retention doesn't satisfy compliance

Kafka's retention.ms and retention.bytes keep data inside the same failure domain that produced it. A deleted topic, a corrupting bug, or a misapplied config change destroys the "record" and its replicas together. Replication makes the cluster available; it also copies every bad write to all replicas within milliseconds.

Regulators expect something structurally different: an isolated copy, in separate storage, that can be restored and shown to restore. That is a backup, and it is the foundation the rest of this guide builds on. The Kafka backup best practices guide covers the general architecture; the disaster recovery use case covers the recovery side.

There is a second, quieter failure mode. Teams set topic retention to seven days for disk reasons, while the data in that topic carries a seven-year regulatory obligation. The retention schedule must come from the regulation, not from the storage budget.

What each regulation asks of Kafka backup compliance

The table below maps the major frameworks to Kafka topic data. Retention figures marked "as commonly applied" are established audit practice for principle-based rules rather than a number printed in the statute.

Regulation	What it asks of Kafka data	Retention anchor	Backup control that satisfies it
GDPR (Art. 5, 17, 32)	Restore availability after an incident; test measures regularly; erase on request; keep data no longer than needed	Purpose-limited — no fixed period	Tested restores with evidence; record-level erasure path; per-topic retention
SOX (§404 ITGC)	Integrity monitoring, restore testing, durable audit evidence	7 years (2,555 days), as commonly applied	Scheduled validation runs; checksummed evidence retained 7 years
HIPAA Security Rule	"Retrievable exact copies" of ePHI; documented procedures	6-year documentation retention	Isolated encrypted backups of PHI topics; retained policies and evidence
PCI DSS v4.0	Protect stored cardholder data; keep audit log history	12 months of logs, 3 months immediately available	Backed-up audit-log topics; masking so card data never reaches the bucket
CMMC Level 2 (RE.3.139)	Regularly performed and tested backups, documented	Per assessment scope	Validation checks mapped to the control in the evidence report
MiFID II	Reconstructible records of transactions and communications	5 years, up to 7 on request	Point-in-time restore of transaction topics

GDPR is the most demanding because it pulls in two directions. Article 32 requires "the ability to restore the availability and access to personal data in a timely manner" and regular testing of that ability — an untested backup is arguably non-compliant on its own. At the same time, Article 5's storage-limitation principle and Article 17's right to erasure mean you cannot keep personal data forever just because it sits in a backup. The erasure section below deals with that tension. The full text of Article 32 is short and worth reading.

SOX treats Kafka topics that feed financial reporting as part of IT General Controls. Auditors ask for restore-test results and expect the evidence trail itself to survive for seven years. SHA-256 checksummed reports are accepted as tamper-evident documentation.

HIPAA's Security Rule contains a required implementation specification — the data backup plan — that obliges covered entities to "create and maintain retrievable exact copies" of electronic protected health information. Documentation must be retained for six years. The HHS Security Rule summary is the authoritative reference.

PCI DSS works differently: its first instinct is that you should not be storing cardholder data at all beyond defined need. For Kafka, that usually means masking or tokenizing card fields before they reach a backup bucket, while the audit-log topics that prove access control carry the 12-month log-history requirement.

Design the retention schedule

Compliance retention is a per-topic property, not a cluster property. Inventory which topics carry regulated data — payment streams, audit logs, PHI — and give them a dedicated backup policy with its own schedule, bucket prefix, and lifecycle rules.

mode: backup
backup_id: "compliance-${DATE}"

source:
  topics:
    include:
      - financial-transactions
      - audit-log

backup:
  compression: zstd
  compression_level: 9  # Maximum compression for archival
  include_offset_headers: true
  source_cluster_id: "production"

storage:
  backend: s3
  bucket: compliance-archives
  prefix: kafka/daily

Zstd at level 9 trades CPU for the smallest archives, which matters when copies live for seven years. The config.yaml reference documents every field.

Long retention does not mean expensive retention. Age the copies through storage classes with a lifecycle policy:

{
  "Rules": [
    {
      "ID": "compliance-retention",
      "Status": "Enabled",
      "Transitions": [
        { "Days": 90, "StorageClass": "GLACIER" },
        { "Days": 365, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 2555 }
    }
  ]
}

A workable tiering baseline, by data type:

Data type	Hot (S3 Standard)	Warm (S3-IA)	Cold (Glacier)	Archive
Audit logs	30 days	90 days	1 year	7 years
Transactions	90 days	1 year	3 years	7 years
Personal data	30 days	—	On deletion	Per GDPR

Archive tiers are fine for the aged copies as long as your recovery objectives account for retrieval time. Glacier restores take minutes to hours, so keep recent backups in a class you can restore from within your RTO — the RTO/RPO planning guide shows how to set those numbers.

Make backups tamper-evident

An auditor's third question, after "do backups exist" and "do they restore", is "how do I know nobody edited them". Two controls answer it.

First, make the storage itself immutable. S3 Object Lock in COMPLIANCE mode prevents deletion or overwrite by anyone, including the root account, until the retention date passes:

aws s3api put-object-lock-configuration \
  --bucket compliance-backups \
  --object-lock-configuration '{
    "ObjectLockEnabled": "Enabled",
    "Rule": {
      "DefaultRetention": {
        "Mode": "COMPLIANCE",
        "Years": 7
      }
    }
  }'

Keep compliance backups in their own bucket — ideally their own account — so a compromise of the production account cannot reach the archives.

Second, make the restore evidence verifiable. OSO Kafka Backup's validation runs check a restored cluster against the backup manifest and emit JSON and PDF reports with SHA-256 checksums and detached ECDSA-P256-SHA256 signatures, mapped to SOX, CMMC, and GDPR controls. The compliance evidence documentation covers what the reports contain, and the backup verification guide covers how to structure the restore drills that generate them.

Encryption and access boundaries

Every framework in the table expects encryption at rest and in transit, and an access trail. All three are available with the MIT-licensed core because they lean on the storage provider.

At rest, set default encryption on the bucket with SSE-KMS and a dedicated key, then scope IAM so the backup writer can only put objects encrypted with that key. In transit, the source connection uses SASL_SSL and object-storage traffic is HTTPS:

source:
  security:
    security_protocol: SASL_SSL
    ssl_ca_location: /certs/ca.crt

storage:
  backend: s3
  bucket: compliance-backups
  # SSE-KMS is enforced by bucket policy; all S3 traffic is HTTPS

Give read access to a separate principal tagged for the compliance team, and turn on S3 server access logging so every retrieval is itself a record. The security setup guide walks through the certificates and credentials.

Product-level controls — separate backup-operator and restore-operator roles, immutable operation logs, and masking — are Enterprise features. For SOX and CMMC scopes without personal data, the storage-level controls above usually carry the audit on their own.

The GDPR erasure problem

Immutable seven-year backups and a right to erasure sound incompatible. In practice there are three honest ways to reconcile them, and the right one is a decision for your data-protection officer, not your platform team.

The first is expiry-based: scope personal data to backup sets with retention matched to the processing purpose, and let erasure be satisfied by scheduled deletion. The second is crypto-shredding: encrypt records per key scope and destroy the key to render the data unreadable. The third is record-level deletion — removing specific records from existing backups by key — which is what the Enterprise right-to-be-forgotten feature implements, so honoring a request does not mean destroying whole backup sets.

There is also a fourth option that avoids the problem: data masking at backup time, so regulated fields never reach the bucket. Field-level redaction removes one field while preserving the rest of the record. Both are Enterprise capabilities, and both shrink the erasure surface to zero for the masked fields.

The audit-readiness checklist

Work through this before the auditor asks. Each unchecked box is a finding waiting to happen.

• Regulated topics inventoried, with the applicable regulation named per topic
• Retention period per topic derived from the regulation, not the disk budget
• Backups isolated in a dedicated bucket or account, outside the cluster's failure domain
• Object Lock or equivalent immutability enabled on compliance buckets
• Encryption at rest (SSE-KMS) and in transit (TLS) enforced, not assumed
• IAM scoped separately for writers and readers; storage access logging on
• Restore validation scheduled, producing signed evidence reports
• Evidence reports retained for the longest applicable period (typically 7 years)
• Erasure path for personal data documented and agreed with legal
• A restore drill on the calendar — evidence ages, and auditors check dates

Turn compliance requirements into config

The Kafka backup compliance solution page pairs this guide with copy-ready configs: a backup policy scoped to regulated topics, a validation setup that generates signed evidence, and the schedule that runs both. Or start from zero with the getting started guide.

Frequently asked questions

Does Kafka topic retention satisfy compliance retention requirements?

No. Topic retention keeps data inside the cluster that produced it, so deletion, corruption, or cluster loss destroys the record and its replicas together. Regulators expect an isolated, restorable copy in separate storage with evidence that restores work.

How long should Kafka backups be kept for HIPAA?

HIPAA requires retrievable exact copies of electronic protected health information and six-year retention of Security Rule documentation, including policies and evidence. Retention for the PHI itself follows your documented policies, so most programs keep PHI topic backups at least six years unless legal guidance sets a different period.

Does the GDPR right to erasure apply to Kafka backups?

Yes, personal data in backups remains personal data. Programs reconcile erasure with retention through purpose-matched backup expiry, crypto-shredding of key-scoped records, or record-level deletion from backup sets. Which approach satisfies a given request is a data-protection decision, so agree the path with legal before the first request arrives.

Can compliance backups be stored in Glacier or archive tiers?

Yes. Aged compliance copies suit archive tiers because they are rarely restored and must exist for years. Keep recent backups in a storage class you can restore within your recovery time objective, since archive retrievals take minutes to hours.

Which Kafka topics need compliance backups?

The topics carrying regulated data: payment and transaction streams under SOX or PCI DSS, audit and access logs, and any topic holding personal data or PHI. Inventory these, name the applicable regulation per topic, and give them a dedicated backup policy instead of one blanket rule for the cluster.

Wrapping up

Every framework in this guide asks the same four things of Kafka data: retain it for a defined period, restore it on demand, prove both, and erase what the law says must go. Broker retention answers none of them; an isolated, immutable, validated backup answers all four. Scope the policies per topic, automate the evidence, and an audit becomes a file transfer instead of a fire drill.