Skip to main content

Example: SOX Compliance Evidence

This example walks through a complete SOX Section 404 (IT General Controls) compliance scenario — from backup to signed evidence report.

Scenario​

You're a platform engineer at a financial services company. Your SOX auditor requires:

  • Weekly proof that Kafka backups containing financial transaction data can be restored
  • SHA-256 checksums as tamper-evident documentation
  • Evidence retained for 7 years in write-once storage
  • A machine-readable report they can feed into their GRC platform

Step 1: Set Up the Backup​

sox-backup.yaml
mode: backup
backup_id: "financial-data-weekly"

source:
  bootstrap_servers:
    - kafka-prod-0.internal:9092
    - kafka-prod-1.internal:9092
    - kafka-prod-2.internal:9092
  security:
    security_protocol: SASL_SSL
    sasl_mechanism: SCRAM-SHA-512
    sasl_username: backup-service
    sasl_password: "${KAFKA_BACKUP_PASSWORD}"
  topics:
    include:
      - transactions
      - settlements
      - audit-log
      - "ledger-*"

storage:
  backend: s3
  bucket: sox-compliance-backups
  region: us-east-1
  prefix: production/weekly

backup:
  compression: zstd
  compression_level: 9          # Maximum compression for archival
  include_offset_headers: true  # Required for offset restoration
  source_cluster_id: "prod-us-east-1"
  stop_at_current_offsets: true # Snapshot mode

Run the backup:

$ kafka-backup backup --config sox-backup.yaml

Step 2: Restore to a Validation Cluster​

sox-restore.yaml
mode: restore
backup_id: "financial-data-weekly"

target:
  bootstrap_servers:
    - validation-kafka:9092

storage:
  backend: s3
  bucket: sox-compliance-backups
  region: us-east-1
  prefix: production/weekly

restore:
  create_topics: true
  topic_mapping:
    transactions: validation-transactions
    settlements: validation-settlements
    audit-log: validation-audit-log
$ kafka-backup restore --config sox-restore.yaml

Step 3: Generate Signing Keys​

# One-time setup — store the private key securely
$ openssl ecparam -genkey -name prime256v1 -noout | \
    openssl pkcs8 -topk8 -nocrypt -out /etc/kafka-backup/sox-signing-key.pem

$ openssl ec -in /etc/kafka-backup/sox-signing-key.pem \
    -pubout -out /etc/kafka-backup/sox-signing-key-pub.pem

# Restrict permissions
$ chmod 600 /etc/kafka-backup/sox-signing-key.pem

Step 4: Run Validation with Evidence Generation​

sox-validation.yaml
backup_id: "financial-data-weekly"

storage:
  backend: s3
  bucket: sox-compliance-backups
  region: us-east-1
  prefix: production/weekly

target:
  bootstrap_servers:
    - validation-kafka:9092

checks:
  message_count:
    enabled: true
    mode: exact
    topics:
      - transactions
      - settlements
      - audit-log
    fail_threshold: 0
  offset_range:
    enabled: true
  consumer_group_offsets:
    enabled: false  # Not restoring consumer groups in this scenario

evidence:
  formats: [json, pdf]
  signing:
    enabled: true
    private_key_path: "/etc/kafka-backup/sox-signing-key.pem"
  storage:
    prefix: "evidence-reports/sox/"
    retention_days: 2555  # 7 years (SOX requirement)

notifications:
  slack:
    webhook_url: "${SLACK_SOX_CHANNEL_WEBHOOK}"

triggered_by: "weekly-sox-validation-cron"
$ kafka-backup validation run --config sox-validation.yaml

Expected output:

=== Validation Results ===
Overall: PASSED
Checks: 2/2 passed, 0 failed, 0 skipped
Duration: 45ms

  [PASSED] MessageCountCheck — 3 topics; 2,841,293 messages expected, 2,841,293 restored; 0 discrepancies
  [PASSED] OffsetRangeCheck — 9 partitions checked; 9 passed; 0 issues

JSON evidence uploaded: evidence-reports/sox/validation-a1b2c3d4/2026/04/validation-a1b2c3d4.json
PDF evidence uploaded: evidence-reports/sox/validation-a1b2c3d4/2026/04/validation-a1b2c3d4.pdf
Signature uploaded: evidence-reports/sox/validation-a1b2c3d4/2026/04/validation-a1b2c3d4.sig

Step 5: Schedule Weekly Runs​

/etc/cron.d/sox-validation
# Run every Sunday at 02:00 UTC
0 2 * * 0 kafka-backup validation run --config /etc/kafka-backup/sox-validation.yaml

After 52 weeks, your auditor has a URL to a year of uninterrupted weekly evidence — no manual intervention required.

Step 6: Verify Evidence (Auditor Workflow)​

Your auditor downloads the evidence and verifies independently:

# Download the report and signature
$ kafka-backup validation evidence-get \
    --path s3://sox-compliance-backups \
    --report-id validation-a1b2c3d4 \
    --format json --output report.json

# Verify integrity
$ kafka-backup validation evidence-verify \
    --report report.json \
    --signature report.sig \
    --public-key sox-signing-key-pub.pem
SHA-256 checksum: VALID
ECDSA signature: VALID
Evidence report integrity: VERIFIED

What the Auditor Sees​

The JSON evidence report contains a compliance_mappings.sox_itgc section:

{
  "sox_itgc": {
    "control": "IT General Controls - Backup and Recovery",
    "satisfied_by": ["MessageCountCheck", "OffsetRangeCheck"],
    "evidence_retention_required_years": 7,
    "evidence_retention_configured_days": 2555
  }
}

The PDF report contains:

  • Page 1: Overall result badge (PASSED), report ID, generation timestamp
  • Page 2: Per-check results table with record counts and durations
  • Page 3: SHA-256 checksums, signature details, SOX ITGC control mapping with retention confirmation

S3 Object Lock (Optional)​

For WORM compliance (immutable evidence):

$ aws s3api put-object-lock-configuration \
    --bucket sox-compliance-backups \
    --object-lock-configuration '{
      "ObjectLockEnabled": "Enabled",
      "Rule": {
        "DefaultRetention": {
          "Mode": "COMPLIANCE",
          "Years": 7
        }
      }
    }'

Next Steps​