Metrics Reference

OSO Kafka Backup v0.15.11 exposes Prometheus text-format metrics while a backup or restore is running.

Endpoint configuration

metrics:
  enabled: true
  port: 8080
  bind_address: "0.0.0.0"
  path: "/metrics"
  update_interval_ms: 500
  keep_alive_seconds: 60
  max_partition_labels: 100

Field	Default	Description
enabled	true	Start the HTTP metrics server
port	8080	Listener port
bind_address	0.0.0.0	Listener address
path	/metrics	Metrics path
update_interval_ms	500	Metrics recalculation interval
keep_alive_seconds	0	Continue serving after a one-shot operation completes
max_partition_labels	100	Maximum unique per-partition label sets; 0 is unlimited

Use keep_alive_seconds for short-lived Jobs so the final values live through at least one scrape. A practical minimum is twice your Prometheus scrape interval. Shutdown signals still stop the process promptly during this window.

Cardinality behavior

The partition limit applies to unique topic/partition/backup_id label sets used by kafka_backup_lag_records, kafka_backup_lag_bytes, and kafka_backup_lag_seconds.

• Repeated updates to an admitted partition do not consume more slots.
• Admitted partitions continue updating after the limit is reached.
• 0 disables the limit; use it only when the number of partitions is bounded.
• Aggregate lag and finite snapshot progress include all partitions, including those omitted by the per-partition limit.

Backup and progress metrics

Metric	Type	Labels	Description
kafka_backup_lag_records	Gauge	topic, partition, backup_id	Current records behind for a partition
kafka_backup_lag_bytes	Gauge	topic, partition, backup_id	Estimated bytes behind for a partition
kafka_backup_lag_seconds	Gauge	topic, partition, backup_id	Time lag for a partition
kafka_backup_lag_records_max	Gauge	backup_id	Maximum current partition lag
kafka_backup_lag_records_sum	Gauge	backup_id	Current lag summed across every partition
kafka_backup_snapshot_records_target	Gauge	backup_id	Records in the captured finite-backup range
kafka_backup_snapshot_records_remaining	Gauge	backup_id	Records left before the finite backup completes
kafka_backup_throughput_records_per_sec	Gauge	backup_id, topic	Current record throughput
kafka_backup_throughput_bytes_per_sec	Gauge	backup_id, topic	Current byte throughput
kafka_backup_records_total_total	Counter	backup_id	Records backed up
kafka_backup_bytes_total_total	Counter	backup_id	Uncompressed bytes backed up
kafka_backup_duration_seconds	Histogram	backup_id, status	Backup duration
kafka_backup_last_successful_commit	Gauge	backup_id, topic, partition	Unix timestamp of the last offset commit
kafka_backup_consumer_rebalance_total_total	Counter	backup_id	Consumer group rebalances

The doubled _total_total suffix is the emitted name for counters whose registered family name already ends in _total.

Storage, compression, and error metrics

Metric	Type	Labels	Description
kafka_backup_storage_write_latency_seconds	Histogram	backend, operation	Storage write latency
kafka_backup_storage_read_latency_seconds	Histogram	backend, operation	Storage read latency
kafka_backup_storage_write_bytes_total_total	Counter	backend, backup_id	Bytes written to storage
kafka_backup_storage_read_bytes_total_total	Counter	backend, backup_id	Bytes read from storage
kafka_backup_storage_errors_total_total	Counter	backend, error_type	Storage errors
kafka_backup_compression_ratio	Gauge	algorithm, backup_id	Uncompressed-to-compressed ratio
kafka_backup_compressed_bytes_total_total	Counter	algorithm, backup_id	Compressed bytes written
kafka_backup_uncompressed_bytes_total_total	Counter	algorithm, backup_id	Bytes before compression
kafka_backup_errors_total_total	Counter	backup_id, error_type	Operation errors
kafka_backup_retries_total_total	Counter	backup_id, operation	Retry attempts

Storage operations are labelled segment, manifest, checkpoint, or other. Error categories include broker_connection, consumer_timeout, deserialization, storage_io, codec, offset_invalid, auth, timeout, not_found, quota, and unknown.

Restore metrics

Metric	Type	Labels	Description
kafka_restore_duration_seconds	Histogram	backup_id, status	Restore duration
kafka_restore_progress_percent	Gauge	restore_id, backup_id	Completion percentage
kafka_restore_eta_seconds	Gauge	restore_id	Estimated seconds remaining
kafka_restore_throughput_records_per_sec	Gauge	restore_id	Current record throughput
kafka_restore_records_total_total	Counter	restore_id	Records restored

Useful PromQL

Finite snapshot completion percentage:

100 * (
  1 - kafka_backup_snapshot_records_remaining
      / clamp_min(kafka_backup_snapshot_records_target, 1)
)

Current total lag without a high-cardinality sum:

kafka_backup_lag_records_sum

Record throughput over five minutes:

rate(kafka_backup_records_total_total[5m])

99th-percentile storage write latency:

histogram_quantile(
  0.99,
  sum by (le, backend, operation) (
    rate(kafka_backup_storage_write_latency_seconds_bucket[5m])
  )
)

Errors during the last five minutes:

sum by (backup_id, error_type) (
  increase(kafka_backup_errors_total_total[5m])
)

Scrape configuration

scrape_configs:
  - job_name: kafka-backup
    static_configs:
      - targets: ["kafka-backup-host:8080"]
    metrics_path: /metrics
    scrape_interval: 30s

For Kubernetes Jobs managed by the Strimzi Backup Operator, use its Job PodMonitor configuration. A ServiceMonitor cannot discover a short-lived Job pod unless a matching Service is also created.