Installation

Install the OSO Kafka Backup Operator in your Kubernetes cluster.

Prerequisites

• Kubernetes 1.20+
• Helm 3.0+
• kubectl configured to access your cluster
• Cluster admin permissions

Helm Installation

Add Helm Repository

helm repo add oso https://charts.oso.sh
helm repo update

Install Operator

# Create namespace
kubectl create namespace kafka-backup

# Install with default values
helm install kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup

Verify Installation

# Check operator pod
kubectl get pods -n kafka-backup

# Check CRDs installed
kubectl get crds | grep kafka.oso.sh

# Expected output:
# kafkabackups.kafka.oso.sh
# kafkarestores.kafka.oso.sh
# kafkaoffsetresets.kafka.oso.sh
# kafkaoffsetrollbacks.kafka.oso.sh

Installation Options

Custom Values

helm install kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup \
  --set replicaCount=2 \
  --set resources.requests.memory=256Mi \
  --set metrics.enabled=true

Values File

values.yaml

replicaCount: 2

image:
  repository: osodevops/kafka-backup-operator
  tag: "1.0.0"
  pullPolicy: IfNotPresent

resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

metrics:
  enabled: true
  serviceMonitor:
    enabled: true

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 1000

helm install kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup \
  --values values.yaml

Install Specific Version

helm install kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup \
  --version 0.1.0

CRD Installation

Default (with Helm)

CRDs are installed automatically by Helm.

CRDs Only

If you need to install CRDs separately (e.g., for GitOps):

# Install CRDs only
kubectl apply -f https://raw.githubusercontent.com/osodevops/kafka-backup-operator/main/deploy/crds/

Keep CRDs on Uninstall

By default, CRDs are deleted when uninstalling. To keep them:

crds:
  install: true
  keep: true  # Don't delete on uninstall

RBAC Configuration

Default Service Account

The operator creates a service account with required permissions:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kafka-backup-operator
  namespace: kafka-backup

Required Permissions

The operator needs these permissions:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kafka-backup-operator
rules:
  # Manage CRDs
  - apiGroups: ["kafka.oso.sh"]
    resources: ["kafkabackups", "kafkarestores", "kafkaoffsetresets", "kafkaoffsetrollbacks"]
    verbs: ["*"]
  - apiGroups: ["kafka.oso.sh"]
    resources: ["kafkabackups/status", "kafkarestores/status", "kafkaoffsetresets/status", "kafkaoffsetrollbacks/status"]
    verbs: ["get", "patch", "update"]

  # Create jobs for backups
  - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["*"]

  # Read secrets
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "watch"]

  # Manage pods (for job pods)
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

  # Events
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "patch"]

Custom Service Account

serviceAccount:
  create: true
  name: "my-kafka-backup-sa"
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/kafka-backup-role

Cloud Provider Setup

AWS (EKS)

IAM Role for Service Account (IRSA)

# Create IAM policy
aws iam create-policy \
  --policy-name KafkaBackupPolicy \
  --policy-document file://policy.json

# Create IAM role with OIDC
eksctl create iamserviceaccount \
  --name kafka-backup-operator \
  --namespace kafka-backup \
  --cluster my-cluster \
  --attach-policy-arn arn:aws:iam::123456789:policy/KafkaBackupPolicy \
  --approve

policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::kafka-backups",
        "arn:aws:s3:::kafka-backups/*"
      ]
    }
  ]
}

Helm Values for IRSA

serviceAccount:
  create: false
  name: kafka-backup-operator
  # Service account created by eksctl

Azure (AKS)

Workload Identity

# Enable workload identity on AKS
az aks update \
  --resource-group myResourceGroup \
  --name myAKSCluster \
  --enable-oidc-issuer \
  --enable-workload-identity

# Create managed identity
az identity create \
  --name kafka-backup-identity \
  --resource-group myResourceGroup

# Get identity client ID
CLIENT_ID=$(az identity show \
  --name kafka-backup-identity \
  --resource-group myResourceGroup \
  --query clientId -o tsv)

# Assign Storage Blob Data Contributor role
az role assignment create \
  --assignee $CLIENT_ID \
  --role "Storage Blob Data Contributor" \
  --scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<account>

# Create federated credential
az identity federated-credential create \
  --name kafka-backup-federated \
  --identity-name kafka-backup-identity \
  --resource-group myResourceGroup \
  --issuer $(az aks show --name myAKSCluster --resource-group myResourceGroup --query oidcIssuerProfile.issuerUrl -o tsv) \
  --subject system:serviceaccount:kafka-backup:kafka-backup-operator \
  --audience api://AzureADTokenExchange

Azure Key Vault CSI Secrets Store Driver

Install the CSI Secrets Store Driver for Azure Key Vault integration:

# Add the Helm repo
helm repo add csi-secrets-store-provider-azure \
  https://azure.github.io/secrets-store-csi-driver-provider-azure/charts
helm repo update

# Install the Azure provider
helm install csi-secrets-store-provider-azure \
  csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \
  --namespace kube-system

Grant the managed identity access to Key Vault:

# Get the managed identity principal ID
PRINCIPAL_ID=$(az identity show \
  --name kafka-backup-identity \
  --resource-group myResourceGroup \
  --query principalId -o tsv)

# Grant secret access to Key Vault
az keyvault set-policy \
  --name <key-vault-name> \
  --object-id $PRINCIPAL_ID \
  --secret-permissions get list

Helm Values for Azure

serviceAccount:
  create: true
  name: kafka-backup-operator
  annotations:
    azure.workload.identity/client-id: <managed-identity-client-id>
  labels:
    azure.workload.identity/use: "true"

# For Key Vault integration
deployment:
  tenantId: <azure-tenant-id>
  workloadIdentityClientId: <managed-identity-client-id>
  syncSecrets:
    keyVaultName: <key-vault-name>
    envSecrets:
      kafka-sasl-username: KAFKA_SASL_USERNAME
      kafka-sasl-password: KAFKA_SASL_PASSWORD

GCP (GKE)

Workload Identity

# Enable workload identity on GKE
gcloud container clusters update my-cluster \
  --workload-pool=PROJECT_ID.svc.id.goog

# Create GCP service account
gcloud iam service-accounts create kafka-backup-sa

# Bind to Kubernetes service account
gcloud iam service-accounts add-iam-policy-binding \
  kafka-backup-sa@PROJECT_ID.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:PROJECT_ID.svc.id.goog[kafka-backup/kafka-backup-operator]"

High Availability

Multiple Replicas

replicaCount: 2

# Leader election is enabled by default
leaderElection:
  enabled: true
  leaseDuration: 15s
  renewDeadline: 10s
  retryPeriod: 2s

Pod Disruption Budget

podDisruptionBudget:
  enabled: true
  minAvailable: 1

Affinity Rules

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: kafka-backup-operator
          topologyKey: kubernetes.io/hostname

Upgrade

Helm Upgrade

# Update repo
helm repo update

# Upgrade operator
helm upgrade kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup \
  --values values.yaml

CRD Updates

CRDs are updated automatically during Helm upgrade. To update CRDs manually:

kubectl apply -f https://raw.githubusercontent.com/osodevops/kafka-backup-operator/main/deploy/crds/

Uninstall

Helm Uninstall

helm uninstall kafka-backup-operator --namespace kafka-backup

Clean Up CRDs

If CRDs were not kept:

kubectl delete crds kafkabackups.kafka.oso.sh
kubectl delete crds kafkarestores.kafka.oso.sh
kubectl delete crds kafkaoffsetresets.kafka.oso.sh
kubectl delete crds kafkaoffsetrollbacks.kafka.oso.sh

Clean Up Namespace

kubectl delete namespace kafka-backup

Troubleshooting Installation

Operator Not Starting

# Check pod logs
kubectl logs -n kafka-backup deployment/kafka-backup-operator

# Check events
kubectl get events -n kafka-backup --sort-by='.lastTimestamp'

CRDs Not Found

# Verify CRDs exist
kubectl get crds | grep kafka

# Reinstall CRDs
helm upgrade kafka-backup-operator oso/kafka-backup-operator \
  --namespace kafka-backup \
  --set crds.install=true

Permission Denied

# Check RBAC
kubectl auth can-i create kafkabackups --as=system:serviceaccount:kafka-backup:kafka-backup-operator

# Check cluster role binding
kubectl get clusterrolebinding | grep kafka-backup

Next Steps

• Configuration - Configure operator settings
• Secrets Guide - Set up credentials
• KafkaBackup CRD - Create your first backup