Kafka on AWS: EC2, MSK, MSK Serverless or Confluent Cloud?

Running Kafka on AWS means choosing who operates the brokers. Self-managed EC2 leaves the install, the quorum, and the disks with your team. Amazon MSK provisioned and MSK Serverless leave broker operations with AWS. Confluent Cloud on AWS leaves them with Confluent.

Topic data stays your responsibility on every option. A live replica is not a restore point. Multi-AZ replication, MirrorMaker, and MSK Replicator copy deletes and bad writes with the good records. AWS Backup's supported services do not include MSK.

Quick Answer

Amazon MSK provisioned is the usual default: AWS runs the brokers and you size them. Choose MSK Serverless for spiky traffic inside its quotas, or EC2 when you must pin the Kafka version. Choose Confluent Cloud when you want Confluent to operate the cluster on AWS. None of the four keeps a point-in-time copy of topic data.

Should you run Kafka on EC2 or MSK?

Self-managed EC2 is the right home when you must match a release, a plugin, or a broker setting. Managed catalogs for Apache Kafka on AWS do not offer every combination.

You pick the version. You run the KRaft controllers, or ZooKeeper on an older release. You own patching and disk replacement. That control is the reason to take the operational load.

Amazon MSK is the right home when you want AWS-hosted Kafka inside your own account. A provisioned cluster lets you pick a supported version, a broker type, and a broker count. AWS patches the fleet, replaces failed brokers, and includes the metadata quorum at no extra charge. Metadata management describes that KRaft quorum.

Use the table when the choice is still open. The Amazon MSK pricing guide walks each meter in detail.

The cells below name the pricing model. They do not quote a rate. Read the linked pricing page for the Region you will run. The MSK meters match the Amazon MSK pricing page as of 3 November 2026.

	Self-managed on EC2	MSK provisioned	MSK Serverless	Confluent Cloud on AWS
Who operates brokers	You	AWS	AWS	Confluent
Scaling model	You add brokers and disks	You add brokers. Standard storage is provisioned. Express storage grows with use.	AWS scales with throughput, inside per-cluster quotas	Dedicated clusters scale by adding CKUs. Other types scale on eCKU usage.
Pricing model	EC2 instance time, EBS storage, and data transfer. EC2 pricing. EBS pricing.	Broker hours plus storage. Standard bills the GB you provision. Express also bills data written and storage you use. MSK pricing.	Cluster hours, partition hours, storage consumed, and data in and out. MSK pricing.	CKU or eCKU hours, plus ingress, egress, and storage. Confluent pricing.
KRaft and version	Any Kafka release you install. You run the quorum.	You pick an MSK-supported version. 3.9.x is the recommended release.	AWS sets broker configs. You can edit a short topic list only.	You pick a cluster type. Confluent operates the brokers.
Tiered storage	KIP-405 only if the release you install supports it	Standard brokers, on 2.8.2.tiered or 3.6.0 and later. Express storage is fully managed and separate.	Not offered. Tiered storage applies to provisioned clusters.	Freight writes to object storage as its primary store. That store is not a second copy.
Private connectivity	Your VPC, subnets, and security groups	Brokers in your VPC. Public access can be added later. Multi-VPC access uses PrivateLink.	PrivateLink. IAM is required on every connection.	Standard clusters use public endpoints. Enterprise and Dedicated can use PrivateLink.
Data protection	Replication and EBS snapshots do not restore a topic to an earlier timestamp	Multi-AZ copies and MSK Replicator follow the log, including deletes. AWS Backup does not list MSK.	Same gap. There is no topic snapshot.	Cluster Linking follows the log. A deleted topic cannot be restored from the service.

EC2 wins when the managed version list blocks you. An EBS snapshot is a disk image taken at one moment. It is not a topic restore that puts consumer groups back on the right offsets. The EBS snapshots comparison draws that line.

MSK provisioned is the usual production choice for steady traffic. Standard brokers use EBS you size, up to 16,384 GiB per broker, per the MSK quota page.

Express brokers replace that disk with pay-as-you-go storage. You cannot change a live cluster from Standard to Express. Create a new cluster of the other type. Broker types state that limit.

MSK Serverless drops the broker count. It fits traffic that spikes and still stays inside the published quotas. Those limits are in the next section.

Confluent Cloud fits teams that want Confluent to run the cluster, including on AWS. Cluster types run from Basic through Dedicated, plus Freight. Private networking and Cluster Linking come with the types that support them. The backup gap is still yours.

The MSK pricing page also says you are not charged for replication traffic between brokers, or between metadata nodes and brokers. On EC2, traffic between your instances follows EC2 data transfer pricing. That difference belongs in the cost model even though this table does not quote either rate.

Which Kafka version can you run?

On EC2 you install the Apache Kafka release you need, including a current KRaft build. You also run the controller quorum yourself.

On MSK provisioned, you choose from the supported versions. The page marks 3.9.x as recommended. It is the last version that still offers both ZooKeeper and KRaft.

The same page lists 3.6.0 support as ending on 1 June 2026, and 3.7.x support as ending on 1 September 2026. Kafka 4.0 on MSK deprecates ZooKeeper metadata management. Version 4.2.x is available on Express brokers only.

KRaft mode on MSK starts at 3.7.x. Controllers are included at no extra charge, and you do not operate them.

ZooKeeper to KRaft migration is an in-place update once the cluster is on 3.9.x in ZooKeeper mode. New clusters can be created in KRaft directly.

Moving the data onto a new KRaft cluster is a separate path. The MSK KRaft migration runbook uses kafka-backup Enterprise for that cutover.

MSK Serverless does not give you a version picker. Serverless configuration says AWS sets broker properties. You may change only the listed topic settings.

When does MSK Serverless make sense?

MSK Serverless is a strong fit when you do not want to size brokers at all. AWS provisions capacity and moves partitions as throughput changes. You pay for use, not for idle broker hours.

The Serverless guide describes that model. It also lists the Regions where the cluster type exists. It is not in every Region.

The same guide requires IAM access control on every cluster. Kafka ACLs are not supported. Clients reach the cluster through AWS PrivateLink. The service integrates with AWS Glue Schema Registry, Managed Service for Apache Flink, and Lambda.

Stay inside the per-cluster quotas or pick provisioned instead. Ingress is capped at 200 MBps and egress at 400 MBps. A cluster allows 3,000 client connections and messages up to 8 MiB.

Non-compacted topics can have 2,400 partitions. Compacted topics can have 120. Consumer groups stop at 500. You can attach at most five client VPCs.

Some of those quotas can be raised with a support case. The page says which ones.

Topic settings are a short list. retention.ms defaults to seven days and retention.bytes defaults to 250 GiB. Both can be set to unlimited.

cleanup.policy can be set only when the topic is created. Broker configs are closed.

Tiered storage is not part of Serverless. The feature page says it applies only to provisioned clusters. Long retention on Serverless means keeping bytes in the serverless store and paying for that storage. It does not give you a copy outside the cluster.

Choose provisioned, or EC2, when you need SASL/SCRAM, mutual TLS, a pinned Kafka version, or tiered storage. The backup example later in this guide uses SASL/SCRAM, which Serverless does not offer.

When do Express brokers fit?

Express brokers are still MSK provisioned. AWS positions them for high throughput with less storage work. Storage is elastic and pay-as-you-go. You do not pick a disk size.

The Express brokers page says each broker can deliver up to 3x the throughput of a Standard broker. AWS also says they scale up to 20x faster and recover 90% quicker. Those figures are AWS's comparison, and they assume the instance sizes in that page.

Express clusters run in three Availability Zones only. Supported Kafka versions on Express are 3.6, 3.8, 3.9, and 4.2. KRaft on Express starts at 3.9. Version 4.2.x on MSK is Express-only today.

Express brokers do not yet fully support the Kafka Streams API. KIP-932 queues are not supported either.

Standard brokers remain the flexible choice. They keep EBS, tiered storage, and a wider config surface. Tiered storage needs cluster version 3.6.0 or higher, or 2.8.2.tiered.

It does not support compacted topics, and the low-cost tier has a three-day minimum retention. The tiered storage comparison explains why that remote tier is retention economics, not a backup.

Express pricing, on the MSK pricing page, is three meters. You pay broker instance hours, the storage you use, and a per-GB charge for data written. Standard pricing is broker hours plus the storage you provision, with provisioned throughput as a further meter. Neither meter is a backup product.

When does Confluent Cloud on AWS fit?

Confluent Cloud is the right pick when you want Confluent to operate Kafka and you are willing to leave the AWS MSK console. Every cluster type includes managed replica placement, encryption at rest, TLS in transit, Kafka ACLs, and managed connectors. Cluster types split the rest.

Basic is for development. Standard is for production on public networking. Enterprise is for production that needs private networking. Dedicated is for workloads that need reserved capacity and manual scale.

Freight is for high volume that can accept latency on the order of seconds. Freight writes directly to object storage, such as Amazon S3, instead of local broker disks. That write path is the cluster's store. It is not an independent backup.

Basic, Standard, Enterprise, and Freight scale elastically in eCKUs. You pay for the capacity used in an hour, with a minimum when the cluster is in use, and nothing at zero consumption. Dedicated clusters scale by the CKU count you set, and you pay for that count even when idle.

Billing dimensions define both units. Current rates are on Confluent pricing.

On AWS, Enterprise clusters that use PrivateLink are limited to 10 eCKU. Private Network Interface networking is what raises that ceiling to 32 eCKU. Dedicated clusters on AWS can be much larger. The cluster-types page lists the CKU ceiling.

Standard clusters are the public-endpoint option. If you turn on PrivateLink, the cluster has no public endpoints. AWS PrivateLink for Confluent Cloud covers inbound access for Dedicated and Enterprise.

Cluster Linking is Confluent's replication tool, and it is good at that job. It keeps a mirror topic for failover across regions or clusters.

Disaster recovery on Confluent Cloud is explicit about the limit. Recovery point is the mirroring lag. Synchronous replication, which would be required for a zero recovery point, is not offered. A delete on the source is a delete in the log the mirror is copying.

Topic management says a deleted topic cannot be restored. On 2 November 2026 the Confluent Cloud release notes added a full-record backup mode to the managed Google Cloud Storage sink and source connectors.

That mode can write record envelopes to GCS and restore them later. It is a connector you configure. It is not a cluster snapshot, and the note names GCS rather than Amazon S3.

How do you connect privately to Kafka on AWS?

Provisioned MSK places brokers in subnets you choose, inside your VPC, across Availability Zones. Clients in that VPC use the bootstrap string and a security group. That is the simplest private path, and it is the one most backup clients should use.

Public access is available on provisioned clusters running Kafka 2.6.0 or later. You cannot turn it on while you create the cluster. The subnets must be public, in-cluster encryption must be on, and plaintext client traffic must be off.

At least one of IAM, SASL/SCRAM, or mutual TLS must be on. Public access lists every condition.

Other VPCs and other accounts in the same Region can use multi-VPC private connectivity. It is PrivateLink, managed by MSK. It needs Kafka 2.7.1 or higher, and IAM, mutual TLS, or SASL/SCRAM.

Unauthenticated clusters cannot use it. It does not cross Regions, and it does not support t3.small. The pricing page bills it per authentication scheme per hour, plus a per-GB data-processing charge, on top of standard PrivateLink charges.

MSK Serverless is PrivateLink-only. IAM is the only client auth. The Amazon MSK integration documents the provisioned listeners most backup clients use: SASL/SCRAM on port 9096 and mutual TLS on port 9094.

The configuration reference lists SCRAM-SHA-256, SCRAM-SHA-512, and TLS. It does not list MSK IAM as a SASL mechanism. Plan on SCRAM or mutual TLS for a provisioned cluster you intend to back up with kafka-backup.

Confluent Cloud on AWS can stay public, on a Standard cluster, or private. Enterprise and Dedicated support AWS PrivateLink. Match the client network to the cluster type before you promise a VPC-only path.

How do you back up Kafka on AWS?

AWS keeps the brokers healthy on MSK. Confluent keeps them healthy on Confluent Cloud. You keep the ability to return to an earlier topic state.

Replication inside a cluster protects you from a dead broker or a lost Availability Zone. It forwards every write, including a bad one, within milliseconds.

What does a live replica fail to protect?

Failure	What the live copy does	What still has to exist
Broker or Availability Zone loss	MSK places brokers across zones. In-cluster replication keeps serving.	A replication factor above 1, with replicas in more than one zone
Region loss	MSK Replicator or Cluster Linking can hold a standby cluster	A tested failover. The standby is seconds behind, and it is still not a backup.
Topic or records deleted	The delete becomes the current log. Confluent will not restore a deleted topic.	A copy taken before the delete
Bad application writes	Every replica stores the bad records	A restore to a timestamp from before the write
Retention expires data	The log drops it on schedule	A copy outside the cluster retention window

MSK Replicator is the managed way to copy between MSK clusters in the same account, in one Region or across Regions. It also copies from self-managed Kafka onto MSK provisioned. It replicates topic data, topic configuration, ACLs, and consumer group offsets.

That is what you want for a warm standby. It cannot return a topic to an earlier hour. The MSK Replicator comparison and the MSK Replicator guide cover failover. MirrorMaker 2 can do the same class of copy when you operate it yourself.

Tiered storage and a Kafka Connect S3 sink are also not that restore. Tiered storage keeps older segments cheaply, and they still expire. An S3 sink is a strong way to land records in a lake. It was not built to recreate topics and consumer groups at a chosen timestamp.

AWS Backup still has no MSK resource type. Broker EBS volumes on MSK are not volumes in your account, so you cannot snapshot them yourself. On EC2 you can snapshot EBS, and that snapshot is still a disk image, not a Kafka restore.

How do you store a point-in-time copy?

Run the backup in the same VPC as the brokers, on EC2 or on EKS. Authenticate to a provisioned cluster with SASL/SCRAM or mutual TLS. Write the archive to S3 in the same Region.

S3 credentials can come from the instance profile or from an IRSA role, so the process does not need a static access key. The Kafka backup on AWS guide walks the IAM policy, the bucket layout, and the compute placement. The backup to S3 walkthrough is the step-by-step version.

mode: backup
backup_id: "msk-prod-aws"

source:
  bootstrap_servers:
    - b-1.mymsk.abcd12.c2.kafka.us-east-1.amazonaws.com:9096
    - b-2.mymsk.abcd12.c2.kafka.us-east-1.amazonaws.com:9096
  security:
    security_protocol: SASL_SSL
    sasl_mechanism: SCRAM-SHA512
    sasl_username: ${MSK_SCRAM_USER}
    sasl_password: ${MSK_SCRAM_PASSWORD}
  connection:
    tcp_keepalive: true
  topics:
    include:
      - "*"
    exclude:
      - "__consumer_offsets"

storage:
  backend: s3
  bucket: my-kafka-backups
  region: us-east-1
  prefix: msk/production

backup:
  compression: zstd
  include_offset_headers: true
  consumer_group_snapshot: true

Every key in that file is defined in the configuration reference. consumer_group_snapshot writes consumer group offsets after each cycle. include_offset_headers stores the original offset and timestamp on each record.

Compression defaults to Zstd. Storage setup for the bucket is in the AWS S3 docs.

Restore from that archive when you need a timestamp, not just the latest log. OSO Kafka Backup restores to a millisecond timestamp. The point-in-time recovery notes describe how record timestamps are filtered.

Set the recovery point from the business tolerance, not from the tool. The RTO and RPO guide and the disaster recovery use case show how to pick that window per topic class.

The copy MSK and Confluent Cloud do not keep

AWS and Confluent keep the brokers up. They do not keep yesterday's topic state. OSO Kafka Backup copies topic records and consumer group offsets to S3, with Zstd compression and millisecond point-in-time restore. Get started.

Pick the operator, then add a restore point

For most teams, Kafka in AWS should be MSK provisioned. Use Standard brokers when you want EBS and tiered storage. Use Express when you want elastic storage and you can accept the Express version list.

Use Serverless when the quotas fit and IAM is acceptable. Use EC2 when you must own the release. Use Confluent Cloud when you want Confluent's cluster types and private networking on AWS.

Then add a backup that is not another replica. Replication, MirrorMaker, MSK Replicator, and Cluster Linking answer an outage. A point-in-time copy in S3 answers a delete, a bad deploy, and a retention window that already closed.

Frequently asked questions

What is the best way to run Kafka on AWS?

Amazon MSK provisioned is the usual production default, because AWS runs the brokers and you still choose the version and size. Use MSK Serverless for spiky traffic inside its quotas. Use EC2 when you must own the install, and Confluent Cloud when you want Confluent to operate the cluster. None of the four stores an independent point-in-time copy of topic data.

Should you run Kafka on EC2 or Amazon MSK?

Run Kafka on EC2 when you must pick a release, a plugin, or a broker setting that MSK does not offer. Choose Amazon MSK when you want AWS to patch brokers, replace failed nodes, and run the metadata quorum. Provisioned MSK still lets you pick a supported version. EC2 leaves upgrades, disks, and the quorum on your team.

When does MSK Serverless make sense?

MSK Serverless makes sense when throughput changes a lot and you want AWS to add capacity without a broker count. It requires IAM on every connection, uses PrivateLink, and caps ingress at 200 MBps and egress at 400 MBps per cluster. You cannot edit broker configs, and tiered storage is not available.

How is Confluent Cloud on AWS different from Amazon MSK?

Confluent operates Confluent Cloud, with cluster types from Basic through Dedicated and Freight. Amazon MSK is an AWS service in your account, with provisioned brokers or Serverless. Both can use private networking on AWS. Cluster Linking and MSK Replicator both copy the live log, and a deleted topic still needs a separate backup.

Is MSK Replicator a Kafka backup?

No. MSK Replicator copies topic data, topic configuration, ACLs, and consumer group offsets to another cluster. A delete or a bad write is copied with the rest of the log. Use it for regional failover and migration. Keep a separate backup when you need an earlier copy of the data.

How do you back up Kafka on AWS?

Run a backup client in the brokers VPC and store topic records plus consumer group offsets in S3. On a provisioned cluster, connect with SASL/SCRAM or mutual TLS. AWS Backup does not list MSK, and MSK Replicator is replication rather than a restore point. OSO Kafka Backup provides this copy with Zstd compression and millisecond point-in-time restore.