pkutaj/2023-10-18-Explaining-Diffs-between-Apache-Kafka-and-AWS-Kinesis.md

Created October 18, 2023 09:13

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/pkutaj/a453360fd64312db325c4ee1731f3067.js"></script>
Save pkutaj/a453360fd64312db325c4ee1731f3067 to your computer and use it in GitHub Desktop.

2023-10-18-Explaining-Diffs-between-Apache-Kafka-and-AWS-Kinesis.md

Raw

2023-10-18-Explaining-Diffs-between-Apache-Kafka-and-AWS-Kinesis.md

    
Kafka
AWS Kinesis

Operational Model
System of brokers (servers that store and process messages)
Managed service provided by AWS

Data Writing
Writes data to disk, I/O operations carried out by OS
Synchronously brokers data streams, writes and replicates ingested data into three different AWS machines

Configurability
More control to the operator
Machine configuration is abstracted away

Architecture
Topics (all related events are stored), Partitions (based on cluster specifications)
Streams (equivalent of Kafka topic), Shards

	Kafka	AWS Kinesis
Operational Model	System of brokers (servers that store and process messages)	Managed service provided by AWS
Data Writing	Writes data to disk, I/O operations carried out by OS	Synchronously brokers data streams, writes and replicates ingested data into three different AWS machines
Configurability	More control to the operator	Machine configuration is abstracted away
Architecture	Topics (all related events are stored), Partitions (based on cluster specifications)	Streams (equivalent of Kafka topic), Shards