Skip to content

Instantly share code, notes, and snippets.

@jahe

jahe/kafka-notes.txt

Created May 12, 2017 19:30
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jahe/ff2bc3550563672bb2f68774cdd1c2ee to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jahe/ff2bc3550563672bb2f68774cdd1c2ee to your computer and use it in GitHub Desktop.
Download ZIP
Kafka Notes
Raw
kafka-notes.txt
Producer - Publishes data on a specific topic into the Kafka Cluster
Consumer - Consumes data from a specific topic
Broker - One Node in the Kafka Cluster which is responsible for specific topics
Kafka Cluster - Multiple machines running Kafka
Topic - Subject where Producers + Consumers can communicate on
Commit Log - Messages stored in a broker identified by an ID
Retention Policy - What to do in case of lacking storage:
1. Remove some old ones
2. Remove the old versions of messages and only leave the latest versions of the messages
When a Consumer dies and restarts after an amount of time
it gets the missing data from the Broker by an id (Commit Log ID) which indicates the last received message.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment