Skip to content

Instantly share code, notes, and snippets.

@justdoit0823

justdoit0823/kafka_consumer_group_rebalance.md

Created March 22, 2019 07:28
Show Gist options
  • Save justdoit0823/561bbd722e86fbcdb9469405e6772bab to your computer and use it in GitHub Desktop.
Save justdoit0823/561bbd722e86fbcdb9469405e6772bab to your computer and use it in GitHub Desktop.
Download ZIP
Kafka consumer group rebalance process.
Raw
kafka_consumer_group_rebalance.md

Kafka commands

  • list consumer group members
kafka-consumer-groups --describe  --bootstrap-server localhost:9092 --group g1 --members
  • show consumer group state
kafka-consumer-groups --describe  --bootstrap-server localhost:9092 --group g1 --state

Kafka design

Phase

  • Phase 1: Joining the Group

  • Phase 2: Synchronizing Group State

Coordinator state machine

Coordinator state machine

FAQ

  • Leader Failures

The leader of each group is responsible for initiating group synchronization when topic metadata changes. A leader failure is detected by the coordinator through the expiration of its session timeout. The coordinator will respond by forcing all members to rejoin, which will allow a new leader to be elected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment