understanding about Kafka

Platform for real-time distributed streaming which just behaves like any other MESSAGING system

Handson using kafka docker image

This will pull the fast-data-dev image from lensesio and will have the image runnable

$ docker run --rm --net=host lensesio/fast-data-dev

List all the topics

$ kafka-topics --zookeeper 127.0.0.1 --list

List all the topics without zookeeper

$ kafka-topics --bootstrap-server 127.0.0.1:9092 --list

Create a topic in the kafka broker

$ kafka-topics --zookeeper 127.0.0.1 --topic pap --create

create topic with partitions and replication-factor

$ kafka-topics --zookeeper 127.0.0.1 -topic pap --create --partitions 3 --replication-factor 1

Describe the details about topic

$ kafka-topics --zookeeper 127.0.0.1 --topic pap --describe

Delete a topic

$ kafka-topics --zookeeper 127.0.0.1 --topic pap --delete

Producer which produce the data

$ kafka-console-producer --broker-list 127.0.0.1:9092 --topic pap

Consumer to consume the data

$ kafka-console-consumer --bootstrap-server 127.0.0.1:9092  --topic pap --from-beginning

List all the consumer groups

$ kafka-consumer-groups  --bootstrap-server localhost:9092 --list

Consume the data from a specific topic and a specific group

$ kafka-consumer-groups --bootstrap-server localhost:9092 --group myPap --reset-offsets --to-earliest --execute --topic pap

Delete a kafka topic without using zookeeper reference

$ kafka-topics --bootstrap-server localhost:9092 --topic pap --delete