CS--Kafka Useful commands

//Install on ubuntu 14-04
https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-14-04

cd ~/kafka 

//Start the server 
 nohup bin/kafka-server-start.sh config/server.properties > kafka.log 2>&1 &

//Write to a topic TutorialTopic
echo "Hello, World" | bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TutorialTopic > /dev/null

//Start a consumer to read from topic TutorialTopic (for kafka before 2.8.1)
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic TutorialTopic --from-beginning

//Start a consumer to read from topic TutorialTopic (for kafka after 2.8.1)
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic TutorialTopic --from-beginning

//Stop the server 
just list the process for kafka and kill them 

//Create a topic
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic dx_aep_ups_edge_segmentation_sync_to_hub --replication-factor 1 --partitions 3

//To clear a topic 
stop kafka server as described above //on command line
rm -Rf /tmp/kafka-logs/<topic_name>* //on command line 
rmr /brokers/topics/UserActivitySDM  //in zookeeper client -> zkCli.sh

//To count the numder of elements in a topic
- One way is to start the consumer with --from-beginning flag (As described above) and see how many messages it processes

//References
https://www.digitalocean.com/community/tutorials/how-to-install-apache-kafka-on-ubuntu-14-04