Installation and Getting started with Apache-Kafka for OSX

install_kafka.md

Apache Kafka is a highly-scalable publish-subscribe messaging system that can serve as the data backbone in distributed applications. With Kafka’s Producer-Consumer model it becomes easy to implement multiple data consumers that do live monitoring as well persistent data storage for later analysis. You can assume that RabbitMQ is similar to Kafka, You do get an option for message-sending or Broadcasting. The installation process for OSX is as below.

1. The best way to install the latest version of the Kafka server on OS X and to keep it up to date is via Homebrew.

    brew search kafka
    brew install kafka 
   

2. The above commands will install a dependency called zookeeper which is required to run kafka. Start the zookeeper service.

    zkserver start
   

3. While Zookeeper is running, Start the kafka server which handles both producer and consumer parts. Locate the bin folder below cd /usr/local/Cellar/kafka/<kafka-version-number>/bin. In my example, the version which I am running is 0.11.0.0. This will keep kafka running.

    cd /usr/local/Cellar/kafka/0.11.0.0/bin
    sh kafka-server-start /usr/local/etc/kafka/server.properties
   

4. In a new terminal, Start the consumer which will recieve events or messages.

    cd /usr/local/Cellar/kafka/0.11.0.0/bin
    sh kafka-console-consumer --zookeeper localhost:2181 --topic sample_topic --from-beginning
   

5. Note that, the topic determines to where the producer is producing the event, and where the consumer is listening to. In simple words, Keep the topic same in both producer and consumer. Here the topic is sample_topic

6. In a new terminal, Start the producer which will produce the events.

    cd /usr/local/Cellar/kafka/0.11.0.0/bin
    sh kafka-console-producer --broker-list localhost:9092 --topic sample_topic
   

7. In the producer's terminal, You will get an angle prompt >. Type your messages and hit enter

8. Receive your message in the consumer terminal.

I think this is working if your kafka server is running on "localhost:9092"
check the exact your "host:port" at the "/usr/local/etc/kafka/server.properties"
sh kafka-console-consumer --bootstrap-server localhost:9092 --topic sample_topic --from-beginning