Install Kafka in Mac

kafka_setup_mac.md

Installation

Install kafka

brew install kafka

The above will install kafka and it's dependency zookeeper

You can start in the background by

brew services start zookeeper
brew services start kafka

You can ensure those services started by,

brew services list

If the services have not started, look at the logs at

tail -f /usr/local/var/log/kafka/kafka_output.log

Local UI

There are bunch of options. One of them is Kafdrop. Download the latest release and you can run that locally with,

java --add-opens=java.base/sun.nio.ch=ALL-UNNAMED -jar ~/Downloads/kafdrop-3.27.0.jar

And visit http://localhost:9000/

Optional

One more optional tool to play with kafka (publishing, subscribing, metadata, etc.) is Kafkacat. Install with

brew install kafkacat

Troubleshooting:

If things are not running out of the box and you're seeing connection errors, it's likely due to (listner configs):"https://medium.com/@Ankitthakur/apache-kafka-installation-on-mac-using-homebrew-a367cdefd273".

Example error messages:

Disconnecting from node 0 due to request timeout. (org.apache.kafka.clients.NetworkClient)
Cancelled in-flight UPDATE_METADATA request with correlation id 3 due to node 0 being disconnected
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes

Update or uncomment the listeners value in /usr/local/etc/kafka/server.properties OR /opt/homebrew/etc/kafka/server.properties to be: listeners=PLAINTEXT://0.0.0.0:9092 and advertised.listeners=PLAINTEXT://localhost:9092 then restart kafka:

brew services start kafka