Kafka Development Cheat Sheet

KafkaDevelopmentCheatSheet.md

Kafka Development Cheat Sheet

These instructions assume you have Docker and Docker Compose installed.

To run a local development instance of Kafka, first create the following docker-compose.yml file:

version: '2'
services:
    zookeeper:
        image: wurstmeister/zookeeper:3.4.6
        ports:
          - 2181:2181
    kafka:
        image: wurstmeister/kafka:1.0.0
        environment:
            KAFKA_ADVERTISED_HOST_NAME: localhost # change to 192.168.99.100 if using Docker Toolbox
            KAFKA_ADVERTISED_PORT: 9092
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_CREATE_TOPICS: "topic-jhipster:1:1"
        ports:
            - 9092:9092

Now start it up.

$ docker-compose up -d

Installing Kafkacat (Linux and Mac only)

Next, install kafkacat. For Linux...

$ sudo apt-get install kafkacat

...or for Mac...

$ sudo chown -R $(whoami) /usr/local/lib/pkgconfig
$ brew install kafkacat

Using Kafkacat (Linux and Mac only)

Now you can produce messages like this (enter messages after the command, then ^C to quit):

$ kafkacat -P -b localhost:9092 -t my-topic
This is a test message
This is another message
^C

And you can consume messages like this (messages will appear after the command, then ^C to quit):

$ kafkacat -C -b localhost:9092 -t my-topic
This is a test message
This is another message
^C

Using Kafkacat in a Docker Container (Windows, Linux, and Mac)

You can also use kafkacat in a Docker container (to be completed - look here for now).