Kafka-based pub/sub demo in Chapter 6 of "Microservices in Go" by Alexander Shuiskov

README.md

Running Chapter 6's pub/sub demo using Kafka

Here's how to setup Kafka as a pub/sub engine for Alexander Shuiskov's excellent Microservices with Go, and how to create a producer and consumer for your instance in a CLI

running Docker Compose

• create a Docker Compose YAML file in cmd/ratingingester

run Kafka in a Docker container (from movieapp/src/cmd/ratingingester):

• docker-compose -d up
• docker exec -it kafka /bin/sh
• cd /opt/kafka_2.13-2.8.1/bin
• kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic ratings

send a message (in a separate terminal):

• docker exec -it kafka /bin/sh
• cd /opt/kafka_2.13-2.8.1/bin
• kafka-console-producer.sh --bootstrap-server localhost:9092 --topic ratings
• (type and send your messages at the command prompt)

run the book's ingester application (in a separate terminal):

• cd movieapp/src/cmd/ratingingester
• go run main.go

consume messages (in a separate terminal):

• docker exec -it kafka /bin/sh
• cd /opt/kafka_2.13-2.8.1/bin
• kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic ratings --from-beginning --max-messages 20
• watch the messages appear! :)

docker-compose.yml

version: '3'

services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

Thanks for the example, Jason! I'm working on 2nd edition of the book that will include some improvements to the existing chapters, as well as 3-4 new ones — do you mind if I include this into the updated edition? In that case I would certainly call out your contribution — really helpful.

Absolutely, @b441berith ! Happy to help...and great work! I really enjoyed your book. :)