kafka docker-compose minimal

docker-compose-minimal.yml

version: '2'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:4.1.0
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    restart: always

  kafka:
    image: confluentinc/cp-kafka:4.1.0
    hostname: kafka
    container_name: kafka
    extra_hosts:
    - "moby:127.0.0.1"
    depends_on:
      - zookeeper
    ports:
      - '9092:9092'
    environment:
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://127.0.0.1:9092'
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
      LOG4J_LOGGER_KAFKA_AUTHORIZER_LOGGER: INFO
      KAFKA_CREATE_TOPICS: "test:1:1"
    restart: always

== Prerequisites

docker version >= 18.03.0
docker-compose version >= 1.19.0

== Install standalone kafka

We will need a couple of scripts only from kafka to check that everything works

wget http://ftp.man.poznan.pl/apache/kafka/1.1.0/kafka_2.11-1.1.0.tgz
tar -xzf kafka_2.11-1.1.0.tgz
cd kafka_2.11-1.1.0

Now you can developed kafka applications

You can use prepared docker images to run kafka as well

== Kafka in docker ================================

== 1. Minimal kafka configuration

start up zookeeper and kafka

docker-compose -f docker-compose-minimal.yml up -d

to see logs, type

docker-compose logs -f

start producer and type lines

cd kafka_2.11-1.1.0
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
111111
222222

start consumer. it should read messages from topic

cd kafka_2.11-1.1.0
bin/kafka-simple-consumer-shell.sh --broker-list localhost:9092 --topic test
111111
222222

stop all kafka services

docker-compose -f docker-compose-minimal.yml down

Now kafka in docker is accessible on localhost:9092