Skip to content

Instantly share code, notes, and snippets.

@hoghweed
Last active June 11, 2024 09:55
Show Gist options
  • Save hoghweed/025d3c005e812d7d0231130fae297b50 to your computer and use it in GitHub Desktop.
Save hoghweed/025d3c005e812d7d0231130fae297b50 to your computer and use it in GitHub Desktop.
Kafka support console

Sample Kafka Console

To run the console:

  1. rename the env.sample to .env
  2. based on the kafka's configuration to use, comment/uncomment the required env vars
  3. in case of a secured kafka, remember to configure the right SASL MECHANISM
  4. run the console using docker-compose up -d
  5. open the browser at http://localhost:8081

In case you need to test the console with a local running dev kafka you can:

inside the docker-compose file

  1. uncomment the kafka-broker
  2. uncomment the kafka-coordinator
  3. uncomment the networks
  4. uncomment the depends-on and networks in the kafka-console

inside the .env file

  1. set the KAFKA_BROKERS to kafka-broker:19092

inside the cointaining folder

  1. create a folder dev
  2. create a dev subfolder .datastore
version: '3.8'
services:
kafka-console:
container_name: kafka-console
image: docker.redpanda.com/redpandadata/console:latest
restart: on-failure
ports:
- '8081:8080'
# depends_on:
# - kafka-broker
env_file:
- .env
# networks:
# - tooling
# kafka-coordinator:
# platform: linux/amd64
# image: confluentinc/cp-zookeeper:7.1.6
# container_name: kafka-coordinator
# hostname: zookeeper
# environment:
# ZOOKEEPER_CLIENT_PORT: 2181
# ports:
# - 2181:2181
# volumes:
# - ./dev/.datastore/kafka-coordinator/log/version-2/:/var/lib/zookeeper/log/version-2
# - ./dev/.datastore/kafka-coordinator/data/version-2/:/var/lib/zookeeper/data/version-2
# networks:
# - tooling
# kafka-broker:
# image: confluentinc/cp-kafka:7.1.6
# container_name: kafka-broker
# hostname: kafka-broker
# depends_on:
# - kafka-coordinator
# ports:
# - 19092:19092
# volumes:
# - ./dev/.datastore/kafka-broker/data:/var/lib/kafka/data
# - ./dev/.datastore/kafka-broker/etc:/etc/kafka
# environment:
# KAFKA_BROKER_ID: 1
# KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
# KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-broker:9092,CONNECTIONS_FROM_HOST://localhost:19092
# KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
# KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
# KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
# networks:
# - tooling
# networks:
# tooling:
# Simple local configuration
# Local kafka with no TLS and no SASL auth configured
KAFKA_BROKERS=localhost:19092
# KAFKA_BROKERS=kafka-broker:19092
KAFKA_TLS_ENABLED=false
KAFKA_SASL_ENABLED=false
# Secured kafka
# KAFKA_BROKERS=hostname:port
# KAFKA_TLS_ENABLED=true
# KAFKA_SASL_MECHANISM=saslmechanism-to-use
# KAFKA_SASL_ENABLED=true
# KAFKA_SASL_USERNAME=
# KAFKA_SASL_PASSWORD=
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment