https://github.com/confluentinc/cp-docker-images/wiki/Getting-Started
cli
to find how many consumers and what each consumer hold of paritions
sudo docker run --net=host --rm confluentinc/cp-kafka:4.1.2-2 kafka-consumer-groups --describe --members --group fixed-lines-kafka-group --bootstrap-server localhost:9092
create a kafka topics
####### NOTE: --net=host is making the network of the container the same as the my host create topics docker run --net=host --rm confluentinc/cp-kafka:4.1.2-2 kafka-topics --create --topic foo --partitions 1 --replication-factor 1 --if-not-exists --zookeeper localhost:2181
decribe topics docker run --net=host --rm confluentinc/cp-kafka:4.1.2-2 kafka-topics --describe --topic foo --zookeeper localhost:2181
produce messages docker run -it --net=host --rm confluentinc/cp-kafka:4.1.2-2 kafka-console-producer --broker-list localhost:9092 --topic foo
+++++
docker run --net=host --rm confluentinc/cp-kafka:4.1.2-2 kafka-console-consumer --bootstrap-server localhost:9092 --topic foo --from-beginning
to execute command inside a created running container from docker-compose sudo docker-compose exec some-kafka kafka-topics --create --zookeeper some-zookeeper:2181 --replication-factor 1 --partitions 1 --topic users
docker run --net=host --rm confluentinc/cp-kafka:5.2.3 kafka-topics --create --topic foo --partitions 1 --replication-factor 1 --if-not-exists --zookeeper localhost:32181
info about a topic sudo docker-compose exec some-kafka kafka-topics --describe --topic users --zookeeper some-zookeeper:2181
execute bash on kafka from docker-compose sudo docker-compose exec some-kafka bash -c "seq 42 | kafka-console-producer --broker-list localhost:29092 --topic users && echo 'Produced 42 messages.'"
run from docker only zookeeper docker run -d --net=host --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 -e ZOOKEEPER_TICK_TIME=2000 confluentinc/cp-zookeeper:5.2.3
kafka docker run -d --net=host --name=kafka -e KAFKA_ZOOKEEPER_CONNECT=localhost:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 confluentinc/cp-kafka:4.1.2-2
docker run -d --name zookeeper -p 2181:2181 confluentinc/cp-zookeeper:5.2.3
why are we using zookeeper here for creating a topic ?
because zookeeper is what notify and updates kafka brokers (servers)
follow the cli proudcer in google doc https://docs.google.com/document/d/1qS8NRe8FJrku4ofyuueiqDapPO-j4xUgW2ztnHwmSaU/edit
---
version: "2"
services:
some-zookeeper:
image: confluentinc/cp-zookeeper:5.2.3
container_name: some-zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
some-kafka:
# "`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-
# An important note about accessing Kafka from clients on other machines:
# -----------------------------------------------------------------------
#
# The config used here exposes port 9092 for _external_ connections to the broker
# i.e. those from _outside_ the docker network. This could be from the host machine
# running docker, or maybe further afield if you've got a more complicated setup.
# If the latter is true, you will need to change the value 'localhost' in
# KAFKA_ADVERTISED_LISTENERS to one that is resolvable to the docker host from those
# remote clients
#
# For connections _internal_ to the docker network, such as from other services
# and components, use kafka:29092.
#
# See https://rmoff.net/2018/08/02/kafka-listeners-explained/ for details
# "`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-'"`-._,-
#
image: confluentinc/cp-kafka:4.1.2-2
container_name: some-kafka
depends_on:
- some-zookeeper
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: some-zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://some-kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
```yml