Skip to content

Instantly share code, notes, and snippets.

@cinhtau
Last active April 28, 2022 22:50
Show Gist options
  • Save cinhtau/75a310dbe8389483d7a0e1a86da31cfd to your computer and use it in GitHub Desktop.
Save cinhtau/75a310dbe8389483d7a0e1a86da31cfd to your computer and use it in GitHub Desktop.
Ansible playbook to start a minimum HA Apache Kafka + ZooKeeper cluster of 3 nodes with docker containers, on host 1 Kafka Manager is started
# playbook for Apache Kafka deployment
# with docker containers
---
- hosts: all
vars:
ids:
host01: 1
host02: 2
host03: 3
mappings:
KAFKA_ZOOKEEPER_CONNECT: "host01:2181,\
host02:2181,\
host03:2181"
KAFKA_BROKER_ID: "{{ids[ansible_hostname]}}"
KAFKA_ADVERTISED_HOST_NAME: "{{ansible_hostname}}"
KAFKA_JMX_OPTS: "-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Djava.rmi.server.hostname={{ansible_hostname}} \
-Dcom.sun.management.jmxremote.rmi.port=9099"
JMX_PORT: 9099
KAFKA_LOG_DIRS: "/kafka/kafka-logs"
tasks:
- name: deploy
docker_container:
env:
"{{mappings}}"
name: kafka
image: wurstmeister/kafka
ports:
- "9092:9092"
- "9099:9099"
pull: true
log_opt:
max-file: "3"
max-size: 25m
state: started
restart: yes
restart_policy: always
restart_retries: 10
volumes:
- /var/opt/kafka:/kafka
- name: deploy manager
docker_container:
env:
ZK_HOSTS: "{{mappings.KAFKA_ZOOKEEPER_CONNECT}}"
name: kafka-manager
image: cinhtau/docker-kafka-manager
ports:
- "9000:9000"
log_opt:
max-file: "1"
max-size: 10m
state: started
restart: yes
restart_policy: always
restart_retries: 10
command: "-Djava.net.preferIPv4Stack=true"
when: inventory_hostname == 'host01'
# with docker containers
---
- hosts: all
vars:
image: zookeeper
ports:
- "2181:2181"
- "2888:2888"
- "3888:3888"
ids:
host01: 1
host02: 2
host03: 3
mappings:
ZOO_SERVERS: >
server.1=host01:2888:3888
server.2=host02:2888:3888
server.3=host03:2888:3888
ZOO_MY_ID: "{{ids[ansible_hostname]}}"
tasks:
- name: deploy
docker_container:
env:
"{{mappings}}"
name: zookeeper
image: zookeeper
network_mode: host
pull: true
log_opt:
max-file: "3"
max-size: 25m
state: started
restart: yes
restart_policy: always
restart_retries: 10
volumes:
- "/var/opt/zookeeper:/data"
- "/var/log/zookeeper:/datalog"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment