Skip to content

Instantly share code, notes, and snippets.

@bepcyc

bepcyc/kafka_topics_sizes.sh

Last active October 27, 2022 12:16
Show Gist options
  • Save bepcyc/c443859f92541d2b82142591c2fd082c to your computer and use it in GitHub Desktop.
Save bepcyc/c443859f92541d2b82142591c2fd082c to your computer and use it in GitHub Desktop.
Download ZIP
Get kafka topic sizes in GB and sort them by size in ascending order
Raw
kafka_topics_sizes.sh
#!/usr/bin/env bash
topic-size() { kafka-log-dirs --command-config /opt/kafka/ssl/client.txt --bootstrap-server server:9093 --topic-list ${1} --describe | tail -n1 | jq '.brokers[0].logDirs[0].partitions | map(.size/1000000000) | add' | xargs echo ${1} =; }
list-topics() { kafka-topics --command-config /opt/kafka/ssl/client.txt --bootstrap-server server:9093 --list; }
export -f topic-size
TEMP_FILE=$(mktemp)
list-topics | xargs -I{} bash -c 'topic-size "{}"' > $TEMP_FILE
sort -g -k3 $TEMP_FILE
rm $TEMP_FILE
@somu123
Copy link

somu123 commented Sep 6, 2022

This will list only size of logDir 0 in broker 0 right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment