Using kcat with jq to read a kafka topic message

kcat.md

Given a message like this:

{ "message": {"accountId":1,"timestamp":"1694709688878","amount":1001,"sentFromClientAt":1694709688913}, "delivered_at": 1694709688954 }

The kcat command can be used with jq to format message, even a calculated attribute:

kcat -C -b localhost:31090 -f '{ "message": %s, "delivered_at": %T } \n' -t input-topic -u | jq '. + { "took": "\(.delivered_at - .message.sentFromClientAt) ms" }'

And this can be used as an "alias" function putting this in your .zshrc, for example:

kmsg() {
    kcat -C -b localhost:31090 -f '{ "message": %s, "delivered_at": %T } \n' -t $1 -u | jq '. + { "took": "\(.delivered_at - .message.sentFromClientAt) ms" }'
}

So you can do:

kmsg my-kafka-topic