Skip to content

Instantly share code, notes, and snippets.

View kmozum's full-sized avatar

kmozum

View GitHub Profile
@kmozum
kmozum / benchmark-commands.md
Created July 28, 2019 16:21 — forked from ueokande/benchmark-commands.md
Kafka Benchmark Commands

Benchmark commands

Producer

Setup

bin/kafka-topics.sh \
  --zookeeper zookeeper.example.com:2181 \
  --create \
@kmozum
kmozum / athena.rst
Created May 26, 2019 00:45 — forked from chrisdpa-tvx/athena.rst
Create an Athena database, table, and query

All Your Data Does Not Belong In a Database

Businesses are machines producing mountains of data about sales, usage, customer, costs, etc... Traditionally data processing is highly centralised with teams of staff and computer running hot a whirling ready to process. We can do better than moving the mountain of data into the corporate data machine - so long as that machinary is light enough to be moved to the data.

Don't move the mountain - Bring the processing to the data

We've had this problem; a huge directory of files in CSV format, conataining vital information for our business. But it's in CSV, requires analysis, and don't you don't feel like learning sed/grep/awk today - besides it's 2017 and no-one thinks those tools are easy to use.