Skip to content

Instantly share code, notes, and snippets.

View kavimaluskam's full-sized avatar

Alex Kam kavimaluskam

View GitHub Profile
const zlib = require('zlib');
const concatStream = require('concat-stream');
const BufferHelper = require('bufferhelper');
const { debugLogger } = require('../../utils/logger');
/**
* Modify the response
* @param res {Object} The http response
* @param contentEncoding {String} The http header content-encoding: gzip/deflate
* @param callback {Function} Custom modified logic
@kavimaluskam
kavimaluskam / curl.sh
Created January 30, 2019 10:34
Schema Registry expected curl outputs
# Check list of available schemas
curl http://localhost:8081/subjects
# Expected output:
# ["app-users-value", "app-movies-value"]
# Check detail of app-users-value schema
curl http://localhost:8081/subjects/app-users-value/versions/latest
# Expected output:
@kavimaluskam
kavimaluskam / console-con.sh
Created January 30, 2019 09:57
Cheat sheet for Kafka Console Consumer
# Enter Kafka Shell Client
make kafka-shell
# Assign topic name in variable
TOPIC=test
# Consumer for console text output, no stream offset
kafka-console-consumer --topic=$TOPIC --bootstrap-server=$BROKERS --from-beginning
# Console consumer will print each records in the topic
@kavimaluskam
kavimaluskam / aws-msk-cli.sh
Last active December 11, 2022 10:04
Cheat sheet for AWS MSK CLI
# Assign 1 public subnetID from above subnet setup
SUBNET_ID=${subnetID}
# List all kafka cluster for corrsponding region
aws kafka list-clusters --region us-east-1
# Expected output
# {
# "ClusterInfoList": [
# {
@kavimaluskam
kavimaluskam / kafak-connect-avro.sh
Last active January 30, 2019 10:23
Cheat sheet for Standalone Kafka Connect with Avro serialisation
# 0. Delete the existing topics: app-users, app-movies
make kafka-shell
kafka-topics --delete --topic app-users --zookeeper $ZK
kafka-topics --delete --topic app-movies --zookeeper $ZK
kafka-topics --create --topic app-users --partitions 2 --zookeeper $ZK --replication-factor 2
kafka-topics --create --topic app-movies --partitions 2 --zookeeper $ZK --replication-factor 2
exit
@kavimaluskam
kavimaluskam / kafka-connect-json.sh
Last active January 30, 2019 09:59
Cheat sheet for Standalone Kafka Connect with JSON deserialisation
# Enter Kafka Connect Shell Client
make connect-shell
# Kafka Connect with connect-standalone setting, connecting via postgres
connect-standalone dev/connect-standalone.properties dev/source-postgres-incremental.properties
# Expected console-consumer output from topic: app-users
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":101,"first_name":"Cass","last_name":"Truss","email":"[email protected]"}
@kavimaluskam
kavimaluskam / console-pro.sh
Last active January 30, 2019 09:58
Cheat sheet for Kafka Console Producer
# Enter Kafka Shell Client
make kafka-shell
# Assign topic name in variable
TOPIC=test
# Producer for console text input
kafka-console-producer --topic=$TOPIC --broker-list=$BROKERS
###############################################
@kavimaluskam
kavimaluskam / topic-crud.sh
Last active January 30, 2019 09:49
Cheat sheet for Kafka Topic CRUD
# Enter Kafka Shell Client
make kafka-shell
# Assign testing topic name in variable
TOPIC=test
# Describe all topic
kafka-topics --describe --zookeeper $ZK
# Create topic
@kavimaluskam
kavimaluskam / k8s-pi.md
Created October 30, 2018 19:53 — forked from alexellis/k8s-pi.md
K8s on Raspbian

Kubernetes on (vanilla) Raspbian Lite

Yes - you can create a Kubernetes cluster with Raspberry Pis with the default operating system called Raspbian. This means you can carry on using all the tools and packages you're used to with the officially-supported OS.

This is part of a blog post Serverless Kubernetes home-lab with your Raspberry Pis written by Alex Ellis.

Copyright disclaimer: Please provide a link to the post and give attribution to the author if you plan to use this content in your own materials.

Pre-reqs:

@kavimaluskam
kavimaluskam / ItemTypes.js
Created January 24, 2018 07:33
React-DND & Material-ui-next: List Item
export default {
LIST_ITEM: 'listItem',
};