Skip to content

Instantly share code, notes, and snippets.

View chicagobuss's full-sized avatar

Joshua Buss chicagobuss

View GitHub Profile
@chicagobuss
chicagobuss / How-to.txt
Last active March 23, 2026 03:03
lambda-to-kafka
# First I created a working directory
~$ mkdir s3-kafka
# Then I installed the dependency I needed in that directory with pip
~$ cd s3-kafka
~$ pip install kafka-python -t $(pwd)
# Then I put my code into a file called s3-kafka.py
~$ vi s3-kafka.py
@chicagobuss
chicagobuss / keybase.md
Created August 30, 2017 19:50
keybase proof

Keybase proof

I hereby claim:

  • I am chicagobuss on github.
  • I am chicagobuss (https://keybase.io/chicagobuss) on keybase.
  • I have a public key ASBeYneyoRUzB6cOq47eSqgO3GOxBSF6PL6XqZxo6GWbggo

To claim this, I am signing this object:

@chicagobuss
chicagobuss / squid.conf
Created September 11, 2017 19:43
Simple Squid Reverse Proxy for Vault
# Vault forwarding
http_port 8200 accel defaultsite=vault
cache_peer 10.10.10.41 parent 8200 0 proxy-only name=vault1
cache_peer 10.10.50.52 parent 8200 0 proxy-only name=vault2
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
http_access allow localnet
http_access allow localhost
http_access deny all
@chicagobuss
chicagobuss / elevated.sh
Created October 20, 2017 22:13
how to find elevated containers
#!/bin/bash
#
# Author: Joshua Buss (@chicagobuss)
# Usage: elevated.sh [-d]
# Example:
# ./elevated.sh | grep 'is p'
# ./elevated.sh -d | grep 'is not'
for id in $(docker ps -a | grep "Up" | awk '{print $1}'); do
name=$(docker ps -a | grep "${id}" | awk '{print $2}')
@chicagobuss
chicagobuss / apply_ecr_lifecycle_policy.sh
Created December 14, 2017 16:53
a sane ecr lifecycle policy
#!/bin/bash
LIFECYCLE_POLICY='{"rules":[{"rulePriority":10,"description":"keeps 50 latest tagged images","selection":{"tagStatus":"tagged","countType":"imageCountMoreThan","countNumber":50,"tagPrefixList":["v"]},"action":{"type":"expire"}},{"rulePriority":20,"description":"keeps 5 latest untagged images","selection":{"tagStatus":"untagged","countType":"imageCountMoreThan","countNumber":5},"action":{"type":"expire"}},{"rulePriority":30,"description":"keeps latest 20 numeric-tagged images","selection":{"tagStatus":"tagged","countType":"imageCountMoreThan","tagPrefixList":["0","1","2","3","4","5","6","7","8","9"],"countNumber":20},"action":{"type":"expire"}},{"rulePriority":40,"description":"keeps latest 20 a-f tagged images","selection":{"tagStatus":"tagged","countType":"imageCountMoreThan","tagPrefixList":["a","b","c","d","e","f"],"countNumber":20},"action":{"type":"expire"}}]}'
aws ecr put-lifecycle-policy --region ${AWS_REGION} --repository-name ${REPO} --lifecycle-policy-text ${LIFECYCLE_POLICY} || echo "Fa
@chicagobuss
chicagobuss / k8s-gke-perms.txt
Created January 31, 2018 14:06
perms in gke k8s 1.8
# If you're having problems creating an admin role binding in k8s on GKE,
# make sure you're a project owner first in the project the cluster is in
# Then you can do this:
~$ kubectl create clusterrolebinding foo-cluster-admin-binding --clusterrole=cluster-admin --user="[email protected]"
clusterrolebinding "foo-cluster-admin-binding" created
@chicagobuss
chicagobuss / export.py
Created February 2, 2018 22:04
github export
```#!/usr/bin/env python
"""
Exports Issues from a specified repository to a CSV file
Uses basic authentication (Github username + password) to retrieve Issues
from a repository that username has access to. Supports Github API v3.
"""
import csv
import requests
import sys
@chicagobuss
chicagobuss / policy.json
Created February 12, 2018 21:39
example s3 bucket policy with kms enforced
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::prod-secrets/*",
"Condition": {
@chicagobuss
chicagobuss / metric-consumer.go
Last active May 15, 2018 18:19
metric-consumer
package main
import (
"encoding/json"
"fmt"
"time"
"github.com/confluentinc/confluent-kafka-go/kafka"
"github.com/prometheus/common/model"
)
@chicagobuss
chicagobuss / all_jmx_metrics
Created February 26, 2019 02:25
all jmx metrics
This file has been truncated, but you can view the full file.
#####################################
Instance: 10.74.147.10:7203
#####################################
Not Matching: Bean name: kafka.log:type=Log,name=Size,topic=lkc-4vndj__confluent-ksql-ksql-prodtransient_5013047390911865834_1548275444821-KSTREAM-AGGREGATE-STATE-STORE-0000000006-repartition,partition=1 - Attribute name: Value - Attribute type: java.lang.Object
Not Matching: Bean name: kafka.cluster:type=Partition,name=InSyncReplicasCount,topic=__consumer_offsets,partition=37 - Attribute name: Value - Attribute type: java.lang.Object
Not Matching: Bean name: kafka.network:type=RequestMetrics,name=ResponseQueueTimeMs,request=ListGroups - Attribute name: Count - Attribute type: long
Not Matching: Bean name: kafka.network:type=RequestMetrics,name=ResponseQueueTimeMs,request=ListGroups - Attribute name: Min - Attribute type: double
Not Matching: Bean name: kafka.network:type=RequestMetrics,name=ResponseQueueTimeMs,request=ListGroups - Attribute name: 50thPercentile - A