Skip to content

Instantly share code, notes, and snippets.

@keyan
keyan / gtfs_rt_proto_parser.py
Created February 15, 2019 18:21
A small helper script for quickly parsing agency GTFS-RT feeds for debugging
"""
A small helper script for quickly parsing agency GTFS-RT feeds for debugging.
Usage:
$ service_venv python3 parse_gtfs_rt_feed.py --help
"""
import argparse
import requests
from google.transit import gtfs_realtime_pb2
@keyan
keyan / nested_map.py
Created November 1, 2018 22:13
Apply a func to a target class to any fields and items nested within an object
def nested_map(obj: object, func, target_class) -> Any:
"""
Accepts any object type and calls .func() on any nested fields of the specified class.
"""
def _nested_map(obj: object) -> None:
"""
Traverse all object fields and list items, calling func when possible.
"""
# Objects without __dict__ cannot have vars() called
if not hasattr(obj, '__dict__'):
@keyan
keyan / ssl_disable.md
Last active March 19, 2018 16:31
Disabling Python's SSL verification

It is pretty confusing how to add custom root certificates to Python's trusted cert list as it doesn't use system certs. Instead I just disabled verification entirely. Note this is for local development only and SHOULD NOT BE USED IN PRODUCTION:

$ export PYTHONHTTPSVERIFY=0

This is helpful for capturing and inspecting outbound packets using Charles proxy or a similar tool.

@keyan
keyan / config_management_terms.md
Last active March 22, 2024 15:55
Chef <-> Salt <-> Puppet terminology equivalence

Recently I have had to deal with Salt configuration. I am doing product work so I didn't want to focus too much on deeply grasping Salt, but instead knowing the terminology equivalents to Chef/Puppet which I am comfortable with.

Salt Chef Puppet
state resource resource
states recipes manifests
formula* cookbook module
pillars databags hiera
grains ohai facter
@keyan
keyan / fair_lock.py
Created January 19, 2018 03:23
Implement a "fair lock"
"""
A "fair lock" is a lock which guarantees threads are given access in
the order that they requested it. The whole idea here is just to play
with the concurrency control ideas, because Python's Queue
implementation is already threadsafe, therefore acheiving the same
behavior as the FIFO lock below.
"""
import threading
@keyan
keyan / halyard.md
Created January 4, 2018 16:51
Some notes on how to setup spinnaker and use halyard

Installation

  • Halyard must be run on Ubuntu 14.04 (Trusty), you will likely use a VM to run it:
$ vagrant init ubuntu/trusty64; vagrant up --provider virtualbox
  • Sync your local kubeconfig with the guest VM by adding the following line to your newly created Vagrantfile:
@keyan
keyan / email_todo_authors.py
Last active January 4, 2018 18:24
A script you can run in your repo to scan for TODO messages and email the authors asking them to fix it! Can be easily adapted for other search strings.
#!/usr/bin/python
"""
This script searches the git repo looking for TODO messages, determines the
author of the line, then emails that author asking them to repair the issue.
It is intended to be run at a regular interval on a CI server or cron-job
server. By changing the `SEARCH_STRING` the script can be easily adapted to
trigger on any undesired code.
Usage:
@keyan
keyan / task1.exs
Created October 30, 2017 19:06 — forked from moklett/task1.exs
Elixir Task - Crash Handling
# This demonstrates that, when using async/await, a crash in the task will crash the caller
defmodule Tasker do
def good(message) do
IO.puts message
end
def bad(message) do
IO.puts message
raise "I'm BAD!"
end
@keyan
keyan / etcd_upgrade.md
Last active October 16, 2019 06:17
Upgrading etcd cluster from APIv2 data storage to APIv3 data storage for Kubernetes

Checklist for offline etcd v2 API -> v3 API

Adapted from https://coreos.com/etcd/docs/latest/op-guide/v2-migration.html#migrate-data

Note that this strategy results in kube-apiserver downtime, but this should not affect the state of the cluster, it will only prevent changes during the migration

  • Ensure there is no v3 data in etcd as this will be overwritten during the migration, verify that the output of the following command is 0:
ETCDCTL_API=3 etcdctl get "" --from-key --keys-only --limit 1 | wc -l

Installation

  • Halyard must be run on Ubuntu 14.04 (Trusty), you will likely use a VM to run it:
$ vagrant init ubuntu/trusty64; vagrant up --provider virtualbox
  • Sync your local kubeconfig with the guest VM by adding the following line to your newly created Vagrantfile: