Skip to content

Instantly share code, notes, and snippets.

@monkut
monkut / github_org_projects_info.graphql
Last active June 13, 2018 23:48
A github graphql query to obtain information on originzational projects
query {
organization(login:"your-org-name"){
name
projects(first:25, states:OPEN) {
nodes {
name,
columns(first:10){
nodes{
name,
cards (first:50){
@monkut
monkut / compare_directories.py
Created June 12, 2018 07:23
Compare two directory's content size with percentage
"""
This is a script to compare the content of two directories.
It was used to determine how complete a series of cp commands were done after the cp process was already started
"""
import os
def directory_size(path):
total = 0
for entry in os.scandir(path):
if entry.is_file():
@monkut
monkut / get_package_subclassed_classes.py
Created May 31, 2018 03:40
Voodoo function to discover classes in given 'package' that subclass the given 'target_baseclass'
def _get_package_defined_classes(package, target_baseclass) -> dict:
"""
Voodoo function to discover classes in given 'package' that subclass the given 'target_baseclass'
:param package: Python Package Object
:param target_baseclass: Python Class Object
:return:
{
CLASS_NAME: CLASS_OBJECT,
...
}
VALID_PYTHON_EXTENSIONS = ('.py', )
def get_userdefined_class(directory: str, target_baseclass: object) -> list:
"""
Load Classes that sub-classed the given 'target_baseclass' for modules in the given directory
:param directory: directory containing user-defined classes subclassing 'target_baseclass'
:param target_baseclass: the ABC class that the user class subclasses
:return: (class) [UserDefinedClass, ...]
"""
@monkut
monkut / lazy_LRUish_redis_cache_decorator.py
Last active December 22, 2021 23:46
A lazy implementation of an lru-ish redis based cache decorator
import json
from functools import wraps
import redis
REDIS_HOST = 'localhost'
REDIS_PORT = 6379
REDIS_CONNECTION_POOL = redis.ConnectionPool(host=REDIS_HOST, port=REDIS_PORT)
CACHE_EXPIRE_SECONDS = 5000
@monkut
monkut / serving-sphinx-docs-on-s3.md
Created May 8, 2018 05:35
Serving Sphinx Documentation on s3

Build your sphinx document

Assumes you already have a sphinx project installed and a project created

make html

By default this command will build documentation to _build/html

@monkut
monkut / sample.ipynb
Last active December 8, 2017 05:06 — forked from egradman/sample.ipynb
Simple Google Spreadsheets to Pandas DataFrame in IPython Notebook
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@monkut
monkut / ec2_metadata_manager.py
Created November 15, 2017 07:24
Provides a simple (top-level only) wrapper around the AWS ec2 instance metadata interface
import requests
METADATA_URL = 'http://169.254.169.254/latest/meta-data/'
class AmiMetaDataManager:
"""
Provides a simple wrapper around the AWS ec2 instance metadata interface
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
"""
@monkut
monkut / aws_and_dask-gettingstarted.md
Last active September 11, 2017 04:10
Getting Started with AWS and Dask

Prereqs

NOTE: this assumes python >= 3.6

On ubuntu make sure that you have the python3.6-dev package installed in order to build the necessary packages

  • python3.6
  • jq

Preparation

@monkut
monkut / github_archive_retriever.py
Created August 14, 2017 01:31
parallel retrieve of github arhive data
"""
Retrieve github archieve data from:
https://www.githubarchive.org/
"""
import datetime
from concurrent.futures import ThreadPoolExecutor
from tqdm import tqdm