wware / log_caller_frame.py

Last active November 28, 2017 19:51

Sometimes you want a Python function that includes a `logging.info()` or `logging.debug()` call, but you'd really prefer the logged filename and line number to be from the caller of your function, not your function itself. This is the solution to that problem, at least for Python 2.7. Also see https://stackoverflow.com/questions/12980512/custom-…

	import os
	import logging
	from contextlib import contextmanager

	logging.basicConfig(
	format='%(asctime)-15s %(levelname)s %(filename)s:%(lineno)d %(message)s',
	level=logging.INFO
	)

	def __LINE__():

wware / bash_without_getopt_s.sh

Last active February 1, 2018 21:31

Apparently getopt(1) for Bash is buggy, according to http://mywiki.wooledge.org/BashFAQ/035#getopts, where getopts is OK. But it's not that hard to just write scripts that don't use either of them.

	#!/bin/bash

	help() {
	echo "Bash script that parses cmd line args without getopt(s)"
	echo "Adapted from https://stackoverflow.com/questions/192249"
	echo " -h, --help print this help message"
	echo " -f, --foo set FOO variable"
	echo " -b, --bar set BAR variable"
	echo " -d, --debug enable debug (doesn't really do anything here)"
	echo " <anything else> becomes a positional argument"

wware / memoize.py

Last active May 7, 2018 16:11

Handy memoization decorator for Python functions and methods, also see https://github.com/wware/python-hacks/tree/master/memoize

	import unittest

	_cache = {}


	def memoize(func):
	"""
	Decorator that memoizes a function. The memoized function can use "fresh=True"
	to avoid using cached results.
	@param func: the function to be memoized

wware / pye.sh

Last active July 6, 2017 22:50

A shell script that manages my Python virtualenv, rebuilds and re-pip-installs my code, helps me with pdb debugging and performance profiling.

	#!/bin/bash

	PYTHON=python

	help() {
	echo "This is a virtualenv swiss army knife script."
	echo ""
	echo "Usage: $0 [options] # -S, -s, -i -c"
	echo " $0 [options] foobar.py --arg1 --arg2"
	echo ""

wware / running.py

Created June 27, 2017 03:12

Generate mu, sigma for a stream of numbers, incrementally.

	import collections


	class RunningAverage(object):
	"""
	>>> r = RunningAverage()
	>>> r
	<__main__.RunningAverage object at 0x...>
	>>> r.update(1)
	>>> r.update(2)

wware / perf.py

Last active May 18, 2017 19:35

An ill-designed performance measuring thing for Python, use at your own risk - I'd prefer line_profiler except I'm on Python 2.7 and cannot upgrade.

	import pprint
	import re
	import logging
	import timeit
	import traceback
	from functools import wraps
	from contextlib import contextmanager

	_WIDTH = 50

wware / mem.py

Created March 16, 2017 03:33

On my job, I need to debug what appears to be a memory leak in a Python program that spins up a bunch of workers using the multiprocessing module and then it uses managed queues to send them work and get back results. But the memory is going crazy. This is my attempt to better wrap my head around what Python does with memory.

	# A litle library of memory diagnostics

	# In no particular order of usefulness:
	# https://pymotw.com/2/gc/
	# https://code.tutsplus.com/tutorials/understand-how-much-memory-your-python-objects-use--cms-25609
	# https://www.huyng.com/posts/python-performance-analysis
	# http://stackoverflow.com/questions/23369937

	import gc
	import os

wware / 0README.python-swig

Last active November 28, 2016 15:58

Reminder of how to use SWIG to wrap C functions for use in Python

	Do this:

	make
	gdb -x hook.x `which python`

	More info: http://www.swig.org/Doc1.3/Python.html

wware / HackingAES.py

Last active November 10, 2016 15:23

	#!/usr/bin/env python
	# -- coding: utf-8 --
	import logging
	import random
	import os
	from Crypto.Cipher import AES

	def _make_binary(x):
	for c in list(x):
	assert c in '0123456789ABCDEFabcdef'

wware / SQLAlchemyVibes.md

Last active July 19, 2017 16:22

The Magic of SQLACodeGen

SQLACodeGen is a work of genius. Here is the command you use to generate the model classses for your database. Lately I'm working with MySQL, previously I was working with PostgreSQL.

sqlacodegen mysql://user:password@server/database > mymodels.py

Had I been aware of SQLACodeGen three months ago, I would have used SQLAlchemy for a project where instead I wrote a lot of raw SQL.

Will Ware wware

The Magic of SQLACodeGen