luke’s gists

luke / bulkupsert.py

Last active February 15, 2025 06:26

I needed to upsert (insert or update) bajillions of records into postgresql. After trying various libs including upsert (which was slow as hell) I ended up doing a bit of research and trying 3 different methods. This one won. While I'm manually building the sql string no user data is passed in. Its loaded via the copy from statement as CSV. Call…

	import logging
	import cStringIO
	import csv

	DEBUG = False

	def data2csv(data):
	si = cStringIO.StringIO()
	cw = csv.writer(si, delimiter='\t',lineterminator="\n")
	for row in data:

luke / migrate_zerigo_to_route53.py

Last active February 6, 2016 19:44

Migration script for moving from Zerigo DNS to Amazon Route 53.

	import logging

	import zerigodns
	import boto
	from boto.route53.record import ResourceRecordSets
	from boto.s3.website import RedirectLocation

	# There is no API for these so we have to embed and lookup
	# https://forums.aws.amazon.com/thread.jspa?threadID=116724
	# http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

luke / flask_ses_log_handler.py

Created June 25, 2013 10:28

Special log handler for use in Flask which sends emails using Amazon SES. Wraps stream handler using StringIO Configure with log level and trigger level (at which to send emails)

	#!/usr/bin/env python

	import string, logging
	from logging import StreamHandler
	from flask import g
	import boto
	import StringIO

	DEFAULT_FORMAT = "%(asctime)s %(levelname)-5s %(message)s"
	DEFAULT_SUBJECT = "FlaskBufferingSESHandler log"

luke / picloud_health_check.py

Created July 17, 2013 09:19

Quick and dirty script to check status of last 100 jobs on picloud.

	import logging
	import requests
	import lxml.html
	import collections

	def picloud_health_check(username, password):

	# setup session
	session = requests.Session()
	session.verify = False

luke / test_devices_list.py

Created November 20, 2013 20:38

Make list of uuids not already in provisioning profile. For copying UUIDs out of testflight and into apple.


	import re

	testflight_file = './testflight_devices.txt'
	testflight_data = open(testflight_file).read()

	apple_file = './apple_devices.txt' # copy and paste from provisioning portal
	apple_data = open(apple_file).read()
	apple_uuids = re.findall('[a-z0-9]{30,50}',apple_data)

luke / redis_lua_smembers_hgetall_list.lua

Created July 3, 2014 19:21

lua script to get members of set then return hash for each key

EVAL "local members = redis.call('smembers',KEYS[1])\n local results = {}\n for index,key in ipairs(members) do results[index] = redis.call('hgetall',KEYS[1] .. '/' .. key) end \n return results" 1 accounts

luke / redis_sorted_set_delayed_queue.py

Created April 16, 2015 12:53

	import math
	import logging
	import time
	import random
	from multiprocessing import Process
	from redis import StrictRedis, WatchError

	"""
	RedisSortedSetDelayedQueue - uses zset to keep maintain a time ordered queue of items
	"""

luke / chrome-node-http-server.js

Created September 18, 2015 12:31


	// copy from node 0.12 source lib folder and add to your package.json
	//
	// "browser": {
	// "net": "chrome-net",
	// "http": "./http/http.js",
	// "_http_agent": "./http/_http_agent.js",
	// "_http_client": "./http/_http_client.js",
	// "_http_common": "./http/_http_common.js",
	// "_http_incoming": "./http/_http_incoming.js",

luke / gist:b6c65adae8a559bab11b

Created October 22, 2015 13:41

	-----BEGIN PGP PUBLIC KEY BLOCK-----
	Version: Mailvelope v1.2.2
	Comment: https://www.mailvelope.com

	xsFNBFYo5qUBEACigdbnvIeCSvWnyeX2Vea10OxWkkBk06Y8KCFWTOXRGq4O
	8U9gU3/sX4ub4Fu5wkRlplPEII0xujE0ZhRBrzSheGbS79yQsn1hGelLqV3x
	djJjSER830FcZyELy5iCc0gqD3VJ3bORUm12zradzCpJU2px1w2yaJIuWiaE
	Jg6GQgEu9ftJkycS8tqd9sR8GSLXbSgtbiVuB/8NGlhsob5Qz7Udf6/5uI8D
	3G8OSk5evorVUHwc/XOfBgyq/mG4ygBNINR2f8wGMuZBOY1EEtwHwGjmoV5o
	ABqVnszOMwfZnVAAkcekP/XqEGC+Oygw+kc8uC3jsZEETJ4IvWp1avjQk7BL

luke / youtube_play_mobile_embed.html

Last active September 27, 2016 11:28

	<!DOCTYPE html>
	<html>
	<body>
	<div id="player"></div>

	<script>

	// code taken from yt iframe api example
	var tag = document.createElement('script');
	tag.src = "https://www.youtube.com/iframe_api";

Luke luke