Nathan Milford nmilford

nmilford / splash_metrics.py

Created June 2, 2016 17:12

Splash metric collection

	#!/usr/bin/env python
	from statsd import StatsClient
	import requests
	import json

	statsd = StatsClient()
	url = 'http://127.0.0.1:8050/_debug'
	resp = requests.get(url=url)
	metrics = json.loads(resp.text)

nmilford / knife_completion.sh

Created September 7, 2016 03:04

knife bash completion with knife ec2 plugin completion (extended from https://github.com/stevendanna/knife-hacks)

nmilford / deps.sh

Created August 9, 2017 13:27

Create Native deb packages for Splash 3.0 dependencies for Ubuntu 16.04

	# Make sure to have fpm installed as well as dependancies described here: https://github.com/scrapinghub/splash/blob/master/dockerfiles/splash/provision.sh

	# Environment Setup
	sudo mkdir -p /tmp/{downloads,build,target}
	sudo mkdir -p /tmp/{downloads,build,target}

	# Build qt-5.9.1 deb
	sudo curl -L -o /tmp/downloads/qt-installer-noninteractive.qs https://raw.githubusercontent.com/scrapinghub/splash/master/dockerfiles/splash/qt-installer-noninteractive.qs
	sudo sed -i 's\|/opt/qt59\|/tmp/target/opt/qt59\|g' /tmp/downloads/qt-installer-noninteractive.qs
	sudo curl -L -o /tmp/downloads/qt-installer.run http://download.qt.io/official_releases/qt/5.9/5.9.1/qt-opensource-linux-x64-5.9.1.run

nmilford / performline_example.py

Last active June 6, 2018 18:05

Sample script to build a report from PerformLine API and Export

	#!/usr/bin/env python
	import performline.embedded.stdlib.clients.rest.exceptions
	from performline.client import Client
	from glob import iglob
	import argparse
	import json
	import sys

	class PlineExportReport(object):
	def __init__(self, token=None, path='.'):

nmilford / build_redis_deb.sh

Last active June 4, 2018 18:35

redis 5.0 deb with rebloom and redis graph

	# Create build root
	mkdir -p /tmp/build/etc/redis/
	mkdir -p /tmp/build/opt/redis/data/
	mkdir -p /tmp/build/opt/redis/modules/
	mkdir -p /tmp/build/etc/systemd/system/

	# Get Redis
	wget https://github.com/antirez/redis/archive/5.0-rc1.tar.gz
	tar zxvf 5.0-rc1.tar.gz
	cd redis-5.0-rc1/

nmilford / api-batching-to-csv.py

Created September 3, 2019 20:41

Example on how to use limit/offset to collect and export data from the PerformLine API in bulk.

	# This code demonstrates how to loop through an API using limit+offset.
	#
	# It will get all of the results as it pages through the data by limit and
	# offset preventing the backend that serves it from getting overwhelemed.
	#
	# It will then put all of that collected data into a CSV for later processing.
	#
	# This code was thrown together as a demonstration, no warrenties are given or
	# implied.
	import requests