Skip to content

Instantly share code, notes, and snippets.

View simonw's full-sized avatar

Simon Willison simonw

17.1k followers · 141 following
View GitHub Profile
@simonw
simonw / python_logging.py
Created December 4, 2014 23:22
How to make Python logging output everything to the console
import logging, sys
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
@simonw
simonw / splunk_user_agents.py
Created January 5, 2015 19:22
Generate splunk query for extracting user agents
browser_mapping = (
('MSIE 7.*Trident/4.0', 'Internet Explorer 8.0'),
('MSIE 6.0', 'Internet Explorer 6.0'),
('MSIE 7.0', 'Internet Explorer 7.0'),
('MSIE 8.0', 'Internet Explorer 8.0'),
('MSIE 9.0', 'Internet Explorer 9.0'),
('MSIE 10.0', 'Internet Explorer 10.0'),
('Trident/7.0; rv:11.0','Internet Explorer 11.0'),
('droid', 'Android'),
('Chrome', 'Chrome'),
@simonw
simonw / example.sql
Created January 27, 2015 22:55
Useful MySQL pattern for quickly finding the most recent 10 rows that match a complex where clause in a giant table
select * from Table FORCE INDEX (primary)
where /* lots of complex where clauses */
order by id desc limit 10;
@simonw
simonw / loading_data.js
Created January 28, 2015 02:57
Using gist.github.com to persist data from JavaScript running on any webpage
jQuery.get('https://api.github.com/gists/3a065d804c44b0149eac', function(data) {
console.log(data.files['file1.txt'].content)
});
@simonw
simonw / eventbrite_to_slack.py
Created March 7, 2015 18:08
Send new eventbrite orders to a Slack channel (Django view function)
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
import requests, json
# Get an OAuth token from https://www.eventbrite.com/myaccount/apps/
# Create a new app, then expand the expandy-thingy and copy out the
# 'Your personal OAuth token' value.
EVENTBRITE_OAUTH_TOKEN = '...'
# Should look something like https://yoursite.slack.com/services/hooks/incoming-webhook?token=xxx...xxx
@simonw
simonw / flatten.py
Last active November 12, 2022 09:08
flatten()/unflatten() - now available as json-flatten on PyPI, see https://github.com/simonw/json-flatten
"""
flatten() and unflatten()
A pair of functions that can convert an arbitrary JSON object into a
flat name/value pair dictionary and back again, preserving type
information and handling both nested lists and nested dictionaries.
For example:
{
@simonw
simonw / csv_example.py
Created July 10, 2015 18:24
Here's how to use Python to output an Excel-compatible CSV (actually TSV) file from a command-line script. You can then use this pattern: "python csv_example.py | pbcopy" to copy the output to your clipboard, then just focus on a cell in Excel or Google Sheets and hit "paste" to copy the data into a bunch of cells.
import csv, sys
def fetch_data():
# Returning some example data - hit a JSON API or something here
return [{
"name": "Name 1",
"age": 32,
"description": "Look, we can use commas and\ttabs & stuff in here!",
}, {
"name": "Name 2",
@simonw
simonw / hello.py
Created July 16, 2015 01:20
A django view in a single file, adapted from https://raw.githubusercontent.com/halfnibble/django-light/master/hello.py
from django.conf import settings
from django.http import HttpResponse
from django.conf.urls import url
settings.configure(
DEBUG = True,
SECRET_KEY = 'yourrandomsecretkey',
ROOT_URLCONF = __name__,
MIDDLEWARE_CLASSES = (),
)
@simonw
simonw / tryout.sh
Created July 24, 2015 23:48
Script for quickly trying out Python modules - run "tryout.sh jsonschema" to pip install jsonschema into your scratch virtualenv and run ipython with that module loaded.
#!/bin/bash
# Create scratch with 'virtualenv --python=python2.7 scratch && scratch/bin/pip install ipython'
scratch/bin/pip install $1
scratch/bin/ipython -c "import $1; print '\nimported $1'" -i
@simonw
simonw / oldest-lines-example-output.txt
Last active May 12, 2025 16:33
How to order the lines in a file by their age according to git blame (useful for checking things like which are the oldest lines in a Python requirements.txt file)
2014-10-22: https://pypi.python.org/packages/source/l/librabbitmq/librabbitmq-1.5.2.tar.gz#md5=842aea204fcfb5d7a541ae72d5ad38bc
2014-10-28: https://pypi.python.org/packages/source/r/raven/raven-3.5.2.tar.gz#md5=a3fe6c823d01cca0802b78d43cc4953a
2014-10-31: https://pypi.python.org/packages/source/e/exam/exam-0.10.5.tar.gz#md5=cb5a5848f3779283054d5556a6c16f55