Skip to content

Instantly share code, notes, and snippets.

View simonw's full-sized avatar

Simon Willison simonw

View GitHub Profile
{
"install": "pip3 install --user -r requirements.txt",
"start": "PYTHONUNBUFFERED=true python3 server.py",
"watch": {
"ignore": [
"\\.pyc$"
],
"install": {
"include": [
"^requirements\\.txt$"
@simonw
simonw / pool.py
Created October 5, 2019 17:46
Earlier attempt at Datasette connection pooling, Version from 8am 2nd October 2019
import contextlib
import threading
import sqlite3
class Connection:
def __init__(self, name, db_conn):
self.name = name
self.db_conn = db_conn
self.available = True
@simonw
simonw / log_to_sqlite.py
Created September 30, 2019 16:22
Class for logging structlog entries to a SQLite database
# Use it like this:
#
# structlog.configure(
# processors=[
# structlog.processors.JSONRenderer(),
# LogToSqlite("/tmp/logs.db")
# ]
# )
# log = structlog.get_logger()
# log.msg("say-hello", whom="world", num=[random.randint(1,55)])
@simonw
simonw / starlette_demo.py
Created July 14, 2019 01:37
datasette_auth_github middleware example, from https://simonwillison.net/2019/Jul/14/sso-asgi/
from datasette_auth_github import GitHubAuth
from starlette.applications import Starlette
from starlette.responses import HTMLResponse
import uvicorn
app = Starlette(debug=True)
@app.route("/")
async def homepage(request):
@simonw
simonw / catjq
Created June 22, 2019 19:26
I have this in ~/bin/catjq - chmod 755 - from https://news.ycombinator.com/item?id=20246957
#!/usr/local/bin/jq -rf
tostream
| select(length > 1)
| (
.[0] | map(
if type == "number"
then "[" + tostring + "]"
else "." + .
end
) | join("")
@simonw
simonw / installing-mysqldb.md
Created June 21, 2019 04:17
Installing MySQLdb mysqlclient with Homebrew and pip in OS X Mojave

Installing MySQLdb mysqlclient with Homebrew and pip in OS X Mojave

First I used Homebrew to install the MySQL server along with mysql_config, needed to compile the Python extension:

brew install mysql

I had to then use this incantation to get pip install mysqlclient to work:

LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/ pip install mysqlclient
@simonw
simonw / example-Locations.xml
Last active July 21, 2019 06:19
Convert Locations.kml (pulled from an iPhone backup) to SQLite
<?xml version="1.0" encoding="utf-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
<Placemark>
<TimeStamp>
<when>2015-12-18T19:12:32</when>
</TimeStamp>
<name>2015-12-18 19:12:32 Source: WhatsApp</name>
<Point>
<coordinates>-0.120970480144024,51.510383605957</coordinates>
@simonw
simonw / CSV conf CSV schedule.ipynb
Created May 9, 2019 18:45
Code for scraping the CSVConf schedule. This is pretty messy - I wrote most of it on a plane with no internet connection so I had to get it working against the offline data I had accidentalyl cached.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
# For a sample Starlette app
from starlette.applications import Starlette
from starlette.responses import JSONResponse
import sys
import sqlite3
application = Starlette()
@application.route("/")
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.