simonw

Configuring CORS on Amazon S3 for direct image upload from JavaScript

I needed to add CORS headers so that client-side JavaScript could upload images directly to an S3 bucket.

I used the Amazon S3 console tool https://console.aws.amazon.com/s3/home?region=us-east-1#&bucket=name-of-bucket&prefix=

Permissions -> Add CORS Configuration

simonw

Here's the Datasette instance: https://washington-post-school-shootings.now.sh - using data from https://github.com/washingtonpost/data-school-shootings

$ wget https://raw.githubusercontent.com/washingtonpost/data-school-shootings/master/school-shootings-data.csv

I manually edited the first line of the CSV file, renaming the lat column to latitude and the long column to longitude. This is needed to get the datasette-cluster-map plugin to work correctly.

Then I created the SQLite database using csvs-to-sqlite, specifing specific fields that should be added to a full-text search index:

$ csvs-to-sqlite school-shootings-data.csv school-shootings-data.db \

-f school_name -f district_name -f city -f state -f county

simonw

<link rel="stylesheet" href="http://leaverou.github.io/awesomplete/awesomplete.css">
<script src="http://leaverou.github.io/awesomplete/awesomplete.js"></script>
<script>
document.addEventListener('DOMContentLoaded', () => {
    const input = document.querySelector('input.blah');
    const awesomplete = new Awesomplete(input);
    input.addEventListener("awesomplete-selectcomplete", (ev) => console.log(ev.text));
    input.addEventListener('keyup', (e) => {
        var code = (e.keyCode || e.which);      
        if (code === 37 || code === 38 || code === 39 || code === 40 || code === 27 || code === 13) {

simonw

import json
import os
import sqlite3


def init_db(filename):
    if os.path.exists(filename):
        return

    conn = sqlite3.connect(filename)

simonw

Installing the Presto command-line tool

I had to install Java to get presto working

brew cask install java

From https://prestodb.io/docs/current/installation/cli.html I did:

wget https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.202/presto-cli-0.202-executable.jar

mv presto-cli-0.202-executable.jar presto

simonw

Really basic load test

I ran Apache Bench like so:

ab -n 1000 -c 100 'http://127.0.0.1:8000/'

uvicorn

Server Software:        uvicorn
Server Hostname: 127.0.0.1

simonw

Notes on using rbenv to install a gem application without needing root

I don't like running sudo gem install X - here's how I installed a gem packaged app without it.

Install rbenv using homebrew

brew install rbenv

Download Ruby 2.5.1 using it

simonw

Installing hackable Python packages

This is a handy recipe for installing a Python package in a way that makes it easy to modify locally:

Assuming you have a virtualenv in venv/:

source venv/bin/activate
pip install -e git+git@github.com:simonw/datasette-cluster-map#egg=datasette-cluster-map

This will install code from that GitHub repository in the venv/src/ folder and set it up so changes you make there will be visible to other code in that virtual environment.

simonw

How to update an existing vega view with new fetched data

This took quite a bit of figuring out because the web is littered with Vega 2 examples but Vega 3 is a bit different.

https://vega.github.io/vega/docs/api/view/#view_change

var spec = {
  data: {"url": "https://fivethirtyeight.datasettes.com/fivethirtyeight-45d758d/most-common-name%2Fsurnames.json?_shape=array"},
  mark: "bar",

simonw

{
  "data": {
    "url": "https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators.csv?_size=max&_sort_desc=replies",
    "format": {
      "type": "csv"
    }
  },
  "mark": "bar",
  "encoding": {
    "x": {