Skip to content

Instantly share code, notes, and snippets.

View tslumley's full-sized avatar
😐

Thomas Lumley tslumley

😐
View GitHub Profile
@tslumley
tslumley / README.md
Created December 3, 2017 20:58
Non-transitive dice

An example of the non-transitivity of the Wilcoxon test, using Efron's non-transitive dice.

refron() generates numbers from a set of four non-transitive dice, so that a beats b beats c beats d beats a two-thirds of the time, with optional Gaussian smoothing.

example.R does Wilcoxon rank-sum tests to show that a > b > c > d > a

<!DOCTYPE html>
<html lang="en" >
<head>
<meta charset="UTF-8">
<title>Epiphany Clock</title>
<link rel='stylesheet prefetch' href='http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css'>
@tslumley
tslumley / wellybus.R
Last active August 13, 2018 22:48
Slowly accumulating map of Wellington buses (rate-limited to 1 route per 40sec)
library(tidyverse)
library(leaflet)
library(jsonlite)
library(ratelimitr)
library(htmlwidgets)
library(htmltools)
allroutes<-read_csv("~/Downloads/WLG-google-transit/routes.txt") %>% filter(route_type==3)
download_route<-limit_rate(
@tslumley
tslumley / read-glove.R
Last active September 23, 2018 06:38
Read GloVe word embeddings
# Based on https://gist.github.com/tjvananne/8b0e7df7dcad414e8e6d5bf3947439a9
# Rewritten to work chunk by chunk, so I can read the 42B file with only 8GB memory
# input .txt file, exports list of list of values and character vector of names (words)
proc_pretrained_vec <- function(filename, chunksize=1000, guess_size=100000) {
size<-guess_size
here<-0
# initialize space for values and the names of each word in vocab
vals <- vector(mode = "list", length(size))
@tslumley
tslumley / app.R
Created January 4, 2019 01:13
Shiny app for exploring posterior distributions given surprising data
library(shiny)
# Define UI for application that draws a histogram
ui <- fluidPage(
# Application title
titlePanel("Bayesian Surprise"),
# Sidebar with a slider input for number of bins
@tslumley
tslumley / withPV.R
Created April 21, 2019 00:56
Plausible values in surveys
withPV<-function(mapping, design, action, ...) UseMethod("withPV",design)
withPV.survey.design<-function(mapping, design, action,...){
if(inherits(mapping,"formula")) mapping<-list(mapping)
if (!is.list(mapping))
stop("'mapping' must be a list of formulas")
if (!all(sapply(mapping, length)==3))
@tslumley
tslumley / net-real-estate.R
Created May 2, 2019 23:41
Gross and net transfers of NZ real estate.
## from: https://www.stats.govt.nz/information-releases/property-transfer-statistics-march-2019-quarter
re<-read.table(text=
"year quarter citbuy citsell resbuy ressell fornbuy fornsell corpbuy corpsell knowbuy knowsell unkbuy unksell total
2017 Mar 23592 23490 2130 1101 621 375 3060 4407 29406 29370 4287 4320 33690
2017 Jun 30414 30678 3063 1512 930 486 4188 5946 38595 38625 450 426 39048
2017 Sep 27123 27021 2703 1458 783 441 3618 5316 34230 34239 126 117 34356
2017 Dec 28632 28125 2862 1491 1038 468 3663 6111 36195 36192 84 87 36279
2018 Mar 25881 25947 2625 1401 1083 501 3255 4998 32841 32847 39 36 32880
2018 Jun 31044 31173 3171 1656 1116 492 4281 6285 39606 39606 21 21 39627
2018 Sep 28284 27684 2982 1557 717 378 3630 5997 35613 35613 21 21 35634
@tslumley
tslumley / bus-bot-school.R
Created May 3, 2019 00:56
Real-time board for Auckland school buses
## startup
library(jsonlite)
library(httr)
library(knitr)
library(kableExtra)
options(stringsAsFactors=FALSE)
## These two files are part of the static GTFS information
routes<-read.csv("./routes.txt")
routes$route_id<-substr(as.character(routes$route_id),1,5) ## remove versioning info
@tslumley
tslumley / hoffci.R
Last active June 7, 2019 03:40
Perverting Burris-Hoff confidence intervals for fun and profit.
## https://doi.org/10.1093/jssam/smz010
make.s<-function(mu, sigma,tau2,alpha){
g<-make.g(alpha)
invg<-make.ginv(alpha)
function(theta) invg(2*sigma*(theta-mu)/tau2)
}
make.g<-function(alpha){
@tslumley
tslumley / svyivreg.R
Created July 15, 2019 01:04
survey-weighted two-stage least squares for instrumental variables
library(AER)
svyivreg<-function(formula, design, ...) UseMethod("svyivreg",design)
svyivreg.survey.design<-function(formula, design){
.data<-model.frame(design)
.data$.weights<-weights(design,"sampling")
model<- ivreg(formula, data=.data, weights=.weights)