Michael Chirico MichaelChirico
MichaelChirico
/ xml_read_efficiency.R
Created
September 6, 2020 03:59
Efficiency of reading XML documents with relative/absolute addresses
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| test_xml = ' | |
| <div> | |
| <div> | |
| <div> | |
| <div> | |
| <p>1</p> | |
| <p>2</p> | |
| <p>3</p> | |
| </div> | |
| </div> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # NB: not fully reproducible since I've had to censor some stuff / obscure credentialling | |
| library(data.table) | |
| mtcars = cbind(car = rownames(mtcars), mtcars) | |
| mtcars = lapply(mtcars, function(x) if (is.character(x)) sQuote(x, "'") else x) | |
| colnames = toString(names(mtcars)) | |
| mtcars$sep = ', ' | |
| # \n\t are bells&whistles to make cat(query) look nicer | |
| mtcars$collapse = '),\n\t(' | |
| query = sprintf("select * from (values\n\t(%s)\n)\nt(%s)", do.call(paste, mtcars), colnames) | |
| presto_rest_endpoint = file.path('/path/to/rest', 'statement') |
MichaelChirico
/ posix_java_switch.R
Created
July 6, 2020 09:57
Translate between Java SimpleDateFormat and POSIX time format
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # sources: | |
| # https://prestodb.io/docs/current/functions/datetime.html | |
| # https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html | |
| # NB: when multiple matches are possible (e.g. %h=%I), the higher one is "preferred" | |
| # NB: %r/%T are basically blocked out from being returned Java->Presto by putting | |
| # them at the bottom, as a way of preferring the more verbose "full" form | |
| # NB: the following supported formats don't have an exact equivalent: %w, %x, %v | |
| time_fmt_mapping = fread('posix,java | |
| %Y,yyyy | |
| %y,yy |
MichaelChirico
/ loan_repayments_random_search
Last active
May 31, 2020 13:56
A simple simulation of loan repayments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| P0 = c(...) # initial principal | |
| R = c(...) # annualized interest | |
| r = (1+R)^(1/12)-1 # monthly interest | |
| np = length(P0) | |
| X = ... # monthly willingness-to-pay | |
| BB = 1e7 # repetitions | |
| # slightly more principled... find how many months it takes if |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(data.table) | |
| library(microbenchmark) | |
| get_reduce = function(FUN, ELEMENT) { | |
| function(..., na.rm=FALSE) { | |
| l = list(...) | |
| if (length(l) == 1L && is.list(l[[1L]])) l = l[[1L]] | |
| if (length(l) == 1L && (identical(FUN, `|`) || identical(FUN, `&`))) l[[1L]] = as.logical(l[[1L]]) | |
| if (na.rm) { | |
| # TODO: nafill to support complex input, then use nafill here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(data.table) | |
| library(rtweet) | |
| library(ggplot2) | |
| library(ggrepel) | |
| nyt_words = get_timeline( | |
| 'NYT_first_said', | |
| n = 900, | |
| exclude_replies = TRUE, | |
| include_rts = FALSE |
MichaelChirico
/ write_type_regression_test.R
Last active
April 30, 2020 06:35
Regression tests for writeObject & writeType refactor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| old_env = new.env() | |
| URL = 'https://raw.githubusercontent.com/apache/spark/master/R/pkg/R/serialize.R' | |
| source(URL, old_env) | |
| new_env = new.env() | |
| # run from ./R/pkg on the write_type_object_s3 branch | |
| source('R/serialize.R', new_env) | |
| # missing: jobj | |
| test_objects = list( |
MichaelChirico
/ rproj
Created
April 16, 2020 03:30
open the .Rproj file in a directory, e.g. `rproj ~/path/to/dir`; no arguments = PWD
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| if [ $# = 0 ] | |
| then | |
| check_dir=. | |
| else | |
| check_dir=$1 | |
| fi | |
| rproj_files=$(ls -1 $check_dir/*.Rproj 2> /dev/null) |
MichaelChirico
/ euler_spiral.R
Last active
April 13, 2020 18:24
Playing with the idea in https://twitter.com/matthen2/status/1249611168265547776
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(data.table) | |
| library(viridisLite) | |
| NN = 1e6 | |
| col = viridis(NN - 1L) | |
| ll = 1 | |
| points = data.table(x = numeric(NN), y = numeric(NN)) | |
| # one cumsum accumulates the rotational angle, the other cumsum | |
| # accumulates the accumulated angle | |
| points[ , theta := cumsum(cumsum(rep(runif(1L, 0, pi/2), .N)))] |
MichaelChirico
/ ft_nyt_covid_plots.R
Last active
July 23, 2020 12:20
Generate Financial Times-like COVID plots for US states by county using NYT county-level data
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(data.table) | |
| library(colourvalues) | |
| library(RColorBrewer) | |
| # palette from: https://www.color-hex.com/color-palette/40131 | |
| # raw = c('#eeaf61', '#fb9062', '#ee5d6c', '#ce4993', '#6a0d83') | |
| # bump_col = function(col, n) do.call(rgb, as.list(pmax(pmin((col2rgb(col) + n)/256, 1), 0))) | |
| # dput(sapply(raw, function(col) c(bump_col(col, 16), bump_col(col, -16)))) | |
| sunset = c( | |
| "#FDBE71", "#DD9E51", | |
| "#FF9F72", "#EA8052", |