Jas Sohi j450h1

Eurostat, World Bank and others: https://ikashnitsky.github.io/2017/data-acquisition-two/
Star Wars data: in dplyr, http://dplyr.tidyverse.org/reference/starwars.html
Baby names data: babynames package, https://cran.r-project.org/web/packages/babynames/index.html
Movies data: https://cran.r-project.org/web/packages/ggplot2movies/index.html ggplot2movies package
Game of Thrones screen time: https://github.com/Preetish/GoT_screen_time
Open Bike Data: https://github.com/ropensci/bikedata
Tons of data through 538: https://cran.r-project.org/web/packages/fivethirtyeight/vignettes/fivethirtyeight.html
Public health data England: fingertipsR, https://cran.r-project.org/web/packages/fingertipsR/
Financial data via Quandl: https://www.quandl.com/tools/r
Cyclones: https://github.com/ropensci/rrricanesdat

Using Selenium and Python to screenshot a javascript-heavy page

As websites become more JavaScript heavy, it's harder to automate things like screenshotting for archival purposes. I've seen examples and suggestions to use PhantomJS for visual testing/archiving of websites, but have run into issues such as the non-rendering of webfonts. I've never tried out Selenium until today...and while I'm not thinking about performance implications yet, Selenium seems far more accurate than PhantomJS...which makes sense since it actually opens a real browser. And it's not too hard to script to do complex interactions: here's an [example of how to log in to Twitter, write a tweet, upload an image, and send a tweet via Selenium and DOM element selection](https://gist.github.com/dannguyen/8a6fa49253c1d6a0eb92

	library(tidyverse)
	library(paletteer)
	library(gt)

	pizzaplace %>%
	mutate(type = case_when(
	type == "chicken" ~ "chicken (pizzas with chicken as a major ingredient)",
	type == "classic" ~ "classic (classical pizzas)",
	type == "supreme" ~ "supreme (pizzas that try a little harder)",
	type == "veggie" ~ "veggie (pizzas without any meats whatsoever)",

	def import_to_big_query(data, context, dataset='UPDATE_DATASET_HERE', table='UPDATE_TABLE_HERE', verbose=True):
	def vprint(s):
	if verbose:
	print(s)

	vprint('Event ID: {}'.format(context.event_id))
	vprint('Event type: {}'.format(context.event_type))
	vprint('Importing required modules.')

	from google.cloud import bigquery

	#!/bin/bash

	SOURCE="$1"
	if [ "${SOURCE}" == "" ]; then
	echo "Must specify a source url"
	exit 1
	fi

	DEST="$2"
	if [ "${DEST}" == "" ]; then

	# Setup Ubuntu
	sudo apt update --yes
	sudo apt upgrade --yes

	# Get Miniconda and make it the main Python interpreter
	wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
	bash ~/miniconda.sh -b -p ~/miniconda
	rm ~/miniconda.sh

	export PATH=~/miniconda/bin:$PATH

	# Day 0: Mean, Median, and Mode

	# Selecting standard input
	con <- file('stdin', open='r')

	# We don't need the first input
	data_line <- readLines(con)[[2]]

	# splitting the data into individual string
	split_data <- strsplit(data_line, " ")

	-- Active issues
	-- Count of total active issues in the specified time frame
	-- Source: githubarchive public data set via Google BigQuery http://githubarchive.org/

	SELECT
	COUNT(DISTINCT JSON_EXTRACT_SCALAR(events.payload, '$.issue.id')) AS events_issue_count
	FROM (SELECT * FROM TABLE_DATE_RANGE([githubarchive:day.],TIMESTAMP('2015-09-01'),TIMESTAMP('2016-08-31')))
	AS events

	-- 10,723,492 active issues

	#' Email a user a report is ready
	#'
	#' Requires an account at Mailgun: https://mailgun.com
	#' Pre-verification can only send to a whitelist of emails you configure
	#'
	#' @param email Email to send to
	#' @param mail_message Any extra info
	#'
	#' @return TRUE if successful email sent
	#' @import httr

	# BB-8 --------------------------------------------------------------------

	# Inspired by Brian Hough in http://codepen.io/bhough/pen/wawrPL



	# Packages ----------------------------------------------------------------

	library("dplyr")
	library("ggplot2")