Eric Leung erictleung

Spark, R, and Databricks Notes

Here are some notes on using R on Databricks and making use of the underlying Spark tooling.

Contents

date	alt	tree	id	bloom
2025-04-14	Prunus ‘Taki-nioi’	taki_nioi	163	First Bloom
2025-04-14	Prunus pendula ‘Pendula Rosea ’	pendula	128	Peak Bloom
2025-04-14	Prunus pendula ‘Pendula Plena Rosea’	yae_beni_shidare	126	Peak Bloom
2025-04-14	Prunus × sieboldii	sieboldii	160	First Bloom
2025-04-14	Prunus ‘Hata-zakura’	hatazakura	106	First Bloom
2025-04-14	Prunus ‘Ariake’	ariake	154	First Bloom
2025-04-14	Prunus ‘Ukon’	ukon	162	Prebloom
2025-04-14	Prunus × sieboldii	sieboldii	161	First Bloom
2025-04-14	Prunus ‘Fudan-zakura’	fudan_zakura	107	Post-Peak Bloom

	library(tidyverse)
	library(rvest)
	library(ggtext)
	library(extrafont)

	rt <- read_html("https://editorial.rottentomatoes.com/guide/best-christmas-movies/")

	film_regex <- regex(
	"^([A-Za-z0-9.' ,-:&]* \\([0-9]{4,4}\\))\\n\\s+([0-9]{0,4})\\%\\n\\s+([0-9]{0,4})\\%"
	)

	#!/usr/bin/env python3

	"""
	Help audit, remove, and update musician infoboxes.

	https://en.wikipedia.org/wiki/Category:Pages_using_infobox_musical_artist_with_associated_acts
	"""

	import re
	import requests

	#!/usr/bin/env bash

	# Description: Extract unique URLs from the plain text output from Zoom chat and
	# save links to new file.
	# Usage: bash extract_zoom_urls.sh

	# Set path to where Zoom chat is, or `cd` to the directory itself
	FILE=meeting_saved_chat.txt

	# Get list of unique links shared in Zoom chat

	# Follow instructions here for more:
	# https://exercism.io/cli-walkthrough
	# Releases:
	# https://github.com/exercism/cli/releases

	# Download and move around
	# Change URL as needed for new versions
	wget https://github.com/exercism/cli/releases/download/v3.1.0/exercism-3.1.0-linux-x86_64.tar.gz
	mkdir bin
	mv exercism bin/

	# Make logo
	library(hexSticker)
	library(showtext)

	# Add Google Font
	font_add_google(name = "Open Sans", family = "Open Sans")
	showtext_auto() # Use this font in all rendering

	# Picture of book that was quickly screenshot
	imgurl <- "~/Downloads/Screenshot 2022-06-13 092720.png"

	library(dplyr)
	library(stringr)
	library(ggplot2)

	d <- rtweet::search_tweets("from:stats_feed", n = 30) %>%
	filter(status_id == "1521871339908263937") %>%
	pull(text) %>%
	str_split("\n") %>%
	as_tibble(.name_repair = "universal") %>%
	rename("data" = "...1") %>%