Skip to content

Instantly share code, notes, and snippets.

@slopp
slopp / README.md
Last active October 12, 2022 15:16
Airflow Usage Minutes

Dagster Cloud pricing is based on usage. If you are using Airflow, you can get an estimate of this usage by querying the Airflow metadata database.

For SQLite:

# navigate to your airflow DB, normally ~/airflow
sqlite3
.open airflowdb
SELECT SUM((julianday(end_date) - julianday(start_date))* 24 * 60) AS run_minutes FROM dag_run;
@slopp
slopp / README.md
Last active July 14, 2024 16:35
BigQuery and Google Maps

Introduction

With BigQuery's new remote user defined functions (in preview) it is now possible to bring the power of Google Maps to your analytic data warehouse. Using Google Maps API endpoints in Cloud Functions called by BigQuery you can:

  • Geocode Addresses
  • Determine drive time distance between locations
  • Supplementing address or location data with Google Map's data such as elevation or place descriptions

By enriching location datasets in BigQuery you can accomplish advanced spatial analysis including:

@slopp
slopp / violin.js
Last active December 21, 2021 22:00
test of d3 custom viz for looker
!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.violin=t():e.violin=t()}(window,(function(){return function(e){var t={};function n(r){if(t[r])return t[r].exports;var o=t[r]={i:r,l:!1,exports:{}};return e[r].call(o.exports,o,o.exports,n),o.l=!0,o.exports}return n.m=e,n.c=t,n.d=function(e,t,r){n.o(e,t)||Object.defineProperty(e,t,{enumerable:!0,get:r})},n.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},n.t=function(e,t){if(1&t&&(e=n(e)),8&t)return e;if(4&t&&"object"==typeof e&&e&&e.__esModule)return e;var r=Object.create(null);if(n.r(r),Object.defineProperty(r,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var o in e)n.d(r,o,function(t){return e[t]}.bind(null,o));return r},n.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return
@slopp
slopp / rhel_7_rpm_deps.md
Created August 20, 2021 17:00
A list of dependencies required for RStudio Team

If you are installing RStudio Team onto an air-gapped RHEL environment you may need to first install  additional system dependencies before you will be able to install the RStudio product rpms. Some of these dependencies are available through the standard RedHat yum repo, others may need to be pulled from the EPEL.

The following list names the dependencies by product. The list is based off of RedHat 7 rpms for the corresponding version of RStudio Team.

RStudio Workbench version 1.4.1717-3

rrdtool
sqlite

@slopp
slopp / README.md
Created June 21, 2021 14:04
Python Install

Install Python

This script is designed to make it easy to install multiple versions of Python onto a Linux system. The resulting versions are stored side-by-side in /opt/Python/

Instructions

  1. Copy the bash script to install-python,sh
  2. Make the script executable: sudo chmod +x install-python.sh
  3. Run the script for each desired python version, e.g. sudo ./install-python.sh 3.8.3
@slopp
slopp / penguins.csv
Created March 31, 2021 20:07
Palmer Penguins Dataset as CSV
rowid species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g sex year
1 Adelie Torgersen 39.1 18.7 181 3750 male 2007
2 Adelie Torgersen 39.5 17.4 186 3800 female 2007
3 Adelie Torgersen 40.3 18 195 3250 female 2007
4 Adelie Torgersen NA NA NA NA NA 2007
5 Adelie Torgersen 36.7 19.3 193 3450 female 2007
6 Adelie Torgersen 39.3 20.6 190 3650 male 2007
7 Adelie Torgersen 38.9 17.8 181 3625 female 2007
8 Adelie Torgersen 39.2 19.6 195 4675 male 2007
9 Adelie Torgersen 34.1 18.1 193 3475 NA 2007
@slopp
slopp / steps.md
Last active January 12, 2021 23:14
Remote "local" k8s development
@slopp
slopp / housing.csv
Created October 26, 2020 02:42
AmesHousing
We can't make this file beautiful and searchable because it's too large.
Order,PID,MS SubClass,MS Zoning,Lot Frontage,Lot Area,Street,Alley,Lot Shape,Land Contour,Utilities,Lot Config,Land Slope,Neighborhood,Condition 1,Condition 2,Bldg Type,House Style,Overall Qual,Overall Cond,Year Built,Year Remod/Add,Roof Style,Roof Matl,Exterior 1st,Exterior 2nd,Mas Vnr Type,Mas Vnr Area,Exter Qual,Exter Cond,Foundation,Bsmt Qual,Bsmt Cond,Bsmt Exposure,BsmtFin Type 1,BsmtFin SF 1,BsmtFin Type 2,BsmtFin SF 2,Bsmt Unf SF,Total Bsmt SF,Heating,Heating QC,Central Air,Electrical,1st Flr SF,2nd Flr SF,Low Qual Fin SF,Gr Liv Area,Bsmt Full Bath,Bsmt Half Bath,Full Bath,Half Bath,Bedroom AbvGr,Kitchen AbvGr,Kitchen Qual,TotRms AbvGrd,Functional,Fireplaces,Fireplace Qu,Garage Type,Garage Yr Blt,Garage Finish,Garage Cars,Garage Area,Garage Qual,Garage Cond,Paved Drive,Wood Deck SF,Open Porch SF,Enclosed Porch,3Ssn Porch,Screen Porch,Pool Area,Pool QC,Fence,Misc Feature,Misc Val,Mo Sold,Yr Sold,Sale Type,Sale Condition,SalePrice
1,0526301100,020,RL,141,31770,Pave,NA,IR1,Lvl,AllPub,Corner,Gtl,NAmes,Norm
@slopp
slopp / check.R
Created May 27, 2020 17:02
Check PPM for a binary
library(shiny)
library(httr)
check_binary <- function(pkg,
version,
distro, # one of windows, xenial, bionic, centos7, centos8, opensuse42, opensuse15
r_version,
repo_base = "https://packagemanager.rstudio.com/cran/",
verbose = FALSE){
@slopp
slopp / license_search.R
Created April 23, 2020 15:29
Get packages by license
library(dplyr)
library(tibble)
REPO <- "http://cran.rstudio.com"
LICENSE <- "AGPL"
p <- available.packages(repos = REPO)
p_tbl <- as_tibble(p)
p_tbl %>%
filter(License == !!LICENSE) %>%