Skip to content

Instantly share code, notes, and snippets.

View dantonnoriega's full-sized avatar

Danton Noriega-Goodwin dantonnoriega

View GitHub Profile
@dantonnoriega
dantonnoriega / get-etf-holdings-data.R
Last active March 4, 2024 19:26
[development] prototype functions to programmatically pull ETF holdings (portfolio) data; likewise which ETFs hold a specific stock and its allocation (data source: etf.com)
# SETUP -----------------
library(httr2)
api_base_url = "https://api-prod.etf.com/private"
hdrs <- list(
`x-limit` = 10000,
`User-Agent` = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15',
`Origin` = 'https://www.etf.com',
`Referer` = 'https://www.etf.com/',
`Accept` = '*/*',
@dantonnoriega
dantonnoriega / example_simulated-difference-in-differences.py
Last active April 9, 2024 06:27
A simple example of a difference-in-difference analysis in python using simulated data
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
# Set random seed for reproducibility
np.random.seed(42)
# Generate synthetic data
n_obs = 100
@dantonnoriega
dantonnoriega / est-tax-etrade-GL-collapsed.md
Last active April 11, 2024 15:27
instructions on how to download gains and losses sheet from etrade and then estimate capital gains tax

Quick Way to Estimate Taxes Owed on Capital Gains (E*Trade)

This short explainer goes through

  1. where to download your Gains and Losses sheet on one E*Trade account.
  2. how to quickly estimate taxes owned.

(1) Downloading Gains & Losses Sheet (G&L_collapsed.xlsx) from E*Trade

@dantonnoriega
dantonnoriega / zsh_history_cat.md
Last active October 26, 2022 05:45
how to append old zsh_history files into a single one, keeping things in order, and updating $HISTFILE

Helpful Resources

WARNING: using fc -p will replace your current history file with a blank one. Tread carefully.

GOAL: merge (or concat or join or append) a set of old zsh_history files into a single one.

Imagine you have 3 zsh history files, with timestamps from oldest to newest: zsh_history0, zsh_history1, zsh_history2.

@dantonnoriega
dantonnoriega / databricks-jdbc-instructions.md
Created March 5, 2022 02:09
instructions for using JDBC SIMBA drivers in R to connect to Databricks

Databricks - JDBC

This is the preferred method if you're collecting small objects from the Spark Cluster because it is otherwise easy to set up, doesn't appear to cause many "out of memory" errors, and requires fewer dependencies.

Setup

  1. Install and configure Java.
    • Install Java 8

brew tap AdoptOpenJDK/openjdk

@dantonnoriega
dantonnoriega / rsc_app_specific_total_and_unique_user_visits_reprex.R
Created February 15, 2022 20:52
an example of pulling total and unique visits for a specific Rstudio Connect application Raw
# The CONNECT_SERVER URL must have a trailing slash.
connectServer <- "https://some-rsc-endpont.com"
connectAPIKey <- Sys.getenv("RSC_API_KEY")
# the content GUID of interest
content_guid_3pc <- "0a0b1c2d-1234-4321-a1c2-5678aaa9d9bb" # I made this one up
resp <- httr::GET(
paste0(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=100"),
httr::add_headers(Authorization = paste("Key", connectAPIKey)),
@dantonnoriega
dantonnoriega / rsc_user_activity_reprex.R
Last active February 15, 2022 20:53
a script for determining the number of active users in some RSC instance
# The CONNECT_SERVER URL must have a trailing slash.
connectServer <- "https://some-rsc-url.com/"
# Save RSC API key to ~/.Renviron as RSC_API_KEY
# - https://docs.rstudio.com/connect/1.7.4/user/cookbook.html
connectAPIKey <- Sys.getenv("RSC_API_KEY")
# Request a page of up to 25 usage records.
resp <- httr::GET(
file.path(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=100"),
httr::add_headers(Authorization = paste("Key", connectAPIKey))
@dantonnoriega
dantonnoriega / random-datatable-if_any-testing.R
Created February 10, 2022 23:09
a simple script that tests out different was of implementing the `if_any` logic in native data.table
library(tidyverse)
dat <- as_tibble(mtcars) %>%
mutate(vs = as.character(vs),
am = as.character(am)) #just to make some non-numeric
dd0 <- dat %>%
select(where(is_numeric)) %>%
filter(if_any(disp:wt, ~ .x > 100))
dd0
library(data.table)
@dantonnoriega
dantonnoriega / carryforward.R
Created September 2, 2021 19:02
base R way to carry forward data
carryforward <- function(x, blank = is.na) {
# SOURCE: https://stackoverflow.com/a/32536507/3987905
# Find the values
if (is.function(blank)) {
isnotblank <- !blank(x)
} else {
isnotblank <- x != blank
}
# Fill down
x[which(isnotblank)][cumsum(isnotblank)]
@dantonnoriega
dantonnoriega / gitlfs-github-api-content-extraction.sh
Created September 2, 2021 19:00
Steps to use the git-lfs Batch API to download content from GitHub (enterprise) when the raw content is actually on git-lfs
# github + git lfs api contenct extraction
# GOAL: pull a large file that was sent to git lfs (cannot download directly from github.com)
## git-lfs API: https://github.com/git-lfs/git-lfs/tree/main/docs/api
## github API: https://docs.github.com/en/rest/reference/repos#get-repository-content
OWNER=dantonnoriega
REPO=some-repo
BRANCH=develop
# -----------------