Edgar Ruiz edgararuiz
edgararuiz
/ spark_vetiver.R
Last active
June 19, 2024 19:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Connect to Spark and create table pointer | |
| library(sparklyr) | |
| library(dplyr) | |
| sc <- spark_connect(method = "databricks_connect") | |
| lendingclub_dat <- tbl(sc, dbplyr::in_catalog("hive_metastore", "default", "lendingclub")) | |
| # Prepare the data, replicating what was done in the first article | |
| lendingclub_prep <- lendingclub_dat |> | |
| select(term, bc_util, bc_open_to_buy, all_util) |> | |
| mutate(term = trimws(substr(term, 1,4))) |> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pkgs <- | |
| c("bonsai", "doParallel", "embed", "finetune", "lightgbm", "lme4", | |
| "plumber", "probably", "ranger", "rpart", "rpart.plot", "rules", | |
| "splines2", "stacks", "text2vec", "textrecipes", "tidymodels", | |
| "vetiver", "remotes", "usethis") | |
| install.packages(pkgs) | |
| if(dir.exists("~/tidymodels")) stop("El folder ya existe") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ```{r setup, include = FALSE} | |
| suppressPackageStartupMessages(library(tidyverse)) | |
| suppressPackageStartupMessages(library(rlang)) | |
| ``` | |
| ```{r, echo = FALSE, results='asis'} | |
| walk( | |
| names(mtcars), | |
| ~{ |