Srikanth K S talegari

Problem: When linking to the raw version of a gist, the link changes with each revision.

Solution:

To return the first file from a gist: https://gist.github.com/[gist_user]/[gist_id]/raw/

Example: https://gist.github.com/atenni/5604522/raw/
Works even when you change the filename.

To get a file from multi–file gist: https://gist.github.com/[gist_user]/[gist_id]/raw/[file_name]

R to python data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. The beauty of dplyr is that, by design, the options available are limited. Specifically, a set of key verbs form the core of the package. Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package).

dplyr is organised around six key verbs:

Explainable Deep Learning

Overview of Explainable Deep Learning

Three major research directions in explainable deep learning: understanding, debugging, and refinement/steering

Model understanding

aims to explain the rationale behind model predictions and the inner workings of deep learning models, and it attempts to make these complex models at least partly understanding

Perturbation experiments (CVPR2014): Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we address both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. We also perform an ablation study to discover the performance contribution from different model layers. This enables us to find model archite

Create cluster

$ gcloud beta dataproc clusters create [CLUSTER-NAME] \
                              --enable-component-gateway \
                              --bucket bucket-name \
                              --region us-central1 \
                              --subnet default \
                              --zone us-central1-a \
                              --master-machine-type n1-standard-4 \

--master-boot-disk-size 500 \

	#!/usr/bin/env bash
	# memusg -- Measure memory usage of processes
	# Usage: memusg COMMAND [ARGS]...
	#
	# Author: Jaeho Shin <[email protected]>
	# Created: 2010-08-16
	############################################################################
	# Copyright 2010 Jaeho Shin. #
	# #
	# Licensed under the Apache License, Version 2.0 (the "License"); #

	#' Create SQL statement from rpart rules
	#'
	#' Rpart rules are changed to sql CASE statement.
	#'
	#' @param df data frame used for rpart model
	#' @param model rpart model
	#' @export
	#' @examples
	#' parse_tree(df=kyphosis,model=rpart(data=kyphosis,formula=Kyphosis~.))
	#' parse_tree(df=mtcars,model=rpart(data=mtcars,formula=am~.))

	.Rproj.user
	.Rhistory
	.RData
	*.Rproj
	*.html


	# ---- Multiple dispatch functions -----
	multi_dispatch <- function(gen_name) {
	calling_env <- parent.frame()
	parent_call <- sys.call(sys.parent())
	calling_fun <- sys.function(sys.parent())

	arg1 <- eval(parent_call[[2]], calling_env)
	arg2 <- eval(parent_call[[3]], calling_env)

	library(dplyr)
	library(readr)
	library(DBI)
	library(RSQLite)

	read.csv2sqlite <- function(csv_file, sqlite_file, table_name, batch_size = 10000) {

	## establish a connection to the database
	condb <- dbConnect(SQLite(), sqlite_file)

	library(tidycensus)
	library(ggiraph)
	library(tidyverse)
	library(patchwork)

	vt_income <- get_acs(
	geography = "county",
	variables = "B19013_001",
	state = "VT",
	year = 2019,