Devon turingDH

turingDH / sparklyr_cv_pipeline_example.R

Created April 5, 2018 05:02 — forked from eddjberry/sparklyr_cv_pipeline_example.R

An example of creating a Spark pipeline with sparklyr

	# Load packages
	library(dplyr)
	library(sparklyr)

	# Set up connect
	sc <- spark_connect(master = "local")

	# Create a Spark DataFrame of mtcars
	mtcars_sdf <- copy_to(sc, mtcars)

turingDH / gist:54fa4d3a712760ccba15ccb7ebaea8d1

Created March 30, 2018 17:34 — forked from conormm/r-to-python-data-wrangling-basics.md

R to Python: Data wrangling with dplyr and pandas

	R to python useful data wrangling snippets

	The dplyr package in R makes data wrangling significantly easier.
	The beauty of dplyr is that, by design, the options available are limited.
	Specifically, a set of key verbs form the core of the package.
	Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe.
	Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R.
	The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package).

	dplyr is organised around six key verbs