turingDH

turingDH

## Credit to LucidProgramming https://www.youtube.com/watch?v=u2jTn-Gj2Xw for the walkthrough.  My changes weren't terribly significant.  

import os # to get core count on machine
import time  # to time the duration

from multiprocessing import Pool  # to instantiate a Pool of workers to distribute the process across the cores in CPU


def sum_square(number):
    s=0

turingDH

library(dplyr)
library(DT)
library(purrr)
library(tidyr)

phraseToSearch <- 'this|that'

scriptFiles <-
  bind_rows(
    map_dfc('~/R', list.files, full.names = T) %>% rename(fileNames = V1),

turingDH

# Load packages
library(dplyr)
library(sparklyr)

# Set up connect
sc <- spark_connect(master = "local")

# Create a Spark DataFrame of mtcars
mtcars_sdf <- copy_to(sc, mtcars)

turingDH

R to python useful data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. 
The beauty of dplyr is that, by design, the options available are limited. 
Specifically, a set of key verbs form the core of the package. 
Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. 
Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. 
The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package). 

dplyr is organised around six key verbs

turingDH

import numpy as np
import pandas as pd

greeks = [chr(code) for code in range(945,970)]
Greeks = ['alpha', 'beta', 'gamma', 'delta', 'epsilon', 'zeta', 'eta', 'theta', 'iota', 'kappa', 'lambda', 'mu', 'nu', 'xi', 'omicron', 'pi', 'rho', 'word-final sigma', 'sigma', 'tau', 'upsilon', 'phi', 'chi', 'psi', 'omega']

df = pd.DataFrame(greeks, Greeks).reset_index().reset_index()
df.rename(columns={df.columns[0]:"chr_val", df.columns[1]:"greek text", df.columns[2]:"greek symbol"}, inplace=True)
df['chr_val'] += 945
print(df)

turingDH

turingDH