Kristian Alexander krisalexander200

krisalexander200 / spark-with-sql.py

Created January 13, 2016 18:36

Spark SQL with Python

	from pyspark import SparkConf, SparkContext
	from pyspark.sql import SQLContext, Row
	import collections

	conf = SparkConf().setMaster("local").setAppName("RatingsHistogram")
	sc = SparkContext(conf = conf)
	sqlContext = SQLContext(sc)

	def mapper(line):
	li = line.split(',')