Gabriel Ciuloaica devsprint

Advanced Functional Programming with Scala - Notes

1. Mastering Functions

A function is a mapping from one set, called a domain, to another set, called the codomain. A function associates every element in the domain with exactly one element in the codomain. In Scala, both domain and codomain are types.

val square : Int => Int = x => x * x

Introduction to Kafka

Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.

Terminology:

Producers send messages to brokers
Consumers read messages from brokers
Messages are sent to a topic

slow

This bash script offers quick shortcuts to simulate slower network connections. It is useful when you need to simulate a wireless network on a Linux network server, especially when you are using a virtual machine guest on your local machine or in the cloud.

slow 3G                   # Slow network on default eth0 down to 3G wireless speeds
slow reset                # Reset connection for default eth0 to normal
slow vsat --latency=500ms # Simulate satellite internet  with a high latency
slow dsl -b 1mbps         # Simulate DSL with a slower speed than the default

slow modem-56k -d eth0 # Simulate a 56k modem on the eth1 device. eth0 is unchanged.

	object ServerSparkContext {

	private[this] lazy val _sqlContext = {
	val conf = new SparkConf()
	.setAppName("....")
	val sc = new SparkContext(conf)

	// TODO: Bug in Spark: http://stackoverflow.com/questions/30323212
	val ctx = new HiveContext(sc)
	ctx.setConf("spark.sql.hive.convertMetastoreParquet", "false")

	import spray.json._
	import spray.http.HttpBody
	import spray.httpx.marshalling._
	import spray.httpx.unmarshalling._
	import spray.http.MediaTypes._
	import spray.routing.directives._
	import spray.routing._
	import spray.http._
	import spray.json.DefaultJsonProtocol
	import spray.httpx.SprayJsonSupport

	# to install the latest stable version:
	brew install scala --with-docs

	# to install scala-2.10.0:
	brew install https://raw.github.com/gist/4456110/scala.rb --with-docs

	# to switch versions (from https://github.com/mxcl/homebrew/wiki/External-Commands):
	# brew switch scala 2.9.2
	brew switch scala 2.10.0