Skip to content

Instantly share code, notes, and snippets.

View r-wheeler's full-sized avatar

Ryan Wheeler r-wheeler

20 followers · 22 following
View GitHub Profile
@r-wheeler
r-wheeler / scala and state monad: no free lunch part 2.md
Last active August 29, 2015 14:20 — forked from aappddeevv/scala and state monad: no free lunch part 2.md

As a follow up to my last post about no free lunch I wanted to show more state monad usage and recognize that you can change function return types in the middle of a for-comprehension. This allows each function to return different types of values from the computation along the way.

In the code below, you compose a function by using the for-comprehension. The yield part of the for-comprehension is a State object. When that state object is called, it returns a tuple of (state, value). You can use ._1 or ._2 to obtain the value of interest.

The last composed function shows an example of changing the state type (instead of changing the returned value type mentioned above) in the middle of the for-comprehension using scalaz IndexedState's iPut and iModify. This allows you to change the type of state as you go to adapt to different function's API needs.

import scalaz._
import scalaz.State._
@r-wheeler
r-wheeler / StreamingHLL.scala
Last active August 29, 2015 14:19 — forked from MLnick/StreamingHLL.scala
import spark.streaming.StreamingContext._
import spark.streaming.{Seconds, StreamingContext}
import spark.SparkContext._
import spark.storage.StorageLevel
import spark.streaming.examples.twitter.TwitterInputDStream
import com.twitter.algebird.HyperLogLog._
import com.twitter.algebird._
/**
* Example of using HyperLogLog monoid from Twitter's Algebird together with Spark Streaming's