compile.md

** Adapted from an answer in the FS2 gitter channel by Fabio Labella: https://gitter.im/functional-streams-for-scala/fs2?at=5e962ebb6823cb38acd12ebd

What is Stream.compile and what does it do? Is it actually compiling something? Optimizing the streaming somehow?

At its core, Stream[I, O].compile is nothing more than a namespace for related methods that return the same type wrapper, I. It's not compiling anything or doing any optimization. For example, all the methods on (s: Stream[IO, Int]).compile generally have the return type IO.

In FP there is a technique of design through algebras (speaking here in the most general sense, even more general than tagless final) and it basically works like this:

• you have some type, for example Option
• some introduction forms (ways of getting "into" the algebra, e.g., Option.empty, Option.some; this is often referred to as "lifting" into a type)
• some combinators (building programs in your algebra from smaller programs, like Option.map, Option.flatMap, Option.getOrElse, etc)
• and potentially laws (e.g. anyOption.flatMap(Option.empty) == Option.empty)

Basically it's a way to write programs in a mini-language (in this case the Option language), which affords high compositionality through combinators that helps us write larger programs in a Lego way.

But what does it mean to "run" programs in Option? "Running" an algebraic programs amounts to transforming the carrier type (Option), into a different type (say B). The shape of this transformation depends on the shape of the language, and in the case of Option, it's basically getOrElse or fold.

This trasformation function is called an eliminator. These functions transform your algebra type to another type, and correspond to our notion of "running".

Unsurprisingly, Stream also follows this pattern:

• Stream is the type
• emit, eval are the introduction forms (take types that aren't Stream, and convert them to Stream)
• ++, flatMap, concurrently are combinators
• and lastly you have the elimination form, which begins with compile

For pure streams, we might convert them from Stream[Pure, A] -> List[A]. For effectful streams, we transform them from Stream[IO, A] -> IO[A]. The name compile is meant to evoke this translation process in which the Stream language gets compiled down to the IO language. It used to be called run in previous versions, but compile is more evocative of this transformation.