- Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
- Models and Issues in Data Stream Systems
- Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
- Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
- [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# bash function | |
defcomp () { | |
local name="$1" && shift | |
local functionName="$( echo "${name}_completion" | tr - _ )" | |
source <(cat <<EOM | |
$functionName () { | |
local cur | |
_get_comp_words_by_ref -n =: cur | |
COMPREPLY=( \$(compgen -W "\$( $@ )" -- "\$cur") ) | |
} |
Slightly disorganized but reasonably complete notes on the algorithms, strategies and optimizations of the Akka Cluster implementation. Could use a lot more links and context etc., but was just written for my own understanding. Might be expanded later.
Links to papers and talks that have inspired the implementation can be found on the 10 last pages of this presentation.
This is the Gossip state representation:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Trivia problem: Generate primes between 1 and 100 million with Scalding | |
Input: Numbers from 1 to 100 million, 1 per line - use a simple C program for this | |
Output: Filter input for primes using the filter function on pipes in Scalding | |
Execution time: 13:11:00 to 13:25:54 = 15 minutes in hdfs-local mode | |
Part files: 27 | |
$ ls | |
part-00000 part-00002 part-00004 part-00006 part-00008 part-00010 part-00012 part-00014 part-00016 part-00018 part-00020 part-00022 part-00024 part-00026 part-00001 part-00003 part-00005 part-00007 part-00009 part-00011 part-00013 part-00015 part-00017 part-00019 part-00021 part-00023 part-00025 | |
$ tail part-00026 | |
99999787 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import scala.language.higherKinds | |
import scala.language.implicitConversions | |
import scala.language.reflectiveCalls | |
trait Forall[M[_]] { | |
def apply[A]: M[A] | |
} | |
object Maybe { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Welcome to Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_05). | |
Type in expressions to have them evaluated. | |
Type :help for more information. | |
scala> def foo = { println("foo"); "foo" } | |
foo: String | |
scala> def bar = { println("bar"); "bar" } | |
bar: String |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/*©2013 Viktor Klang*/ | |
package akka.util | |
import java.util.concurrent.atomic.AtomicReference | |
import scala.concurrent.{ Future, ExecutionContext } | |
import scala.annotation.tailrec | |
class Cache[K, V](__ec: ExecutionContext, throughput: Int) extends AtomicReference[Map[K, V]] { | |
implicit val executor = SerializedSuspendableExecutionContext(throughput)(__ec) | |
@tailrec final def update(f: Map[K, V] ⇒ Map[K, V]): Map[K, V] = { | |
val v = get | |
val nv = f(v) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Companies hiring Scala developers in the Bay Area. | |
Created in response to a thread on scala-base. | |
My favorites: | |
- CloudPhysics | |
- Wordnik | |
Unbiased list: | |
- 10Gen | |
- Audax Health |
Have seen a few play apps' source and it feels like massive boilerplate and leaky promise/future/async crud everywhere.
Lets take this:
case class ArticlePage(article: Article, storyPackage: List[Trail], edition: String)
object ArticleController extends Controller with Logging {
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
object Algebra { | |
// Build up increasingly complex algebras | |
trait Magma[T] { | |
def add(x : T, y : T) : T | |
} | |
trait Monoid[T] extends Magma[T] { | |
def zero : T | |
} |