arosien

rrdtool create redirector.rrd --step 60 DS:registrations:COUNTER:600:U:U RRA:AVERAGE:0.5:1:3000 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797

arosien

package asr.nestedsets;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import java.util.SortedSet;

arosien

h1. Using Norbert for cluster management

h2. What is a cluster?

In Norbert parlance, a cluster is a set of Nodes with a given name. A Node is a host, port and list of partitions for which the node can handle requests. The set of member Nodes in a given cluster is centrally stored in ZooKeeper. Additionally, a Node can advertise that it is available to process requests. In general, a Node can be in one of three states:

[arosien: Most people don't know the word 'parlance'. I'd say 'In Norbert, a cluster...'.]
[arosien: So Nodes have a name, and Nodes with the same name are a Cluster? It may be easier to understand if a Node has a clusterName and all nodes with the same clusterName are the de-facto Cluster.]
[arosien: What are partitions? How are they represented? What's an example of a partition?]
[arosien: I'd use bullet points for the properties of a Node.]

arosien

import org.specs2.mutable.Specification
import akka.actor.Actor
import akka.actor.FSM
import akka.testkit.TestFSMRef
import org.specs2.specification.Scope

class AkkaFSMEventSpec extends Specification {

  case class A(msg: String)

arosien

import scalaz._
import Scalaz._

case class Foo(value: String)
case class Bar(value: String)
case class Baz(foo: Foo, bar: Bar)
type E = Map[String, String]

type K[M[_],A,B] = Kleisli[M,A,B]
      

arosien

F[A]           // A is a Functor
M[A]           // A is a Monad
fa: F[A], a: A // variables used below

fa1 |+| fa2 // binary append (SemiGroup)
fa1  >| fa2 // map           (Functor)
fa1 >>= fa2 // flatmap       (Bind)
fa1 |>| fa2 // foreach       (Each)

fa <**> fb { (a, b) => c }                       // apply, produces F[C]

arosien

Monad     | effect                                   | sequences the effect as | M[A]         | bind: M[A] => (f: A => M[B]) => M[B]

Identity  | nothing                                  | continue                | Id[A]        | f(a)
Option    | zero or one value (anonymous exception)  | halt if None            | Option[A]    | if Some(a) f(a)
Either    | exception with error type or one value   | halt if Left            | Either[L, A] | if Right(a) f(a)
List      | any # of values (non-determinism)        | halt if empty           | List[A]      | join(each f(a))
Reader    | an environment; dependency-injection     | function composition    | R => A       | (r: R) => f(a(r))
Writer    | logging                                  | append log value W      | Writer[W, A](log: W, value: A) | k = f(a.value); Writer(a.log |+| k.log, k.value)
State     | state                                    | new state from old      | State[S, A](s: S => (S, A))    | 
Responder | continuation-passing

arosien

• screenshot of eclipse type inference
• one-to-one principle: one type used per function signature
• Rather than thinking about how to jam our ideas into the trappings of the language--class hierarchies and such--we "merely" work with the things as types, transforming them with commonly known functions like map, fold, and so on, then adding context and effects to them in a few well-known ways (dependency-injection = Reader, accumulate logging information = Writer/Logger, perform a transformation using the current state and produce a new state = State, etc.).
• tension between not explicitly typing, because the compiler can properly infer, and being able to inspect the type, via the editor via the compiler
• adding more types potentially creates N more adapters to that type, so you want to minimize the amount of new methods signatures you will maybe need to adapt, so having a set of reusable (semantically and structurally well-known) methods reduces the cost of new classes, otherwise we'd

arosien

case class WithZk(path: String, client: ZooKeeperClient) {
  import WithZk._
  import scala.util.control.Exception._

  lazy val dir = path.split('/').reverse.tail.reverse.mkString("/")
  private val catcher = catching(classOf[KeeperException])

  def reader[A: Reads] = new Reader[A]
  def writer[A: Writes](value: A) = new Writer[A](value)
  def deleter = new Deleter

arosien

The Basics

We're going to be linking the following traits and classes together:

trait FileSystem {
  def roots: Traversable[File]
}

class PathWalker(fs: FileSystem) {