kachayev

Channels Are Not Enough

... or Why Pipelining Is Not That Easy

Golang Concurrency Patterns for brave and smart.

By @kachayev

Intro

freeman-lab

package thunder.streaming

import org.apache.spark.{SparkConf, Logging}
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkContext._
import org.apache.spark.streaming._
import org.apache.spark.streaming.dstream.DStream
import org.apache.spark.mllib.clustering.KMeansModel

import scala.util.Random.nextDouble

elithrar

A quick "how to" on what you need to do to both setup AND recover a single-server PostgreSQL database using WAL-E

• WAL-E: https://github.com/wal-e/wal-e
• Assuming Ubuntu 12.04 LTS ("Precise")
• We'll be using S3. Make sure you have an IAM in a group with GetObject, ListBucket and PutObject on the bucket you want to use (and that it's not public).

Setup:

1. These packages:

fedesilva

val o1 = Some("string")
//o1: Some[String] = Some(string)

val o2 = Some("")
//o2: Some[String] = Some()

val o3: Option[String] = None
//o3: Option[String] = None

o3.filter (_ != "").fold( "Bad Bad Bad")( x => s"Good, we got $x" )

fedesilva

scala> val rp = pipeline(rq) 
rp: scala.concurrent.Future[spray.http.HttpResponse] = scala.concurrent.impl.Promise$DefaultPromise@597aa1d

scala> rp fp2013-08-216 20:18:26.560 INFO  akka.actor.DeadLetterActorRef - Message [spray.http.HttpResponse] from Actor[akka://test/user/IO-HTTP/host-connector-0/0#229322353] to Actor[akka://test/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2013-08-216 20:18:26.561 INFO  akka.actor.LocalActorRef - Message [akka.io.Tcp$Write] from Actor[akka://test/user/IO-HTTP/group-0/0#-1483684089] to Actor[akka://test/system/IO-TCP/selectors/$a/0#634853586] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2013-08-216 20:18:26.561 INFO  akka.actor.LocalActorRef - Message [akka.io.Tcp$ConfirmedClos

kputnam

Using dtruss on vim's :w

Ever wonder what happens when you save a file in vim? There's quite a lot more happening than open, write, and close. I found this out while working on some code to monitor changed files using the node.js fs.watch API, which abstracts (or obscures) Mac OS X's kqueue API.

To find out exactly what vim is doing when saving a file, we'll use a tool included with DTrace that reports on another process's system calls. We need the process ID of vim, which is already open and editing my file:

puffnfresh

Where TDD Fails

I've just gotten back from the awesome mloc.js conference. There was a talk about compiling C# to JavaScript and one of the benefits explained was static types. Someone from the audience asked, who needs types when you do Test Driven Development?

I tried to address the question in my talk on Roy but I talked to some developers afterwards and they thought that TDD

gclaramunt

trait Entity
trait User extends Entity
trait Product extends Entity

case class Id[T<:Entity](id:String)


def buy(pId:Id[Product],uId:Id[User])="Bought product %s for user %s".format(pId.id,uId.id)

val pId=new Id[Product]("1")

fedesilva

upstream artifacts {
    server 127.0.0.1:8081 fail_timeout=0;
}

server {

    listen 80; 
    root /var/www;
    server_name artifacts.inconcert artifacts;

fedesilva

// Mersenne Twister 19937
// Based on code from: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html
// Note: This implementation is not thread-safe!
final class MersenneTwister (seed: Int = 5489) {
  private val N = 624
  private val M = 397

  private val MatrixA = 0x9908b0dfL

  private val UpperMask = 0x80000000L