Sean Parsons seanparsons

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

http://www.youtube.com/watch?v=JeK979aqqqc

Why he doesn't use JavaScript to show examples of closure is beyond me.

JavaScript is an awful language to both demonstrate and practice functional programming.

You have no idea what you're talking about. Functions are first-class in JavaScript. Nine times out of ten, those who hate JavaScript just don't understand it and/or suck at it.

Ugh, well this is going to get silly.

Backend Architectures

Twitter

ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats

Introduce Gizzard of Twitter - 트위터의 새로운 분산 관리 라이브러리 Gizzard를 소개합니다.
The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds
Scaling Twitter: Making Twitter 10000 Percent Faster

Opening and closing an SSH tunnel in a shell script the smart way

I recently had the following problem:

From an unattended shell script (called by Jenkins), run a command-line tool that accesses the MySQL database on another host.
That tool doesn't know that the database is on another host, plus the MySQL port on that host is firewalled and not accessible from other machines.

We didn't want to open the MySQL port to the network, but it's possible to SSH from the Jenkins machine to the MySQL machine. So, basically you would do something like

ssh -L 3306:localhost:3306 remotehost

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD

	// a.scala
	// Fri Jun 29 16:15:08 PDT 2012

	import scala.reflect.makro.Context
	import collection.mutable.ListBuffer
	import collection.mutable.Stack
	import language.experimental.macros

	object Macros {
	val conversionChars = "%BbCcdEefGgHhinosuXx"

	@channingwalton Recently read »I, Robot« from Asimov. The short stories are entertaining and very thoughtful.

	@channingwalton I would like a wizard where I tell it I like Asimov (especially Daneel and Giskard) and Clarke, and it recommends others.

	@channingwalton pretty much anything by Alastair Reynolds. House of Suns is particularly great.

	@channingwalton I’ve enjoyed the Laundry novels by Charles Stross.

	@channingwalton "Wool" series by Hugh Howey.

	scala> val o: Option[Int] = Some(5)
	o: Option[Int] = Some(5)

	scala> val l: List[Int] = List(10)
	l: List[Int] = List(10)

	scala> l flatMap (_ => o)
	res1: List[Int] = List(5)

	scala> o flatMap (_ => l)

	object BenchmarkCommon {
	import scala.util.Random

	val DatasetSize = 10000
	val Iterations = 10000
	val ArrayPoolSize = 1000
	val ArrayPool = {
	def randomArray(): Array[Int] = {
	val array = new Array[Int](DatasetSize)

	class Outer {
	class Inner
	type Type
	}
	trait Trait
	object Object extends Outer {
	val inner = new Inner
	}
	class OuterP[A] {
	class InnerP[B]