I have some early benchmark results for our work on a high performance NATS server in Go.
Quick Summary:
We can process ~2M msgs/sec through the system, and the ingress and egress are fairly well balanced.
The basics of the architecture are intelligent buffering and IO calls, fast hashing algorithms and subject distributor/routing, and a zero-allocation hand-written protocol parser.
In addition, I used quite a bit of inlining to avoid function overhead, no use of defer, and little to no object allocation within the fast path. I will share more details and the code at a future date.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import re, sys | |
keys = ('abc', 'def', 'ghi', 'jkl', 'mno', 'pqrs', 'tuv', 'wxyz') | |
m = dict((l, str(n)) for n, letters in enumerate(keys, start=2) for l in letters) | |
data, ocurrences = {}, {} | |
wmatch = re.compile('[^a-z]+') | |
def learn(word): | |
num = ''.join(m[c] for c in word) | |
for i in xrange(1, len(word) + 1): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
# Assuming you use gs(1) and dep(1), this program | |
# implodes your .gs directory and installs all gems | |
# listed in your .gems file with a single call to gem(1). | |
# | |
# gs(1): https://github.com/soveran/gs | |
# dep(1): https://github.com/cyx/dep | |
set -e |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
Created on Jun 25, 2012 | |
@author: andres.rangel | |
''' | |
import logging | |
import os | |
import re | |
from redis.client import Redis, BasePipeline, ConnectionError, StrictRedis, Script, NoScriptError | |
from redis.connection import ConnectionPool, Connection, DefaultParser |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[{"event"=>"inbound", | |
"ts"=>1348084981, | |
"msg"=> | |
{"raw_msg"=> | |
"Received: from mail-pb0-f50.google.com (mail-pb0-f50.google.com [209.85.160.50])\n\tby app02.transact (Postfix) with ESMTPS id 113FA160008E\n\tfor <[email protected]>; Wed, 19 Sep 2012 16:03:01 -0400 (EDT)\nReceived: by pbcmd12 with SMTP id md12so3511454pbc.37\n for <[email protected]>; Wed, 19 Sep 2012 13:03:00 -0700 (PDT)\nX-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=google.com; s=20120113;\n h=mime-version:from:date:message-id:subject:to:content-type\n :content-transfer-encoding:x-gm-message-state;\n bh=nj7aRozzB6DfuzLS5/RWru2ni7V73l9sEZWbQhPgc4Y=;\n b=TyTdKUhjpecuS7QeXUN8cP23fuG8nar8EdkyHuNqyDPBVm8n5R1bH+YLJEL2KhZ2kf\n EskBsvgjKW2qallcEcN+yoI6JR16g6mXzRTGXENMCJ2fI9sbTo1a3K3ZQ9AfFbjOTtDf\n F8L1PF8HpHtXRDAYsPRSlc6JCaAzWMfscE0l9g63DMx0jxC6hkpEGzNBlBUqK+hV7y6w\n FeJ6njxlS+1x2/UfjYj8Y9rKZNWCbElGnHOIvpDeq8AE/RXWXs06/G8ggSB8lr90MfCj\n u |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
check process redis-server | |
with pidfile "/var/run/redis.pid" | |
start program = "/etc/init.d/redis-server start" | |
stop program = "/etc/init.d/redis-server stop" | |
if 2 restarts within 3 cycles then timeout | |
if totalmem > 100 Mb then alert | |
if children > 255 for 5 cycles then stop | |
if cpu usage > 95% for 3 cycles then restart | |
if failed host 127.0.0.1 port 6379 then restart | |
if 5 restarts within 5 cycles then timeout |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$stack, $draws = [], {} | |
def method_missing *args | |
return if args[0][/^to_/] | |
$stack << args.map { |a| a or $stack.pop } | |
$draws[$stack.pop(2)[0][0]] = args[1] if args[0] == :< | |
end | |
class Array | |
def +@ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env sh | |
## | |
# This is script with usefull tips taken from: | |
# https://github.com/mathiasbynens/dotfiles/blob/master/.osx | |
# | |
# install it: | |
# curl -sL https://raw.github.com/gist/2108403/hack.sh | sh | |
# |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class Book < Ohm::Model | |
collection :authors, :Author | |
end | |
class Author < Ohm::Model | |
reference :book, :Book | |
attribute :name | |
attribute :mood | |
index :mood | |
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require "cuba" | |
run Cuba.new { on(root) { res.write "hello, world" } } |