errordeveloper

Overview

Logplex is the Heroku logging multiplexor. Components throughout the Heroku platform publish syslog packets that are received and processed by a grid of logplex nodes. Users may retrieve their logs through requests to the logplex API.

Logplex relies on channel, token, drain and session objects. This data is stored in redis, but also cached for fast lookups in a normalized format in ETS tables. When logplex boots, all of the keys are loaded from redis and cached in ETS. When an object is created or deleted through the logplex API, other logplex nodes are notified of the change (via Erlang rpc calls) so that they may update their local ETS cache. We would like to explore the possibility of using nsync to manage replicating the dataset into ETS. This would speed up logplex boot times and save us from making Erlang RPC calls to update local caches on remote nodes.

To get started, fork logplex on github and follow the directions below to get logplex running locally.

Once you are up and runni

shayarnett

Generate ctags with `rake ctags`
Run snipper.rb
Put resulting ruby.snippets file where your snippet plugin can pick it up. (I'm using snipmate)

jboner

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

sfate

#!/usr/bin/env bash
curl https://s3.amazonaws.com/heroku-jvm-buildpack-vi/vim-7.3.tar.gz --output vim.tar.gz
mkdir vim && tar xzvf vim.tar.gz -C vim
export PATH=$PATH:/app/vim/bin

tenderlove

rubiii

$ rspec --format MacVimFormatter --color spec

gen_event_caster.erl

-module(gen_event_caster).
-author('Fernando Benavides <[email protected]>').

-behaviour(gen_event).

-export([start/2]).
-export([init/1, handle_event/2, handle_call/2, handle_info/2, terminate/2, code_change/3]).

-type receiver() :: atom() | pid().

ryandotsmith

Tales From a Heroku User

Here are some things I have learned along the way.

Last Updated: 2013-02-08

Original Audience: Hack Reactor

About

skierkowski

require 'json'
require 'rest-client'
require 'heroku'
require "heroku/command/base"
require "anvil"
require "anvil/engine"
require "uri"

class Cisaurus

ragingwind

Backend Architectures

Twitter

ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats

• Introduce Gizzard of Twitter - 트위터의 새로운 분산 관리 라이브러리 Gizzard를 소개합니다.
• The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds
• Scaling Twitter: Making Twitter 10000 Percent Faster