apg

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

apg

#lisp.py
#
#An interpreter for a simple scheme like lexically 
#scoped language implemented as single python expression.
#
#The interpreter is limited to the lambda special form and a
#sum function. The data types supported are positive integers
#and symbols.
#

apg

package main

import (
	"bufio"
	"io"
	"regexp"
	"strings"
	"sync"
)

apg

import socket
import itertools
(lambda port, s=socket.socket(socket.AF_INET, socket.SOCK_STREAM):
          s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) == None and 
          s.bind(('', port)) == None and
          s.listen(5) == None and
          list(map(lambda c: 
                   c[0].sendall(c[0].recv(1024)) and
                   c[0].close(),
                   (s.accept() for _ in itertools.count(1)))) != None and

apg

Ricon 2014 Notes

Keynote: Marten Mickos (SVP and General Manager, HP Cloud): Open Source Wins

Started with RDBMSes and needed to “scale out”.

• Why? Well, we couldn’t scale up anymore. (talking about MySQL)

Composibility of simple components

Distributed (not monolithic)

Design for failure (Assume everything is always going to fail)

• Shift in trust. (> more reliable)

apg

Listen Poller

Overview

The listen poller is a poller that allows external sources to record metrics through a single shh instance. This means that any process that can produce output, can record metrics through shh. While this is happening, shh can concurrently collect and publish it's own metrics (from other pollers).

apg

import re, itertools
from PIL import Image

def str2pt(s):
    L = len(s) - 1
    x, y = 1, 1
    for n in s:
        y += 2**L if n in '34' else 0
        x += 2**L if n in '14' else 0
        L -= 1

apg

var PENDING_INVITE = '#fff2cc';
var INVITED = '#00ff00';
var CONFIRMED = '#ffff00';
var NOT_THIS_TIME = '#00ffff';
var NOT_THIS_TIME_SENT = '#4a86e8';

function markPerson(person, newtyp) {
  var nameCell = person['range'].getCell(1, 1);
  nameCell.setBackground(newtyp); 
};

apg

# This inflates things pretty horribly since it's looking at 
# raw lines (including comments, documentation, etc) rather 
# than lines of code metrics that you might get from a tool 
# like sloccount, or cloc

AUTHOR=apg

for n in {1..100}; do
   PAGER=cat git log --numstat --since="$n days ago" --until="$(expr $n - 1) days ago" --author=$AUTHOR | grep '^[0-9]' | awk 'BEGIN { lines = 0 } { lines += ($1 - $2) } END { print lines }'
done 

apg

#!/usr/bin/env python2.7
"""html2text: Turn HTML into equivalent Markdown-structured text."""
__version__ = "2.38"
__author__ = "Aaron Swartz ([email protected])"
__copyright__ = "(C) 2004-2008 Aaron Swartz. GNU GPL 3."
__contributors__ = ["Martin 'Joey' Schulze", "Ricardo Reyes", "Kevin Jay North"]

# TODO:
#   Support decoded entities with unifiable.