Ryan Dooley 3rdman

#System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

##Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

Studying for a Tech Interview Sucks, so Here's a Cheat Sheet to Help

This list is meant to be a both a quick guide and reference for further research into these topics. It's basically a summary of that comp sci course you never took or forgot about, so there's no way it can cover everything in depth. It also will be available as a gist on Github for everyone to edit and add to.

Data Structure Basics

###Array ####Definition:

Stores data elements based on an sequential, most commonly 0 based, index.
Based on tuples from set theory.

Keybase proof

I hereby claim:

I am 3rdman on github.
I am 3rdman (https://keybase.io/3rdman) on keybase.
I have a public key whose fingerprint is 7C18 C3A2 7788 0757 9F7A C7EB 4EAD F6F1 45EC 760E

To claim this, I am signing this object:

Easy parallel python with concurrent.futures

As of version 3.3, python includes the very promising concurrent.futures module, with elegant context managers for running tasks concurrently. Thanks to the simple and consistent interface you can use both threads and processes with minimal effort.

For most CPU bound tasks - anything that is heavy number crunching - you want your program to use all the CPUs in your PC. The simplest way to get a CPU bound task to run in parallel is to use the ProcessPoolExecutor, which will create enough sub-processes to keep all your CPUs busy.

We use the context manager thusly:

with concurrent.futures.ProcessPoolExecutor() as executor:

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

Does the design expect failures to happen regularly and handle them gracefully?
Have we kept things as simple as possible?

How to Install Vector on Ubuntu 14

Vector is an on-host performance monitoring framework which exposes hand picked high resolution metrics to every engineer’s browser. More here.

Host machine

PCP Requirements

Vector requires PCP to collect data.

	Latency Comparison Numbers
	--------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD

	# Kernel sysctl configuration file for Red Hat Linux
	#
	# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
	# sysctl.conf(5) for more details.

	# Controls source route verification
	net.ipv4.conf.default.rp_filter = 1

	# Do not accept source routing
	net.ipv4.conf.default.accept_source_route = 0

	'Twas the night before Christmas, when all through the racks
	Not a server was alerting, not even Compaqs.
	The backups were written to tapes with care
	In hopes that later the data would be there.

	The machines were nestled all snug in their sleds
	Whilst visions of vengeance danced in their heads;
	And oncall in his three-wolf and I in my rack.
	Had just settled down for some syn and some ack.

	1. Monolithic applications and architectures can vary in their monolithness. This is an under-specified description.
	2. Microservice applications and architectures can vary in their microness. This is an under-specified description.
	3. Microservices and monolithic architectures have both benefits and disadvantages.
	4. Organizations will exploit those benefits while working around any weaknesses.
	5. Success of the business is a large influence on the exploitation of benefits and implementation and costs of workarounds.
	6. All benefits and work arounds are context-sensitive. Meaning that they are both technically and socially constructed by the organization that navigates them.
	7. Path dependency is a thing. History matters and manifests in these architectural decisions and evolution in an organization.
	8. Patterns exist to inform practice, not dictate it. Zealous adherence to an architectural pattern brings peril when it is to the exclusion of cultural context in actual practice.
	9. Architectural patterns w