Andrew Kornilov frutik

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

Thread Pools

Thread pools on the JVM should usually be divided into the following three categories:

CPU-bound
Blocking IO
Non-blocking IO polling

Each of these categories has a different optimal configuration and usage pattern.

Set the default size for new Docker for Mac disk images

If you are getting the error: No space left on device

Configuring the qcow2 size cap is possible in the current versions:

# my disk is currently 64GiB
$ /Applications/Docker.app/Contents/MacOS/qemu-img info ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
image: /Users/djs/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

Next Steps

Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
Take hot threads snapshots under read+write, read-only, write-only (TD - done)
Adjust refresh time to 10s (from 1s) and see how load changes (TD)
Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
Specify routing on query -- make it choose same node for each shard each time (MD)
GC new generation size (TD)
Warmers

measure before/after of client query time with and without warmers (MD)

	function permutations(list)
	{
	// Empty list has one permutation
	if (list.length == 0)
	return [[]];


	var result = [];

	for (var i=0; i<list.length; i++)

	/jslint sloppy:true, white:true, vars:true, plusplus:true /

	var permutation = function (collection){
	var current,
	subarray,
	result = [],
	currentArray = [],
	newResultArray = [];

	if (collection.length){

	# cat /etc/redhat-release
	CentOS release 6.5 (Final)

	golang from EPEL:

	# rpm -qi golang
	Name : golang Relocations: (not relocatable)
	Version : 1.1.2 Vendor: Fedora Project
	Release : 4.el6 Build Date: Fri 20 Sep 2013 16:21:49 BST
	Install Date: Thu 09 Jan 2014 15:14:13 GMT Build Host: buildvm-17.phx2.fedoraproject.org

	# Run me with:
	#
	# $ nginx -p /path/to/this/file/ -c nginx.conf
	#
	# All requests are then routed to authenticated user's index, so
	#
	# GET http://user:password@localhost:8080/_search?q=*
	#
	# is rewritten to:
	#

	A warning occurred (42 apples)
	An error occurred