JK.Ryan jkryanchou

Next Steps

Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
Take hot threads snapshots under read+write, read-only, write-only (TD - done)
Adjust refresh time to 10s (from 1s) and see how load changes (TD)
Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
Specify routing on query -- make it choose same node for each shard each time (MD)
GC new generation size (TD)
Warmers

measure before/after of client query time with and without warmers (MD)

Tuning Storm+Trident

Tuning a dataflow system is easy:

The First Rule of Dataflow Tuning:
* Ensure each stage is always ready to accept records, and
* Deliver each processed record promptly to its destination

Performance of Flask, Tornado, GEvent, and their combinations

Wensheng Wang, 10/1/11

Source: http://blog.wensheng.org/2011/10/performance-of-flask-tornado-gevent-and.html

When choosing a web framework, I pretty much have eyes set on Tornado. But I heard good things about Flask and Gevent. So I tested the performance of each and combinations of the three. I chose something just a little more advanced than a "Hello World" program to write - one that use templates. Here are the codes:

1, Pure Flask (pure_flask.py)

Moved

Now located at https://github.com/JeffPaine/beautiful_idiomatic_python.

Why it was moved

Github gists don't support Pull Requests or any notifications, which made it impossible for me to maintain this (surprisingly popular) gist with fixes, respond to comments and so on. In the interest of maintaining the quality of this resource for others, I've moved it to a proper repo. Cheers!

Example outputs:

$ python dynamic.py DynamicRequirements --a 4 --b 8

DEBUG: Checking if DynamicRequirements(a=4, b=8) is complete
INFO: Scheduled DynamicRequirements(a=4, b=8)
DEBUG: Checking if SomeTask(number=4) is complete
INFO: Scheduled SomeTask(number=4)
INFO: Done scheduling tasks

DEBUG: Asking scheduler for work...

	import os
	import inspect
	from functools import wraps


	_file_cache = {}


	def cache_file(path):
	def cache_is_fresh(name):

	(* EXPORT ALL SKIM NOTES TO EVERNOTE WITH HYPERLINKS

	-- Stephen Margheim
	-- 9/7/13
	-- open source

	REQUIRED PROGRAMS:
	- Skim (pdf viewer and annotator)
	- Evernote (cloud based note app)

	// Macro - Set the flag for block completion
	#define StartBlock() __block BOOL waitingForBlock = YES

	// Macro - Set the flag to stop the loop
	#define EndBlock() waitingForBlock = NO

	// Macro - Wait and loop until flag is set
	#define WaitUntilBlockCompletes() WaitWhile(waitingForBlock)

	// Macro - Wait for condition to be NO/false in blocks and asynchronous calls

	# This is not used unless SQLALCHEMY_BINDS is not present
	SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8'

	SQLALCHEMY_BINDS = {
	'master': 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8',
	'slave': 'mysql+pymysql://{username}:{password}@{hostname}/{database}?charset=utf8'
	}

	#!/bin/zsh
	# pyenv install for CentOS 6.5 x86_64

	yum install -y gcc gcc-c++ make git patch openssl-devel zlib-devel readline-devel sqlite-devel bzip2-devel

	git clone git://github.com/yyuu/pyenv.git ~/.pyenv

	export PATH="$HOME/.pyenv/bin:$PATH"
	eval "$(pyenv init -)"