Kurt Griffiths kgriffs

Teaching Falcon to Fly

Falcon is a new web framework specifically designed for creating lean, high-performance HTTP APIs.

The difference between efficient and inefficient code can mean the difference between a good user experience and a poor one, or the difference between a profitable product and a financial disaster. In this talk, I'll introduce the Falcon web framework, and share some of the performance patterns I discovered while developing the framework.

Appendix D: Conflict or Complimentary?

Web Frameworks

Django — Flask
Rails — Sinatra
Hapi — Express
Pecan — Falcon

Libraries

In order to trace the execution of our code we can use Python's profile hooks mechanism.

import sys 

def tracer(frame, event, arg):
    print(frame.f_code.co_name, event) 

sys.setprofile(tracer)

Using timestamps as IDs

Helps scale writes:

For a single master, means don't have to query to find last inserted ID.
Collisions can be practically impossible
Avoids having to retry using an optimistic algorithm, in the case that a parallel request beat me to the punch and I have a collision of IDs.

Caveats:

Timestamp must be granular enough to be unique for expected max message insertion rate per queue. For example, given a goal of 100,000 writes/sec to a specific queue, to avoid collisions the timestamps would need to be stored with at least 100 ms granularity. However, one must also allow for outliers, where occasionally the stars will align and several requests will arrive and request timestamps at the same moment. [wWth that in mind, what is a the practical granularity required? microseconds? 100-nanoseconds?]
Requires specialized hardware to avoid significant drift. However, NTP may be "good enough" if ~1 ms drift is acceptable.

	import functools
	import inspect
	import warnings


	# NOTE(kgriffs): We don't want our deprecations to be ignored by default,
	# so create our own type.
	class DeprecatedWarning(UserWarning):
	pass

	In [6]: callable?
	Type: builtin_function_or_method
	String Form:<built-in function callable>
	Namespace: Python builtin
	Docstring:
	callable(object) -> bool

	Return whether the object is callable (i.e., some kind of function).
	Note that classes are callable, as are instances with a __call__() method.

	N = 5
	V = 0.001

	for i in range(N):
	print V * (2 ** i)

	# solom/__init__.py
	THREAD_LOCAL = threading.local()


	# In some pecan helper module...
	def before_hook(self):
	# Use thread-local storage so we don't have to
	# add two extra params to virtuall every single
	# function in our app!
	solum.THREAD_LOCAL.ctx = context.RequestContext(...)

	In [4]: response
	Out[4]:
	{'messages': {'claimed': 2409,
	'free': 146929,
	'newest': {'age': 12,
	'created': '2013-08-12T20:45:46Z',
	'href': 'queues/fizbit/messages/50b68a50d6f5b8c8a7c62b01'},
	'oldest': {'age': 63,
	'created': '2013-08-12T20:44:55Z',
	'href': 'queues/fizbit/messages/50b68a50d6f5b8c8a7c62b01'},

	from __future__ import division

	import multiprocessing as mp
	import argparse
	import random
	import time

	from gevent import monkey as curious_george
	curious_george.patch_all(thread=False, select=False)