Artem Kanev nixorn

On Tue Oct 27, 2015, history.state.gov began buckling under load, intermittently issuing 500 errors. Nginx's error log was sprinkled with the following errors:

2015/10/27 21:48:36 [crit] 2475#0: accept4() failed (24: Too many open files)

2015/10/27 21:48:36 [alert] 2475#0: *7163915 socket() failed (24: Too many open files) while connecting to upstream...

An article at http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/ provided directions that mostly worked. Below are the steps we followed. The steps that diverged from the article's directions are marked with an *.

* Instead of using su to run ulimit on the nginx account, use ps aux | grep nginx to locate nginx's process IDs. Then query each process's file handle limits using cat /proc/pid/limits (where pid is the process id retrieved from ps). (Note: sudo may be necessary on your system for the cat command here, depending on your system.)
Added fs.file-max = 70000 to /etc/sysctl.conf

Context

I've built sizeable products (~100,000+ LOC) using Rails at Cleartrip, as well as my own startup Vacation Labs. Even at Cleartrip, before settling on Rails, I tried very hard to make Common Lisp work for us. But, it felt like a losing battle. Instead of solving business problems we were writing reams of boilerplate code to make Lisp interact with the DB in a sensible manner. At Vacation Labs, I have a hunch that rewriting core parts of our app in Haskell will lead to safer code, fewer bugs, lesser dependancy on testing, and eventually faster product evolution. However, history seems to be repeating itself when it comes to accessing a large, complicated DB schema (100+ tables) using Haskell (Persistent).

I'm currently grappling with audit logs - a core requirement for any app dealing with sensitive customer info (especially e-commerce or payments). Unforuntaly, apart from [Snaplet ActionLog](http://devblog.soostone.com/posts/2013-05-1

title	author
Passing a DB connection to handlers in Servant	Oleg

This write-up is motivated by discussion in servant/#704 issue. I try to summarize the main points.

As this is a literate haskell file, we'll need to do a small prelude dance:

	##############################################################################
	#
	# Simple Python program to benchmark several Python Excel writing modules.
	#
	# python bench_excel_writers.py [num_rows] [num_cols]
	#
	#

	import sys
	from time import clock

	# This is an example of file uploading using scrapy to a server that uses uberuploader.
	# The spider logins to the page to make the upload, some webpages dont need to be logged in order to upload.
	# IMPORTANT : You should increment DOWNLOAD_TIMEOUT in settings.py, but by the time this snippet was wrote isnt working fine. I recompile the whole scrapy changing the default of 3min.
	# Observations about my snippet:
	# Is possible this is not the best code, please comment corrections.
	# This could or should be implemented in a downloader middleware or pipeline?
	# Don't show uploading state.
	# Mime message creating could or should be in other place.

	class fileUploadSpider(CrawlSpider):