bcambel’s gists

bcambel / reducers to the resque!

Created November 4, 2013 12:44 — forked from mishok13/reducers to the resque!

	user> (defn c [p]
	(with-open [rdr (clojure.java.io/reader p)]
	(reduce + (map count (line-seq rdr)))))
	#'user/c
	user> (bench/quick-bench
	(c "/tmp/test"))
	WARNING: Final GC required 356.7754050400232 % of runtime
	Evaluation count : 6 in 6 samples of 1 calls.
	Execution time mean : 525.040480 ms
	Execution time std-deviation : 29.017225 ms

bcambel / .gitconfig

Created October 25, 2013 12:37 — forked from bvsatyaram/.gitconfig

	[user]
	name = Pavan Kumar Sunkara
	email = pavan.sss1991@gmail.com
	[core]
	editor = vim
	whitespace = fix,-indent-with-non-tab,trailing-space,cr-at-eol
	[sendemail]
	smtpencryption = tls
	smtpserver = smtp.gmail.com
	smtpuser = pavan.sss1991@gmail.com

bcambel / INSTALL.txt

Created June 24, 2013 21:37 — forked from mrjoes/INSTALL.txt

bcambel / message_queue_pipeline.py

Created December 23, 2012 18:41

	from scrapy.xlib.pydispatch import dispatcher
	from scrapy import signals
	from scrapy.exceptions import DropItem
	from scrapy.utils.serialize import ScrapyJSONEncoder

	from carrot.connection import BrokerConnection
	from carrot.messaging import Publisher

	from twisted.internet.threads import deferToThread

bcambel / gist:4365127

Created December 23, 2012 18:37 — forked from anonymous/gist:4365124

	# Many times when crawling we run into problems where content that is rendered on the page is generated with Javascript and therefore scrapy is unable to crawl for it (eg. ajax requests, jQuery craziness). However, if you use Scrapy along with the web testing framework Selenium then we are able to crawl anything displayed in a normal web browser.
	#
	# Some things to note:
	# You must have the Python version of Selenium RC installed for this to work, and you must have set up Selenium properly. Also this is just a template crawler. You could get much crazier and more advanced with things but I just wanted to show the basic idea. As the code stands now you will be doing two requests for any given url. One request is made by Scrapy and the other is made by Selenium. I am sure there are ways around this so that you could possibly just make Selenium do the one and only request but I did not bother to implement that and by doing two requests you get to crawl the page with Scrapy too.
	#
	# This is quite powerful

bcambel / supervisord-example.conf

Created December 7, 2012 02:35 — forked from didip/supervisord-example.conf

Example for supervisord conf file

	; Sample supervisor config file.

	[unix_http_server]
	file=/tmp/supervisor.sock ; (the path to the socket file)
	;chmod=0700 ; sockef file mode (default 0700)
	;chown=nobody:nogroup ; socket file uid:gid owner
	;username=user ; (default is no username (open server))
	;password=123 ; (default is no password (open server))

	;[inet_http_server] ; inet (TCP) server disabled by default

bcambel / supervisor.conf

Created December 6, 2012 17:07 — forked from tsabat/supervisor.conf

Sample supervisor config file

	; Sample supervisor config file.

	[unix_http_server]
	file=/tmp/supervisor.sock ; (the path to the socket file)
	;chmod=0700 ; sockef file mode (default 0700)
	;chown=nobody:nogroup ; socket file uid:gid owner
	;username=user ; (default is no username (open server))
	;password=123 ; (default is no password (open server))

	;[inet_http_server] ; inet (TCP) server disabled by default

bcambel / sample.config

Created July 31, 2012 21:28 — forked from sidupadhyay/sample.config

Sample HA Proxy Config for Tornado/Socket.io Backends

	global
	maxconn 10000 # Total Max Connections. This is dependent on ulimit
	nbproc 2

	defaults
	mode http
	option redispatch
	maxconn 2000
	contimeout 5000
	clitimeout 50000

bcambel / index.html

Created June 8, 2012 23:42 — forked from ralphbean/index.html

d3 viz for narcissus

	<!DOCTYPE html>
	<html>
	<head>
	<title>Proof of concept for d3 viz + narcissus</title>
	<link rel="stylesheet" type="text/css" href="spider.css"></style>
	<script type="text/javascript" src="http://mbostock.github.com/d3/d3.min.js"></script>
	<script type="text/javascript" src="http://mbostock.github.com/d3/d3.geom.min.js"></script>
	<script type="text/javascript" src="http://mbostock.github.com/d3/d3.layout.min.js"></script>
	<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js"></script>
	<script type="text/javascript" src="spider.js"></script>

bcambel / latency.txt

Created June 3, 2012 22:37 — forked from jboner/latency.txt

Latency numbers every programmer should know

	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns
	Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
	Read 4K randomly from SSD 150,000 ns 0.15 ms
	Read 1 MB sequentially from memory 250,000 ns 0.25 ms
	Round trip within same datacenter 500,000 ns 0.5 ms

Bahadir Cambel bcambel