Danil Ivanov nonamenix

nonamenix / crawler.py

Created January 22, 2017 13:01 — forked from jmoiron/crawler.py

Simple gevent/httplib2 web crawler.

	#!/usr/bin/env python
	# -- coding: utf-8 --

	"""Simple async crawler/callback queue based on gevent."""

	import traceback
	import logging
	import httplib2

	import gevent