This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// node.js get keypress | |
var stdin = process.stdin; | |
// without this, we would only get streams once enter is pressed | |
//stdin.setRawMode( true ); | |
// resume stdin in the parent process (node app won't quit all by itself | |
// unless an error or process.exit() happens) | |
stdin.resume(); | |
// i don't want binary, do you? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from contextlib import contextmanager | |
from urlparse import urlparse | |
from paramiko import AuthenticationException | |
import pysftp | |
@contextmanager | |
def open_sftp(url, mode='r', n_retries=5, retry_sleep_time=5): | |
"""Context manager to read/write a file via SFTP |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import asyncio | |
import aiohttp | |
import bs4 | |
import tqdm | |
@asyncio.coroutine | |
def get(*args, **kwargs): | |
response = yield from aiohttp.request('GET', *args, **kwargs) | |
return (yield from response.read_and_close(decode=True)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spider_count = 0 | |
# Get a signal that a crawler has finished its job, start another crawler | |
def spider_closed(spider, reason): | |
spider_count += 1 | |
#Only starts another crawler if available | |
if spider_count < len(spiders): | |
reactor.callLater(0, start_crawler, spider=spiders[spider_count]) | |
else: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?php | |
// Change /*CUSTOMIZE_THIS*/ to a unique name (two places). Go ahead, make it long. | |
// Like, your initials, and your full plugin name. | |
// e.g. MTJ_Some_Awesome_Plugin_Controller | |
/*CUSTOMIZE_THIS*/_Controller::init(); | |
class /*CUSTOMIZE_THIS*/_Controller { | |
function init() { |