Skip to content

Instantly share code, notes, and snippets.

Created March 12, 2013 13:35
Show Gist options
  • Save anonymous/5142892 to your computer and use it in GitHub Desktop.
Save anonymous/5142892 to your computer and use it in GitHub Desktop.
Snippet from run scrapy spider inside a python script
def stop_reactor():
reactor.stop() #Stops reactor to prevent script from hanging
if __name__ == '__main__':
#Handles engine_stopped to stop twisted reactor
dispatcher.connect(stop_reactor, signal=signals.engine_stopped)
spider = EstiloMASpider()
crawler = Crawler(Settings()) #Loads the settings
crawler.configure()
crawler.crawl(spider)
crawler.start()
#Start log and twisted reactor
log.start()
reactor.run()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment