I succesfully managed to setup a local development environment for Bookie in my Mac OSX machine and I'd like to share a few notes. The required changes are just small details and I guess it would be easy to edit the Makefile in order to make it work with Mac OSX, but actually I'm not an expert on it...
OS: Mac OSX 10.8.5
Xcode: 4.6.3
| class ScraperManager: | |
| def __init__(self): | |
| self.scraper = ThisIsTheScraperYouWroteWithScrapy() | |
| def start_session(self): | |
| # Loop over all elements that Scrapy finds. | |
| for files, metadata in self.scraper.run(): | |
| # `files`: a list of URLs of actual files to download for the current paper, like: | |
| # ['http://www.narcis.com/paper1-part1.pdf', 'http://www.narcis.com/paper1-part2.pdf'] | |
| # `metadata`: a dictionary of metadata for the current paper, like: |
| # Create an SSH tunnel in Python. | |
| # In this example we suppose we are in a firewalled network where we cannot | |
| # access GMAIL IMAP server on port 993. | |
| # So we create a tunnel to a machine out of the firewalled network and forward | |
| # to GMAIL IMAP server. | |
| # | |
| # Requirements: | |
| # pip install paramiko | |
| # | |
| # Inspired by: |