Quick HOWTO installation | Python's scrapy package | OSX 10.10.5 | Homebrew
- Using homebrew make sure you have all dependencies for python's resolved to prevent any issue from breaking packages.
$brew install python
- If the previous step was satisfactory then create a generic directory for python's projects
$mkdir pydir
$cd pydir
- In order to isolate the project enviroment dependencies install virtualenv
$pip install virtualenv
- If the previous step was satisfactory then initialize the virtual enviroment on the same very path you are in pydir
$virtualenv .
- The skeleton for the enviroment must be bootstraped and now it is time to tell the system where to look for this project.
$source bin/activate
- The route of the directory should be prepended to your $PATH. Make sure the path is there.
$echo $PATH
- Time to install scrapy using PIP
$pip install scrapy
- Start scrapy project
$scrapy startproject crawler
- Finally the skeleton of the scrapy project has been generated and one example is provided for you to get started.
scrapy genspider example example.com
Created spider 'example' using template 'basic' in module:
crawler.spiders.example