Skip to content

Instantly share code, notes, and snippets.

@amferraz

amferraz/spider_start_url.py

Last active December 17, 2015 15:09
Show Gist options
  • Save amferraz/5629475 to your computer and use it in GitHub Desktop.
Save amferraz/5629475 to your computer and use it in GitHub Desktop.
Download ZIP
A simple spider with a custom start url
Raw
spider_start_url.py
class CustomSpider(BaseSpider):
def __init__(self, start_url=None, **kwargs):
super(BaseSpider, self).__init__(*args, **kwargs)
if start_url:
self.start_url = start_url
def start_requests(self):
return [Request(self.start_url)]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment