Skip to content

Instantly share code, notes, and snippets.

@mGalarnyk

mGalarnyk/additionalStartUrlsScrapy.py

Created September 26, 2017 03:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mGalarnyk/b4d0ccaac939441a51771202ac0e1b1d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mGalarnyk/b4d0ccaac939441a51771202ac0e1b1d to your computer and use it in GitHub Desktop.
Download ZIP
How many additional start_urls we want for the blog post: and youtube tutorial: https://youtu.be/O_j3OTXw2_E
Raw
additionalStartUrlsScrapy.py
start_urls = ["https://fundrazr.com/find?category=Health"]
npages = 2
# This mimics getting the pages using the next button.
for i in range(2, npages + 2 ):
start_urls.append("https://fundrazr.com/find?category=Health&page="+str(i)+"")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment