Skip to content

Instantly share code, notes, and snippets.

@SivaArwin
Forked from fnneves/flight_scraper_2.py
Created December 27, 2019 15:18
Show Gist options
  • Save SivaArwin/fed8d5b39607545d3d1592d1676b9837 to your computer and use it in GitHub Desktop.
Save SivaArwin/fed8d5b39607545d3d1592d1676b9837 to your computer and use it in GitHub Desktop.
# Load more results to maximize the scraping
def load_more():
try:
more_results = '//a[@class = "moreButton"]'
driver.find_element_by_xpath(more_results).click()
# Printing these notes during the program helps me quickly check what it is doing
print('sleeping.....')
sleep(randint(45,60))
except:
pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment