Skip to content

Instantly share code, notes, and snippets.

@DanNi0130
Created February 24, 2019 07:18
Show Gist options
  • Save DanNi0130/e211230aa662dfbe418c13118cee27db to your computer and use it in GitHub Desktop.
Save DanNi0130/e211230aa662dfbe418c13118cee27db to your computer and use it in GitHub Desktop.
results = [] # We want our results to come back as a list
for i in range(len(commentlinks)):
thread = requests.get(commentlinks[i]) # Go to each link
cleanthread = bs4.BeautifulSoup(thread.text, 'html.parser')
link, commenter = scrapethread(cleanthread) # Scrape the data and return them to these variables
results.append(link + [commenter]) # Append the results - note that the link actually returns as a list, rather than a string
time.sleep(30)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment