Lowell130
/ ScrapeGoogle.py
Created
January 7, 2018 19:27
— forked from EdmundMartin/ScrapeGoogle.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| from bs4 import BeautifulSoup | |
| import time | |
| USER_AGENT = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'} | |
| def fetch_results(search_term, number_results, language_code): | |
| assert isinstance(search_term, str), 'Search term must be a string' | |
| assert isinstance(number_results, int), 'Number of results must be an integer' |
Lowell130
/ parsing.py
Created
January 7, 2018 19:28
— forked from krmaxwell/parsing.py
Meta tags and BeautifulSoup
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from bs4 import BeautifulSoup | |
| soup = BeautifulSoup(response) | |
| metatags = soup.find_all('meta',attrs={'name':'generator'}) | |
| for tag in metatags: | |
| print tag |