This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| upload_dict = { | |
| "fields": { | |
| "URL" : article.url, | |
| "Title" : article.title, | |
| "Date" : publish_date(article), | |
| "Author(s)" : article.authors, | |
| "Site Name" : article.meta_data['og']['site_name'], | |
| "Media" : determine_media(article), | |
| "Opinion" : determine_opinion(article), | |
| "Keywords" : article.keywords, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import newspaper | |
| article = newspaper.Article(urls[0]) | |
| article.download() | |
| article.parse() | |
| article.nlp() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| headers = {"Authorization" : "Bearer [INSERT AIRTABLE API KEY HERE]"} | |
| params = ( | |
| ('maxRecords', '100'), | |
| ('view', 'Grid view')) | |
| URL = 'https://api.airtable.com/v0/appgaUkvDLKnYOmdA/URLS%20to%20Process' | |
| response = requests.get(URL, params = params, headers = headers) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| import numpy as np | |
| def strange_zeroes(dataset, drop_columns = None, zero = 0): | |
| """Exploratory Data Analysis function that looks to see if there is any correlation between '0' values occurerence, | |
| which would suggest a relationship between the instruments used and their non-detection of a value.""" | |
| # Find all zeroes (can be altered to including other common missing / non-response values as well) | |
| zeroes = (dataset == zero) | |
NewerOlder