This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
upload_dict = { | |
"fields": { | |
"URL" : article.url, | |
"Title" : article.title, | |
"Date" : publish_date(article), | |
"Author(s)" : article.authors, | |
"Site Name" : article.meta_data['og']['site_name'], | |
"Media" : determine_media(article), | |
"Opinion" : determine_opinion(article), | |
"Keywords" : article.keywords, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import newspaper | |
article = newspaper.Article(urls[0]) | |
article.download() | |
article.parse() | |
article.nlp() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
headers = {"Authorization" : "Bearer [INSERT AIRTABLE API KEY HERE]"} | |
params = ( | |
('maxRecords', '100'), | |
('view', 'Grid view')) | |
URL = 'https://api.airtable.com/v0/appgaUkvDLKnYOmdA/URLS%20to%20Process' | |
response = requests.get(URL, params = params, headers = headers) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
import numpy as np | |
def strange_zeroes(dataset, drop_columns = None, zero = 0): | |
"""Exploratory Data Analysis function that looks to see if there is any correlation between '0' values occurerence, | |
which would suggest a relationship between the instruments used and their non-detection of a value.""" | |
# Find all zeroes (can be altered to including other common missing / non-response values as well) | |
zeroes = (dataset == zero) | |
NewerOlder