Igor Brigadir igorbrigadir

Pop open "filter preferences" in adblock plus, and add the following rules to hide mentions from people who don't follow you (and who you don't follow).

For the interactions/notifications page:

twitter.com##.interaction-page [data-follows-you="false"][data-you-follow="false"]:not(.my-tweet)

For the mentions page:

twitter.com##.mentions-page [data-follows-you="false"][data-you-follow="false"]:not(.my-tweet)

Correlation is not causation (???)
No causation without manipulation. (Holland)
All models are wrong, some are useful. (Box)
Statistics is the science of uncertainty. (arguably Tukey)
Statistics is the science of learning from experience, especially experience that arrives a little bit at a time. (Efron)

version 1.0.3 # +

Text Analysis and Entity Resolution

####Entity resolution is a common, yet difficult problem in data cleaning and integration. This lab will demonstrate how we can use Apache Spark to apply powerful and scalable text analysis techniques and perform entity resolution across two datasets of commercial products.

Entity Resolution, or "[Record linkage][wiki]" is the term used by statisticians, epidemiologists, and historians, among others, to describe the process of joining records from one data source with another that describe the same entity. Our terms with the same meaning include, "entity disambiguation/linking", duplicate detection", "deduplication", "record matching", "(reference) reconciliation", "object identification", "data/information integration", and "conflation".

	############
	# Option 1: Get all solutions
	# using brute force
	############

	## This function creates every possible distribution of responses for
	## a likert scale with nlev responses. This is total brute force. There's
	## probably a better way.
	## Argument:
	## v : initially, the total number of responses

	datum,nummer,omschrijving,gebruik,aantal,type,catdt,herkomst,bestemming,waarde,Column 11,Column 12,Column 13,Column 14
	2004-05-01T00:00:00Z,2.3632284E7,(Tweede generatie) beeldversterkerbuizen,Nachtzichtsystemen,,Dual-use,D,Nederland,Chili Via België,25740.0,,,,
	2004-06-01T00:00:00Z,2.3653915E7,Dimethylamine,Vervaardiging van herbiciden,,Dual-use,D,Nederland,Kroatië,15000.0,,,,
	2004-07-01T00:00:00Z,2.3691973E7,Methyldiethanolamine,Tbv de olie-industrie,,Dual-use,D,Onbekend,Zuid-afrika,377834.0,,,,
	2004-08-01T00:00:00Z,2.3674858E7,Fosforoxychloride,Vervaardiging van harsen,,Dual-use,D,Onbekend,Turkije,5100.0,,,,
	2004-08-01T00:00:00Z,2.3711486E7,(Tweede generatie) beeldversterkerbuizen,Demonstratie,,Dual-use,T,Onbekend,Jordanië,6600.0,,,,
	2004-08-01T00:00:00Z,2.3711494E7,(Tweede generatie) beeldversterkerbuizen,Demonstratie,,Dual-use,T,Onbekend,Jordanië,2200.0,,,,
	2004-01-14T00:00:00Z,2.3477165E7,Fosfortrichloride,Vervaardiging van fosfieten,,Dual-use,D,Duitsland,Taiwan,525000.0,,,,
	2004-01-14T00:00:00Z,2.37292

	#!/bin/sh

	if [[ "${1}" == "http"* ]] ; then
	doi="${1}"
	else
	doi="http://dx.doi.org/${1}"
	fi

	# Stopped working around 2015-10-04.
	# curl -sLH "Accept: text/bibliography; style=bibtex" "${doi}" \| sed 's/^ *//'

	##vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv##
	## Download and examine deleted congress tweets ##
	## Data Source: politwoops.sunlightfoundation.com ##
	## Analysis: Katherine Ognyanova at www.kateto.net ##
	## Visualizations: http://kateto.net/politwoops ##
	##vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv##

	library(RJSONIO)
	library(RCurl)
	library(plyr)