Zecca Lehn ZeccaLehn

Web Scraping Workshop

Install our tools (preferably in a new virtualenv):

pip install beautifulsoup4

	text = """Sample #line of #text to demonstrate #hashtags
	extraction. Here is some duplicate ones: #text #hashtags #tags."""


	def get_hashtags(text, order=False):
	tags = set([item.strip("#.,-\"\'&*^!") for item in text.split() if (item.startswith("#") and len(item) < 256)])
	return sorted(tags) if order else tags


	print "\n".join(get_hashtags(text, True))

	This post examines the features of [R Markdown](http://www.rstudio.org/docs/authoring/using_markdown)
	using [knitr](http://yihui.name/knitr/) in Rstudio 0.96.
	This combination of tools provides an exciting improvement in usability for
	[reproducible analysis](http://stats.stackexchange.com/a/15006/183).
	Specifically, this post
	(1) discusses getting started with R Markdown and `knitr` in Rstudio 0.96;
	(2) provides a basic example of producing console output and plots using R Markdown;
	(3) highlights several code chunk options such as caching and controlling how input and output is displayed;
	(4) demonstrates use of standard Markdown notation as well as the extended features of formulas and tables; and
	(5) discusses the implications of R Markdown.

	# Function to implement a version of the improved moving average functionality
	# introduced and tested in:
	#
	# Papailias, Fotis and Thomakos, Dimitrios D.,
	# "An Improved Moving Average Technical Trading Rule,
	# (September 11, 2011). Available at SSRN: http://ssrn.com/abstract=1926376
	#
	# Original code written by Kent Russell @ timelyportfolio.com
	#
	# Cross-checked by Dimitrios Thomakos on 11/29/2011, @ quantf.com

	from numpy import loadtxt, zeros, ones, array, linspace, logspace, mean, std, arange
	from mpl_toolkits.mplot3d import Axes3D
	import matplotlib.pyplot as plt
	from pylab import plot, show, xlabel, ylabel

	#Evaluate the linear regression

	def feature_normalize(X):
	'''
	Returns a normalized version of X where

	mha <- 1 # mean under the alternative
	es <- 2 # observed effect (deviation from mean under H0)
	x <- seq(-6, 6, length=1000)
	dh0 <- dnorm(x, 0, 1)

	show.it <- function(es, mha, verbose=FALSE) {
	dh1 <- dnorm(x, mha, 1)

	plot.new()
	plot.window(xlim=range(x), ylim=c(0,.6))

	sudo apt-get install unzip;
	wget -O /tmp/chromedriver.zip http://chromedriver.googlecode.com/files/chromedriver_linux64_19.0.1068.0.zip && sudo unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/;