neilkod

define x and y

x<-c(2.5,.5,2.2,1.9,3.1,2.3,2,1,1.5,1.1)
y<-c(2.4,.7,2.9,2.2,3.0,2.7,1.6,1.1,1.6,.9)

create a covariance matrix the hard way

cm<-matrix(data=c(cov(x,x),cov(x,y),cov(y,x),cov(y,y)),nrow=2,ncol=2)
> cm
          [,1]      [,2]

neilkod

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
9a:d0:b1:46:88:e5:d2:......
Please contact your system administrator.
Add correct host key in /Users/nkodner/.ssh/known_hosts to get rid of this message.

neilkod

#!/usr/bin/python
import sys
import tweepy
def intersect(a, b):
     return list(set(a) & set(b))


CONSUMER_KEY = 'xxx'
CONSUMER_SECRET = 'yyy'
ACCESS_KEY = 'zzzz'

neilkod

-- load the amazon piggybank.jar and the standard piggybank.jar

register ../piggybank-0.3-amzn.jar
register ../piggybank.jar

-- function definitions.

DEFINE DATE_TIME    org.apache.pig.piggybank.evaluation.datetime.DATE_TIME();
DEFINE EXTRACT org.apache.pig.piggybank.evaluation.string.EXTRACT();
DEFINE FORMAT_DT org.apache.pig.piggybank.evaluation.datetime.FORMAT_DT();

neilkod

neil-kodners-MacBook-Pro:r_recommendation_system nkodner$ ls
README.markdown		data			extract_graphs.R	get_packages.R		install_packages.R	model_assessment.R
cran			example_model.R		get_maintainers.R	get_views.rb		lib
neil-kodners-MacBook-Pro:r_recommendation_system nkodner$ R

R version 2.11.1 (2010-05-31)
Copyright (C) 2010 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R is free software and comes with ABSOLUTELY NO WARRANTY.

neilkod

-- load all of the parsed tweets as (id, timestamp, screenname, tweet)

raw = LOAD 'parsed' USING PigStorage('\t') AS (id:chararray,timestamp:chararray,screenname:chararray,tweet:chararray);

-- limit the twitter data to where screenname matches bieber, case-insensitive
fltr = FILTER raw BY screenname matches '.*?[Bb][Ii][Ee][Bb][Ee][Rr].*?';

-- group by screenname to count the tweets
grpd = GROUP fltr BY screenname;

neilkod

miniserver:conf nkodner$ cat core-site.xml 
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
     <property>
         <name>fs.default.name</name>
         <value>hdfs://hadoop1.local:9000</value>
     </property>

neilkod

ar the malaysian must be eliminated mugatu what no i dont have time for this p
t them clawing their faces for more mugatu sucks support the prime minister mu
tu sucks support the prime minister mugatu uses slave labor down with mugatu y
r mugatu uses slave labor down with mugatu you hate to see something like that
ul people theres no denying jacobim mugatu has used cheap malaysian workers to
ts newsstands tomorrow excuse me mr mugatu mr mugatu matilda jeffries time mag
nds tomorrow excuse me mr mugatu mr mugatu matilda jeffries time magazine any 
loiting cheap malaysian workers hey mugatu screw you and your little dog too l
nding out what that is thank you mr mugatu mr mugatu if i could just have a mo
what that is thank you mr mugatu mr mugatu if i could just have a moment of yo

neilkod

#!/usr/bin/python
import nltk
import string

f = open('zoolander.txt','r').read()

# favorite way to strip punctuation, found on stackoverflow
# http://stackoverflow.com/questions/265960
# it helped the concordance results a little bit but this might
# not be the best approach.  

neilkod

 blue suit and your spiky black hair kill And youre a super hot ninja machine 
ts going on I think Katinka wants to kill you Good I deserve to die if I cant 
ogrammed you But I wont do it I wont kill anybody Its not up to you At the pro
it out Derek has been brainwashed to kill the prime minister of Malaysia And A
k Der wait If you go theyll make you kill that Eurasian dude I dont care Hanse
Do what youve been trained to do and kill the prime minister J ust do it al re
ghting Derek Zoolander just tried to kill the prime minister of Malaysia Thats