- Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
- Models and Issues in Data Stream Systems
- Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
- Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
- [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package thunder.streaming | |
import org.apache.spark.{SparkConf, Logging} | |
import org.apache.spark.rdd.RDD | |
import org.apache.spark.SparkContext._ | |
import org.apache.spark.streaming._ | |
import org.apache.spark.streaming.dstream.DStream | |
import org.apache.spark.mllib.clustering.KMeansModel | |
import scala.util.Random.nextDouble |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudo yum update | |
sudo yum install python27 | |
curl https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py | sudo /usr/bin/python27 | |
sudo easy_install pip | |
echo "alias python='python27'" >> ~/.bashrc | |
source ~/.bashrc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import numpy as np | |
import pylab as pl | |
x = np.random.uniform(1, 100, 1000) | |
y = np.log(x) + np.random.normal(0, .3, 1000) | |
pl.scatter(x, y, s=1, label="log(x) with noise") | |
pl.plot(np.arange(1, 100), np.log(np.arange(1, 100)), c="b", label="log(x) true function") | |
pl.xlabel("x") | |
pl.ylabel("f(x) = log(x)") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- Remove the history from | |
rm -rf .git | |
-- recreate the repos from the current content only | |
git init | |
git add . | |
git commit -m "Initial commit" | |
-- push to the github remote repos ensuring you overwrite history | |
git remote add origin [email protected]:<YOUR ACCOUNT>/<YOUR REPOS>.git |
Locate the section for your github remote in the .git/config
file. It looks like this:
[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = [email protected]:joyent/node.git
Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/*
to this section. Obviously, change the github url to match your project's URL. It ends up looking like this:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# | |
# Copyright (c) 2012 Kyle Gorman | |
# | |
# Permission is hereby granted, free of charge, to any person obtaining a copy | |
# of this software and associated documentation files (the "Software"), to deal | |
# in the Software without restriction, including without limitation the rights | |
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |
# copies of the Software, and to permit persons to whom the Software is | |
# furnished to do so, subject to the following conditions: |
NewerOlder