- Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
- Models and Issues in Data Stream Systems
- Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
- Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
- [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var http = require("http"), | |
url = require("url"), | |
path = require("path"), | |
fs = require("fs") | |
port = process.argv[2] || 8888; | |
http.createServer(function(request, response) { | |
var uri = url.parse(request.url).pathname | |
, filename = path.join(process.cwd(), uri); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var http = require("http"), | |
url = require("url"), | |
path = require("path"), | |
fs = require("fs"), | |
mime = require("mime") | |
port = process.argv[2] || 8888; | |
http.createServer(function(request, response) { | |
var uri = url.parse(request.url).pathname |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# References: | |
# http://blog.mixu.net/2011/08/13/nginx-websockets-ssl-and-socket-io-deployment/ | |
# http://blog.exceliance.fr/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/ | |
# | |
global | |
nbproc 2 | |
maxconn 16384 | |
defaults |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var util = require('util') | |
, Transform = require('stream').Transform | |
, StreamCombiner = require('./streamcombiner'); | |
var chunks1 = []; | |
var stream1 = new Transform(); | |
var soFar = ''; | |
stream1._transform = function(chunk, encoding, done) { | |
chunks1.push(chunk.toString()); | |
var pieces = (soFar + chunk).split('\n'); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var net = require('net'); | |
var util = require('util'); | |
var Writable = require('stream').Writable; | |
/** | |
* Pretends to consume the data written to it. In reality, it just eats data | |
* really slowly. | |
* | |
* @constructor | |
* @extends Writable |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Part of https://github.com/chris-rock/node-crypto-examples | |
var crypto = require('crypto'), | |
algorithm = 'aes-256-ctr', | |
password = 'd6F3Efeq'; | |
function encrypt(buffer){ | |
var cipher = crypto.createCipher(algorithm,password) | |
var crypted = Buffer.concat([cipher.update(buffer),cipher.final()]); | |
return crypted; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Require node modules. | |
var http = require( "http" ); | |
var fileSystem = require( "fs" ); | |
var stream = require( "stream" ); | |
// Create an instance of our http server. | |
var httpServer = http.createServer( | |
function handleRequest( request, response ) { | |
// Create our file-input stream. |
I'm still very new to Kafka, eventsourcing, stream processing, etc. I'm in the middle of building my first production system with this stuff and am writing this at the request of a few folks on Twitter. So if you do have experience, please do me and anyone else reading this a favor by pointing out things I get wrong :)
- The Log — http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
- Turning the database inside out — http://www.confluent.io/blog/2015/03/04/turning-the-database-inside-out-with-apache-samza/
- Why local state is a fundamental primitive in stream processing — http://radar.oreilly.com/2014/07/why-local-state-is-a-fundamental-primitive-in-stream-processing.html
- Samza
- Various functional systems been exposed to over past year or so, React, Flux, RX, etc.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// How to connect 5 publishers with 5 subscribers | |
// over TCP using ZeroMQ's XPUB/XSUB proxy. | |
// sub (connect) | |
// <-8701-> | |
// (bind) xpub <---> xsub (bind) | |
// <-8700-> | |
// (connect) pub | |
var zmq = require('zmq'); |
OlderNewer