Mindaugas Zickus mindis

Build a scalable Twitter clone with Django and GetStream.io

In this tutorial we are going to build a Twitter clone using Django and GetStream.io, a hosted API for newsfeed development. We will show you how easy is to power your newsfeeds with GetStream.io. At the end of this tutorial we will have a Django app with a profile feed, a timeline feed, support for following users, hashtags and mentions.

I assume that you are familiar with Django. If you're new to Django the [official tutorial] (https://docs.Djangoproject.com/en/1.7/intro/install/) explains it very well.

Bootstrap the Django application

Numba Ball Tree

This is a quick attempt at writing a ball tree for nearest neighbor searches using numba. I've included a pure python version, and a version with numba jit decorators. Because class support in numba is not yet complete, all the code is factored out to stand-alone functions in the numba version. The resulting code produced by numba is about ~10 times slower than the cython ball tree in scikit-learn. My guess is that part of this stems from lack of inlining in numba, while the rest is due to some sort of overhead

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

	#!/usr/bin/python
	# -- coding: utf8 --

	# SAMPLE SUBMISSION TO THE BIG DATA HACKATHON 13-14 April 2013 'Influencers in a Social Network'
	# .... more info on Kaggle and links to go here
	#
	# written by Ferenc Huszár, PeerIndex

	from sklearn import linear_model
	from sklearn.metrics import auc_score

	#!/usr/bin/python
	# -- coding: utf8 --

	# SAMPLE SUBMISSION TO THE BIG DATA HACKATHON 13-14 April 2013 'Influencers in a Social Network'
	# .... more info on Kaggle and links to go here
	#
	# written by Ferenc Huszár, PeerIndex

	from sklearn import linear_model
	from sklearn.metrics import auc_score

	# sql.export.rf(): save a randomForest model as SQL
	# v0.03
	# Copyright (c) 2013-2014 Shane Butler <shane dot butler at gmail dot com>
	#
	# sql.export.rf is free software: you can redistribute it and/or modify it
	# under the terms of the GNU General Public License as published by
	# the Free Software Foundation, either version 2 of the License, or
	# (at your option) any later version.
	#
	# sql.export.rf is distributed in the hope that it will be useful, but

	# sql.export.gbm(): save a GBM model as SQL
	# v0.11
	# Copyright (c) 2013-2014 Shane Butler <shane dot butler at gmail dot com>
	#
	# sql.export.gbm is free software: you can redistribute it and/or modify it
	# under the terms of the GNU General Public License as published by
	# the Free Software Foundation, either version 2 of the License, or
	# (at your option) any later version.
	#
	# sql.export.gbm is distributed in the hope that it will be useful, but

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Text;
	using System.Threading.Tasks;

	using System.Web.Http;
	using System.Web.Http.Tracing;
	using System.Reflection;
	using Owin;

	def meanAveragedPrecision(k: Int, model: MatrixFactorizationModel, ratings: RDD[Rating]): Double = {
	def avgPrecisionK(actual: Seq[Int], predicted: Seq[Int], k: Int): Double = {
	val predK = predicted.take(k)
	var score = 0.0
	var numHits = 0.0
	for ((p, i) <- predK.zipWithIndex) {
	if (actual.contains(p) && !predK.take(i).contains(p)) {
	numHits += 1.0
	score += numHits / (i.toDouble + 1.0)
	}