mindis’s gists

mindis / spark_parallel_boost.py

Created October 1, 2015 13:32 — forked from wpm/spark_parallel_boost.py

A simple example of how to integrate the Spark parallel computing framework and the scikit-learn machine learning toolkit. This script randomly generates test and train data sets, trains an ensemble of decision trees using boosting, and applies the ensemble to the test set. The ensemble training is done in parallel.

	from pyspark import SparkContext

	import numpy as np

	from sklearn.cross_validation import train_test_split, Bootstrap
	from sklearn.datasets import make_classification
	from sklearn.metrics import accuracy_score
	from sklearn.tree import DecisionTreeClassifier

	def run(sc):

mindis / infiniteBacon-finished.html

Created October 8, 2015 11:12 — forked from SimplGy/infiniteBacon-finished.html

An infinite scroller built with Bacon using FRP techniques

	<!DOCTYPE html>

	<html>
	<head>
	<meta charset="utf-8">
	<title>FRP Infinite Scroll using Bacon.js</title>
	<script src="//cdnjs.cloudflare.com/ajax/libs/bacon.js/0.7.53/Bacon.min.js"></script>
	<script src="//cdnjs.cloudflare.com/ajax/libs/lodash.js/3.7.0/lodash.min.js"></script>
	<style type="text/css">
	body {

mindis / useful_pandas_snippets.py

Created January 14, 2016 17:45 — forked from bsweger/useful_pandas_snippets.md

Useful Pandas Snippets

	#List unique values in a DataFrame column
	pd.unique(df.column_name.ravel())

	#Convert Series datatype to numeric, getting rid of any non-numeric values
	df['col'] = df['col'].astype(str).convert_objects(convert_numeric=True)

	#Grab DataFrame rows where column has certain values
	valuelist = ['value1', 'value2', 'value3']
	df = df[df.column.isin(value_list)]

mindis / BitSetBenchmark.java

Created June 20, 2016 16:15 — forked from cpatni/BitSetBenchmark.java

Caliper Benchmarks for Analytics using Bitmaps

	package blog;

	import com.google.caliper.SimpleBenchmark;
	import java.util.BitSet;

	public class BitSetBenchmark extends SimpleBenchmark{
	private BitSet bitSet;

	@Override
	protected void setUp() {

mindis / zstdtest2.cpp

Created September 21, 2016 16:51 — forked from Lazin/zstdtest2.cpp

Zstandard test (block compression)

	#include "storage_engine/compression.h"
	#include "perftest_tools.h"

	#define ZSTD_STATIC_LINKING_ONLY
	#include <zstd.h>

	#include <iostream>
	#include <cstdlib>
	#include <algorithm>
	#include <zlib.h>

mindis / anything2vec.list

Created October 3, 2016 11:40 — forked from nzw0301/something2vec.md

	- word2vec https://arxiv.org/abs/1310.4546
	- sentence2vec, paragraph2vec, doc2vec http://arxiv.org/abs/1405.4053
	- tweet2vec http://arxiv.org/abs/1605.03481
	- tweet2vec https://arxiv.org/abs/1607.07514
	- author2vec http://dl.acm.org/citation.cfm?id=2889382
	- item2vec http://arxiv.org/abs/1603.04259
	- lda2vec https://arxiv.org/abs/1605.02019
	- illustration2vec http://dl.acm.org/citation.cfm?id=2820907
	- tag2vec http://ktsaurabh.weebly.com/uploads/3/1/7/8/31783965/distributed_representations_for_content-based_and_personalized_tag_recommendation.pdf
	- category2vec http://www.anlp.jp/proceedings/annual_meeting/2015/pdf_dir/C4-3.pdf

mindis / gist:676e0713c5f5a9a8f181d79ce5f81d01

Created November 4, 2016 14:30 — forked from dnbaker/gist:760d2cd79ae0fc0a67e8

A collection of links for streaming algorithms and data structures

General Background and Overview

Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems
Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

mindis / crop_morphology.py

Created July 10, 2017 15:57 — forked from lpillmann/crop_morphology.py

	#!/usr/bin/env python
	'''Crop an image to just the portions containing text.

	Usage:

	./crop_morphology.py path/to/image.jpg

	This will place the cropped image in path/to/image.crop.png.

	For details on the methodology, see

mindis / GAME.avsc

Created October 5, 2018 13:38 — forked from wjohnson/GAME.avsc

	{
	"type" : "record",
	"name" : "TrainingExample",
	"namespace" : "com.linkedin.metronome.avro.generated",
	"fields" : [ {
	"name" : "uid",
	"type" : [ "null", "string", "long", "int" ],
	"doc" : "a unique id for the training event",
	"default" : null
	}, {

mindis / avazu_ftrl_concurrent.go

Created October 12, 2018 14:19 — forked from ceshine/avazu_ftrl_concurrent.go

Kaggle Avazu Challenge: FTRL-Proximal with L1 & L2 implemented in Go (Concurrent/Multi-threaded)

	// Based on tinrtgu's Python script here:
	// https://www.kaggle.com/c/avazu-ctr-prediction/forums/t/10927/beat-the-benchmark-with-less-than-1mb-of-memory
	package main

	import (
	"encoding/csv"
	"os"
	"strconv"
	"hash/fnv"
	"math"

Mindaugas Zickus mindis