Skip to content

Instantly share code, notes, and snippets.

class PongActor extends Actor {
def act() {
val y = 4
loop {
react {
case Ping(2) =>
println("got ping(2)")
sender ! Pong
case Ping(y) =>
println("got ping(y)")
a !? (2000, WantNext("group", "test")) match {
case m1: Some[NextMessage] =>
println("case Some[NextMessage]")
println(m1)
case m1: Some[NoMessage] =>
println("case Some[NoMessage]")
println(m1)
case m1: Option[Nothing] =>
println("case Option[Nothin]")
println(m1)
@mumrah
mumrah / SolrUpdater.java
Last active December 11, 2015 16:28
Example of multi-threading Solr updates
public class SolrUpdater implements Runnable {
private final SolrInputDocument doc = new SolrInputDocument();
private final UpdateRequest req = new UpdateRequest();
private final SolrServer solr;
private final BlockingQueue<String> strings;
private final AtomicLong id;
public SolrUpdater(SolrServer solr, BlockingQueue<String> strings,
AtomicLong id) {
this.solr = solr;
@mumrah
mumrah / SolrUpdater.java
Last active July 13, 2016 15:13
Example of batching and multi-threading Solr updates
public static class SolrUpdater implements Runnable {
private final UpdateRequest req = new UpdateRequest();
private final SolrServer solr;
private final BlockingQueue<String> strings;
private final AtomicLong id;
private final int batchSize = 100;
private volatile int batchedUpdates = 0;
public SolrUpdater(SolrServer solr, BlockingQueue<String> strings,
AtomicLong id) {
@mumrah
mumrah / gist:5265560
Created March 28, 2013 18:16
Snippet from schema.xml showing docValue fields
<fields>
<field name="id" type="string" indexed="true" stored="true" required="true"/>
<field name="_version_" type="long" indexed="true" stored="true"/>
<!-- DocValue fields -->
<field name="threadId_dv" type="string" indexed="false" stored="false" docValues="true" default=""/>
<field name="docId_dv" type="tint" indexed="false" stored="false" docValues="true" default="0"/>
<field name="wordId_dv" type="tint" indexed="false" stored="false" docValues="true" default="0"/>
<field name="word_dv" type="string" indexed="false" stored="false" docValues="true" default=""/>
<field name="count_dv" type="tint" indexed="false" stored="false" docValues="true" default="0"/>
@mumrah
mumrah / Indexer.java
Created March 28, 2013 18:21
Java class to index NYTimes data into Solr. Data available here http://archive.ics.uci.edu/ml/datasets/Bag+of+Words. Beware of hard coded paths and urls!
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.File;
import java.io.IOException;
length, 29: 00 00 00 1d
api key: 00 07
api version: 00 00
correlation 42: 00 00 00 2a
clientId "foo": 00 03 66 6f 6f
group "test-group": 00 0a 74 65 73 74 2d 67 72 6f 75 70
array length: 00 00 00 00
@mumrah
mumrah / serialize.py
Created June 8, 2013 01:52
Serializing operations in Python, with blocking behavior on the caller side
from threading import Thread, Event
from Queue import Queue
class Proc(Thread):
def __init__(self, in_queue):
Thread.__init__(self)
self.in_queue = in_queue
self.die = Event()
def stop(self):
@mumrah
mumrah / AddToMap.java
Created June 11, 2013 02:39
A Pig UDF that allows you to modify a map by adding additional key/value pairs
import java.io.IOException;
import java.util.Map;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.DataType;
import org.apache.pig.data.Tuple;
/**
* Simple UDF to allow modifying an existing map[] datum
*
* Usage:
@mumrah
mumrah / script.pig
Last active December 18, 2015 10:29 — forked from rohit-parimi/gist:5768968
/*Pig script to convert the user,movie,rating,timestamp data to a user-user graph for running adsorption algorithm.
The format of the input data is
1::122::5::838985046
*/
/*Loading the data into a table. The delimiter might be different for different inputs. */