Skip to content

Instantly share code, notes, and snippets.

View esammer's full-sized avatar

Eric Sammer esammer

  • Decodable
  • San Francisco, CA USA
  • X @esammer
View GitHub Profile
@esammer
esammer / OptionBuilder.java
Created November 3, 2013 20:11
The lack of a good way to build (full featured) Option instances with Apache Commons CLI kills me. I use this.
/*
* Copyright 2013 Cloudera Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
python -c 'import random
r = random.Random()
ops = [ "fuck", "shit", "stack", "put it in a glass bowl", "take it off the wall" ]
for i in range(0, 10):
for j in range(0, 10):
print ops[int(round(r.uniform(0, len(ops) - 1)))],
print "\n"
'
// The MR contract says we're guaranteed to get:
// 1. each key exactly once.
// 2. keys in sorted order.
// 3. all values for each key.
//
// We are not guaranteed to get values in any particular order, however.
// Let's fix that.
void reduce(Text key, Iterator<LongWritable> values, Context<Text, LongWritable> context) {
// Create a copy buffer and slurp all values out of the iterator.
@esammer
esammer / gist:4680106
Last active December 11, 2015 23:58
damn scala you busted
scala> :silent
Switched off result printing.
scala> class Bad {
| var toStringCount: Int = 0
| override def toString(): String = { toStringCount += 1 ; super.toString() }
| }
scala> val b = new Bad
/*
* A simple example of user demographic dataset generation in scala.
*
* This highlights the following scala bits:
* - Java integration
* - Annotations
* - Tail recursion optimization
* - Enums
* - Vals vs. vars
* - Working with singleton objects
@esammer
esammer / gist:4042578
Created November 8, 2012 23:25
No description fully describes what this is.
} catch (NullPointerException e) {
throw npe(e, " of "+schema.getFullName());
}
}
/** Helper method for adding a message to an NPE. */
protected NullPointerException npe(NullPointerException e, String s) {
NullPointerException result = new NullPointerException(e.getMessage()+s);
result.initCause(e.getCause() == null ? e : e.getCause());
return result;
@esammer
esammer / gist:3777642
Created September 24, 2012 19:00
Eclipse template for generating a private static final slf4j logger
// Wire this up to an eclipse template called slf4j_logger and never type it again.
private static final Logger logger = LoggerFactory.getLogger(${enclosing_type}.class);
@esammer
esammer / gist:3716323
Created September 13, 2012 18:11
Eclipse template for generating fluent (builder) methods.
public ${enclosing_type} ${name:field}(${type:elemType(name)} ${name}) {
this.${name} = ${name};
return this;
}
@esammer
esammer / AvroProjectionMapFn.java
Created August 31, 2012 01:27
A Crunch MapFn that can project Avro generic record fields
public class AvroProjectionMapFn extends MapFn<Record, Record> {
private static final long serialVersionUID = 1L;
private List<String> fieldNames;
private Schema sourceSchema;
private Schema projectionSchema;
@Override
@esammer
esammer / fix-uid-gid.py
Created July 24, 2012 19:06
Load two /etc/passwd files and find all users and groups that need to be fixed on a machine.
#!/usr/bin/python
import sys
def load_passwd(path):
lookup = { }
f = file(path)
for line in f:
line = line.rstrip()