Skip to content

Instantly share code, notes, and snippets.

@esammer
Created May 29, 2013 22:27
Show Gist options
  • Select an option

  • Save esammer/5674345 to your computer and use it in GitHub Desktop.

Select an option

Save esammer/5674345 to your computer and use it in GitHub Desktop.
// The MR contract says we're guaranteed to get:
// 1. each key exactly once.
// 2. keys in sorted order.
// 3. all values for each key.
//
// We are not guaranteed to get values in any particular order, however.
// Let's fix that.
void reduce(Text key, Iterator<LongWritable> values, Context<Text, LongWritable> context) {
// Create a copy buffer and slurp all values out of the iterator.
List<Long> bufferedValues = new ArrayList<Long>();
for (LongWritable value : values) {
bufferedValues.add(value.get());
}
// Sort the values (if you want) and iterator over them.
for (Long value : Collections.sort(bufferedValues)) {
context.output(key, value);
}
}
Assuming the input is:
reduce("a", [ 4, 3, 2 ]);
reduce("b", [ 1, 2 ]);
The output is:
"a", 2
"a", 3
"a", 4
"b", 1
"b", 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment