Skip to content

Instantly share code, notes, and snippets.

View butlermh's full-sized avatar

Mark H. Butler butlermh

  • Santa Clara, United States
View GitHub Profile
@butlermh
butlermh / hadoopexample3
Created June 9, 2011 10:30
Changing Mapper for the new Hadoop API 2
@Override
public void map(InputKey key, InputValue value,
OutputCollector<OutputKey, OutputValue> output, Reporter reporter)
throws IOException {
BECOMES
@Override
public void map(InputKey key, InputValue value,
Mapper<InputKey, InputValue, OutputKey, OutputValue>.Context context)
throws IOException, InterruptedException {
@butlermh
butlermh / hadoopexample4
Created June 9, 2011 10:34
Changing mapper to new Hadoop API 3
reporter.incrCounter(family, type, 1);
BECOMES
context.getCounter(family, type).increment(1);
output.collect(key, value);
BECOMES
context.write(key, value);
@butlermh
butlermh / hadoopexample5
Created June 9, 2011 10:37
Changing mapper to new Hadoop API 4
@Override
public void close() throws IOException {
BECOMES
@Override
public void cleanup(Context context) throws IOException, InterruptedException {
@butlermh
butlermh / hadoopexample6
Created June 9, 2011 10:39
Changing Mapper for new Hadoop API 5
@Overide
public void configure(JobConf job) {
super.configure(job);
config = job;
BECOMES
@Override
public void setup(Mapper<InputKey, InputValue, OutputKey, OutputValue>.Context context)
throws InterruptedException, IOException {
super.setup(context);
config = context.getConfiguration();
@butlermh
butlermh / hadoopexample7
Created June 9, 2011 10:42
Changing mapper for new Hadoop API 6
String application_value = job.get("app.value");
BECOMES
String application_value = config.get("app.value");
Path[] localArchives = DistributedCache.getLocalCacheArchives(job);
BECOMES
Path[] localArchives = DistributedCache.getLocalCacheArchives(config);
Class handlerClass = job.getClass(handlerName, SomeProcessor.class);
BECOMES
@butlermh
butlermh / hadoopexample8
Created June 9, 2011 10:48
Changing Driver for new Hadoop API 1
JobConf job = new JobConf(getConf());
BECOMES
Job job = new Job(getConf());
@butlermh
butlermh / hadoopexample9
Created June 9, 2011 10:50
Changing Driver for new Hadoop API 2
job.setInputFormat(SequenceFileInputFormat.class);
job.setOutputFormat(SequenceFileOutputFormat.class);
BECOMES
job.setInputFormatClass(SequenceFileInputFormat.class);
job.setOutputFormatClass(SequenceFileOutputFormat.class);
@butlermh
butlermh / hadoopexample10
Created June 9, 2011 10:52
Changing Driver for the new Hadoop API 3
DistributedCache.addCacheArchive(new URI(path), job);
BECOMES
DistributedCache.addCacheArchive(new URI(path), job.getConfiguration());
job.set("some_property", property_value);
BECOMES
job.getConfiguration().set("some_property", property_value);
@butlermh
butlermh / hadoopexample11
Created June 9, 2011 10:54
changing driver for the new Hadoop API 4
JobClient.runJob(job);
BECOMES
job.waitForCompletion(true);
@butlermh
butlermh / build.gradle
Created June 11, 2011 17:29
Building ElasticSearch in Eclipse 1
dependsOn(':test-testng')
apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'eclipse'