This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@Overide | |
public void configure(JobConf job) { | |
super.configure(job); | |
config = job; | |
BECOMES | |
@Override | |
public void setup(Mapper<InputKey, InputValue, OutputKey, OutputValue>.Context context) | |
throws InterruptedException, IOException { | |
super.setup(context); | |
config = context.getConfiguration(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@Override | |
public void close() throws IOException { | |
BECOMES | |
@Override | |
public void cleanup(Context context) throws IOException, InterruptedException { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
reporter.incrCounter(family, type, 1); | |
BECOMES | |
context.getCounter(family, type).increment(1); | |
output.collect(key, value); | |
BECOMES | |
context.write(key, value); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@Override | |
public void map(InputKey key, InputValue value, | |
OutputCollector<OutputKey, OutputValue> output, Reporter reporter) | |
throws IOException { | |
BECOMES | |
@Override | |
public void map(InputKey key, InputValue value, | |
Mapper<InputKey, InputValue, OutputKey, OutputValue>.Context context) | |
throws IOException, InterruptedException { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public class MyMapper extends MapReduceBase implements | |
Mapper<InputKey, InputValue, OutputKey, OutputValue> { | |
BECOMES | |
public class MyMapper extends | |
Mapper<InputKey, InputValue, OutputKey, OutputValue> { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import org.apache.hadoop.mapred.JobConf; | |
import org.apache.hadoop.mapred.Mapper; | |
BECOME | |
import org.apache.hadoop.mapreduce.Job; | |
import org.apache.hadoop.mapreduce.Mapper; | |
import org.apache.hadoop.mapred.FileInputFormat; | |
import org.apache.hadoop.mapred.SequenceFileInputFormat; | |
BECOME | |
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<build> | |
<plugins> | |
<plugin> | |
<artifactId>maven-antrun-plugin</artifactId> | |
<executions> | |
<execution> | |
<phase>compile</phase> | |
<configuration> | |
<tasks> | |
<property name="runtime_classpath" refid="maven.runtime.classpath" /> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<target name="compile" depends="resolve" description="--> compile the project" unless="module.uptodate"> | |
<mkdir dir="${classes.dir}" /> | |
<javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="lib.path.id" debug="true" includeantruntime="false"/> | |
</target> | |
<target name="jar" depends="version, compile, copyclasses" description="--> make a jar file for this project" unless="module.uptodate"> | |
<jar destfile="${jar.file}"> | |
<fileset dir="${classes.dir}" /> | |
<manifest> | |
<attribute name="Built-By" value="${user.name}"/> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<target name="check.retrieve.necessary" description="Only retrieve jars with Ivy if necessary"> | |
<ivy:resolve file="${ivy.file}" transitive="true"/> | |
<uptodate property="libs.uptodate"> | |
<srcfiles dir="." includes="${ivy.file}"/> | |
<mapper type="merge" to="{lib.dir}/.done"/> | |
</uptodate> | |
</target> | |
<target name="resolve" depends="check.retrieve.necessary" description="--> resolve and retrieve dependencies with ivy" | |
unless="libs.uptodate"> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!-- resolve the job and test dependencies to different directories --> | |
<target name="resolve" depends="clean-lib" description="--> resolve and retrieve dependencies with ivy"> | |
<mkdir dir="${job.lib.dir}"/> <!-- not usually necessary, ivy creates the directory IF there are dependencies --> | |
<mkdir dir="${test.lib.dir}"/> | |
<ivy:resolve file="${ivy.file}" transitive="true"/> | |
<ivy:retrieve pattern="${job.lib.dir}/[artifact]-[revision].[ext]" conf="job" /> | |
<ivy:retrieve pattern="${test.lib.dir}/[artifact]-[revision].[ext]" conf="test" /> | |
</target> |