Created
June 23, 2016 08:42
-
-
Save lazyval/55a60dc1d77d5fb70b4d2e373296962e to your computer and use it in GitHub Desktop.
Map Reduce 101
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/** | |
* Collects results of map phase, persists it to temporary storage | |
* results with same key (globally) can be accessed together | |
*/ | |
interface Emitter<M> { | |
void emit(String key, M result); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import java.util.Iterator; | |
/** | |
* Runs locally on each file | |
*/ | |
interface Mapper<M> { | |
void map(Iterator<String> lines, Emmiter<M> output); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import java.util.Iterator; | |
/** | |
* Runs on <some> host, one reducer for each group (group is defined by key at map phase), group is assembled from | |
* results from all hosts | |
*/ | |
interface Reducer<R> { | |
R reduce(Iterator<String> lines, Emmiter<M> output); | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment