Skip to content

Instantly share code, notes, and snippets.

@drio
Created May 14, 2009 15:43
Show Gist options
  • Save drio/111723 to your computer and use it in GitHub Desktop.
Save drio/111723 to your computer and use it in GitHub Desktop.
I am trying to get my head around the sorting example. I use this as a input:
$ hadoop fs -cat /input/small*
9971681
9686036
2592322
4518219
1467363
607354
....
I use the identity classes for the map/reduce tasks:
$ cat sort_ident.sh
rm -rf ./output
hadoop fs -rmr /output
hadoop jar /home/drio/hadoop-0.20.0/contrib/streaming/hadoop-0.20.0-streaming.jar \
-mapper org.apache.hadoop.mapred.lib.IdentityMapper \
-reducer org.apache.hadoop.mapred.lib.IdentityReducer \
-input /input \
-output /output
The map tasks die with this exception:
java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment