Skip to content

Instantly share code, notes, and snippets.

@kiichi
Created December 24, 2015 21:00
Show Gist options
  • Select an option

  • Save kiichi/2a07dce7c9a500ba26e5 to your computer and use it in GitHub Desktop.

Select an option

Save kiichi/2a07dce7c9a500ba26e5 to your computer and use it in GitHub Desktop.
Simple Hadoop File IO Java Example
////////////////////////////////////////////////////////////////////////////////////////
// Environment:
// Cloudera Quickstart VM
//
// How to compile:
// export CLASSPATH=$CLASSPATH:.:/usr/lib/crunch/lib/hadoop-common.jar:/usr/lib/crunch/lib/hadoop-annotations.jar
// javac MyHadoopIOTest.java
// jar cvf MyHadoopIOTest.jar MyHadoopIOTest.class
// /usr/bin/hadoop jar MyHadoopIOTest.jar MyHadoopIOTest
//
// Reference:
// http://free-hadoop-tutorials.blogspot.com/2011/04/using-hdfs-programmatically.html
//
////////////////////////////////////////////////////////////////////////////////////////
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
public class MyHadoopIOTest {
public static void main(String[] args){
try {
System.out.println("Writing a file .... ");
Path path = new Path("hello.txt");
// Write a text file
FileSystem fs = FileSystem.get(new Configuration());
FSDataOutputStream fso = fs.create(path);
fso.writeUTF("hello world");
fso.close();
// Read the text file
FSDataInputStream fsi = fs.open(path);
String greeting = fsi.readUTF();
fsi.close();
System.out.println(greeting);
}catch (Exception ex){
System.out.println("Some error: " + ex.toString());
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment