Install Cygwin (http://www.cygwin.com/install.html)
Add the Cygwin bin directory to your PATH environment variable (e.g., setx PATH "%PATH%;C:\cygwin\bin")
Add the Conga, Inc. patch for Windows file permission exceptions to your project (https://github.com/congainc/patch-hadoop_7682-1.0.x-win)
Add code like this to your project:
SqoopOptions opts = new SqoopOptions();
opts.setConnectString(CONNECT_STRING);
opts.setUsername(USERNAME);
opts.setPassword(PASSWORD);
opts.setTableName(TABLE_NAME);
opts.setColumns(COLUMNS);
opts.setWhereClause(WHERE_CLAUSE);
opts.setTargetDir(TARGET_DIR);
opts.setNumMappers(1);
opts.setFileLayout(FileLayout.TextFile);
// Do use fields-terminated-by during import and
// input-fields-terminated-by during export. Don't reverse them
// http://www.slideshare.net/kate_ting/habits-of-effective-sqoop-users
opts.setFieldsTerminatedBy(DELIMITER);
Configuration conf = new Configuration();
// use this patch to allow Sqoop to run on Windows similar to
// http://stackoverflow.com/questions/15161984/use-apache-cascading-in-windows
if (System.getProperty("os.name").toLowerCase().indexOf("win") >= 0) {
conf.set("fs.file.impl",
"com.conga.services.hadoop.patch.HADOOP_7682.WinLocalFileSystem");
}
new Sqoop(new ImportTool(), conf, opts).run(null);If you want to run Sqoop from Eclipse, you may need to add a new JRE definition to Eclipse and mark it as the default. The JRE definition should point to the JRE folder in your JDK installation. For more info: http://www.codejava.net/ides/eclipse/managing-jre-installations-in-eclipse