Skip to content

Instantly share code, notes, and snippets.

@cwensel
Created April 5, 2013 21:25
Show Gist options
  • Select an option

  • Save cwensel/5322746 to your computer and use it in GitHub Desktop.

Select an option

Save cwensel/5322746 to your computer and use it in GitHub Desktop.
String statement = "select *\n"
+ "from \"example\".\"sales_fact_1997\" as s\n"
+ "join \"example\".\"employee\" as e\n"
+ "on e.\"EMPID\" = s.\"CUST_ID\"";
Tap empTap = getPlatform().getDelimitedFile( ",", "\"", new SQLTypeResolver(), DATA_EMPLOYEE, SinkMode.KEEP );
Tap salesTap = getPlatform().getDelimitedFile( ",", "\"", new SQLTypeResolver(), DATA_SALESFACT, SinkMode.KEEP );
Tap resultsTap = getPlatform().getDelimitedFile( ",", "\"", new SQLTypeResolver(), getOutputPath( "dynamic" ), SinkMode.REPLACE );
FlowDef flowDef = FlowDef.flowDef()
.setName( "sql flow" )
.addSource( "employee", empTap )
.addSource( "sales_fact_1997", salesTap )
.addSink( "results", resultsTap );
SQLPlanner sqlPlanner = new SQLPlanner()
.setDefaultSchema("example")
.setSql( statement );
flowDef.setAssemblyPlanner( sqlPlanner );
Flow flow = getPlatform().getFlowConnector().connect( flowDef );
flow.complete();
@cwensel
Copy link
Author

cwensel commented Apr 5, 2013

Ignore the getPlatform() stuff, that's part of the Cascading test harness for JUnit.

so yes, in case you just realized, you can write Hadoop independent tests in Cascading, even when using SQL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment