Skip to content

Instantly share code, notes, and snippets.

@jayunit100
Created February 9, 2015 17:23
Show Gist options
  • Save jayunit100/6d8de94c8c9ea7fea6bb to your computer and use it in GitHub Desktop.
Save jayunit100/6d8de94c8c9ea7fea6bb to your computer and use it in GitHub Desktop.
/**
* Transform the non-sparksql mappable calendar
* into a spark sql freindly field.
*/
val mappableTransactions:RDD[TransactionM] =
r._5.map(trans =>
TransactionM(trans.customerId, trans.transactionId, trans.storeId, Map("dayOfWeek"->"Wed"), trans.productId));
mappableTransactions.registerTempTable("transactionsM");
val results: SchemaRDD = sql("SELECT * FROM transactionsM")
return Statistics(results.count());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment