Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save dgadiraju/7222ca51f511a39066f99face5b1f0a9 to your computer and use it in GitHub Desktop.
Save dgadiraju/7222ca51f511a39066f99face5b1f0a9 to your computer and use it in GitHub Desktop.
val orders = sc.textFile("/public/retail_db/orders")
val ordersMap = orders.
map(o => (o.split(",")(1), 1))
ordersMap.
reduceByKeyLocally((agg, ele) => agg + ele).
take(10).
foreach(println)
val orderItems = sc.textFile("/public/retail_db/order_items")
val orderItemsMap = orderItems.
map(oi => (oi.split(",")(1).toInt, oi.split(",")(4).toFloat))
orderItemsMap.
reduceByKeyLocally((agg, ele) => agg.min(ele)).
take(10).
foreach(println)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment