Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save dgadiraju/15403b894b745a003339e63787c53c00 to your computer and use it in GitHub Desktop.
Save dgadiraju/15403b894b745a003339e63787c53c00 to your computer and use it in GitHub Desktop.
val orders = sc.textFile("/public/retail_db/orders")
val ordersMap = orders.
map(o => (o.split(",")(1), 1))
ordersMap.
reduceByKey((agg, ele) => agg + ele).
take(10).
foreach(println)
val orderItems = sc.textFile("/public/retail_db/order_items")
val orderItemsMap = orderItems.
map(oi => (oi.split(",")(1).toInt, oi.split(",")(4).toFloat))
orderItemsMap.
reduceByKey((agg, ele) => agg.min(ele)).
take(10).
foreach(println)
val orderItems = sc.textFile("/public/retail_db/order_items")
val orderItemsMap = orderItems.
map(oi => (oi.split(",")(1).toInt, (oi.split(",")(4).toFloat, 1)))
orderItemsMap.
reduceByKey((agg, ele) => (agg._1 + ele._1, agg._2 + ele._2)).
take(10).
foreach(println)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment