Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save dgadiraju/2c0960542f99003980745d4da2909f39 to your computer and use it in GitHub Desktop.
Save dgadiraju/2c0960542f99003980745d4da2909f39 to your computer and use it in GitHub Desktop.
val orderItems = sc.textFile("/public/retail_db/order_items")
val orderItemsMap = orderItems.
map(oi => (oi.split(",")(1).toInt, oi.split(",")(4).toFloat))
val revenueAndCountPerOrder = orderItemsMap.
aggregateByKey((0.0, 0))(
(agg, ele) => (agg._1 + ele, agg._2 + 1),
(total, inter) => (total._1 + inter._1, total._2 + inter._2)
)
revenueAndCountPerOrder.
take(10).
foreach(println)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment