Skip to content

Instantly share code, notes, and snippets.

@dgadiraju

dgadiraju/spark-core-cogroup.scala

Created September 10, 2019 13:38
Show Gist options
  • Save dgadiraju/dee40a413dffbef1afb24d1171b9857b to your computer and use it in GitHub Desktop.
Save dgadiraju/dee40a413dffbef1afb24d1171b9857b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-core-cogroup.scala
val orders = sc.textFile("/public/retail_db/orders")
val ordersMap = orders.
map(o => (o.split(",")(0).toInt, o))
val orderItems = sc.textFile("/public/retail_db/order_items")
val orderItemsMap = orderItems.
map(oi => (oi.split(",")(1).toInt, oi))
ordersMap.
cogroup(orderItemsMap).
take(10).
foreach(println)
ordersMap.
cogroup(orderItemsMap).
count
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment