Created
March 15, 2017 13:52
-
-
Save juriad/f8929117048bd86330d5c645820891c5 to your computer and use it in GitHub Desktop.
Create a histogram of people's balances using Spark
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class CDA { | |
| String creditor; | |
| String debtor; | |
| int amount; | |
| } | |
| public List<Tuple2<Integer, Integer>> createHistogram(JavaRDD<CDA> inputRDD) { | |
| return inputRDD | |
| .flatMapToPair(cda -> Arrays.asList( | |
| new Tuple2<>(cda.creditor, cda.amount), | |
| new Tuple2<>(cda.debtor, -cda.amount))) | |
| .reduceByKey(Integer::sum) | |
| .mapToPair(pa -> new Tuple2<>(pa._2 / 1000, 1)) | |
| .reduceByKey(Integer::sum) | |
| .sortByKey(true, 1) | |
| .collect(); | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment