Created
March 8, 2018 20:13
-
-
Save s1ck/8731e4b1094b52b212ef1f17b9d2e12b to your computer and use it in GitHub Desktop.
Demonstrates loading a graph from Neo4j into CAPS and running a GraphX algorithm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// 1) Create CAPS session | |
implicit val session: CAPSSession = CAPSSession.local() | |
// 2) Load a graph from a running Neo4j instance. | |
implicit val neo4jConfig = Neo4jConfig(uri = URI.create("bolt://localhost"), password = Some("password")) | |
// 3) Create a data source | |
val neo4jSource = new Neo4jPropertyGraphDataSource(neo4jConfig) | |
// 4) Load the whole Neo4j DB | |
val graph = neo4jSource.graph(GraphName("graph")) | |
// 5) Get nodes for GraphX graph construction | |
val nodes = graph.cypher( | |
"""|MATCH (n:Person) | |
|RETURN id(n), n.name""".stripMargin) | |
val rels = graph.cypher( | |
"""|MATCH (:Person)-[r]->(:Person) | |
|RETURN startNode(r), endNode(r) | |
""".stripMargin) | |
// 4) Create GraphX compatible RDDs from nodes and relationships | |
val graphXNodeRDD = nodes.getRecords.asDataFrame.rdd.map(row => row.getLong(0) -> row.getString(1)) | |
val graphXRelRDD = rels.getRecords.asDataFrame.rdd.map(row => Edge(row.getLong(0), row.getLong(1), ())) | |
// 5) Compute Page Rank via GraphX | |
val graph = Graph(graphXNodeRDD, graphXRelRDD) | |
val ranks = graph.pageRank(0.0001).vertices | |
// 6) Print Page Ranks | |
ranks.show() |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment