Skip to content

Instantly share code, notes, and snippets.

@s1ck
Created March 8, 2018 20:13
Show Gist options
  • Save s1ck/8731e4b1094b52b212ef1f17b9d2e12b to your computer and use it in GitHub Desktop.
Save s1ck/8731e4b1094b52b212ef1f17b9d2e12b to your computer and use it in GitHub Desktop.
Demonstrates loading a graph from Neo4j into CAPS and running a GraphX algorithm
// 1) Create CAPS session
implicit val session: CAPSSession = CAPSSession.local()
// 2) Load a graph from a running Neo4j instance.
implicit val neo4jConfig = Neo4jConfig(uri = URI.create("bolt://localhost"), password = Some("password"))
// 3) Create a data source
val neo4jSource = new Neo4jPropertyGraphDataSource(neo4jConfig)
// 4) Load the whole Neo4j DB
val graph = neo4jSource.graph(GraphName("graph"))
// 5) Get nodes for GraphX graph construction
val nodes = graph.cypher(
"""|MATCH (n:Person)
|RETURN id(n), n.name""".stripMargin)
val rels = graph.cypher(
"""|MATCH (:Person)-[r]->(:Person)
|RETURN startNode(r), endNode(r)
""".stripMargin)
// 4) Create GraphX compatible RDDs from nodes and relationships
val graphXNodeRDD = nodes.getRecords.asDataFrame.rdd.map(row => row.getLong(0) -> row.getString(1))
val graphXRelRDD = rels.getRecords.asDataFrame.rdd.map(row => Edge(row.getLong(0), row.getLong(1), ()))
// 5) Compute Page Rank via GraphX
val graph = Graph(graphXNodeRDD, graphXRelRDD)
val ranks = graph.pageRank(0.0001).vertices
// 6) Print Page Ranks
ranks.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment