mneedham

name := "playground"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"

libraryDependencies += "net.sf.opencsv" % "opencsv" % "2.3"

ikwattro

Github Events Analysis with Neo4j

On July 22, Github announced the 3rd Annual Github Data Challenge presenting multiple sources of data available.

This sounded to me a good opportunity to use their available data and import it in Neo4j in order to have a lot of fun at analyzing the data that fits naturally in a graph.

As I work mainly offline or behind military proxies that do not permit me to use the ReST API, I decided to go for the Github Archive available here, you can then download json files representing Github Events on a daily/hour basis.

jexp

BEARER='...'
def load_tweets(query,since_id=nil,lang="en",page=1,rpp=100)
  res=RestClient.get('https://api.twitter.com/1.1/search/tweets.json',
   {:params=> {:q=>query, :lang=>lang,:count=>rpp,:result_type=>:recent,:since_id=>since_id},
    :accept=>:json,
    :Authorization => "Bearer #{BEARER}"})
  puts "query '#{query}'\n since id #{since_id} result #{res.code}"
  return [] unless res.code==200

  data=JSON.parse(res.to_str)

jexp

# How to run Neo4j GraphGists locally?

git clone https://github.com/neo4j-contrib/rabbithole
cd rabbithole
mvn clean test-compile 
mvn exec:java&

sleep 20

# GraphGists only run in your browser, so in your asciidoc-file use the attribute at the top :neo4j-version: local

prb

<!-- Fragment of pom.xml -->

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.2</version>
                <configuration>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>

jexp

var r=require("request")
function cypher(query,params,cb) { 
  r.post({uri:"http://localhost:7474/db/data/transaction/commit",
          json:{statements:[{statement:query,parameters:params}]}},
        function(err,res) { cb(err,res.body)}) 
}

var query="MATCH (n:User) RETURN n, labels(n) as l LIMIT {limit}"
var params={limit: 10}
var cb=function(err,data) { console.log(JSON.stringify(data)) }

tekiegirl

Contributor Community Graph

Developers using Neo4j are currently working alone when they should be working together, but they don’t know who is working on the same technologies. This graph aims to solve this by linking developers with similar interests, projects and events.

Setup of known data

kbastani

// Delete duplicate nodes as a list collected from the output of neo4j-cypher-duplicate-get-node.txt

START n=node(1120038,1120039,1120040,1120042,1120044,1120048,1120049,1120050,1120053,1120067,1120068)
// Replace IDs above with the IDs from CommaSeparatedListOfIds in neo4j-duplicate-get-node.txt
MATCH n-[r]-()
DELETE r, n

entaroadun

Movies Recommendation:

• MovieLens - Movie Recommendation Data Sets http://www.grouplens.org/node/73
• Yahoo! - Movie, Music, and Images Ratings Data Sets http://webscope.sandbox.yahoo.com/catalog.php?datatype=r
• Jester - Movie Ratings Data Sets (Collaborative Filtering Dataset) http://www.ieor.berkeley.edu/~goldberg/jester-data/
• Cornell University - Movie-review data for use in sentiment-analysis experiments http://www.cs.cornell.edu/people/pabo/movie-review-data/

Music Recommendation:

• Last.fm - Music Recommendation Data Sets http://www.dtic.upf.edu/~ocelma/MusicRecommendationDataset/index.html