Skip to content

Instantly share code, notes, and snippets.

@okram
Last active August 29, 2015 14:02
Show Gist options
  • Save okram/506a005f8b3bdd42a4fa to your computer and use it in GitHub Desktop.
Save okram/506a005f8b3bdd42a4fa to your computer and use it in GitHub Desktop.
$ bin/gremlin.sh
\,,,/
(o o)
-----oOOo-(3)-oOOo-----
gremlin> :use install com.tinkerpop giraph-gremlin 3.0.0-SNAPSHOT
==>loaded: [com.tinkerpop, giraph-gremlin, 3.0.0-SNAPSHOT] - restart the console to use [giraph]
gremlin> :q
$ bin/gremlin.sh
\,,,/
(o o)
-----oOOo-(3)-oOOo-----
loaded: giraph
gremlin> :remote connect giraph ../../../giraph-gremlin/conf/giraph-graphson.properties
==>giraphgraph[GraphSONVertexInputFormat->GraphSONVertexOutputFormat]
gremlin> :> g.V.map{it['name'][1]}.groupBy{it}
INFO com.tinkerpop.gremlin.giraph.process.computer.GiraphGraphRunner - GiraphGremlin: TraversalVertexProgram[GiraphGraphStep, MapStep, GroupByStep]
INFO org.apache.giraph.job.GiraphJob - run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
INFO org.apache.hadoop.mapred.JobClient - Running job: job_201406160857_0264
INFO org.apache.hadoop.mapred.JobClient - map 0% reduce 0%
INFO org.apache.hadoop.mapred.JobClient - map 100% reduce 0%
INFO org.apache.hadoop.mapred.JobClient - Job complete: job_201406160857_0264
INFO org.apache.hadoop.mapred.JobClient - Counters: 32
INFO org.apache.hadoop.mapred.JobClient - Map-Reduce Framework
INFO org.apache.hadoop.mapred.JobClient - Spilled Records=0
INFO org.apache.hadoop.mapred.JobClient - Map input records=3
INFO org.apache.hadoop.mapred.JobClient - SPLIT_RAW_BYTES=132
INFO org.apache.hadoop.mapred.JobClient - Map output records=0
INFO org.apache.hadoop.mapred.JobClient - Total committed heap usage (bytes)=240648192
INFO org.apache.hadoop.mapred.JobClient - Giraph Timers
INFO org.apache.hadoop.mapred.JobClient - Total (milliseconds)=8511
INFO org.apache.hadoop.mapred.JobClient - Superstep 2 (milliseconds)=40
INFO org.apache.hadoop.mapred.JobClient - Shutdown (milliseconds)=462
INFO org.apache.hadoop.mapred.JobClient - Superstep 1 (milliseconds)=120
INFO org.apache.hadoop.mapred.JobClient - Input superstep (milliseconds)=6555
INFO org.apache.hadoop.mapred.JobClient - Superstep 0 (milliseconds)=341
INFO org.apache.hadoop.mapred.JobClient - Setup (milliseconds)=892
INFO org.apache.hadoop.mapred.JobClient - Superstep 3 (milliseconds)=48
INFO org.apache.hadoop.mapred.JobClient - Superstep 4 (milliseconds)=49
INFO org.apache.hadoop.mapred.JobClient - File Input Format Counters
INFO org.apache.hadoop.mapred.JobClient - Bytes Read=0
INFO org.apache.hadoop.mapred.JobClient - Giraph Stats
INFO org.apache.hadoop.mapred.JobClient - Aggregate finished vertices=0
INFO org.apache.hadoop.mapred.JobClient - Aggregate edges=0
INFO org.apache.hadoop.mapred.JobClient - Sent messages=0
INFO org.apache.hadoop.mapred.JobClient - Current workers=2
INFO org.apache.hadoop.mapred.JobClient - Last checkpointed superstep=0
INFO org.apache.hadoop.mapred.JobClient - Current master task partition=0
INFO org.apache.hadoop.mapred.JobClient - Superstep=5
INFO org.apache.hadoop.mapred.JobClient - Aggregate vertices=6
INFO org.apache.hadoop.mapred.JobClient - FileSystemCounters
INFO org.apache.hadoop.mapred.JobClient - HDFS_BYTES_READ=2488
INFO org.apache.hadoop.mapred.JobClient - FILE_BYTES_WRITTEN=241404
INFO org.apache.hadoop.mapred.JobClient - HDFS_BYTES_WRITTEN=3464
INFO org.apache.hadoop.mapred.JobClient - Job Counters
INFO org.apache.hadoop.mapred.JobClient - Launched map tasks=3
INFO org.apache.hadoop.mapred.JobClient - SLOTS_MILLIS_REDUCES=0
INFO org.apache.hadoop.mapred.JobClient - Total time spent by all reduces waiting after reserving slots (ms)=0
INFO org.apache.hadoop.mapred.JobClient - SLOTS_MILLIS_MAPS=46895
INFO org.apache.hadoop.mapred.JobClient - Total time spent by all maps waiting after reserving slots (ms)=0
INFO org.apache.hadoop.mapred.JobClient - File Output Format Counters
INFO org.apache.hadoop.mapred.JobClient - Bytes Written=0
INFO com.tinkerpop.gremlin.giraph.process.computer.GiraphGraphRunner - GiraphGremlin: GiraphGroupByStep[SideEffect Calculation]
WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
WARN org.apache.hadoop.io.compress.snappy.LoadSnappy - Snappy native library not loaded
INFO org.apache.hadoop.mapred.JobClient - Running job: job_201406160857_0265
INFO org.apache.hadoop.mapred.JobClient - map 0% reduce 0%
INFO org.apache.hadoop.mapred.JobClient - map 100% reduce 0%
INFO org.apache.hadoop.mapred.JobClient - map 100% reduce 33%
INFO org.apache.hadoop.mapred.JobClient - map 100% reduce 100%
INFO org.apache.hadoop.mapred.JobClient - Job complete: job_201406160857_0265
INFO org.apache.hadoop.mapred.JobClient - Counters: 26
INFO org.apache.hadoop.mapred.JobClient - Map-Reduce Framework
INFO org.apache.hadoop.mapred.JobClient - Spilled Records=12
INFO org.apache.hadoop.mapred.JobClient - Map output materialized bytes=222
INFO org.apache.hadoop.mapred.JobClient - Reduce input records=6
INFO org.apache.hadoop.mapred.JobClient - Map input records=6
INFO org.apache.hadoop.mapred.JobClient - SPLIT_RAW_BYTES=234
INFO org.apache.hadoop.mapred.JobClient - Map output bytes=198
INFO org.apache.hadoop.mapred.JobClient - Reduce shuffle bytes=222
INFO org.apache.hadoop.mapred.JobClient - Reduce input groups=4
INFO org.apache.hadoop.mapred.JobClient - Combine output records=6
INFO org.apache.hadoop.mapred.JobClient - Reduce output records=4
INFO org.apache.hadoop.mapred.JobClient - Map output records=6
INFO org.apache.hadoop.mapred.JobClient - Combine input records=6
INFO org.apache.hadoop.mapred.JobClient - Total committed heap usage (bytes)=592969728
INFO org.apache.hadoop.mapred.JobClient - File Input Format Counters
INFO org.apache.hadoop.mapred.JobClient - Bytes Read=3464
INFO org.apache.hadoop.mapred.JobClient - FileSystemCounters
INFO org.apache.hadoop.mapred.JobClient - HDFS_BYTES_READ=3698
INFO org.apache.hadoop.mapred.JobClient - FILE_BYTES_WRITTEN=247932
INFO org.apache.hadoop.mapred.JobClient - FILE_BYTES_READ=216
INFO org.apache.hadoop.mapred.JobClient - HDFS_BYTES_WRITTEN=30
INFO org.apache.hadoop.mapred.JobClient - Job Counters
INFO org.apache.hadoop.mapred.JobClient - Launched map tasks=2
INFO org.apache.hadoop.mapred.JobClient - Launched reduce tasks=1
INFO org.apache.hadoop.mapred.JobClient - SLOTS_MILLIS_REDUCES=8408
INFO org.apache.hadoop.mapred.JobClient - Total time spent by all reduces waiting after reserving slots (ms)=0
INFO org.apache.hadoop.mapred.JobClient - SLOTS_MILLIS_MAPS=7407
INFO org.apache.hadoop.mapred.JobClient - Total time spent by all maps waiting after reserving slots (ms)=0
INFO org.apache.hadoop.mapred.JobClient - Data-local map tasks=2
INFO org.apache.hadoop.mapred.JobClient - File Output Format Counters
INFO org.apache.hadoop.mapred.JobClient - Bytes Written=30
==>a [a, a]
==>e [e]
==>i [i]
==>o [o, o]
gremlin>
gremlin> hdfs.ls()
==>rwxr-xr-x marko supergroup 0 (D) output
==>rw-r--r-- marko supergroup 2356 tinkerpop-classic-adjlist.ldjson
gremlin> hdfs.ls('output')
==>rw-r--r-- marko supergroup 0 _SUCCESS
==>rwxr-xr-x marko supergroup 0 (D) _logs
==>rwxr-xr-x marko supergroup 0 (D) cap
==>rw-r--r-- marko supergroup 1876 part-m-00001
==>rw-r--r-- marko supergroup 1588 part-m-00002
gremlin> hdfs.ls('output/cap')
==>rw-r--r-- marko supergroup 0 _SUCCESS
==>rwxr-xr-x marko supergroup 0 (D) _logs
==>rw-r--r-- marko supergroup 30 part-r-00000
gremlin>
gremlin>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment