Skip to content

Instantly share code, notes, and snippets.

@tuxdna
Last active January 30, 2019 18:55
Show Gist options
  • Select an option

  • Save tuxdna/166dd41902c59ca0470252b5bf4f3dcd to your computer and use it in GitHub Desktop.

Select an option

Save tuxdna/166dd41902c59ca0470252b5bf4f3dcd to your computer and use it in GitHub Desktop.
Setup JanusGraph with HBASE backend

Setup JanusGraph with HBASE backend

Install following

  • hbase-1.2.9
  • janusgraph-0.2.2-hadoop2

HBASE Setup and inspection

HBASE setup on local machine: - https://www.guru99.com/hbase-installation-guide.html

wget -c https://www.apache.org/dyn/closer.lua/hbase/hbase-1.2.9/hbase-1.2.9-bin.tar.gz
tar zxf hbase-1.2.9-bin.tar.gz
cd hbase-1.2.9
export HBASE_HOME=`pwd`

Edit conf/hbase-site.xml and update hbase data directory:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>file:///tmp/HBASE/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/tmp/HBASE/zookeeper</value>
  </property>
</configuration>

Edit conf/hbase-env.sh to set Java Home, regionserver and zooker setting:

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers
export HBASE_MANAGES_ZK=true

Start HBASE

bin/start-hbase.sh 

Open HBASE Shell to verify that the server is up and running

$ bin/hbase shell
list

Setup JanusGraph 0.3.1

wget -c https://github.com/JanusGraph/janusgraph/releases/download/v0.3.1/janusgraph-0.3.1-hadoop2.zip
unzip janusgraph-0.3.1-hadoop2.zip
cd janusgraph-0.3.1-hadoop2

Setup JanusGraph 0.2.2

wget -c https://github.com/JanusGraph/janusgraph/releases/download/v0.2.2/janusgraph-0.2.2-hadoop2.zip
unzip janusgraph-0.2.2-hadoop2.zip
cd janusgraph-0.2.2-hadoop2

Update JanusGraph configuration files

Set HBase configuration for graph backend

cp conf/janusgraph-hbase.properties conf/my-janusgraph.properties

Edit conf/my-janusgraph.properties to set following values

gremlin.graph=org.janusgraph.core.JanusGraphFactory
storage.backend=hbase
storage.hostname=127.0.0.1
storage.hbase.table=my_graph
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.5

Create Gremlin Server configuration for HBase Backend and WebSocket + HTTP:

cp conf/gremlin-server/gremlin-server-configuration.yaml conf/gremlin-server/gremlin-server-configuration-hbase.yaml

Edit conf/gremlin-server/gremlin-server-configuration-hbase.yaml with following things to note:

  • channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
  • graph: conf/my-janusgraph.properties
host: 0.0.0.0
port: 8182
scriptEvaluationTimeout: 30000
# channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer
graphManager: org.janusgraph.graphdb.management.JanusGraphManager
graphs: {
  # ConfigurationManagementGraph: conf/janusgraph-cql-configurationgraph.properties
  graph: conf/my-janusgraph.properties
}
plugins:
  - janusgraph.imports
scriptEngines: {
  gremlin-groovy: {
    imports: [java.lang.Math],
    staticImports: [java.lang.Math.PI],
    scripts: [scripts/empty-sample.groovy]}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
processors:
  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000, maxParametes: 18 }}
  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
metrics: {
  consoleReporter: {enabled: true, interval: 180000},
  csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
  jmxReporter: {enabled: true},
  slf4jReporter: {enabled: true, interval: 180000},
  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
  graphiteReporter: {enabled: false, interval: 180000}}
maxInitialLineLength: 4096
maxHeaderSize: 8192
maxChunkSize: 8192
maxContentLength: 65536
maxAccumulationBufferComponents: 1024
resultIterationBatchSize: 64
writeBufferLowWaterMark: 32768
writeBufferHighWaterMark: 65536

Now we can start JanusGraph Gremlin Server:

bin/gremlin-server.sh conf/gremlin-server/gremlin-server-configuration-hbase.yaml 

Verify that it is working:

$ curl -X POST -d "{\"gremlin\":\"g.V().count()\"}" "http://localhost:8182" | python -mjson.tool
{
    "requestId": "f630b377-9126-44c1-9040-2e089018a4ff",
    "result": {
        "data": [
            0
        ],
        "meta": {}
    },
    "status": {
        "attributes": {},
        "code": 200,
        "message": ""
    }
}

Finally verify that we can connect via gremlin console as well:

bin/gremlin.sh
gremlin> :remote connect tinkerpop.server conf/remote.yaml session
==>Configured localhost/127.0.0.1:8182-[2170da23-31fe-4ea8-b9d4-d8e5ff205b3b]
gremlin> :remote console
==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182]-[2170da23-31fe-4ea8-b9d4-d8e5ff205b3b] - type ':remote console' to return to local mode
gremlin> g.V().count()
==>0
gremlin> :remote close
==>Removed - Gremlin Server - [localhost/127.0.0.1:8182]-[2170da23-31fe-4ea8-b9d4-d8e5ff205b3b]
gremlin> :exit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment