Ali Bajwa abajwa-hw

Phoenix/Spark demo

These demo is part of a webinar on 'HBase for Mission Critical Applications'
The webinar recording /slides are available at http://hortonworks.com/partners/learn/#hbase2
The Ambari service for OpenTSDB is available at https://github.com/abajwa-hw/opentsdb-service
The full list of available workshops is available at http://hortonworks.com/partners/learn

Option 1: prebuilt VM

There is a prebuilt Centos 6.5 VM with the below components installed:

HDP 2.3.0.0-1754
Spark 1.3.1

Hbase indexing to solr in HDP 2.3

Background:

The HBase Indexer provides the ability to stream events from HBase to Solr for near real time searching. The HBase indexer is included with HDPSearch as an additional service. The indexer works by acting as an HBase replication sink. As updates are written to HBase, the events are asynchronously replicated to the HBase Indexer processes, which in turn creates Solr documents and pushes them to Solr.

References:
- http://doc.lucidworks.com/hdpsearch23/JobJarGuide.html#_hbase_indexer
- https://github.com/NGDATA/hbase-indexer/wiki/Tutorial

Index documents using HDPSearch

Lab Overview

In this lab, we will learn to:

Configure Solr to store indexes in HDFS
Create a solr cluster of 2 solr instances running on port 8983 and 8984
Index documents in HDFS using the Hadoop connectors
Use Solr to search documents

LAB

This lab is part of a 'Sql on Hadoop' webinar. The recording and slides can be found here

Purpose

How/when to use Hive vs Phoenix vs SparkSQL

HDF workshop

Setup already done

Download and import latest HDP 2.3 sandbox VM from http://hortonworks.com/sandbox and import into VMware Fusion

Deployed and install VNC ambari service (to be able to 'remote desktop' in and use eclipse)

sudo git clone https://github.com/hortonworks-gallery/ambari-vnc-service.git /var/lib/ambari-server/resources/stacks/HDP/2.3/services/VNCSERVER

Build LLAP on HDP 2.3.x

Below are sample steps on how to build LLAP using Gopal's tez-autobuild repo on HDP 2.3.x cluster.
Note that due to HIVE-12446 for now a slightly older version of tez (with tag: 0.8.1-alpha) was built.

Steps

SSH into HDP cluster
Install C/C++ compiler

yum install -y gcc gcc-cpp gcc-c++

EDW optimization and Single view lab

EDW optimization

Goal: demonstrate how you can bulk import data from EDW/RDBMS into Hive and then incrementally keep the Hive tables updated periodically

Pre-requisites

1. Download contoso data set and MSSQL schema from here into /tmp on sandbox

Add HDB (HAWQ) to HDP 2.4.2 with Zeppelin

Goals:

Install a 4 node cluster running HDP 2.4.2 using Ambari 2.2.2.0 (including Zeppelin and HDB) using Ambari bootstrap via blueprints or Ambari install wizard
Configure HAWQ for Zeppelin
Configure Zeppelin for HAWQ
Run HAWQ queries via Zeppelin

Notes:

	<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
	http://maven.apache.org/xsd/settings-1.0.0.xsd">
	<localRepository/>
	<interactiveMode/>
	<usePluginRegistry/>
	<offline/>
	<pluginGroups/>
	<servers/>

	#find jar containing class org.apache.hive.jdbc.HiveStatement
	find / -iname '*.jar' \| xargs -i bash -c "jar -tvf {} \| tr / . \| grep org.apache.hive.jdbc.HiveStatement && echo {}"