Skip to content

Instantly share code, notes, and snippets.

View sankars's full-sized avatar

Sankar sankars

  • Dubai
View GitHub Profile
@sankars
sankars / hbase_commands.sh
Created August 9, 2013 07:01
Bash to Shell
## execute hbase commands from bash
echo list | hbase shell
echo list \'1012\' | hbase shell
## View contents of HFile
@sankars
sankars / notes.txt
Created August 9, 2013 07:15 — forked from nipra/notes.txt
# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode
# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode
# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode
# On Ubuntu and other Debian systems
nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ??
nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems
# Install CDH4
Install Java JDK 6.0 update 31 on Ubuntu 12.04 LTS
Introduction
The first question is why are we installing an old JDK. The answer is that Oracle JDK 6.0 update 31 is the JDK recommended by Cloudera when installing CDH4 (Cloudera Distribution Hadoop v4).
This is an update to an older version of this post. Mainly I have changed the JDK from 1.6.0_26 to 1.6.0_31 as this is the recommended JDK for CDH4 .
Install Java
I have a 64 bit version of Ubuntu 12.04 LTS installed, so the instructions below only apply to this OS.
##########
# For verification, you can display the OS release.
##########
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=11.10
DISTRIB_CODENAME=oneiric
DISTRIB_DESCRIPTION="Ubuntu 11.10"
##########
=================================================================
SETTING UP SSHD AS A SERVICE FOR RUNNING HADOOP DAEMONS ON WINDOWS 7
=================================================================
Steps:
1. Download 'setup.exe' from Cygwin website
2. Right-click on 'setup.exe'
3. Leave settings as they are, click through until you come to the plugin selection window
3.1 - Make sure that the installation directory is 'C:\cygwin'
@sankars
sankars / region_split.sh
Created August 10, 2013 09:23
Region Splitter
hbase org.apache.hadoop.hbase.util.RegionSplitter test_table HexStringSplit -c 10 -f f1
@sankars
sankars / coprocessor.rb
Created August 12, 2013 11:15
Hbase shell commands to load/unload coprocessors
## To load
disable 'Table'
alter 'Table', METHOD => 'table_att', 'COPROCESSOR' => '/path/to/coprocessors.jar|com.org.xxxx.Coprocessorclass|3|'
enable 'Table'
## To check coprocessor is loaded
describe 'Table'
@sankars
sankars / sampledata.rb
Created August 12, 2013 11:29
Jruby hbase shell script to generate sample data
for i in '0'..'9' do for j in '0'..'9' do \
for k in '0'..'9' do put 'testtable', "row-#{i}#{j}#{k}", \
"colfam1:#{j}#{k}", "#{j}#{k}" end end end
@sankars
sankars / secondary_index.java
Created August 12, 2013 11:47
Coprocessor for creating secondary index in Hbase
public class TestCoprocessor extends BaseRegionObserver{
private HTablePool pool = null;
private final static String INDEX_TABLE = "INDEX_TBL";
private final static String SOURCE_TABLE = "SOURCE_TBL";
@Override
public void start(CoprocessorEnvironment env) throws IOException {
pool = new HTablePool(env.getConfiguration(), 10);
@sankars
sankars / migration.sh
Created August 21, 2013 14:27
CDH3 to CDH4 data migration
## @CDH3
hadoop jar hbase-0.90.6-cdh3u4.jar export table_name /exports/directory_name
## @CDH4
hadoop fs -cp hftp://cdh3_namenode:50070/ /exports
hadoop jar $HBASE_HOME/hbase-0.94.6-cdh4.3.0-security.jar import table_name /exports/directory_name