This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## execute hbase commands from bash | |
echo list | hbase shell | |
echo list \'1012\' | hbase shell | |
## View contents of HFile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode | |
# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode | |
# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode | |
# On Ubuntu and other Debian systems | |
nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb | |
nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ?? | |
nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems | |
# Install CDH4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Install Java JDK 6.0 update 31 on Ubuntu 12.04 LTS | |
Introduction | |
The first question is why are we installing an old JDK. The answer is that Oracle JDK 6.0 update 31 is the JDK recommended by Cloudera when installing CDH4 (Cloudera Distribution Hadoop v4). | |
This is an update to an older version of this post. Mainly I have changed the JDK from 1.6.0_26 to 1.6.0_31 as this is the recommended JDK for CDH4 . | |
Install Java | |
I have a 64 bit version of Ubuntu 12.04 LTS installed, so the instructions below only apply to this OS. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
########## | |
# For verification, you can display the OS release. | |
########## | |
$ cat /etc/lsb-release | |
DISTRIB_ID=Ubuntu | |
DISTRIB_RELEASE=11.10 | |
DISTRIB_CODENAME=oneiric | |
DISTRIB_DESCRIPTION="Ubuntu 11.10" | |
########## |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
================================================================= | |
SETTING UP SSHD AS A SERVICE FOR RUNNING HADOOP DAEMONS ON WINDOWS 7 | |
================================================================= | |
Steps: | |
1. Download 'setup.exe' from Cygwin website | |
2. Right-click on 'setup.exe' | |
3. Leave settings as they are, click through until you come to the plugin selection window | |
3.1 - Make sure that the installation directory is 'C:\cygwin' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hbase org.apache.hadoop.hbase.util.RegionSplitter test_table HexStringSplit -c 10 -f f1 | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## To load | |
disable 'Table' | |
alter 'Table', METHOD => 'table_att', 'COPROCESSOR' => '/path/to/coprocessors.jar|com.org.xxxx.Coprocessorclass|3|' | |
enable 'Table' | |
## To check coprocessor is loaded | |
describe 'Table' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
for i in '0'..'9' do for j in '0'..'9' do \ | |
for k in '0'..'9' do put 'testtable', "row-#{i}#{j}#{k}", \ | |
"colfam1:#{j}#{k}", "#{j}#{k}" end end end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public class TestCoprocessor extends BaseRegionObserver{ | |
private HTablePool pool = null; | |
private final static String INDEX_TABLE = "INDEX_TBL"; | |
private final static String SOURCE_TABLE = "SOURCE_TBL"; | |
@Override | |
public void start(CoprocessorEnvironment env) throws IOException { | |
pool = new HTablePool(env.getConfiguration(), 10); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## @CDH3 | |
hadoop jar hbase-0.90.6-cdh3u4.jar export table_name /exports/directory_name | |
## @CDH4 | |
hadoop fs -cp hftp://cdh3_namenode:50070/ /exports | |
hadoop jar $HBASE_HOME/hbase-0.94.6-cdh4.3.0-security.jar import table_name /exports/directory_name |