Whirr-ec2

gistfile1.txt

http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html

thrift repo : http://people.apache.org/~rawson/repo

git clone https://github.com/apache/whirr.git

git checkout -b branch-0.7 -t origin/branch-0.7

git clone git://github.com/apache/whirr.git  && cd whirr && git co branch-0.7

mvn install

mvn package assembly:assembly

tar xvfz whirr/target/whirr-0.8.0-SNAPSHOT.tar.gz

ln -s whirr-0.8.0-SNAPSHOT whirr-080

cd whirr-080

cp conf/credentials.sample ~/.whirr/credentials

bin/whirr launch-cluster --config recipes/hbase-cdh.properties --private-key-file ~/.ssh/id_rsa_whirr

vi ~/.whirr/credentials

ROVIDER=aws-ec2
IDENTITY=AKIAJFPIVZ6QZPFS2SGQ
CREDENTIAL=o2cRotlKbpIFII98H42tlredKDwE0hFrtz/bQK6Z
vi recipes/hbase-cdh.properties
hbase-env.HBASE_MASTER_OPTS=-Xms1000m -Xmx1000m -Xmn256m -XX:+UseConcMarkSweepGC -XX:+AggressiveOpts -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/data/hbase/logs/hbase-master-gc.log
hbase-env.HBASE_REGIONSERVER_OPTS=-Xms2000m -Xmx2000m -Xmn256m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=88 -XX:+AggressiveOpts -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/data/hbase/logs/hbase-regionserver-gc.log

bin/whirr launch-cluster --config recipes/hbase-cdh.properties
Finished running configure phase scripts on all cluster instances
Completed configuration of hbase-cdh
Hosts: ec2-23-20-101-86.compute-1.amazonaws.com:2181
Completed configuration of hbase-cdh role hadoop-namenode
Namenode web UI available at http://ec2-23-20-101-86.compute-1.amazonaws.com:50070
Wrote Hadoop site file /Users/sindhuc/.whirr/hbase-cdh/hadoop-site.xml
Wrote Hadoop proxy script /Users/sindhuc/.whirr/hbase-cdh/hadoop-proxy.sh
Completed configuration of hbase-cdh role hadoop-jobtracker
Jobtracker web UI available at http://ec2-23-20-101-86.compute-1.amazonaws.com:50030
Completed configuration of hbase-cdh
Web UI available at http://ec2-23-20-101-86.compute-1.amazonaws.com
Wrote HBase site file /Users/sindhuc/.whirr/hbase-cdh/hbase-site.xml
Wrote HBase proxy script /Users/sindhuc/.whirr/hbase-cdh/hbase-proxy.sh
Completed configuration of hbase-cdh role hadoop-datanode
Completed configuration of hbase-cdh role hadoop-tasktracker
Finished running start phase scripts on all cluster instances
Started cluster of 2 instances
Cluster{instances=[Instance{roles=[hadoop-datanode, hadoop-tasktracker, hbase-regionserver], publicIp=23.20.13.175, privateIp=10.194.142.144, id=us-east-1/i-76f8f713, nodeMetadata=[id=us-east-1/i-76f8f713, providerId=i-76f8f713, group=hbase-cdh, name=hbase-cdh-76f8f713, location=[id=us-east-1d, scope=ZONE, description=us-east-1d, parent=us-east-1, iso3166Codes=[US-VA], metadata={}], uri=null, imageId=us-east-1/ami-ab36fbc2, os=[name=null, family=ubuntu, version=10.04, arch=paravirtual, is64Bit=false, description=099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20110930], state=RUNNING, loginPort=22, hostname=ip-10-194-142-144, privateAddresses=[10.194.142.144], publicAddresses=[23.20.13.175], hardware=[id=m1.small, providerId=m1.small, name=null, processors=[[cores=1.0, speed=1.0]], ram=1740, volumes=[[id=null, type=LOCAL, size=150.0, device=/dev/sda2, durable=false, isBootDevice=false], [id=vol-d3088abf, type=SAN, size=null, device=/dev/sda1, durable=true, isBootDevice=true]], supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,Not(is64Bit())), tags=[]], loginUser=ubuntu, userMetadata={Name=hbase-cdh-76f8f713}, tags=[]]}, Instance{roles=[zookeeper, hadoop-namenode, hadoop-jobtracker, hbase-master], publicIp=23.20.101.86, privateIp=10.98.14.203, id=us-east-1/i-74f8f711, nodeMetadata=[id=us-east-1/i-74f8f711, providerId=i-74f8f711, group=hbase-cdh, name=hbase-cdh-74f8f711, location=[id=us-east-1d, scope=ZONE, description=us-east-1d, parent=us-east-1, iso3166Codes=[US-VA], metadata={}], uri=null, imageId=us-east-1/ami-ab36fbc2, os=[name=null, family=ubuntu, version=10.04, arch=paravirtual, is64Bit=false, description=099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20110930], state=RUNNING, loginPort=22, hostname=ip-10-98-14-203, privateAddresses=[10.98.14.203], publicAddresses=[23.20.101.86], hardware=[id=m1.small, providerId=m1.small, name=null, processors=[[cores=1.0, speed=1.0]], ram=1740, volumes=[[id=null, type=LOCAL, size=150.0, device=/dev/sda2, durable=false, isBootDevice=false], [id=vol-ad088ac1, type=SAN, size=null, device=/dev/sda1, durable=true, isBootDevice=true]], supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,Not(is64Bit())), tags=[]], loginUser=ubuntu, userMetadata={Name=hbase-cdh-74f8f711}, tags=[]]}], configuration={hbase.zookeeper.quorum=ec2-23-20-101-86.compute-1.amazonaws.com:2181, hadoop.rpc.socket.factory.class.default=org.apache.hadoop.net.SocksSocketFactory, hadoop.socks.server=localhost:6666, hbase.zookeeper.property.clientPort=2181}}
You can log into instances using the following ssh commands:
'ssh -i /Users/sindhuc/.ssh/id_rsa -o "UserKnownHostsFile /dev/null" -o StrictHostKeyChecking=no [email protected]'
'ssh -i /Users/sindhuc/.ssh/id_rsa -o "UserKnownHostsFile /dev/null" -o StrictHostKeyChecking=no [email protected]'
On ec2 boxes:
sudo jps
Get ZooKeeper HostName

sudo vi /etc/hbase/conf/hbase-site.xml

sudo vi /etc/zookeeper/zoo.cfg (on both servers)

server.0=ec2-23-20-101-86.compute-1.amazonaws.com:2888:3888

hbase shell

ls ~/.whirrbin/whirr destroy-cluster --config recipes/hbase-cdh.properties