Skip to content

Instantly share code, notes, and snippets.

@tilakpatidar

tilakpatidar/README.md

Last active January 25, 2018 21:43
Show Gist options
  • Save tilakpatidar/5b12bc46a600d6419dbbf4cf972b631c to your computer and use it in GitHub Desktop.
Save tilakpatidar/5b12bc46a600d6419dbbf4cf972b631c to your computer and use it in GitHub Desktop.
Download ZIP
Uninstall Cloudera and install HDP 2.6
Raw
README.md

Install HDP 2.6 and remove Cloudera 5.12

Uninstall cloudera

Follow the guide to uninstall. Link Also, remove the existing folders and users.

#Remove conf and logs
rm -rf rm -rf /etc/hadoop
rm -rf rm -rf /etc/hbase
rm -rf rm -rf /etc/hive
rm -rf rm -rf /etc/sqoop
rm -rf rm -rf /etc/hue
rm -rf rm -rf /etc/zookeeper
rm -rf rm -rf /etc/hive-hcatalog
rm -rf rm -rf /etc/hive-webhcat
rm -rf rm -rf /etc/mahout
rm -rf rm -rf /etc/spark
rm -rf rm -rf /etc/pig
rm -rf /var/log/hive
rm -rf /var/log/hadoop-hdfs
rm -rf /var/log/hadoop-yarn
rm -rf /tmp/hive
rm -rf /var/log/oozie
rm -rf /var/log/hue
rm -rf /var/log/zookeeper
rm -rf /var/log/hadoop-mapreduce
rm -rf /var/log/spark
rm -rf /var/lib/hue

#Remove users
userdel -r flume
userdel -r hdfs
userdel -r zookeeper
userdel -r mapred
userdel -r sqoop
userdel -r yarn
userdel -r kms
userdel -r hive
userdel -r sqoop2
userdel -r oozie
userdel -r hbase
userdel -r spark
userdel -r hue
userdel -r httpfs
userdel -r solr
userdel -r infra-solr
userdel -r cloudera-scm

#Remove cloudera parcels
rm -rf /opt/cloudera/parcels/*
rm -rf /etc/hive/conf.cloudera.hive
rm -rf /etc/hadoop/conf.cloudera.yarn
rm -rf /etc/spark2/conf.cloudera.spark2_on_yarn
rm -rf /etc/spark/conf.cloudera.spark_on_yarn

# Remove packages
yum remove -y cloudera-manager-agent
yum remove -y cloudera-manager-daemons
yum remove -y jdk
yum remove -y oracle-j2sdk1.7

Ambari server setup guide

Follow the official guide. Installation is prefferable by using a HDP PPA. Installation was done using a Ambari server and automatic agent registeration. See the Link

The prerequisites for installing Ambari:

  • Set Up Password-less SSH
  • Set Up Service User Accounts
  • Enable NTP on the Cluster and on the Browser Host Link for prerequisites

Just install the ambari server using yum do not start the ambari-server setup yet. Few custom configurations have to be done before running the ambari-server setup. These custom configurations are discussed below.

Setup postgresql guide

Once the yum install for ambari server is finished. Setup your postgresql for ambari using the following. guide. After, the below given customizations are done and ambari-server setup is ran choose option 4 postgresql in database section.

Setup java home

During ambari-server setup the server asks for java home. Use the option 1. Do not specify your own JAVA_HOME as it also require Java Cryptography extension which might not be available in your JAVA_HOME.

Using option 1 installs java in /usr/java/jdk1.8.0_60 adjust the path if it install somewhere else.

Set the new JAVA_HOME in /etc/profile.d/java_home.sh. No need to modify ~/.bashrc.

If ssl errors in ambari-server start

After, ambari server setup if you get following error on server start.

09 Oct 2017 10:32:46,911  WARN [main] AbstractLifeCycle:204 - FAILED org.eclipse.jetty.server.Server@2fcad739: MultiException[java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big., java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big.]
MultiException[java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big., java.io.IOException: DerInputStream.getLength(): lengthTag=127, too big.]

It means the ssl certificate has to be generated. The following steps are taken from.

cd /var/lib/ambari-server/keys/
Insert the following in ca.config file
[req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = VA
L = SomeCity
O = MyCompany
OU = MyDivision
CN = www.company.com
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = www.company.com
DNS.2 = company.com
DNS.3 = www.company.net
DNS.4 = company.net
openssl req -config /var/lib/ambari-server/keys/ca.config -new -x509 -keyout /var/lib/ambari-server/keys/ca.key -out /var/lib/ambari-server/keys/ca.crt -days 3650

# Create keystore.p12
openssl pkcs12 -export -in '/var/lib/ambari-server/keys/ca.crt' -inkey '/var/lib/ambari-server/keys/ca.key' -certfile '/var/lib/ambari-server/keys/ca.crt' -out '/var/lib/ambari-server/keys/keystore.p12' -password file:'/var/lib/ambari-server/keys/pass.txt'

# Review keystore.p12
openssl pkcs12 -info -in '/var/lib/ambari-server/keys/keystore.p12' -password file:'/var/lib/ambari-server/keys/pass.txt'

On first time ambari-server start

First, time start will ask you to deploy the cluster. The configuration is pretty straightforward in Ambari UI.

When agents registeration is completed do check the warnings all the instructions to resolve them will be given by the UI. However, a default cleanup can be run by using.

# Use this for cleanup or our script :)
python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent --skip=users

Once, you reach the configuration for each service where the wizard will ask you for the passwords in various services like hive, atlas, oozie etc. Setup your postgresql to be used as metastore for various services.

Download the jdbc driver for postgres and use the below command to copy jar to all ambari agents.

ambari-server setup --jdbc-db=postgres --jdbc-driver=/path/to/postgres/postgresql.jar
Select Postgres as your metastore db for hive(password=>bigdata)

Run the following in psql.

CREATE DATABASE hive;
CREATE USER hive WITH PASSWORD 'bigdata';
GRANT ALL PRIVILEGES ON DATABASE hive TO hive;
CREATE SCHEMA hive AUTHORIZATION hive;
ALTER SCHEMA hive OWNER TO hive;
ALTER ROLE hive SET search_path to 'hive', 'public';

And add the entry for your server in postgresql.

echo "hostnossl    hive             hive             192.168.120.1/21            md5" >> /var/lib/pgsql/data/pg_hba.conf

Select postgresql as your oozie db

Run the following in psql.

CREATE DATABASE oozie;
CREATE USER oozie WITH PASSWORD 'bigdata';
GRANT ALL PRIVILEGES ON DATABASE oozie TO oozie;
CREATE SCHEMA oozie AUTHORIZATION oozie;
ALTER SCHEMA oozie OWNER TO oozie;
ALTER ROLE oozie SET search_path to 'oozie', 'public';

And add the entry for your server in postgresql.

echo "hostnossl    oozie             oozie             192.168.120.1/21            md5" >> /var/lib/pgsql/data/pg_hba.conf

If, during deployment UID exists issue

We ran into creating a user with UID which already exists. Check the logs at /var/log/ambari-server/ambari-server.log. See the command that failed. If there is some create user command which failed try to run it manually. If it throws some UID exists error that means you face the same issue as the prophetic documentation writer.

In all agents edit the file /var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh.

There will be a while loop with some range of 1001 to 2000 try changing the range.

Or, see the original issue posted here.

Enable Ambari views

For enabling ambari views following steps have to be followed

Enable HTTPS for Ambari server

You can follow the steps below or for more reference see the official guide.

# On the Ambari Server host, edit the ambari-env.sh file:
vi /var/lib/ambari-server/ambari-env.sh
# For the AMBARI_JVM_ARGS variable, replace the default -Xmx2048m with the following:
-Xmx4096m -XX:PermSize=128m -XX:MaxPermSize=128m
# Restart Ambari Server for this change to take effect.
ambari-server restart

Run the following in shell

ambari-server stop
cd /var/lib/ambari-server/keys
wserver=y-tst-hdp-cls-1.dnspam
openssl genrsa -des3 -out rootCA.key 2048
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem
# Provide useful information and hostname is mandatory or it will cause CertificateException: No name matching ssl.someUrl.de found
# You are about to be asked to enter information that will be incorporated
# into your certificate request.
# What you are about to enter is what is called a Distinguished Name or a DN.
# There are quite a few fields but you can leave some blank
# For some fields there will be a default value,
# If you enter '.', the field will be left blank.
# -----
# Country Name (2 letter code) [XX]:Italy
# string is too long, it needs to be less than  2 bytes long
# Country Name (2 letter code) [XX]:IT
# State or Province Name (full name) []:
# Locality Name (eg, city) [Default City]:
# Organization Name (eg, company) [Default Company Ltd]:PAM
# Organizational Unit Name (eg, section) []:Datalake
# Common Name (eg, your name or your server's hostname) []:y-tst-hdp-cls-1.dnspam
# Email Address []:[email protected]
# Please enter the following 'extra' attributes
# to be sent with your certificate request
# A challenge password []:*******
# An optional company name []:PAM


echo "[req]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn

[dn]
C=IT
ST=Veneto
L=Some Veneto
O=Some Veneto
OU=Some Veneto
[email protected]
CN = y-tst-hdp-cls-1.dnspam
" > https.csr.cnf

echo "authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = y-tst-hdp-cls-1.dnspam
 " > v3.ext

openssl req -new -sha256 -nodes -out $wserver.csr -newkey rsa:2048 -keyout $wserver.key -config <( cat https.csr.cnf )

openssl x509 -req -in $wserver.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out $wserver.crt -days 500 -sha256 -extfile v3.ext

Provide the appropriate values in confg.

[root@y-tst-hdp-cls-1 keys]# ambari-server setup-security
Using python  /usr/bin/python
Security setup options...
===========================================================================
Choose one of the following options:
  [1] Enable HTTPS for Ambari server.
  [2] Encrypt passwords stored in ambari.properties file.
  [3] Setup Ambari kerberos JAAS configuration.
  [4] Setup truststore.
  [5] Import certificate to truststore.
===========================================================================
Enter choice, (1-5): 1
Do you want to configure HTTPS [y/n] (y)? y
SSL port [8442] ?
Enter path to Certificate: /var/lib/ambari-server/keys/y-tst-hdp-cls-1.dnspam.crt
Enter path to Private Key: /var/lib/ambari-server/keys/y-tst-hdp-cls-1.dnspam.key
Please enter password for Private Key:
WARNING: There is no Common Name in Certificate
WARNING: Unable to validate Certificate hostname
Importing and saving Certificate...done.
Ambari server URL changed. To make use of the Tez View in Ambari please update the property tez.tez-ui.history-url.base in tez-site
Adjusting ambari-server permissions and ownership...

Create truststore for apache ambari

Execute the following in shell. Do check the path of certificate in the command.

cd $JAVA_HOME/bin
./keytool -import -file /var/lib/ambari-server/keys/y-tst-hdp-cls-1.dnspam.crt -alias ambari-server -keystore ambari-server-truststore
mv ambari-server-truststore /var/lib/ambari-server/keys/
cd /var/lib/ambari-server/keys

Now execute ambari-server setup-security and enable truststore.

ambari-server setup-security
Using python  /usr/bin/python
Security setup options...
===========================================================================
Choose one of the following options:
  [1] Enable HTTPS for Ambari server.
  [2] Encrypt passwords stored in ambari.properties file.
  [3] Setup Ambari kerberos JAAS configuration.
  [4] Setup truststore.
  [5] Import certificate to truststore.
===========================================================================
Enter choice, (1-5): 4
Do you want to configure a truststore [y/n] (y)? y
TrustStore type [jks/jceks/pkcs12] (jks):
Path to TrustStore file :/var/lib/ambari-server/keys/ambari-server-truststore
Password for TrustStore:
Re-enter password:
Ambari Server 'setup-security' completed successfully.

Check if ambari views working

If the views are still not working then go to Admin > Manage Ambari > Views. Select a view and check in its settings if cluster name is correct. You can update the cluster name in the view or delete the view and recreate it with correct cluster_name.

Performance tuning for Ambari

The following resources can be followed for performance tunning of ambari:

agent.threadpool.size.max=50
client.threadpool.size.max=100
ecCacheSizeValue=120
http.cache-control=must-revalidate
http.strict-transport-security=max-age=31536000
http.x-content-type-options=nosniff
http.x-frame-options=DENY
http.x-xss-protection=1; mode=block
server.jdbc.connection-pool=internal
server.jdbc.connection-pool.acquisition-size=5
server.jdbc.connection-pool.idle-test-interval=7200
server.jdbc.connection-pool.max-age=0
server.jdbc.connection-pool.max-idle-time=14400
server.jdbc.connection-pool.max-idle-time-excess=0

File /var/lib/ambari-server/ambari-env.sh

export AMBARI_JVM_ARGS="$AMBARI_JVM_ARGS -Xms2048m -Xmx4096m -XX:PermSize=128m -XX:MaxPermSize=128m -XX:MaxPermSize=128m -Djava.security.auth.login.config=$ROOT/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment