Skip to content

Instantly share code, notes, and snippets.

@nsabharwal

nsabharwal/gist:58d2379023407ed2857d4957ad29fb77

Created February 15, 2020 16:54
Show Gist options
  • Save nsabharwal/58d2379023407ed2857d4957ad29fb77 to your computer and use it in GitHub Desktop.
Save nsabharwal/58d2379023407ed2857d4957ad29fb77 to your computer and use it in GitHub Desktop.
Download ZIP
HDP install step - notes
Raw
gistfile1.txt
neeraj_mac:emrfs-cf neerajsab$ ssh -i ~/.ssh/ns-privacera-prod.pem [email protected]
[centos@ip-172-31-41-233 ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/centos/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/centos/.ssh/id_rsa.
Your public key has been saved in /home/centos/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Y3A4qhJucOe2XXxzpdtD6OFcnlCRAYYPPNArIniyUBs [email protected]
The key's randomart image is:
+---[RSA 2048]----+
| .+ .o..o |
| E . *. o |
| . + + . = . |
|. + o..+. . . . |
|o..+o. .S. o. |
|oo.+ o . +oo |
|.o. o o o+o* . |
|.. . o . . o+o+ |
| . . . .. |
+----[SHA256]-----+
[centos@ip-172-31-41-233 ~]$
[centos@ip-172-31-41-233 ~]$ cd .ssh/
[centos@ip-172-31-41-233 .ssh]$ cat id_rsa.pub >> authorized_keys
[centos@ip-172-31-41-233 .ssh]$ chmod 700 ~/.ssh
[centos@ip-172-31-41-233 .ssh]$ chmod 600 ~/.ssh/authorized_keys
[centos@ip-172-31-41-233 .ssh]$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:87HvRkXw4tcRBKR/pBORFMeotGw2BQzlWhWkFF3vc6Q.
ECDSA key fingerprint is MD5:88:a0:bd:da:33:3a:fb:8a:bc:41:5b:6f:73:ff:da:05.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Last login: Sat Feb 15 16:45:23 2020 from 218.42.24.136.in-addr.arpa
[centos@ip-172-31-41-233 ~]$ exit
logout
Connection to localhost closed.
[centos@ip-172-31-41-233 .ssh]$ hostname
ip-172-31-41-233.ec2.internal
[centos@ip-172-31-41-233 .ssh]$ hostname -f
ip-172-31-41-233.ec2.internal
[centos@ip-172-31-41-233 .ssh]$ yum install -y ntp
Loaded plugins: fastestmirror
You need to be root to perform this command.
[centos@ip-172-31-41-233 .ssh]$ sudo su -
[root@ip-172-31-41-233 ~]# yum install -y ntp
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: d36uatko69830t.cloudfront.net
* extras: d36uatko69830t.cloudfront.net
* updates: d36uatko69830t.cloudfront.net
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/4): base/7/x86_64/group_gz | 165 kB 00:00:00
(2/4): extras/7/x86_64/primary_db | 159 kB 00:00:00
(3/4): updates/7/x86_64/primary_db | 6.7 MB 00:00:00
(4/4): base/7/x86_64/primary_db | 6.0 MB 00:00:01
Resolving Dependencies
--> Running transaction check
---> Package ntp.x86_64 0:4.2.6p5-29.el7.centos will be installed
--> Processing Dependency: ntpdate = 4.2.6p5-29.el7.centos for package: ntp-4.2.6p5-29.el7.centos.x86_64
--> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-29.el7.centos.x86_64
--> Running transaction check
---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed
---> Package ntpdate.x86_64 0:4.2.6p5-29.el7.centos will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================================
Installing:
ntp x86_64 4.2.6p5-29.el7.centos base 548 k
Installing for dependencies:
autogen-libopts x86_64 5.18-5.el7 base 66 k
ntpdate x86_64 4.2.6p5-29.el7.centos base 86 k
Transaction Summary
========================================================================================================================================================================================
Install 1 Package (+2 Dependent packages)
Total download size: 701 k
Installed size: 1.6 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/autogen-libopts-5.18-5.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for autogen-libopts-5.18-5.el7.x86_64.rpm is not installed
(1/3): autogen-libopts-5.18-5.el7.x86_64.rpm | 66 kB 00:00:00
(2/3): ntpdate-4.2.6p5-29.el7.centos.x86_64.rpm | 86 kB 00:00:00
(3/3): ntp-4.2.6p5-29.el7.centos.x86_64.rpm | 548 kB 00:00:00
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 9.6 MB/s | 701 kB 00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <[email protected]>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-6.1810.2.el7.centos.x86_64 (installed)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : autogen-libopts-5.18-5.el7.x86_64 1/3
Installing : ntpdate-4.2.6p5-29.el7.centos.x86_64 2/3
Installing : ntp-4.2.6p5-29.el7.centos.x86_64 3/3
Verifying : ntp-4.2.6p5-29.el7.centos.x86_64 1/3
Verifying : ntpdate-4.2.6p5-29.el7.centos.x86_64 2/3
Verifying : autogen-libopts-5.18-5.el7.x86_64 3/3
Installed:
ntp.x86_64 0:4.2.6p5-29.el7.centos
Dependency Installed:
autogen-libopts.x86_64 0:5.18-5.el7 ntpdate.x86_64 0:4.2.6p5-29.el7.centos
Complete!
[root@ip-172-31-41-233 ~]# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
[root@ip-172-31-41-233 ~]#
[root@ip-172-31-41-233 ~]# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
[root@ip-172-31-41-233 ~]# clear
[root@ip-172-31-41-233 ~]# setenforce 0
[root@ip-172-31-41-233 ~]# echo umask 0022 >> /etc/profile
[root@ip-172-31-41-233 ~]# yum install mysql-connector-java*
[root@ip-172-31-41-233 ~]# wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
-bash: wget: command not found
[root@ip-172-31-41-233 ~]# yum install wget
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: d36uatko69830t.cloudfront.net
* extras: d36uatko69830t.cloudfront.net
* updates: d36uatko69830t.cloudfront.net
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-18.el7_6.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================================
Installing:
wget x86_64 1.14-18.el7_6.1 base 547 k
Transaction Summary
========================================================================================================================================================================================
Install 1 Package
Total download size: 547 k
Installed size: 2.0 M
Is this ok [y/d/N]: y
Downloading packages:
wget-1.14-18.el7_6.1.x86_64.rpm | 547 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : wget-1.14-18.el7_6.1.x86_64 1/1
Verifying : wget-1.14-18.el7_6.1.x86_64 1/1
Installed:
wget.x86_64 0:1.14-18.el7_6.1
Complete!
[root@ip-172-31-41-233 ~]# wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
2020-02-15 16:51:41 URL:http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo [376/376] -> "/etc/yum.repos.d/ambari.repo" [1]
[root@ip-172-31-41-233 ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: d36uatko69830t.cloudfront.net
* extras: d36uatko69830t.cloudfront.net
* updates: d36uatko69830t.cloudfront.net
ambari-2.7.3.0 | 2.9 kB 00:00:00
ambari-2.7.3.0/primary_db | 25 kB 00:00:00
repo id repo name status
ambari-2.7.3.0 ambari Version - ambari-2.7.3.0 13
base/7/x86_64 CentOS-7 - Base 10,097
extras/7/x86_64 CentOS-7 - Extras 323
updates/7/x86_64 CentOS-7 - Updates 1,446
repolist: 11,879
[root@ip-172-31-41-233 ~]#
[root@ip-172-31-41-233 ~]# yum install ambari-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: d36uatko69830t.cloudfront.net
* extras: d36uatko69830t.cloudfront.net
* updates: d36uatko69830t.cloudfront.net
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.x86_64 0:2.7.3.0-139 will be installed
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-2.7.3.0-139.x86_64
--> Running transaction check
---> Package postgresql-server.x86_64 0:9.2.24-2.el7_7 will be installed
--> Processing Dependency: postgresql-libs(x86-64) = 9.2.24-2.el7_7 for package: postgresql-server-9.2.24-2.el7_7.x86_64
--> Processing Dependency: postgresql(x86-64) = 9.2.24-2.el7_7 for package: postgresql-server-9.2.24-2.el7_7.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-9.2.24-2.el7_7.x86_64
--> Running transaction check
---> Package postgresql.x86_64 0:9.2.24-2.el7_7 will be installed
---> Package postgresql-libs.x86_64 0:9.2.24-2.el7_7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================================
Installing:
ambari-server x86_64 2.7.3.0-139 ambari-2.7.3.0 352 M
Installing for dependencies:
postgresql x86_64 9.2.24-2.el7_7 updates 3.0 M
postgresql-libs x86_64 9.2.24-2.el7_7 updates 234 k
postgresql-server x86_64 9.2.24-2.el7_7 updates 3.8 M
Transaction Summary
========================================================================================================================================================================================
Install 1 Package (+3 Dependent packages)
Total download size: 359 M
Installed size: 451 M
Is this ok [y/d/N]: y
Downloading packages:
(1/4): postgresql-libs-9.2.24-2.el7_7.x86_64.rpm | 234 kB 00:00:00
(2/4): postgresql-9.2.24-2.el7_7.x86_64.rpm | 3.0 MB 00:00:00
(3/4): postgresql-server-9.2.24-2.el7_7.x86_64.rpm | 3.8 MB 00:00:00
warning: /var/cache/yum/x86_64/7/ambari-2.7.3.0/packages/ambari-server-2.7.3.0-139.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY==-] 59 MB/s | 358 MB 00:00:00 ETA
Public key for ambari-server-2.7.3.0-139.x86_64.rpm is not installed
(4/4): ambari-server-2.7.3.0-139.x86_64.rpm | 352 MB 00:00:05
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 62 MB/s | 359 MB 00:00:05
Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Importing GPG key 0x07513CAD:
Userid : "Jenkins (HDP Builds) <[email protected]>"
Fingerprint: df52 ed4f 7a3a 5882 c099 4c66 b973 3a7a 0751 3cad
From : http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : postgresql-libs-9.2.24-2.el7_7.x86_64 1/4
Installing : postgresql-9.2.24-2.el7_7.x86_64 2/4
Installing : postgresql-server-9.2.24-2.el7_7.x86_64 3/4
Installing : ambari-server-2.7.3.0-139.x86_64 4/4
Verifying : postgresql-server-9.2.24-2.el7_7.x86_64 1/4
Verifying : ambari-server-2.7.3.0-139.x86_64 2/4
Verifying : postgresql-libs-9.2.24-2.el7_7.x86_64 3/4
Verifying : postgresql-9.2.24-2.el7_7.x86_64 4/4
Installed:
ambari-server.x86_64 0:2.7.3.0-139
Dependency Installed:
postgresql.x86_64 0:9.2.24-2.el7_7 postgresql-libs.x86_64 0:9.2.24-2.el7_7 postgresql-server.x86_64 0:9.2.24-2.el7_7
Complete!
[root@ip-172-31-41-233 ~]#
[root@ip-172-31-41-233 ~]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1):
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz
jdk-8u112-linux-x64.tar.gz... 100% (174.7 MB of 174.7 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz
Installing JDK to /usr/jdk64/
Successfully installed JDK to /usr/jdk64/
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to /var/lib/ambari-server/resources/jce_policy-8.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip
Installing JCE policy...
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database ... OK
About to start PostgreSQL
Configuring local database...
Configuring PostgreSQL...
Restarting PostgreSQL
Creating schema and user...
done.
Creating tables...
done.
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@ip-172-31-41-233 ~]# ambari-server start
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...............
Server started listening on 8080
DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment