Created
November 20, 2020 17:07
-
-
Save thanoojgithub/528cbcc042998068add7261dfec93124 to your computer and use it in GitHub Desktop.
How to run HiveServer2 (Hive 2.3.3) on ubuntu 20.04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pre-requisites: | |
1. Expecting Hadoop and Hive is well configured | |
2. hive CLI is working as expected | |
then, we can try running HiveServer2 | |
hduser@thanoojubuntu-Inspiron-3521:~/softwares/apache-hive-2.3.3-bin/conf$ hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console | |
2020-11-20 21:25:32: Starting HiveServer2 | |
SLF4J: Class path contains multiple SLF4J bindings. | |
SLF4J: Found binding in [jar:file:/home/hduser/softwares/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] | |
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. | |
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] | |
2020-11-20T21:25:38,276 INFO [main] session.SessionState: Created HDFS directory: /tmp/hduser/hduser/85cedd8b-b126-4f71-8d0c-2bcfb83fd759 | |
2020-11-20T21:25:38,307 INFO [main] session.SessionState: Created local directory: /tmp/hduser/85cedd8b-b126-4f71-8d0c-2bcfb83fd759 | |
2020-11-20T21:25:38,315 INFO [main] session.SessionState: Created HDFS directory: /tmp/hduser/hduser/85cedd8b-b126-4f71-8d0c-2bcfb83fd759/_tmp_space.db | |
2020-11-20T21:25:38,352 INFO [main] sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=85cedd8b-b126-4f71-8d0c-2bcfb83fd759, clientType=HIVESERVER2] | |
2020-11-20T21:25:38,357 WARN [main] session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory. | |
2020-11-20T21:25:39,136 INFO [main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore | |
2020-11-20T21:25:39,181 INFO [main] metastore.ObjectStore: ObjectStore, initialize called | |
2020-11-20T21:25:41,323 INFO [main] metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" | |
2020-11-20T21:25:43,541 INFO [main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL | |
2020-11-20T21:25:43,547 INFO [main] metastore.ObjectStore: Initialized ObjectStore | |
2020-11-20T21:25:43,708 INFO [main] metastore.HiveMetaStore: Added admin role in metastore | |
2020-11-20T21:25:43,723 INFO [main] metastore.HiveMetaStore: Added public role in metastore | |
2020-11-20T21:25:43,756 INFO [main] metastore.HiveMetaStore: No user is added in admin role, since config is empty | |
2020-11-20T21:25:43,884 INFO [main] metastore.HiveMetaStore: 0: get_all_functions | |
2020-11-20T21:25:43,884 INFO [main] HiveMetaStore.audit: ugi=hduser ip=unknown-ip-addr cmd=get_all_functions | |
2020-11-20T21:25:43,923 INFO [main] service.CompositeService: Operation log root directory is created: /home/hduser/softwares/apache-hive-2.3.3-bin/conf/${system:java.io.tmpdir}/${system:user.name}/operation_logs | |
2020-11-20T21:25:43,930 INFO [main] service.CompositeService: HiveServer2: Background operation thread pool size: 100 | |
2020-11-20T21:25:43,930 INFO [main] service.CompositeService: HiveServer2: Background operation thread wait queue size: 100 | |
2020-11-20T21:25:43,931 INFO [main] service.CompositeService: HiveServer2: Background operation thread keepalive time: 10 seconds | |
2020-11-20T21:25:43,962 INFO [main] metastore.HiveMetaStore: 0: get_all_databases | |
2020-11-20T21:25:43,963 INFO [main] HiveMetaStore.audit: ugi=hduser ip=unknown-ip-addr cmd=get_all_databases | |
2020-11-20T21:25:43,976 INFO [main] metastore.HiveMetaStore: 0: get_all_tables: db=default | |
2020-11-20T21:25:43,976 INFO [main] HiveMetaStore.audit: ugi=hduser ip=unknown-ip-addr cmd=get_all_tables: db=default | |
2020-11-20T21:25:44,002 INFO [main] metastore.HiveMetaStore: 0: get_multi_table : db=default tbls= | |
2020-11-20T21:25:44,003 INFO [main] HiveMetaStore.audit: ugi=hduser ip=unknown-ip-addr cmd=get_multi_table : db=default tbls= | |
2020-11-20T21:25:44,220 INFO [main] server.Server: jetty-7.6.0.v20120127 | |
2020-11-20T21:25:44,258 INFO [Thread-9] thrift.ThriftCLIService: Starting ThriftBinaryCLIService on port 10000 with 5...500 worker threads | |
2020-11-20T21:25:44,301 INFO [main] webapp.WebInfConfiguration: Extract jar:file:/home/hduser/softwares/apache-hive-2.3.3-bin/lib/hive-service-2.3.3.jar!/hive-webapps/hiveserver2/ to /tmp/jetty-0.0.0.0-10002-hiveserver2-_-any-/webapp | |
2020-11-20T21:25:44,534 INFO [main] handler.ContextHandler: started o.e.j.w.WebAppContext{/,file:/tmp/jetty-0.0.0.0-10002-hiveserver2-_-any-/webapp/},jar:file:/home/hduser/softwares/apache-hive-2.3.3-bin/lib/hive-service-2.3.3.jar!/hive-webapps/hiveserver2 | |
2020-11-20T21:25:44,602 INFO [main] handler.ContextHandler: started o.e.j.s.ServletContextHandler{/static,jar:file:/home/hduser/softwares/apache-hive-2.3.3-bin/lib/hive-service-2.3.3.jar!/hive-webapps/static} | |
2020-11-20T21:25:44,625 INFO [main] server.AbstractConnector: Started [email protected]:10002 | |
2020-11-20T21:25:44,630 INFO [main] http.HttpServer: Started HttpServer[hiveserver2] on port 10002 | |
cd $HIVE_HOME/conf | |
gedit hive-site.xml | |
-------------------- | |
<property> | |
<name>hive.server2.enable.doAs</name> | |
<value>false</value> | |
<description> | |
Setting this property to true will have HiveServer2 execute | |
Hive operations as the user making the calls to it. | |
</description> | |
</property> | |
cd $HADOOP_HOME/etc/hadoop/ | |
gedit core-site.xml | |
--------------------------- | |
<property> | |
<name>hadoop.proxyuser.hduser.groups</name> | |
<value>*</value> | |
</property> | |
<property> | |
<name>hadoop.proxyuser.hduser.hosts</name> | |
<value>*</value> | |
</property> | |
------------------------------------------------------------------------------------------- | |
hduser@thanoojubuntu-Inspiron-3521:~/softwares/apache-hive-2.3.3-bin/conf$ diff hive-default.xml.template hive-site.xml | |
39c39 | |
< <value>/tmp/hive</value> | |
--- | |
> <value>/tmp/${user.name}</value> | |
75c75 | |
< <value>${system:java.io.tmpdir}/${system:user.name}</value> | |
--- | |
> <value>/tmp/${user.name}</value> | |
80c80 | |
< <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value> | |
--- | |
> <value>/tmp/${user.name}_resources</value> | |
530c530 | |
< <value>mine</value> | |
--- | |
> <value>Password!23</value> | |
545c545 | |
< <value>jdbc:derby:;databaseName=metastore_db;create=true</value> | |
--- | |
> <value>jdbc:mysql://localhost/metastore_db?createDatabaseIfNotExist=true</value> | |
1020c1020 | |
< <value>org.apache.derby.jdbc.EmbeddedDriver</value> | |
--- | |
> <value>com.mysql.cj.jdbc.Driver</value> | |
1045c1045 | |
< <value>APP</value> | |
--- | |
> <value>hiveusr</value> | |
3976c3976 | |
< <value>${system:java.io.tmpdir}/${system:user.name}/operation_logs</value> | |
--- | |
> <value>/tmp/${user.name}/operation_logs</value> | |
4359c4359 | |
< <value>true</value> | |
--- | |
> <value>false</value> | |
hduser@thanoojubuntu-Inspiron-3521:~/softwares/apache-hive-2.3.3-bin/conf$ | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
hduser - user running HiveServer2