-
-
Save giwa/ed13ac177c1e1a97fba0 to your computer and use it in GitHub Desktop.
$ brew update | |
$ brew install hive |
# ~/.bashrc | |
export HADOOP_HOME=/usr/local/Cellar/hadoop/2.7.0 | |
export HIVE_HOME=/usr/local/Cellar/hive/1.1.1/libexec |
$ tar zxvf mysql-connector-java-5.1.35.tar.gz | |
$ sudo cp mysql-connector-java-5.1.35/mysql-connector-java-5.1.35-bin.jar /usr/local/Cellar/hive/1.1.1/libexec/lib/ |
$ mysql | |
mysql> CREATE DATABASE metastore; | |
mysql> USE metastore; | |
mysql> CREATE USER 'hiveuser'@'localhost' IDENTIFIED BY 'password'; | |
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost'; |
$ cd /usr/local/Cellar/hive/1.1.1/libexec/conf | |
$ cp hive-default.xml.template hive-site.xml |
<property> | |
<name>javax.jdo.option.ConnectionURL</name> | |
<value>jdbc:mysql://localhost/metastore</value> | |
</property> | |
<property> | |
<name>javax.jdo.option.ConnectionDriverName</name> | |
<value>com.mysql.jdbc.Driver</value> | |
</property> | |
<property> | |
<name>javax.jdo.option.ConnectionUserName</name> | |
<value>hiveuser</value> | |
</property> | |
<property> | |
<name>javax.jdo.option.ConnectionPassword</name> | |
<value>password</value> | |
</property> | |
<property> | |
<name>datanucleus.fixedDatastore</name> | |
<value>false</value> | |
</property> | |
<property> | |
<name>hive.exec.local.scratchdir</name> | |
<value>/tmp/hive</value> | |
<description>Local scratch space for Hive jobs</description> | |
</property> | |
<property> | |
<name>hive.downloaded.resources.dir</name> | |
<value>/tmp/hive</value> | |
<description>Temporary local directory for added resources in the remote file system.</description> | |
</property> | |
<property> | |
<name>hive.querylog.location</name> | |
<value>/tmp/hive</value> | |
<description>Location of Hive run time structured log file</description> | |
</property> |
$hive | |
hive > show tables; |
Just to keep things going:
In file1.txt:
Instead of referring to specific versions, I use the links in /usr/local/opt
:
export HADOOP_HOME=/usr/local/opt/hadoop
export HADOOP_COMMON_HOME=/usr/local/opt/hadoop
export HIVE_HOME=/usr/local/opt/hive/libexec
export HCAT_HOME=/usr/local/opt/hive/libexec/hcatalog
I also added HADOOP_COMMON_HOME
and HCAT_HOME
per warning I saw from brew install
and an error from schematool
I am getting the following error while running "hive" after following all the steps:
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME
Unable to determine Hadoop version information.
'hadoop version' returned:
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME
hive
Unable to determine Hadoop version information.
'hadoop version' returned:
hive
Unable to determine Hadoop version information.
'hadoop version' returned:
This worked for me:
export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/libexec
just kill every detail:
chmod 777 /tmp/hive
cp $HADOOP_HOME/share/hadoop/common/lib/guava-27.0-jre.jar $HIVE_HOME/lib/ ; mv guava-19.0.jar guava-19.0.bak # fix version conflicts.
Changes as per the errors I faced:
Note:
wherever you see "hive/1.1.1" replace it with "hive/{version-number}"
in file2.txt
mysql to jdbc drive can be downloaded from:
in file3.txt use:
For the first time users of mysql:
install mysql by: [First Time Users]
brew install mysql
then run the following [First Time Users]
brew services start mysql
[For all users]
in Place of:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE ON metastore.* TO 'hiveuser'@'localhost';
use:
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,ALTER,CREATE, REFERENCES, INDEX ON metastore.* TO 'hiveuser'@'localhost';
in file5.txt:
Also update this property:
in file6.txt:
Also, before running final "hive" command we have to run
./schematool -initSchema -dbType mysql