Start hive-metastore

Question

We are trying to start the hive-metastore on our Linux Server but we are facing an issue. If we try to start the hive-metastore service with this command:

sudo hive --service metastore

we get tons of errors like this:

Caused by: java.sql.SQLException: Access denied for user 'hive'@'sandbox.hortonworks.com' (using password: YES) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:935) at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4101) at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1300) at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2337) at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154) at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:792) at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:49) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:154) at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:254) at com.jolbox.bonecp.BoneCP.(BoneCP.java:305) ... 48 more

We also tried to reset the Password for the user hive but it doesn't help.

The reason why we want to start the hive-metastore is, that the port 9083 is not listening on our Server. We also have no idea why the metastore-service is not running anymore after the restart of the Server.

If you know any other commands to restart the hive metastore or any other possible solutions for this problem please let me know.

Thanks in advance

Edit: I displayed the grants for the user hive:

mysql> show grants for 'hive'@'sandbox.hortonworks.com';
+--------------------------------------------------------------------------------+
| Grants for hive@sandbox.hortonworks.com                                        |
+--------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'hive'@'sandbox.hortonworks.com'                |
| GRANT ALL PRIVILEGES ON `metastore`.* TO 'hive'@'sandbox.hortonworks.com'      |
| GRANT ALL PRIVILEGES ON `hive`.`metastore` TO 'hive'@'sandbox.hortonworks.com' |
+--------------------------------------------------------------------------------+
3 rows in set (0.01 sec)

And here i selected host user and Password of the mysql.user table. There is no Password for the user hive!

mysql> select host, user, password from mysql.user;
+-------------------------+------+----------+
| host                    | user | password |
+-------------------------+------+----------+
| localhost               | root |          |
| sandbox.hortonworks.com | root |          |
| 127.0.0.1               | root |          |
| localhost               |      |          |
| sandbox.hortonworks.com |      |          |
| sandbox.hortonworks.com | hive |          |
| localhost               | hive |          |
+-------------------------+------+----------+
7 rows in set (0.01 sec)

Edit: I found an other error-message:

I connected to Ambari with serverip:8080 and checked the status of some services. I get following error-message if i try to start the hive-metastore manually:

Hive Metastore status
CRITICAL: Error accessing Hive Metastore status [14/08/25 04:41:55 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive

The error occurs at the last of 4 steps of starting the Hive-services. The steps "Hive Metastore start" and "Hiveserver 2 start" already completed without errors. The step named "Hive Check execute" failed. Here is the error-message at the stdout:

notice: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh prepare]/returns: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
err: /Stage[2]/Hdp-hcat::Hcat::Service_check/Exec[hcatSmoke.sh prepare]/returns: change from notrun to 0 failed: su - ambari-qa -c 'sh /tmp/hcatSmoke.sh hcatsmokeidd4b2a272_date472514 prepare' returned 1 instead of one of [0] at /var/lib/ambari-agent/puppet/modules/hdp-hcat/manifests/hcat/service_check.pp:54
notice: /Stage[2]/Hdp-hcat::Hcat::Service_check/Hdp-hadoop::Exec-hadoop[hcat::service_check::test]/Hdp::Exec[hadoop --config /etc/hadoop/conf fs -test -e /apps/hive/warehouse/hcatsmokeidd4b2a272_date472514]/Anchor[hdp::exec::hadoop --config /etc/hadoop/conf fs -test -e /apps/hive/warehouse/hcatsmokeidd4b2a272_date472514::begin]: Dependency Exec[hcatSmoke.sh prepare] has failures: true
notice: /Stage[2]/Hdp-hcat::Hcat::Service_check/Hdp-hadoop::Exec-hadoop[hcat::service_check::test]/Hdp::Exec[hadoop --config /etc/hadoop/conf fs -test -e /apps/hive/warehouse/hcatsmokeidd4b2a272_date472514]/Exec[hadoop --config /etc/hadoop/conf fs -test -e /apps/hive/warehouse/hcatsmokeidd4b2a272_date472514]: Dependency Exec[hcatSmoke.sh prepare] has failures: true

Is it possible that the smoke-test is flawed?

Answer 1

I don't why, but the stored password is not correct. I did the following changes and hive metastore and hive works:

1. Connect to mysql and execute the next command to change the hive user password (original password is encrypted and unknown) for "password":

SET PASSWORD FOR 'hive'@'sandbox.hortonworks.com' = PASSWORD('password');

2. Add the following to hive-site.xml

<property> <name>javax.jdo.option.ConnectionPassword</name> <value>password</value> </property>

3. Start Hive Metastore

hive --service metastore

However, Hive Metastore does not start atomatically anymore when I reboot HDP virtual machine.

Answer 2

Grant permissions to hive user in mysql

GRANT ALL ON metastore.* TO 'hive'@'sandbox.hortonworks.com';
GRANT ALL ON metastore TO 'hive'@'sandbox.hortonworks.com';
FLUSH PRIVILEGES;