Hive 0.14.0 not starting

Question

I have hadoop 1.2.1 and i have install hive 0.14.0 on single node

$ hive

Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-0.14.0.jar!/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxrwxr-x

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxrwxr-x
at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:529)
at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:478)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:430)
... 7 more

The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxrwxr-x.

I use hadoop fs -chmod g+w /tmp/hive but not working.

Answer 1

Check the value for the below tag on hive-site.xml, then change the permission for the folder mentioned

<property>
<name>hive.exec.local.scratchdir</name>
<value>/tmp/mydir</value>
<description>Local scratch space for Hive jobs</description>
</property>

hadoop fs -rmr /tmp/mydir;
hadoop fs -mkdir /tmp/mydir;
hadoop fs -chmod 777 /tmp/mydir;
hadoop fs -chmod -R 777 /tmp/mydir;

Answer 2

I did a little bit of experimentation with this and thought it might be useful to someone.

When hive 0.14.0 is started without first creating /tmp/hive in HDFS, that directory is created with mode 711.

drwx--x--x   - hadoop supergroup          0 2014-12-08 18:47 /tmp/hive

If instead one creates the directory via hadoop dfs -mkdir /tmp/hive it defaults to mode 755.

drwxr-xr-x   - hadoop supergroup          0 2014-12-09 11:13 /tmp/hive

The minimum permissions required to allow hive to start without errors is 733.

hadoop dfs -chmod 733 /tmp/hive

Resulting in the following and hive starting successfully.

drwx-wx-wx   - hadoop supergroup          0 2014-12-09 11:13 /tmp/hive

This leads me to believe that hive 0.14.0 is doing the wrong thing when it creates that directory.

Answer 3

Update the permission of your /tmp/hive HDFS directory using the following command

hadoop fs -chmod 777 /tmp/hive

If so can you remove /tmp/hive on both local and hdfs.

hadoop fs -rm -r /tmp/hive;  
rm -rf /tmp/hive

Only temporary files are kept in this location. No problem even if we delete this, will be created when required with proper permissions.