CODEWITHSUNDEEP
apache-sparkhadoophive
JUNYAN XU
JUNYAN XU

Reputation: 41

How to fix" Permission denied when creating a hive orc table using spark "?

First, I created a dataframe named 'ProDF'. Then I needed to save proDf as a 'orc' format table.I used the code below:

import org.apache.spark.sql.SaveMode
proDF.write.mode(SaveMode.Overwrite).format("orc").saveAsTable("product_orc_table")

However, Spark shell returned a failure:

message:java.security.AccessControlException: Permission denied: user=junyanxu,access=WRITE,inode="/apps/hive/warehouse/product_orc_table":uma_kanagarajan:hdfs:drwxr-xr-x

I tried change the permission of /apps/hive/warehouse/product_orc_table with the code:

hadoop fs -chmod 777 /apps/hive/warehouse/product_orc_table

But this code was denied and said:

user=junyanxu is not the owner of inode=product_orc_table.

I expect I can create the orc table successfully.

Upvotes: 1

Views: 2471

Answers (1)

rbyndoor
rbyndoor

Reputation: 729

user and group ownership for /apps/hive is mentioned here

to get going and execute your command do hadoop fs -chmod 777 /apps/hive/warehouse/product_orc_table

you need to be a hdfs user before executing following command 

sudo su
su hdfs

Beware this is not a better practice, do follow required permission for hive in mentioned link.

Upvotes: 2

Related Questions