CODEWITHSUNDEEP
hadoopconfigurationpermissionshdfs
W.P. McNeill
W.P. McNeill

Reputation: 17026

Can't Delete HDFS Directory Via Web Interface Because I'm Dr. Who

I'm trying to delete the directory deepnlptest/models on a local HDFS system using the web interface at http://localhost:50070/explorer.html. My username is wmcneill and permissions on the HDFS directory I am trying to delete and the enclosing one are drwxr-xr-x owned by wmcneill.

$ hdfs dfs -ls deepnlptest
17/11/16 09:36:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 5 items
drwxr-xr-x   - wmcneill supergroup          0 2017-11-15 17:09 deepnlptest/canonical-folders
drwxr-xr-x   - wmcneill supergroup          0 2017-11-08 14:35 deepnlptest/data
drwxr-xr-x   - wmcneill supergroup          0 2017-11-15 16:51 deepnlptest/embeddings
drwxr-xr-x   - wmcneill supergroup          0 2017-11-15 16:53 deepnlptest/models
drwxr-xr-x   - wmcneill supergroup          0 2017-11-09 10:28 deepnlptest/pretrained_embeddings

I get the following error when I try to use the web interface to delete the directory.

enter image description here

Except for deleting, the web interface works fine for browsing HDFS. I am able to do everything on the HDFS system via the command line command hdfs dfs including deleting the directory.

$ hdfs dfs -rm -r deepnlptest/models
17/11/16 09:37:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Deleted deepnlptest/models

I assume that "dr.who" is a default user and I need to add my username to some Hadoop configuration file, but I can't figure out which one.

I'm running Hadoop 2.8.2 on a Mac installed beneath /usr/local/Cellar/hadoop/2.8.2.

Upvotes: 4

Views: 2367

Answers (2)

Morgan Kobeissi
Morgan Kobeissi

Reputation: 573

In case anyone is still looking at this, you can set your user by setting the property in the core-site.xml as such

<configuration>
...
    <property>
        <name>hadoop.http.staticuser.user</name>
        <value>youruserhere</value>
    </property>
...
</configuration>

That should override the dr.who default user and allow you to delete/upload from the web interface.

Upvotes: 4

Ani Menon
Ani Menon

Reputation: 28209

When you access the Web UI, you are accessing the cluster as Dr.Who, not your user.

Dr. Who doesn't have permission to delete in your dir. It is a user to be used for browsing. 

+------------------------------+-----------+----------------------------------------------------+
|                              |           | The user name to filter as,on static web filters   |
| hadoop.http.staticuser.user  |  dr.who   | while rendering content. An example use is the HDFS|
|                              |           | web UI (user to be used for browsing files).       |
+------------------------------+-----------+----------------------------------------------------+

Refer: Who is Dr.Who?

Additional reference: core-default

Upvotes: 2

Related Questions