CODEWITHSUNDEEP
hadoophdfs
lixinso
lixinso

Reputation: 793

How to find out the 'group' name of a Hadoop user?

User rok uploaded file and set the permission to 770. The file on HDFS looks like this:

-rw-rw---- 3 rok hdfs  filename1

I'm using ksc user to consume the data uploaded by rok user. So first, I'd like to make sure that ksc has permission for that file filename1.

How do I find out the group name of my user ksc? Does user belong to hdfs group in Hadoop?

BTW, if I upload a file to Hadoop, the file permission looks like: 

-rw-r--r-- 3 ksc ksc filename2

The local info on my Linux of ksc user is :

uid=504(ksc) gid=502(ksc) groups=502(ksc)

Upvotes: 6

Views: 25463

Answers (3)

Grizz
Grizz

Reputation: 339

The way that Hadoop maps users to groups is configurable, so HDFS groups may not be the same as the Unix groups. Also note that if your Hadoop configuration does use the Unix user-group mappings, it will use the unix mappings on the NameNode. Also note that the NameNode caches the mappings for a period of time, so any changes you make may not be available until the cache is expired/refreshed.

As for checking, in addition to what is already mentioned you can check the actual system file that contains the mappings like this if you have root access:

grep <user or group> /etc/group

More here: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/GroupsMapping.html

Upvotes: 0

RANJIT M
RANJIT M

Reputation: 161

Use the command below:

$hdfs groups ksc

It gives all of the groups user ksc belongs to.

Upvotes: 14

Ross
Ross

Reputation: 242

HDFS follows the traditional style of Linux file system permssions. To determine the group of ksc, use groups ksc if you are on Linux.

-rw-rw---- 3 rok hdfs filename1 will give you read/write permissions only if you are part of the hdfs group. Judging from your output, I'm thinking you're not.

You will need to do one of the following:

  • Change rok's file permissions to 664 (read permissions for all users), which is pretty insecure
  • Have ksc added to the hdfs group, more secure

The choice is yours...

Consult the following links for more information:

http://www.comptechdoc.org/os/linux/usersguide/linux_ugfilesp.html

https://hadoop.apache.org/docs/r1.1.1/hdfs_permissions_guide.html

Upvotes: 2

Related Questions