CODEWITHSUNDEEP
hadoopusergroups
Aguinore
Aguinore

Reputation: 91

Usergroup for hadoop on windows

Is there any way to add user to hdfs group (any) on Windows platform? I can't assign Windows user itself to group because I have no admin privileges on the machine.

The more wide problem is when I try to run mapred task on hadoop, it sleeps and warns me with "security.UserGroupInformation: No groups available for user". If there is any workaround, I will be glad to know it.

Upvotes: 1

Views: 929

Answers (1)

Chris Nauroth
Chris Nauroth

Reputation: 9844

The more wide problem is when I try to run mapred task on hadoop, it sleeps and warns me with "security.UserGroupInformation: No groups available for user".

As of Apache Hadoop 2.3.0 (and also later versions), Hadoop supports a capability to define static group mappings inside Hadoop's configuration (core-site.xml). Apache JIRA HADOOP-10142 tracked development of this feature.

For example, you could specify in core-site.xml that user "myuser" belongs to a single group: "mygroup". When static group mapping is defined, then all further group lookup mechanisms inside Hadoop are bypassed, so you won't see this error anymore.

Here is the documentation of the property from core-default.xml

<property>
  <name>hadoop.user.group.static.mapping.overrides</name>
  <value>dr.who=;</value>
  <description>
    Static mapping of user to groups. This will override the groups if
    available in the system for the specified user. In otherwords, groups
    look-up will not happen for these users, instead groups mapped in this
    configuration will be used.
    Mapping should be in this format.
    user1=group1,group2;user2=;user3=group2;
    Default, "dr.who=;" will consider "dr.who" as user without groups.
  </description>
</property>

If you'd like to use this feature, then you'll need to restart Hadoop daemons after making the configuration change.

Using this feature is a trade-off, as it will generally be less maintainable than relying on the OS for group resolution. However, it can be helpful for managing the frequent group lookups on service accounts like this, with rarely changing group memberships. In environments that source group membership from LDAP, it also can greatly reduce load on the LDAP infrastructure by preventing those group lookups.

Upvotes: 2

Related Questions