CODEWITHSUNDEEP
javasecurityauthenticationhadoophdfs
Kong
Kong

Reputation: 9596

HDFS from Java - Specifying the User

I'm happily connecting to HDFS and listing my home directory:

Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://hadoop:8020");
conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
FileSystem fs = FileSystem.get(conf);
RemoteIterator<LocatedFileStatus> ri = fs.listFiles(fs.getHomeDirectory(), false);
while (ri.hasNext()) {
    LocatedFileStatus lfs = ri.next();
    log.debug(lfs.getPath().toString());
}
fs.close();

What I'm wanting to do now though is connect as a specific user (not the whois user). Does anyone know how you specify which user you connect as?

Upvotes: 11

Views: 18736

Answers (2)

Roman Nikitchenko
Roman Nikitchenko

Reputation: 13046

As soon as I see this is done through UserGroupInformation class and PrivilegedAction or PrivilegedExceptionAction. Here is sample code to connect to remote HDFS 'like' different user ('hbase' in this case). Hope this will solve your task. In case you need full scheme with authentication you need to improve user handling. But for SIMPLE authentication scheme (actually no authentication) it works just fine.

package org.myorg;

import java.security.PrivilegedExceptionAction;

import org.apache.hadoop.conf.*;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.FileStatus;

public class HdfsTest {

    public static void main(String args[]) {

        try {
            UserGroupInformation ugi
                = UserGroupInformation.createRemoteUser("hbase");

            ugi.doAs(new PrivilegedExceptionAction<Void>() {

                public Void run() throws Exception {

                    Configuration conf = new Configuration();
                    conf.set("fs.defaultFS", "hdfs://1.2.3.4:8020/user/hbase");
                    conf.set("hadoop.job.ugi", "hbase");

                    FileSystem fs = FileSystem.get(conf);

                    fs.createNewFile(new Path("/user/hbase/test"));

                    FileStatus[] status = fs.listStatus(new Path("/user/hbase"));
                    for(int i=0;i<status.length;i++){
                        System.out.println(status[i].getPath());
                    }
                    return null;
                }
            });
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Upvotes: 27

sadhu
sadhu

Reputation: 1479

If I got you correct, all you want is to get home directory of the user if specify and not the whois user.

In you configuration file, set your homedir property to user/${user.name}. Make sure you have a system property named user.name

This worked in my case.

I hope this is what you want to do, If not add a comment.

Upvotes: 0

Related Questions