Is there a way to connect to kerberized service using java client API with just credentials and not keytabs

Question

I have written a code that uses keytabs to access the hadoop but i want it to use just the credentials instead of the keytabs, is there a way to do it? Following is the code I am currently using -

public static void loginToHDFSUsingKeytab(Configuration hdfsConf) {

    hdfsConf.set("hadoop.security.authentication", "kerberos");
    hdfsConf.set("dfs.namenode.kerberos.principal", nameNodePrincipal);

    UserGroupInformation.setConfiguration(hdfsConf);
    try {
        UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(loginUser, keytab);
        UserGroupInformation.setLoginUser(ugi);
        System.out.println("Logged in successfully.");
    } catch (IOException e) {
        e.printStackTrace();
    }
}

I came accross few threads that says there is a method called

UserGroupInformation.loginUserFromSubject(subject)

but I do not see that method in UserGroupInformation class. I tried hadoop-commons with 2.2.0 and 3.1.1 versions but this method was not available in either of them.

Update :

I tried authentication using JAAS with following code:

public static void main(String args[]) throws Exception {

        System.setProperty("kerberos.security.enabled", "true");
        System.setProperty("java.security.auth.login.config", jaasFilePath);
        // System.setProperty("javax.security.auth.login.name", loginUser);
        // System.setProperty("javax.security.auth.login.password", password);

        Configuration conf = new Configuration();
        conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
        conf.set("hadoop.security.authentication", "kerberos");
        conf.set("dfs.namenode.kerberos.principal", nameNodePrincipal);
        loginWithPassword(activeNamenodeURI, conf);
        // usingKeytabs(conf);
    }


    private static void loginWithPassword(final String args, final Configuration conf) throws Exception {

        LoginContext loginContext = new LoginContext("com.sun.security.jgss.initiate", new CallbackHandler() {
            @Override
            public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {

                for (Callback callback : callbacks) {
                    System.out.println("inside handle method" + callback.getClass());
                    if (callback instanceof NameCallback) {
                        NameCallback nameCallback = (NameCallback) callback;
                        nameCallback.setName(loginUser);
                    } else if (callback instanceof PasswordCallback) {
                        System.out.println(">>>>>>>>>>>>>>> password : " + password);
                        PasswordCallback passwordCallback = (PasswordCallback) callback;
                        passwordCallback.setPassword(password.toCharArray());
                    } else {
                        System.out.println("exception  ");
                        throw new UnsupportedCallbackException(callback);
                    }
                }
            }
        });
        loginContext.login();

        Subject mySubject = loginContext.getSubject();
        
        Subject.doAs(mySubject, new PrivilegedExceptionAction() {
            public Object run() throws Exception {
                System.out.println("inside run");
                System.out.println("##################System.getProperty(\"java.security.auth.login.config\") : "
                        + System.getProperty("java.security.auth.login.config"));
                FileSystem fs = null;
                try {
                    fs = FileSystem.get(new URI(activeNamenodeURI), conf);
                    FileStatus[] df = fs.listStatus(new Path("/"));
                    for (FileStatus status : df) {
                        System.out.println(status.getPath().toString());
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
                return conf;
            }
        });

    }

The JAAS file content is -

com.sun.security.jgss.initiate {

  com.sun.security.auth.module.Krb5LoginModule required
  principal="user1@DOMAIN"
  tryFirstPass=true
  debug=true;

};

Following is the output on console and the error -

Debug is  true storeKey false useTicketCache false useKeyTab false doNotPrompt false ticketCache is null isInitiator true KeyTab is null refreshKrb5Config is false principal is flbuser@IMPETUS tryFirstPass is true useFirstPass is false storePass is false clearPass is false
Password from shared state is null
        [Krb5LoginModule] tryFirstPass failed with:Password can not be obtained from sharedstate 
inside handle methodclass javax.security.auth.callback.NameCallback
inside handle methodclass javax.security.auth.callback.PasswordCallback
>>>>>>>>>>>>>>> password : user1
        [Krb5LoginModule] user entered username: user1

principal is user1@DOMAIN
Commit Succeeded 

inside run
##################System.getProperty("java.security.auth.login.config") : /user/jaas.conf
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
    at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1662)
    at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1643)
    at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:640)
    at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:92)
    at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:702)
    at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:698)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:698)
    at com.test.KerberosLogin$2.run(KerberosLogin.java:140)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at com.test.KerberosLogin.loginWithPassword(KerberosLogin.java:132)
    at com.test.KerberosLogin.main(KerberosLogin.java:101)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]
    at org.apache.hadoop.ipc.Client.call(Client.java:1347)
    at org.apache.hadoop.ipc.Client.call(Client.java:1300)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
    at com.sun.proxy.$Proxy7.getListing(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy7.getListing(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:482)
    at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1660)
    ... 12 more

It still didn't work, any help would be appreciated.

Answer 1

Your error is telling haven't configured JAAS to use kerberos.

You should look into Jaas. It's a security paradigm for Java. When configured for kerberos it only relies on your kerberos token being present in the correct place. You could initiate this by initiating your kerberos token via a keytab or with your credentials. It basically ensure the security happens outside your code allowing you to just focus on coding.

Here's some sample code to get you started. This config relies on you creating a ticket cache (Use kinit with keytab/principle) with the keytab before running your app. It's not a great method but it's quick to get started with.

Client {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=false
  useTicketCache=true
  doNotPrompt=true;
};

Once you have that configuration down I suggest that you move to using a keytab in Jaas. Here's more config settings you may want to consider.

You want to move towards something like the following. So that the keytab just 'works'. FYI this isn't a running example just a quick copy paste of the settings I"m aware of.

Client {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  useTicketCache=false
  keyTab=/some/path/to/keytab
  principal=host/testhost.eng.sun.com
  renewTGT=true
};