CODEWITHSUNDEEP
rhadooprstudiokerberosmicrosoft-r
Babu
Babu

Reputation: 41

Kerberos authentication for Windows R

I am trying to connect my HDP cluster from RStudio desktop(Windows) using SparkR package.

Spark init is failing with no credentials error message which seem to be because of missing kerberos credentials. (Exact error messages can be found below) I already have a kerberos ticker but it is found RStudio desktop doesn't support Kerberos authentication and RStudio server pro is what i needed. But, looks like RStudio server Pro can't be installed on Windows.

If i want to stick to my current Windows bases R and RStudio environment, is there any other ways to connect Hadoop?

Also, is there any package in core R itself (without RStudio ) i can use to authenticate with Hadoop cluster?

It looks like i can install Microsoft R on Windows but it doesn't look like it support Kerberos authentication?

sparkR.init(master = "yarn-client",appName = "sparkR", sparkHome = "C:/spark-1.6.2-bin-hadoop2.6",sparkEnvir = list(spark.driver.memory="2g"))

Error in invokeJava(isStatic = TRUE, className, methodName, ...) : 
  java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "HostName/IPAddress"; destination host is: "HostName:PORT; 
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
    at org.apache.hadoop.ipc.Client.call(Client.java:1472)
    at org.apache.hadoop.ipc.Client.call(Client.java:1399)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
    at com.sun.proxy.$Proxy23.getClusterMetrics(Unknown Source)
    at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:202)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethod

Upvotes: 2

Views: 809

Answers (1)

Kirill Glushko - Microsoft
Kirill Glushko - Microsoft

Reputation: 306

Microsoft R Server on Windows, as a client to the Microsoft R Server running on Hadoop does support Kerberos Authentication. It had been tested with Cloudera, Hortonworks HDP, and MapR.

When setting the compute context on Microsoft R Server (client-side) to RxHadoopMR(), R scripts can be executed remotely on the Hadoop Cluster, and, as long as all Nodes have valid Kerberos Tickets, you should be all set.

Please see: https://msdn.microsoft.com/en-us/microsoft-r/rserver-install-hadoop for installing Microsoft R Server on Hadoop.

and: https://msdn.microsoft.com/en-us/microsoft-r/rserver-install-windows for installing Microsoft R Server on Windows.

Upvotes: 1

Related Questions